Sage-Code Laboratory
index<--

EVE Data Processing

By using collections and control structures one can load, modify and store data. In this article we review some of the concepts we already know to create useful design patterns for data processing.

Bookmarks:

Processes

A process is a set of tasks that can pass or fail depending on conditions. Any driver, aspect or module can implement processes. Tasks in a process are execute sequential from top down.

Keywords:

task create one step in the process
pass end current task with result "pass"
fail end current task with result "fail"
skip jump over next task or several tasks
resume continue with next task after the one that failed
retry redo on task that raise a recoverable error
over stop process execution and release resources

Main Subroutine

One way to organize your main routine is to divide it into tasks. Each task can resolve one aspect or several aspects that are related. Tasks are executed in serial mode. No matter if a task pass or fail, the process continue until the end if not intrerupted.

Pattern:

# define main process procedure
routine main:
    //local declaration
process
    ** preconditions
    exit if condition;    
    ....
    ** open resources
    ...
    task c1("description") try
        ...
        skip if condition;  
    task c2("description") try
        ...
        over if condition;          
    task c3("description") try
        ...
        pass if condition;
    task c4("description") try
        ...
        fail  if condition;
recover  
    ** exception region
    ...
    retry if condition;    
    ...
    resume if condition;
    ...
release
    ** close resources
    ...    
report
    ** report the errors
    ...    
return;

Inreruptions

Keywords (abort, exit, raise, over) cab ve used to intrerupt a process. These are also called: "transfer" keywords. Depending on what keyword is used the process can execute the recover, release and report regions or not.

List operations

We can add elements to a list or remove elements from the list using next operations:

List concatenation

List concatenation is ready using operator “++”. This operator represent union. Therefore List union act very similar to append, except we add multiple elements.

routine main:
  List[Symbol]: a = ['a','b','c']; //  initialized collection
  List[Symbol]: b = ['1','2','3']; //  initialized collection
  List[Symbol]: c; //  deferred initialization require store for initialization
process
  store c := a ++ b;
  print c; //  ['a','b','c','1','2','3'];
return;

Join() built-in

The join function receive a list and convert elements into a string separated be specified character.

routine main:
  String: str; //  Null String
process
  store str := join([1,2,3],",");
  print str; // "1,2,3"
return;

Split built-in The join function receive a list and convert elements into a string separated be specified character.

routine main:
  List[Integer]: lst; //  Null List
process
  ** initialize new reference for "lst"
  store lst := split("1,2,3",","); 
  print lst; // (1,2,3)
return;

List as queue

Two operations are possible

List as stack

Two operations are possible

Note: There is no protection about using a List as stack or queue.

Other built-ins

Following other functions should be available

Special attributes A list has properties that can be used in logical expressions:

List.empty();  // true or false
List.full();   // true or false

Collection iteration

A special while loop that is executed for each element belonging to a collection.

pattern

routine main:
  ClassName: element := collection.first();
while element is not null process
  ** statements
  ...
  element := collection.next(element);
repeat;

The "element" is local to iteration and is used as control variable.

Example:

routine main:
  List[Symbol]: my_list; //  this list is Null
process  
  store my_list := ['a','b','c','d','e'];
  routine main:
    String: e;
  for element in my_list process
    write element;
    if element == 'd' process
      break; //  early termination;
    else
      write(',');
    done;
  next;
  print; // c,d
return;

Scanning items

Collections have common methods that enable traversal using for loop.

{List, Map, Set}

built-in:

set iteration Map and Set are similar. We can visit all elements using for loop:

Example:

routine main:
  Map: my_map;
process  
  store my_map := {("a":1),("b":2),("c":3)};
  routine main:
    Symbol: key;
    Binary: value;
  for (key:value) in my_map process
    ** print pairs (key:value)
    print('("' + key + '",' + value +')');
  repeat;
return;

Will print:

("a",1)
("b",2)
("c",3)

Using map table

Maps are sorted in memory by key for faster search. It is more difficult to search by value because is not unique and not sorted. To search by value one must create a loop and verify every element. This is called full scan and is very slow so you should never use this routine.

Example:

routine main:
  Map: my_map; //uninitialized collection
process
  ** initialize my_map with values
  store my_map := {(1:'a'),(2:'b'),(3:'c')};
  
  ** check if a key is present in a map collection
  expect 3 in my_map; // pass
return;  

Example:

** create new elements in the map collection
routine main:
  Map(String, String): animals = {}; //empty collection
process
  ** store is not necessary here
  append animals['Bear'] += 'dog';
  append animals['Kiwi'] += 'bird';

  ** verify effect of append  
  print  animals; 
return;

Output:

{('Bear':'dog'),('Kiwi':'bird')}

Example:

routine main:
  Map: animals = {}; //empty collection
process
  ** establish element types S:U
  append animals['Rover'] += "dog";

  ** use direct assignment to create 2 more element
  append animals['Bear'] += "dog";
  append animals['Kiwi'] += "bird";
  
  ** print the collection to console
  print  animals;
return;

Output:

{('Rover':"dog"),('Bear':"dog"),('Kiwi':"bird")}  

String: concatenation

Strings can be concatenated using:

Example:

** this is example of string concatenation
routine main:
  String: str = "";  //Empty String
process
  ** set string value using different operators
  str := "this " & "_string";  //  "this_string"
  str := "this " + " string";  //  "this string"
  str := "this " . " string";  //  "this/string"
  str := "this " - " "      ;  //  "this"  
return;

path concatenation

Two strings can be concatenated using concatenation operator "/" or "\". This operator is used to concatenate "path" strings or URL locations. Notice "\" is also escape character used for string templates.

routine main:
  String: s = ""; //empty string
process
  s := 'te/' / '/st'; //  "te/st" Linux
  s := 'te/' \ '/st'; //  "te\st" Windows
return;

Text functions

Note: The text also support escape sequences like a normal string. In a text literal we do not have to escape the single quote symbols: "'". However we have to escape the double quotes like: "This is "quoted" text". This is very rare since quoted text should use symbols: "« »" like "«quoted»"

** example of collection iteration
routine main:
    List this = ["a","b","c","d","e"];
    Integer i = 0;
    Symbol  e;
process
    while i < this.length() loop
        e := this[i];
        i := i + 1;
        if e  >= "c" then
            write e 
            if e is not this.tail
                wite (',');
            end if;   
        end if;
    else
        print ('i = ' + i);
    end loop;
return;

Read next: Databases