Sage-Code Laboratory
index<--

Eve Processing

Data processing is an important use case of Eve. A process can modify states, initialize variables, add/modify or remove data in collections, make computations and decision statements to resolve problems. Next we analyse some use cases:

Page bookmarks:

Main Process

A process can do one or more jobs. Each job is a try block. Jobs are sequential. Any job can pass or fail but the process continue with the next job, unless an exception is raised or process is intrerupted early by logic conditions.

In case of exception, the workflow jumps into recover region. If none of the cases is handling the error, the finalization region is executed and the process terminates. Errors can propagate to main process if they are not properly handled.

try blocks

job ... try

Pattern:

Next design pattern show how to create several jobs in a process. Having a process with jobs will enhance your ability to crate robust code that can handle errors and prevent catastrophic defects.


# define process with label
process | routine
    ** local declarations
    ....
    abort if condition;
  job (c1,"description") try
    ...
    raise "error" if condition;
    ...
    exit if condition;
    ...
    pass if condition;
    ...
    fail if condition;
    ...
  job (c2,"description") try
    ...
  job (c3,"description") try
    ...
recover
  ** exception handlers
  case error1 then
    ** exception handler
    ...
    exit; -- terminate process
  case error2 then   
    ...
    retry;
  else
    panic; -- stop the program
finalize
    ** close resources
    ...
    ** report the errors
    ...
return;

Process Keywords

The process is created and controlled by many keywords. Mastering these keywords is important to use the process effective. We have chosen keyword chareful to be easy to memorize.

jobcreate one step in the process
trybegin job executable block
passskip the rest of task and continue next
failskip then rest of task and continue next
retry redo the task that raise a recoverable error
resume execute next job, following the one with error
abort ignore all next jobs and skip the finalization
exit ignore all next jobs and execute the finalization
raisecreate a recoverable error, jump to cases
panicjump to finalize and stop the application

Notes

Process interruption

Interruption keywords (abort, over, panic) can be used to interrupt a process but is not the normal way to terminate a process. These keywords are used to terminate without finalization. Only the exit or raise will trigger the finalization region.

Process execution  A process can be executed using "apply" or "start". For one single argument, or no argument the parentheses are not required. For a list of arguments you specify a list of arguments;

#these are all valid process calls
apply process_name; -- call process without arguments
apply process_name argument_value;      -- call with single argument
apply process_name (value, value, ...); -- call with a list of arguments
apply process_name (param:value,...);   -- call with a argument by name
apply process_name (value, value, param:value); -- mix position with names
apply process_name (value, *list_args);         -- use spreading operator
apply process_name (param:value, *map_args);    -- mix names with hashmap

End of process:

A process end with keyword return. When process is terminated, program execution will continue with the next statement. Keywords {over / exit / panic / abort} can terminate a process early.

Example:


# aspect with one parameter
aspect print_this(p:String):
process 
  print p;
return;

# driver with parameters
driver process_call(*args ()String):
process 
  ** number of arguments received:
  cycle:
    new arg: String;
  for arg in args loop;
    apply print_this(arg);
  repeat;
return;

Multi-Process

You can use processes to run in parallel using multiple aspects. Processes ca run on multi-core processors one for each core. Not all processes have same duration.

Parallel system

Parallel System

Random method

The range and domain are iterable and have a number of methods that are useful for data processing but these are not collections but generators. One method is very useful and is going to be used in many examples: random(). It is implemented for all collections.

Examples:

# domain demo
driver random_demo:

  class Small = [0..1:0.1] <: Range;
  class Large = (0..2^64 ) <: Range;

process
  ** you can call random() like this
  print Small.random(); -- 0.3
  print Large.random(); -- 65535

  ** you can call random() like this
  print random(1..2^32:2);       -- 143453566436
  print random(1..2^32:0.25);    -- 43.75
return;

Note: If you wander how is this possible, remember methods do not need paranthesis when there is a single parameter. So the argument can be a range. This is a single argument not two or three arguments.

Array builder

The array builder is a syntax feature that enable you to populate an array with data in a single statement. An array builder can be based on one or more ranges. Before you can populate the array it must be declared.

Syntax:

You can declare and initialize an array in the global area using the array builder. Array total capacity must match the number of elements created by the range. A function can be used to compute the values of all elements.


** declare an Array initislized using range
   set array = [min..max:ratio]: [?]Type;

** declare a new Array using a map function with range
   set array = [ f(x) | x in (min..max:ratio)]: [n]Type;

** declare a new 2d array that is called matrix
   set matrix = [ f(x) | x in (min..max:ration)]: [n,m]Type;

Example:


driver test_matrix:
  set array  :[4]Integer;
  set matrix :[4,4]Integer;
process
** init a vector Array
   let array := [ x | x in (1..10:3)];
   expect array == [1,4,7,10];
   print array;
   print;

** init a matrix Array
   let matrix := [ x * y | x in (1..4) and y in (1..4)];
   print matrix;
return;

output:

 1 4 7 10

 1 2  3  4
 2 4  6  8
 3 6  9 12
 4 8 12 16

Array deconstructor

The deconsructor is basicly the inverse of a builder. You creste individual variables from an Array. This is useful for copy or borrow specific parts of an Array.


#array deconstructor demo
driver array_deconstruct:
  set a :[?]Integer;
process
  ** initialize array from Range
  let a := [1..10]; 

  ** create new variables using deconstructor
  new x, y, *_, z := a;

  ** check new variables
  expect x == 1;
  expect y == 2;
  expect z == 10;

  ** anonymous variable vanish the value
  expect _ == Null;
return;

Note: The spreading statement must specify what is happening with all the elements. Using *_ you can ignore several elements and reach the last element z. You can have only one rest operator "*" in a statement.

Array slicing

An array can be sliced using slice notation [n:m]. A slice is also an array and can be sliced. Slicing an array create a view over existing elements of an array. Before you can slice an array, it must be already initialized.

Named slices

In next example we use slicing notation to create two named slices. One is a dynamic slice, the other is a static slice. When you modify the dynamic slice, also the base array is modified. If you modify the static slice, the base array is unaffected.


driver test_slice:
  set slice :[?]Symbol;
process
  new base: [20]Symbol;
  let base := "this is a test array".split();

  ** create a dynamic slice
  let slice := base[11:14];

  ** check slice content
  expect slice[1] == 't';
  expect slice[#] == 't';
  expect slice == ['t','e','s','t']

  ** join back an array to a string
  print slice.join(); -- "test"

  ** making a static slice;
  new copy :: base[11:14];

  ** alter two characters in slice
  let slice[1] := 'b';
  let slice[2] := 'u';

  ** the copy is not modified
  expect copy.join(); == "test"

  ** base array is now modified;
  print base.join(); -- "this is a bust array"
return;

Limited slices

A limited slice is a slice that starts at a particular position and goes from that position several elements. You can calculate the last element in the slice using an expression that may depend on the last element "#".


driver test_slice:
  set slice, sreve :[?]Symbol;
process
  ** generate Array elements from a Range
  new base  := [1..10];
  new x := 5;

  ** limited slices
  let slice := base[x: x+3];
  let sreve := base[5: #-3].reverse(); 

  ** check the results
  expect slice == [5,6,7];
  expect sreve == [7,6,5];
return;

Anonymous slices

You can use anonymous slice to modify an Array or to print a section of the array. You do not have to assign a variable to the slice. This is a shortcut, and is very useful for handling Array elements in bulk.


driver ano_slicing:
process
  new base:[10]Integer;

  ** using bulk assignment
  let base[1:5][*] := 0;
  let base[6:#][*] := 2;

  ** create a dynamic slice
  expect base == [1,1,1,1,1,2,2,2,2,2];

  ** print 2 elements from the middle
  print base[5:6]; -- [1,2]
return;

Decontructor with rest

You can use deconstructor with rest to create a slice in the same time extract several values. By using operator ":=" you will create a slice but using "::" you will create a new Array.


# deconstructor with slice
process
  ** generate Array elements from a Range
  new base  := [1..8];

  ** create a rest slice
  new x,y,z, *rest  := base;

  ** check the rest slice
  expect rest == [4,5,6,7,8];
  expect (x,y,z) == (1,2,3);
return;


# deconstructor with clone
process
  ** generate Array elements from a Range
  new base  := [1..8];

  ** create a clone from reversed array
  new a,b,c, *clone :: base.reverse(); 

  ** check the clone elements
  expect clone == [5,4,3,2,1];
  expect (a,b,c) == (8,7,6);
return;

List operations

There are many things you can do with a list. You can add/remove elements at both ends. You can deconstruct a list and you can concatenate lists together. Most important features are implemented as built-in functions or special operators:

List builder

You can populate a list from a range using a function or expression. List builder are similar to Array builders. Also you can append in bulk new elements into an existing list.


#list builder demo
driver list_builder:
process
  ** initialize list from range
  new a := (a   | a in ('A'..'F')); 
  print a;

  ** initialize list from another list
  new b := (x^2 | x in (1, 2, 3, 4)); 
  print b;
return;

'A' 'B' 'C' 'D' 'E' 'F'
 1 4 9 16

List deconstructor

The deconsructor is basicly the inverse of a builder. You creste individual variables from a list. This is useful to clone or borrow specific parts of a list.


#list deconstructor demo
driver list_deconstruct:
  set a : List;
process
  ** initialize list from Range
  let a := (1..10); 

  ** create new variables using clone deconstructor
  new x, y, _, *rest, _ :: a;

  ** check new variables
  expect x == 1;
  expect y == 2;

  ** anonymous variable vanish the value
  expect _ == Null;

  ** check the rest 
  print rest;
  print list;
return;

(4,5,6,7,8,9)
(1,2,3,4,5,6,7,8,9,10)

Note: variable "_" is special. It can consume one single element and is always Null. It can be redeclared several times in the same block. The value received is ignored.

Append to list

List is using operator +> or <+ to grow. This operator represent insert or append. You can insert an elemnt or many elements into a list at the beginning or you can add element or list of elements at the end but you can not insert elements in the middle of the list./p> 


#list demo concatenation
driver list_concat:
process
  ** initialized list
  new a := ('a','b'); 
  new b := ('1','3'); 

  ** concatenate two lists
  new c := a <+ b;
  expect c ==  ('a','b','1','3');

  ** concatenate two lists
  new d := b +> b;
  expect c ==  ('1','3','a','b');
return;

Alter a list

You can use "let" with operators { += -= }, to append or remove elements from beginning of list. You can remove any element from a list using "-=". All elements having specified value will be remove. To remove one single element you must use operator <-, or ->


#altering a list
driver list_alter:
  set lst :()Symbol;
process
  ** initialized list
  let lst := ('a','b','c','b'); 

  ** create a new element at the beginning
  let lst += 'z';
  expect lst ==  ('a','b','c','b','z');

  ** remove all "b" from list
  let lst -= 'b';
  expect lst ==  ('a','c','z');

  ** append element to list 
  let a := a <+ 'x';
  expect a ==  ('a','c','z','x');

  ** enque element in list
  let a := 'v' +> a;
  expect a ==  ('v','a','b','z','x');
return;
The list is prepared to support queue and stack. It has all the opperations required. We can implement a class wraper with the sugestive method names.

Note: There is no protection about using a List as stack or queue. Is developer responsability to append and remove correctly using one of the operators: {+=, -=} {:=, <-, ->}.

Queue (FIFO)

FIFO = First in First Out

For queue, two operations are possible


#remove elements from list
driver list_remove:
  ** initialized queue
  set queue := ()Symbol; 
process
  ** enqueue one element
  let queue += "x";
  expect queue ==  ('x');
  ** engueue several elements
  let queue += ('z','y')
  expect queue ==  ('z', 'y', 'x')

  ** create new element and dequeue
  new e: Symbol; let queue := queue -> e; 
  expect queue ==  ('z', 'y');
  expect e == 'x';
return;

Stack

LIFO = Last in Last Out

For stack, two operations are possible


#remove elements from list
driver list_remove:
  ** initialized stack
  set stack := ('a','b','c'); 
process
  ** append element at the beginning
  let Symbol += 'x';
  expect stack ==  ('x','a','b','c');

  ** extract element from the beginning
  new x <- stack;
  expect stack ==  ('x','a','b','c');
  expect x == 'x';
return;

List methods

A List is a class, so it has all required methods and properties to be used in algorithms and expressions. We do not yet forsee all methods bu maybe these would be useful


List.empty();  -- true/false
List.full();   -- true/false
List.capacity; -- list with capacity

Join() method

The join method receive a list and convert elements into a string separated be specified character.


#call List.join() demo
function list_join:
  new str := (1,2,3).join(',');
  expect str == "1,2,3";
return;

Split() method The split method is actually a method of a String, that create a List of strings by splitting a long string into parts. Usually you split by comma, but any symbol can be used. The separator is removed and will not be part of the resulting list.


** string split demo
process list_split:
  new list: ()Integer; --  empty List
  ** initialize new reference for "lst"
  let lst := "1,2,3".split(',');
  expect lst == (1,2,3);
return;

Collection iteration

We can use a  for loop  that is executed for each element belonging to a collection. All collections are iterable. To be iterable a class need to implement an iterable interface.

Example:


#iteration process
driver test_iteration:
  set collection := ('a','b','c');
process
  cycle: 
    new element: Symbol;
  for element is collection loop
     write element;
     write ',' if element is not collection.last;
  repeat;
  print;
return;

Example:

# list iteration
driver list_iteration:
  set my_list: ()Symbol; --  this list is Null
process
  let my_list := ('a','b','c','d','e');
  cycle:
    new e: Symbol;
  for e in my_list loop
    write e;
    if e == 'd' then
      break; --  early termination;
    else
      write(',');
    done;
  repeat;
  print; -- a,b,c,d
return;

Using DataSet

A DataSet is very useful for it's property to hold unique elements sorted. This enable deduplication of elements in a list by converting the list to a DataSet.


# convert a list to a dataset
process
  new list := (1,2,1,2,3,1,2,3,4);

  ** using operator ":>"
  new dset := list :> DataSet;

  ** check the new set
  expect dset == {1,2,3,4};
  
  ** using operator "+>"
  new alt := list +> {};
  expect alt == {1,2,3,4};
return;

Builder

You can create elements of a DataSet from a range using builder.


# demonstrate how to create a set from a range
process
  new dset := {x^2 | x in (2..8:2)};

  ** check the new set
  expect dset == {4,16,36,64};
return;

Difference

You can extract elements out of a DataSet using a list.


# Extract severa lelements;
process
  new test := {1,2,3,4};
  new rest := test - (3,4);

  ** check remaining elements
  expect (1,2) in rest;
  expect (3,4) not in rest;

  ** use modifier -= 
  let test -= (1,3)
  expect test == {2,4};
return;

Intersection

You can intersect 2 lists but the result will be a DataSet.


# Create intersection using &
driver set_intersection_demo:
  set test: DataSet; -- Null set
process
  let test := (1,2,3,4) & (2,3,4,5);
  expect test == {2,3,4};
return;

Using HashMap

Maps are sorted in memory by key for faster search. It is more difficult to search by value because is not unique and not sorted. To search by value one must create a loop and verify every element. This is called full scan and is very slow so you should never use this process.

Map Initialization

Initialize a collection later than declaration.

# local map initialization
driver map_init:
  set my_map :HashMap; -- uninitialized collection
process
  ** initialize my_map with values
  let my_map := {1:'a',2:'b',3:'c'};

  ** check if a key is present in a map collection
  expect 3 in my_map; -- will pass
return;

Create Elements

** create new elements
process map_append:
  set animals: {}(String, String): ; -- empty collection
process
  ** append several elements
  let animals += {"Rob":"dog", "Kiwi":"bird"};

  ** create single element
  new animals["Toto":"parot"]
  print  animals;
return;

Output:

{"Rob":"dog","Kiwi":"bird","Toto":"parot"}

Modify Elements

#update a map
driver map_update:
  set animals : {}(String, String);
process
  ** print the new collection
  print  animals;

  ** modify the collection
  new animals["dogs"]   := 1; -- new element
  new animals["birds"]  := 2; -- new element
  let animals["birds"]  += 3; -- alter element

  ** print the new collection
  print  animals;
return;

Output:

{}
{"dogs":1,"birds":5};

Iteration of Map items

Collections have common methods that enable traversal using for loop.

{List, Map, Set}

built-in:

set iteration Map and Set are similar. We can visit all elements using for loop:

Example:


# HashMap iteration
driver map_iteration:
process
  new my_map := {'a':1,'b':2,'c':3};
  cycle: 
    new key: Symbol;
    new value: Integer;
  for (key: value) in my_map loop
    ** print pairs (key:value)
    print "(#s:#n)" ? (key, value);
  repeat;
return;

Output:

('a':1)
('b':2)
('c':3)

Generate Elements

You can generate elements for a HashMap using a collection builder. This is similar to a List builder, except you generate pair of values instead of a single value. You can use a function to caclulate the value, or you can search values from another collection.

Example:


# HashMap iteration
driver map_generator:
  set my_map = { (x + y):2^y | x in ('a'..'c') and y in (1..3)}: HashMap;
process
  cycle: 
    new key  :String;
    new value:Integer;
  for (key: value) in my_map loop
    ** print pairs (key:value)
    print "(#s:#n)" ? (key, value);
  repeat;
return;

Output:

('a1':2)
('b1':2)
('c1':2)
('a2':4)
('b2':4)
('c2':4)
('a3':8)
('b3':8)
('c3':8)

Using strings:

In next examples we use type inference to create strings. The length of the string is the number of symbols. You can use operator "*" to initialize a long string with the same symbol or a pattern of symbols.


# string generators
process
  ** examples of empty strings
  new short_string := "short string";-- most common form of literal
  new long_line := '-' * 80;         -- a long line of 80 dashes
  new dash_dot  := "-·" * 40; -- a long line -&midot;-&midot;-&midot;

String: concatenation

Strings can be concatenated using operators and modifiers:

Example:

** example of string concatenation
driver string_concat:
  set str: String;  -- empty String
process
  ** set string value using different operators
  let str := "this " + " string"; expect str == "this  string";
  let str := "this " - " string"; expect str == "this string";
  let str := "this/" / " string"; expect str == "this/string";
return;

path concatenation

Two strings can be concatenated using concatenation operator "/". Notice "/" will remove exra "/" from a path to avoid doubling it.


# path concatenation
process
  new s := ""; -- empty string
  let s := "te/" / "/st"; --  "te/st" 
return;

symbols & numbers

You can concatenate a Symbol with a number and you get a new String. You must always put the string first in the expression. If you put the number first, it will not work.


# path concatenation
process
  ** implicit conversion
  new s := 'a' + 1; 
  expect s == "a1"; -- is a string

  ** explicit conversion
  new c := String(1) + 'a'; 
  expect c == "1a"; -- that works
return;

String: builders

You can convert a string to a list of symbols, and a collection into a string by using operator: :> This trick is used to create a string using a builders.


# different ways to convert data into string
process
  ** Range conversion
  new str1 := (0..9) :> String; 
  expect str1 == "0123456789"

  ** DataSet conversion
  new str2 := { a | a in ("A".."F") } :> String; 
  expect str2 == "ABCDEF";

  ** Join a collection
  new str2 := { ("0x" + a) | a in ("A".."F") }.join(","); 
  expect str2 == "0xA,0xB,0xC,0xD,0xE,0xF";
return;

String split

You can split a string in list of words. When you do this, you can specify a symbol or a regular expression. The content of regular expression is removed and not included in the string.

You can split by regular expression and keep the match substring in the result by using "look arrount" regular expression features. It is not our scope to teach regular expressions.


process
  new str   := "Eve is a cool language.";
  new words := str.split(" ");
  print words;
return;

("Eve","is","a", "cool", "language",".");

Text Literals

Strings can contain multiple lines separated with end of line character. Large text literal can be defined on multiple lines. This kind of literal is enclosed in triple quotes. """...""".

Triple quoted string support inside double quotes and also single quote without escape. On double quoted strings you must use escape backslash \" for double quotes.


# declaration of a text literal
driver text_literal:

process
  ** triple quoted string
  new my_text:=""" 
         "Opportunity is missed by most people
      because it is dressed in overalls
      and looks like work."
  """; 
  print my_text;
return;

Output:

    "Opportunity is missed by most people
because it is dressed in overalls
and looks like work."

Note: In a text literal we do not have to escape the single quote symbols like this 'β'. However we have to escape the double quotes like: "This is \"quoted\" text". This is very rare since quoted text should use symbols: "« »" like this: "this «quoted» string" by convention.

Regular Expressions

Regular expressions are special string templates useful to search inside strings. We implement regular expressions a la carte except the delimitor \" must be escaped. Regular expressions are enclosed in double quotes like strings: "/regex/g". A regular expression always start with "/" and ends with "/flags".

Regex operators

Eve has support for two operators: "~" and "!~". These are logic operators. Represent "is like" respective "is not like". It require a string for the left operand and a regular expression for the right operand.


# compare regular expression
process
  ** search " is " in the string
  new found := "this is like" ~ "/\sis\s/"; 
  expect found; -- True

  ** check if string start with "not"
  new not_found := "this is not like" !~ "/^not/";
  expect not_found; -- True 
return;

Note: Regular expressions are powerful but difficult to master. This is a language and like any other language has complex rules. You can create and test regular expressions using a tool. It is not in our scope to teach you regular expressions.

External Reference

You can learn and build regular expressions on this website: https://regexr.com/

String functions

Full inventory of functions will be design in package specification. We enumerate several functions we wish to have but is not our scope yet to enumerate all possible functions.

Note. These functions can be used with class qualifier "String" but also with data literals. For example: "test".split() is equivalend to String.split("test"). Just a trick you can use to simplify your code sometimes.

Read next: Eve Algorithms