Project Wolframe

A subroutine declaration starts with the Keyword SUBROUTINE followed by the subroutine name and optionally some parameter names in brackets ('(' ')') separated by comma. The declared subroutine name identifies the function in the scope of this sourcefile after this subroutine declaration. The name is not exported and the subroutine not available for other TDL modules. With includes described later we can reuse code. The body of the function contains the following parts:

The following pseudocode example shows the parts of a subroutine declaration:

			SUBROUTINE <name> ( <parameter name list> )
			DATABASE <list of database names>
			BEGIN
				...<instructions>...
			END
			

The DATABASE declaration is optional.

A transaction function declaration starts with the Keyword TRANSACTION followed by the name of the transaction function. This name identifies the function globally. The body of the function contains the following parts:

The following pseudo code snippet shows the explained building blocks in transaction functions together:

	
			TRANSACTION <name>
			AUTHORIZE ( <auth-function>, <auth-resource> )
			DATABASE <list of database names>
			RESULT FILTER <post-filter-name>
			PREPROC
				...<preprocessing instructions>...
			ENDPROC
			BEGIN
				...<instructions>...
			END
			AUDIT CRITICAL <funcname> ( ...<parameter>... )

			

The lines with AUTHORIZE,DATABASE and RESULT FILTER are optional. So is the preprocessing block PREPROC..ENDPROC. A simpler transaction function looks like the following:

	
			TRANSACTION <name>
			BEGIN
				...<instructions>...
			END
			

Main processing instructions defined in the main execution block of a subroutine or transaction function consist of three parts in the following order terminated by a semicolon ';' (the order of the INTO and FOREACH expression can be switched):

				
TRANSACTION insertCustomerAddresses
BEGIN
    DO SELECT id FROM Customer
        WHERE name = $(customer/name);
    FOREACH /customer/address
        DO INSERT INTO Address (id,address)
        VALUES ($RESULT.id, $(address));
END

				

Preprocessing instructions defined in the PREPROC execution block of a transaction function consist similar to the instructions in the main execution block of three parts in the following order terminated by a semicolon ';' (the order of the INTO and FOREACH expression can be switched and has no meaning, e.g. FOREACH..INTO == INTO..FOREACH):

				
TRANSACTION insertPersonTerms
PREPROC
    FOREACH //address/* INTO normalized
        DO normalizeStructureElements(.);
    FOREACH //id INTO normalized
        DO normalizeNumber(.);
ENDPROC
BEGIN
    DO UNIQUE SELECT id FROM Person
        WHERE name = $(person/name);
    FOREACH //normalized DO
        INSERT INTO SearchTerm (id, value)
        VALUES ($RESULT.id, $(.));
END

				

An element of the input or a set of input elements can be selected by a path. A path is a sequence of one of the following elements separated by slashes:

A slash at the beginning of a path selects the root element of the transaction function input tree. Two subsequent slashes express that the following node is (transitively) any descendant of the current node in the tree.

Paths can appear as argument of a FOREACH selector where they specify the set of elements on which the attached command is executed on. Or they can appear as reference to an argument in a command expression where they specify uniquely one element that is passed as argument to the command when it is executed.

When used in embedded database statements, selector paths are referenced with $(<path expression>). When used as database function or subroutine call arguments path expressions can be used in plain without '$' and '(' ')' markers. These markers are just used to identify substitution entities.

			
TRANSACTION selectPerson
PREPROC
    FOREACH /person/name
        INTO normalized DO normalizeName( . );
    FOREACH /person
        INTO citycode DO getCityCode( city );
ENDPROC
BEGIN
    FOREACH person
        DO INSERT INTO Person (Name,NormalizedName,CityCode)
            VALUES ($(name),$(name/normalized),$(citycode));
END


			

Database results of the previous instruction are referenced with a '$RESULT.' followed by the column identifier or column number. Column numbers start always from 1, independent from the database! So be aware that even if the database counts column from 0 you have to use 1 for the first column.

As already explained before, database result sets of cardinality bigger than one cannot be addressed if not bound to a FOREACH selection. In statements potentially addressing more than one result element you have to add a FOREACH RESULT quantifier.

For addressing results of instructions preceding the previous instruction, you have to name them (see next section). The name of the result can then be used as FOREACH argument to select the elements of a set to be used as base for the command arguments of the instruction. Without binding instruction commands with a FOREACH quantifier the named results of an instruction can be referenced as $<name>.<columnref>, for example as $person.id for the column with name 'id' of the result named as 'person'.

The 'RESULT.' prefix in references to the previous instruction result is a default and can be omitted in instructions that are not explicitly bound to any other result than the last one. So the following two instructions are equivalent:

			DO SELECT name FROM Company
			    WHERE id = $RESULT.id
			DO SELECT name FROM Company
			    WHERE id = $id
			

and so are the following two instructions:

			FOREACH RESULT
			    DO SELECT name FROM Company
			        WHERE id = $RESULT.id
			FOREACH RESULT
			    DO SELECT name FROM Company
			        WHERE id = $id
			

The result name prefix of any named result can also be omitted if the instruction is bound to a FOREACH selector naming the result. So the following two statements in the context of an existing database result named "ATTRIBUTES" are equivalent:

			FOREACH ATTRIBUTES
			    DO SELECT name FROM Company
			        WHERE id = $ATTRIBUTES.id
			FOREACH ATTRIBUTES
			    DO SELECT name FROM Company
			        WHERE id = $id
			

Database results can be hold and made referenceable by name with the declaration KEEP AS <resultname> following immediately the instruction with the result to be referenced. The identifier <resultname> references the result in a variable reference or a FOREACH selector expression.

TRANSACTION selectDevices
BEGIN
    DO SELECT id FROM DevIdMap
        WHERE name = $(device/name);
    KEEP AS dev;
    FOREACH dev
        DO SELECT key,name,registration
            FROM Devices WHERE sid=$id;
END


				

Subroutine Parameters are addressed like results but with the prefix PARAM. instead of RESULT. or a named result prefix. "PARAM." is reserved for parameters. The first instruction without FOREACH quantifier can reference the parameters without prefix by name.

SUBROUTINE selectDevice( id)
BEGIN
    INTO device
        DO SELECT name FROM DevIdMap
            WHERE id = $PARAM.id;
END

TRANSACTION selectDevices
BEGIN
    DO selectDevice( id );
END

				

Database commands returning results can have constraints to catch certain errors that would not be recognized at all or too late otherwise. For example a transaction having a result of a previous command as argument would not be executed if the result of the previous command is empty. Nevertheless the overall transaction would succeed because no database error occurring during execution of the commands defined for the transaction.

Constraints on database results are expressed as keywords following the DO keyword of an instruction in the main processing section. If a constraint on database results is violated the whole transaction fails and a rollback occurrs.

The following list explains the result constraints available:

				
TRANSACTION selectCustomerAddress
BEGIN
    DO NONEMPTY UNIQUE SELECT id FROM Customer
        WHERE name = $(customer/name);
    INTO address
        DO NONEMPTY SELECT street,city,country
            FROM Address WHERE id = $id;
END


				

Sometimes internal error messages are confusing and are not helpful to the user that does not have a deeper knowledge about the database internals. For a set of error types it is possible to add a message to be shown to the user if an error of a certain class happens. The instruction ON ERROR <errorclass> HINT <message>; following a database instruction catches the errors of class <errorclass> and add the string <message> to the error message show to the user.

We can have many subsequent ON ERROR definitions in a row if the error classes to be caught are various.

			
TRANSACTION insertCustomer
BEGIN
    DO INSERT INTO Customer (name) VALUES ($(name));
    ON ERROR CONSTRAINT
        HINT "Customers must have a unique name.";
END

			

			unique constaint violation in transaction 'insertCustomer'
			-- Customers must have a unique name.
			

We already learned how to define substructures of the transaction function result with the RESULT INTO directive of a TRANSACTION. But we can also define a scope in the result structure for sub blocks. A sub-block in the result is declared with

				INTO <resulttag>
				BEGIN
					...<instruction list>...
				END
			

All the results of the instruction list that get into the final result will be attached to the substructure with name <resulttag>. The nesting of result blocks can be arbitrary and the path of the elements in the result follows the scope of the sub-blocks.

The result of a transaction consists normally of database command results that are mapped into the result with the attached INTO directive. For printing variable values or constant values you can in certain SQL databases use a select constant statement without specifying a table. Unfortunately select of constants might not be supported in your database of choice. Besides that explicit printing seems to be much more readable. The statement INTO <resulttag> PRINT <value>; prints a value that can be a constant, variable or an input or result reference into the substructure named <resulttag>. The following artificial example illustrates this.

			
TRANSACTION doPrintX
BEGIN
  INTO person
  BEGIN
    INTO name PRINT 'jussi';
    INTO id PRINT '1';
  END
END


			

TDL allows the support of different transaction databases with one code base. For example one for testing and demonstration and one for the productive system. We can tag transactions,subroutines or whole TDL sources as beeing valid for one or a list of databases with the command DATABASE followed by a comma separated list of database names as declared in the configuration. The following example declares the transaction function 'getCustomer' to be valid only for the databases DB1 and DBtest.

			
TRANSACTION getCustomer
DATABASE DB1,DBtest
BEGIN
    INTO customer
        DO SELECT * FROM CustomerData
            WHERE ID=$(id);
END


			

The following example does the same but declares the valid databases for the whole TDL file. In this case the database declaration has to appear as first declaration in the file.

			
DATABASE DB1,DBtest

TRANSACTION getCustomer
BEGIN
    INTO customer DO SELECT *
        FROM CustomerData WHERE ID=$(id);
END


			

To reuse code with different context, for example for doing the same procedure on different tables, subroutine templates can be defined in TDL. Subroutine templates become useful when we want to make items instantiable that are not allowed to be dependent on variable arguments. Most SQL implementations for example forbid tables to be dependent on variable arguments. To reuse code on different tables you can define subroutine templates with the involved table names as template argument. The following example defines a transaction using the template subroutine insertIntoTree on a table passed as template argument. The subroutine template arguments are substituting the identifiers in embedded database statements by the passed identifier. Only whole identifiers and not substrings of identifiers and no string contents are substituted.

			
TEMPLATE <TreeTable>
SUBROUTINE insertIntoTree( parentID)
BEGIN
    DO NONEMPTY UNIQUE SELECT rgt FROM TreeTable
        WHERE ID = $PARAM.parentID;
    DO UPDATE TreeTable
        SET rgt = rgt + 2 WHERE rgt >= $1;
    DO UPDATE TreeTable
        SET lft = lft + 2 WHERE lft > $1;
    DO INSERT INTO TreeTable (parentID, lft, rgt)
        VALUES ( $PARAM.parentID, $1, $1+1);
    DO NONEMPTY UNIQUE SELECT ID AS "ID" from TreeTable
        WHERE lft = $1;
END

TRANSACTION addTag
BEGIN
    DO insertIntoTree<TagTable>( $(parentID) )
    DO UPDATE TagTable
        SET name=$(name),description=$(description)
            WHERE ID=$RESULT.id;
END


			

TDL has the possibility to include files for reusing subroutines or subroutine templates in different modules. The keyword INCLUDE followed by the name of the relative path of the TDL file without the extension .tdl includes the declarations of the included file. The declarations in the included file are treated as they would have been made in the including file instead. The following example swhows the use of include. We assume that the subroutine template insertIntoTree of the example before is defined in a separate include file treeOperations.tdl located in the same folder as the TDL program.

			
INCLUDE treeOperations

TRANSACTION addTag
BEGIN
    DO insertIntoTree<TagTable>( $(parentID) )
    DO UPDATE TagTable
        SET name=$(name),description=$(description)
            WHERE ID=$RESULT.id;
END



			

TDL defines hooks to add function calls for auditing transactions. An audit call is a form function call with a structure build from transaction input and some database results. An auditing function call can be marked as critical, so that the final commit is dependent not only on the transaction success but also on the success of the auditing function call. The following two examples show equivalent calls of audit. One with the function call syntax for calls with a flat structure (only atomic parameters) as parameter and one with the parameter build from a result structure of a BEGIN..END block executed. The later one can be used for audit function calls with a more complex parameter structure.

				
TRANSACTION doInsertUser
BEGIN
	DO INSERT INTO Users (name) values ($(name));
	DO SELECT id FROM Users WHERE name = $(name);
END
AUDIT CRITICAL auditUserInsert( $RESULT.id, $(name) )



				

				
TRANSACTION doInsertUser
BEGIN
	DO INSERT INTO Users (name) values ($(name));
	DO SELECT id FROM Users WHERE name = $(name);
END
AUDIT CRITICAL auditUserInsert WITH
BEGIN
	INTO id PRINT $RESULT.id;
	INTO name PRINT $(name);
END



				

In .NET the building blocks for functions called by Wolframe are classes and method calls. The way of defining callable items for Wolframe is restricted either due to the current state of the Wolframe COM/.NET interoperability implementation or due to general or version dependent restrictions of .NET objects exposed via COM/.NET interop. We list here the restrictions:

Functions callable from Wolframe take an arbitrary number of arguments as input and return a structure (struct) as output. The named input parameters referencing atomic elements or complex structures are forming the input structure of the Wolframe function. A Wolframe function called with a structure containing the elements "A" and "B" is implemented in .NET as function taking two arguments with the name "A" and "B". Both "A" and "B" can represent either atomic elements or arbitrary complex structures. .NET functions that need to call global Wolframe functions, for example to perform database transactions, need to declare a ProcProvider interface from Wolframe namespace as additional parameter. We will describe the ProcProvider interface in a separate section of this chapter.

The following simple example without provider context is declared without marshalling and introspection tags. It can therefore not be called by Wolframe. We explain later how to make it callable. The example just illustrates the structure of the exported object with its interface (example C#):

		
using System;
using System.Runtime.InteropServices;

public struct Address
{
    public string street;
    public string country;
};

public interface FunctionInterface
{
    Address GetAddress( string street, string country);
}

public class Functions : FunctionInterface
{
    public Address GetAddress( string street, string country)
    {
        Address rt = new Address();
        rt.street = street;
        rt.country = country;
        return rt;
    }
}


		

First the asseblies with the functions exported to Wolframe have to be build COM visible. To make the .NET functions called from Wolframe COM visible, you have to tick "Properties/Assembly Information" the switch "Make assembly COM visible". Furthermore every object and method that is part of the exported API (also objects used as parameters) has to be tagged in the source as COM visible with [ComVisible(true)].

Each object that is part of the exported API has to be tagged with a global unique identifier (Guid) in order to be adressable. Modules with .NET functions will have to be globally registered and the objects need to be identified by the Guid because that's the only way to make the record info structure visible for Wolframe. The record info structure is needed to serialize/deserialize .NET objects from another interpreter context that is not registered for .NET. There are many ways to create a Guid and tag an object like this: [Guid("390E047F-36FD-4F23-8CE8-3A4C24B33AD3")].

For marshalling function calls correctly, Wolframe needs tags for every parameter and member of a sub structure of a parameter of methods exported as functions. The following table lists the supported types and their marshalling tags:

Decimal floating point and numeric types (DECIMAL) are not yet supported, but will soon be available.

The following C# module definition repeats the example introduced above with the correct tagging for COM visibility and introspection:

		
using System;
using System.Runtime.InteropServices;

[ComVisible(true)]
[Guid("390E047F-36FD-4F23-8CE8-3A4C24B33AD3")]
public struct Address
{
    [MarshalAs(UnmanagedType.BStr)] public string street;
    [MarshalAs(UnmanagedType.BStr)] public string country;
};

[ComVisible(true)]
public interface FunctionInterface
{
    [ComVisible(true)]  Address GetAddress( [MarshalAs(UnmanagedType.BStr)] string street, [MarshalAs(UnmanagedType.BStr)] string country);
}

[ComVisible(true)]
[ClassInterface(ClassInterfaceType.None)]
public class Functions : FunctionInterface
{
    public Address GetAddress([MarshalAs(UnmanagedType.BStr)] string street, [MarshalAs(UnmanagedType.BStr)] string country)
    {
        Address rt = new Address();
        rt.street = street;
        rt.country = country;
        return rt;
    }
}


		

For making the API introspectable by Wolframe, we have to create a TLB (Type Library) file from the assembly (DLL) after build. The type library has to be recreated every time the module interface (API) changes. The type library is created with the program tlbexp. All created type library (.tlb) file that will be loaded with the same runtime environment have to be copied into the same directory. They will be referenced for introspection in the configuration. The configuration of .NET will be explained later.

The type library created with tlbexp has also to be registered. For this you call the program regtlibv12 with your type library file (.tlb file) as argument. The type libary fegistration has to be repeated when the the module interface (API) changes.

Wolframe does not accept local assemblies. In order to be addressable over the type library interface assemblies need to be put into the global assembly cache (GAC). Unfortunately this has to be repeated every time the assembly binary changes. There is no way around. For the registration in the GAC we have to call the program gacutil /if <assemblypath> with the assembly path <assemblypath> as argument. The command gacutil has to be called from administrator command line. Before calling gacutil, assemblies have to be strongly signed. We refer here to the MSDN documentation for how to sign an application.

We have to register the types declared in the assembly to enable Wolframe to create these types. An example could be a provider function returning a structure that is called from a Wolframe .NET function. The structure returned here has to be build in an unmanaged context. In order to be valid in the managed context, the type has to be registered. For the registration of the types in an assembly we have to call the program regasm <assemblypath> with the assembly path <assemblypath> as argument. The command regasm has to be called from administrator command line.

We already found out that Wolframe .NET modules have to be strongly signed. Each strongly signed assembly has such a public key token that has to be used as attribute when referencing the assembly.

We can get the PublicKeyToken of the assembly by calling sn -T <assemblypath> from the command line (cmd) with <assemblypath> as the path of the assembly. The printed value is the public key to insert as attribute value of PublicKeyToken in the Wolframe configuration for each .NET assembly.

The data type 'datetime' is used as interface for date time values.

The data type 'bignumber' is used to reference fixed point BCD numbers with a precision of 32767 digits between -1E32767 and +1E32767.

The function takes a structure as input and writes the result into an output structure. In this example input and output type are the same, but this is not required. It's just the same here for simplicity.

The elements of the function declaration are put into a structure with four elements. The typedef for the InputType and OutputType structures is required, because the input and output types should be recogniceable without complicated type introspection templates. (Template based introspection might cause spurious and hard to understand error messages when building the module).

The function name returns the name of the function that identifies the function in the Wolframe global scope.

The exec function declared as static function with this signature refers to the function implementation.

		
// ... PUT THE INCLUDES FOR THE "Customer" STRUCTURE DECLARATION HERE !

struct ProcessCustomer
{
    typedef Customer InputType; 
    typedef Customer OutputType; 
    static const char* name() {return "process_customer";}

    static int exec( const proc::ProcessorProvider* provider, InputType& res, const OutputType& param);
};


		

Declares the structure and the serialization description of the structure. Structures may contain structures with their own serialization description.

		
#include "serialize/struct/structDescriptionBase.hpp"
#include <string>

namespace _Wolframe {
namespace example {

struct Customer
{
    int ID;                         // Internal customer id
    std::string name;               // Name of the customer
    std::string canonical_Name;     // Customer name in canonical form
    std::string country;            // Country
    std::string locality;           // Locality

    static const serialize::StructDescriptionBase* getStructDescription();
};

}}//namespace


		

Declares 'ID' as attribute and name, canonical_Name, country, locality as tags. The '--' operator marks the end of attributes section and the start of content section.

		
#include "serialize/struct/structDescription.hpp"

using namespace _Wolframe;

namespace {
struct CustomerDescription :public serialize::StructDescription<Customer>
{
    CustomerDescription()
    {
        (*this)
        ("ID", &Customer::ID)
        --
        ("name", &Customer::name)
        ("canonical_Name", &Customer::canonical_Name)
        ("country", &Customer::country)
        ("locality", &Customer::locality)
        ;
    }
};

const serialize::StructDescriptionBase* Customer::getStructDescription()
{
    static CustomerDescription rt;
    return &rt;
}


		

The module declaration needs to include appdevel.hpp and of course all headers with the function and data structure declarations needed. The module starts with the header macro CPP_APPLICATION_FORM_FUNCTION_MODULE with a short description of the module. What follows are the function declarations declared with the macro CPP_APPLICATION_FORM_FUNCTION. This macro has the following arguments in this order:

The declaration list is closed with the parameterless footer macro CPP_APPLICATION_FORM_FUNCTION_MODULE_END. The following example shows an example module declaration:

		
#include "appDevel.hpp"
// ... PUT THE INCLUDES FOR THE "ProcessCustomer" FUNCTION DECLARATION HERE !

#include "customersFunction.hpp"

using namespace _Wolframe;

WF_MODULE_BEGIN( "ProcessCustomerFunction", "process customer function")
WF_FORM_FUNCTION("process_customer",ProcessCustomer::exec,Customer,Customer)
WF_MODULE_END