6.2. Extending Scilab

The brute force way of getting a better performance is rewriting an existing Scilab script in a low-level language as C, Fortran, or even assembler. This option should be chosen with care, because the rapid prototyping facilities of Scilab are lost. On the other hand if the interface of the function has settled, its performance is known to be crucial and it is of use in future projects then the translation into compiled code could be be worth the time and the grief.

In the first part of this section we compare different ways of integrating an external function into Scilab. We focus on the ease of integration versus the runtime overhead introduced. The second part deals with writing the low-level functions themselves, especially their interfaces.

6.2.1. Comparison Of The Link Overhead

We revive our matrix mirroring example from Section 6.1.2.

Our Fortran-77 version looks like this:


      subroutine mir(n, m, a, dir, b)
*     
*     Mirror n*m-matrix a along direction prescribed
*     by dir.  If dir == 'c' then mirror along the
*     columns, i.e., vertically.  Any other value for
*     dir mirrors along the rows, i.e., horizontally.
*     The mirrored matrix is returned in b. 
*     
      implicit none

*     ARGUMENTS
      integer n, m
      double precision a(n, m)
      character dir*(*)
      double precision b(n, m)

*     LOCAL VARIABLES
      integer i

*     TEXT
      if (dir(1:1) .eq. 'c') then
          do 100, i = 1, m
              call dcopy(n, a(1, m+1-i), 1, b(1, i), 1)
 100      continue
      else
          do 200, i = 1, n
              call dcopy(m, a(n+1-i, 1), n, b(i, 1), n)
 200      continue
      end if

      end

The dcopy(n, x, incx, y, incy, ) is from BLAS level 1, and copies n double precision elements from vector x in increments of incx to y, where it uses increments of incy.

The only thing missing is the glue code between Scilab and mir.


function b = mirf(a, dir)
// interface function for 'mir.f'
// Behavior is the same as mirror()

[n, m] = size(a)
b = zeros(n, m)

if dir == 'r' | dir == 'c' then
    b = fort('mir', ..
             n, 1, 'i', m, 2, 'i', a, 3, 'd', dir, 4, 'c', ..
             'out', ..
             [n, m], 5, 'd')
else
    error('dir must be ''r'' or ''c''')
end

OK, let's lock-and-load. We are ready to rock!


link('mir.o', 'mir')
getf('mirf.sci')

The fast alternative to using fort, which dynamically creates an interface to a C or Fortran function is using intersci, which which creates an interface suitable for static loading.

intersci can create the Fortran glue code for a C or Fortran function to make it callable form the Scilab interpreter. The glue code is compiled (with a Fortran compiler) and linked to Scilab. intersci is described very well in the SCI/doc/Intro.ps. Anyhow, Example 6-4 shows the description (".desc") file for our current example. Finally it will supply us with a Scilab function called mirai(a, dir).

Example 6-4. Sample interface description (".desc")


mirai   a       dir
a       matrix  n       m
dir     string  1
b       matrix  n       m

mir     n       m       a       dir     b
n       integer
m       integer
a       double
dir     char
b       double

out     sequence        b
*
   

We do not want to go into detail here, but a desc-file has three parts separated by blank lines: The description of the Scilab-level function's signature (here: mirai), the same for the low-level function (here: mir), and finally the results' structure. The signatures resemble Fortran or K&R-style C function definitions with the parenthesis missing. The process of passing a desc-file through intersci, compiling the low-level routine and the glue code can be automated. Example 6-5, a snippet of our Makefile.intersci shows the relevant rules.

Example 6-5. Makefile for static Scilab interfaces via intersci


ifdef SCI
SCIDIR := $(SCI)
else
SCIDIR := /site/X11R6/src/scilab
endif

%.f.pre: %.desc
        $(SCIDIR)/bin/intersci $*
        mv $*.f $*.f.pre

%.f: %.f.pre
        perl -pe 's#SCIDIR#$(SCIDIR)#' $< > $@

%.o: %.f
        $(FC) $(FFLAGS) -c $<
   

Running the automatically generated Fortran code through a filter (here: perl) is necessary to fix the lines include 'SCIDIR/routines/stack.h'. After everything is compiled a single Scilab command makes the new routine available to the user.


addinter(['mirai.o', 'mir.o'],  // object files
         'mirai',               // name of interface routine
         'mirai')               // name of new Scilab function

The first argument which almost always is a vector of strings tells Scilab the names of the object files to load. One of them is the interface code made by intersci. The rest are the user routines. The second argument specifies name of entry point into the interface routine. The third parameter is the name the new Scilab function will carry.

Warning Entry point of interface function
 

addinter's second argument must be the name of the interface routine, i.e. the one generated by intersci. Using the low-level function's entry point here causes Scilab to barf (of course).

Why do we go through that tedious process? After all we are in the performance section, so what we want is speed, high speed, or even better the ultimate speed. Now, we compare all the variants in Figure 6-1.

Figure 6-1. Benchmark results for the mirror functions

MIPS vs. matrix size on a 2-way PIII/550

Performance comparison of mirror[1-4], mirf, mirai, and a pure C-program doing the same job on a PIII/550 GNU/Linux box. The straight line between (20 elements, 550 MFLOPS) and (20000 elements, 550 MFLOPS) marks the peak performance of the processor.



If we compare the performance of our three Scilab mirror routines mirror1, mirror2, and mirror3 together with the two incarnations of the hard-coded routine mirf, and mirai, we reach at the following conclusions:

Conclusion: Never underestimate the power of the Emperor^H^H^H^H^H^H^H vectorized Scilab code.

6.2.2. Preparing And Compiling External Subroutines

In this section we will discuss the interfacing of C, C++, Fortran-77, Fortran-9x, or Ada routines with Scilab via link command. We restrict ourselves to the simple case of functions that expect exactly one double precision floating point parameter and return a double precision floating point result. Functions with that signature are required e.g. for the integration routine intg, or the root finder fsolve.

Before we dive into the language specific descriptions, let us point out the main features of Fortran we have be pay attention to when writing an interface in another language.

Function name mangling

A function named FOO (foo, or whatever capitalization is chosen) in the Fortran source can become a different symbol in the object file. This is compiler dependent. Most often an underscore "_" is prepended or appended. Sometimes the name is downcased, sometimes it is upcased.

Tip

The nm(1) command provides easy access to the symbols in an object file.

Call-by-reference

Fortran never passes the value of a parameter, but always a pointer to the parameter.

Arrays in column-major order

Arrays are stored so that their leftmost index varies fastest.

6.2.2.1. Fortran-77

 

Fortran-77 or how do you want to ruin your day?

  L. E. van Dijk

Extending Scilab with Fortran-77 is most straightforward. Scilab is writtin in that language, remember? A Fortran-77 source for function fals could look like this:


      double precision function fals(x)
  
      double precision x
  
      fals = sin(10.0d0 * x)
  
      end

After compilation (e.g. f77 -c fals.f) the compiled code can be linked to Scilab and called with the integration routine.


link('fals.o', 'fals');
[res, aerr, neval, info] = ..
    intals(0.0, 1.0, -0.5, -0.5, 'alg', 'fals')

6.2.2.2. Fortran-9x

 

Fortran-90? Don't worry, it can't get much worse.

  Ch. L. Spiel

A bloated, but portable Fortran-90 source for a function could look like this:


function fsm(x)
    implicit none
    integer, parameter :: idp = kind(1.0d0)

    ! arguments/return value
    real(kind = idp), intent(in) :: x
    real(kind = idp) :: fsm

    ! text
    fsm = exp(x) / (1.0d0 + x*x)
end function fsm

After compilation (e.g. f90 -c fsm.f90) the compiled code can be linked to Scilab and called with an integration routine.


link('fsm.o', 'fsm');
[ires, ierr, neval] = intsm(0.0, 1.0, 'fsm')

6.2.2.3. (ANSI-) C

A simple C function meeting our signature requirements has e.g. this shape:


#include <math.h>
#include "machine.h"

double
C2F(fgen)(const double *x)
{
        if (*x > 0.0)
                return 1.0 / sqrt(*x);
        else
                return 0.0;
}

After compilation (e.g. cc -I/site/X11R6/src/scilab/routines -c fgen.c) the compiled code can be linked to Scilab and called with the integration routine.


link('fgen.o', 'fgen', 'c');
[ires, ierr, neval, info] = intgen(0.0, 1.0, 'fgen')

There are several ways to get the naming convention differences between Fortran and C right. We show three possible solutions for the case where C uses no decoration at all and Fortran appends one underscore.


/* (1) GNU C compiler */
double foo(const double *x) __attribute__((weak, alias ("foo_")));

/* (2) good preprocessor */
#define C2F(name) name##_

/* (3) old preprocessor ;-) */
#define ANOTHERC2F(name) name/**/_

None of the above three examples is portable. Therefore, it is prudent to include SCI/routines/machine.h, which is automatically generated during the Scilab configuration process and thus knosw of the name mangling. Among a lot of other macros it supplies a C-to-Fortran name conversion macro called C2F.

6.2.2.4. C++

A C++ source for a function could look like this:


#include <math.h>

extern "C"
{
        double C2F(fgk)(const double *x);
}

double
C2F(fgk)(const double *x)
{
        return 2.0 / (2.0 + sin(10.0 * M_PI * (*x)));
}

After compilation (e.g. c++ -I/site/X11R6/src/scilab/routines -c fgen.c) the compiled code can be linked to Scilab and called with the integration routine.


link('fgk.o', 'fgk', 'c');
[ires, ierr, neval, info] = ..
    intgk(0.0, 1.0, 'fgk', 0, %eps, '15-31')

See Section 6.2.2.3 for a discussion of the C2F macro.

Further problems arise if the C++ code depends on libraries that have not been linked with Scilab. In the following example myfct_ is correctly declared, but requires sqrt indirectly through a call to subfct.


// linkcxx.cc
#include <complex>

extern "C" {
    void myfct_(const double *re, const double *im);
}

double_complex subfct(double_complex z);

void
myfct_(const double *re, const double *im)
{
    double_complex u(*re, *im);
    double_complex v(subfct(u));
    // do something with v
}

double_complex
subfct(double_complex z)
{
    return 1.0 + 0.5 * sqrt(z);
}

The problem when linking myfct_ with Scilab is not the call to subfct, but the missing complex sqrt function. A listing of the object file's symbols shows the missing function among some functions the (this particular version of g++) compiler silently generates due to inline expansion.


lydia@orion:/home/lydia/tmp $ nm -C linkcxx.o
000000bd t Letext
00000000 ? __FRAME_BEGIN__
00000000 W complex<double> operator/<double>(complex<double> const &, double)
00000000 W complex<float> operator/<float>(complex<float> const &, float)
00000000 W complex<long double> operator/<long double>(complex<long double> const &, long double)
0000008b T main
00000000 T myfct_
         U complex<double> sqrt<double>(complex<double> const &)
00000046 T subfct(complex<double>)

It is up to the programmer to supply all necessary libraries – in the correct order – when linking. For the previous example the following call would succeed (on a libc6 GNU/Linux system):


-->link("linkcxx.o -lstdc++-2-libc6.1-1-2.9.0")
linking files linkcxx.o -lstdc++-2-libc6.1-1-2.9.0  to create a shared executable
shared archive loaded
Link done
 ans  =

    0.

In the case that the compiler documentation lacks information abouth which library defines what symbol, the nm(1) command is the most useful tool to find out.

Additional Caveats

The inclusion of C++ modules into a project whose main() is not written in C++ call for some additional warnings. See also Section 6.3 for a caveat using compilation switches that break the ABI.

Runtime initialization

When it comes to runtime initialization of his/her code, a C++-programmer depends on the linker as a junkie on his dealer. Either the compiler system does it – and does it right, or you have a very very hard time ahead of you. Sidenote: The GNU linker does the Right Thing(tm)!

exceptions

In brief: Get them – all! If the C++ to be linked with Scilab is known to throw exceptions, all interfaced functions of which an exception possibly could escape have to be wrapped in C++-functions that catch these exceptions and translate them into error codes e.g. à la Lapack. Otherwise Scilab is terminated with an abort() call.

6.2.2.5. Ada

For GNAT/Ada the package's interface part pulls in the Fortran interface definitions. Is the simplest case the mathematical functions are only instantiated with the type Double_Precsion. Ada requires to export every function's interface separately, as is clear from the following example.


with Interfaces.Fortran;
use Interfaces.Fortran;
with Ada.Numerics.Generic_Elementary_Functions;

package TestFun is
    package Fortran_Elementary_Functions is new
        Ada.Numerics.Generic_Elementary_Functions(Double_Precision);
    use Fortran_Elementary_Functions;

    function foo(x : Double_Precision) return Double_Precision;
    pragma Export(Fortran, foo);
    pragma Export_Function(Internal => foo,
                           External => "foo_",
                           Mechanism => Reference,
                           Result_Mechanism => Value);
end TestFun;

According to the interface specification the package body looks like this:


package body TestFun is
    function foo(x : Double_Precision) return Double_Precision is
    begin
        return exp(x) / (1.0 + x*x);
    end foo;
end TestFun;

The package is compiled as usual gnatmake -O2 testfun.adb.

Hint: Make sure that there is a GNAT runtime library libgnat-3.12p.so. Your version number may be different, but only the ending ("so") is critical, as libgnat-3.12p.so.1.7 will not make dlopen(3) happy. From now on everything goes downhill, and the function can be linked almost as usual.


link('testfun.o -L/site/gnat-3.12p/lib -lgnat-3.12p', 'foo')

Again, the path to your gnat-library and the version numbers can differ.

In the case of several functions in the package it is preferable to rely on the extended dlopen(3) mechanism, and link the package/library combo with remembering the ID of the shared library.


adacode = link('testfun.o -L/site/gnat-3.12p/lib -lgnat-3.12p', 'foo')

Linking further functions from the library happens by referencing the number of the library.


link(adacode, 'bar')

This saves space (Scilab's TRS) and time (to execute the link). Speaking about saving, users with a loader e.g. GNU ld, capable of incremental linking (e.g. -i, -r, or --relocatable) can of course link testfun.o with the (gnat-)library before linking everything to Scilab. To complete the example, here comes the command-line to archive exactly this:


ld -i -o testfun-lib.o testfun.o -L/site/gnat-3.12p/lib -lgnat-3.12p

In Scilab the arguments to link then reduce to


link('testfun-lib.o', 'foo')

6.2.2.6. Visual C++ by Dave Sidlauskas

This section illustrates the calling of C/C++ routines from the Windows™ version of Scilab using Microsoft™'s Visual C++ compiler. The process is quite simple.

  1. Use VC++ create a DLL containing the C functions.

  2. In Scilab, use link() to load the DLL functions.

  3. Use fort() to run the functions.

In a little more detail:

  1. Use VC++ to create a DLL.

    Start VC++, click FILE, NEW, and select WIN 32 Dynamic Link-Library. Give it a name and location and click OK. Then select Empty DLL and click Finish.

    Prepare a source file and insert it into the project (Project, Add To Project). Then build the project (F7).

    A sample source file is shown below. The declaration extern "C" declspec(dllexport) is critical. Using this, the function name is exported correctly with no name mangling. This type of declaration is covered in the VC++ on-line documentation if you wish more details.

    Also note that C files that are to be executed by a call to fort() are always void, returning no value. Values are returned via pointers in the function parameter list. For example, the parameter *out in matcpy_c is the return value for that function.

    
extern "C" _declspec(dllexport) void matset_c(double *mat,
                                                  const int *nrows,
                                                  const int *row,
                                                  const int *col,
                                                  double *val);
    
    extern "C" _declspec(dllexport) void matcpy_c(const double *in,
                                                  const int *nrow,
                                                  const int *ncol,
                                                  double *out);
    
    
    // matset
    
    // Set element in mat at row and col to val.
    // nrows is number of rows in mat.  Shows row
    // and col reference in a C function.
    // REMEMBER: C row or col = Scilab row or col-1.
    
    void matset_c(double *mat,
                  const int *nrows,
                  const int *row, 
                  const int *col,
                  double *val)
    {
            mat[*row - 1 + (*col - 1)*(*nrows)] = *val;
    }
    
    
    // matcpy
    
    // Function to copy one matrix to another.
    
    void matcpy_c(const double *in,
                  const int *nrow,
                  const int *ncol,
                  double *out)
    {
            int row, col;
    
            for (col = 0; col < *ncol; col++)
                    for (row = 0; row < *nrow; row++)
                            out[row + col*(*nrow)] = in[row + col*(*nrow)];
    
    }
           
    
  2. In Scilab, use link to load the DLL functions.

    
link("path\filename.dll", "FunctionName", "c")
           
    

    The path is wherever you told VC++ to put your output. It is usually something like ProjectName\debug.

    Link uses the WindowsLoadLibrary function to load your DLL. See the VC++ on-line documentation for details.

  3. Use fort() to execute your function.

    Actually it is probably better to prepare a wrapper function to reduce the clutter of fort(). Here is a sample for the matset function above.

    
// Wrapper function for calling C language routine matset_c from SciLab
    
    function mat = matset(mat, row, col, val)
    m = size(mat);
    mat = fort("matset_c",
               mat, 1, "d",
               m(1, 1), 2, "i",
               row, 3, "i",
               col, 4, "i",
               val, 5, "d",
               "out",
               m, 1, "d");
    endfunction
           
    

    A sample Scilab session is shown below:

    
-->link("d:\vc\sci\debug\sci.dll", "matset_c", "c")
    Linking matset_c
    Link done
     ans  =
        0.
    
    -->getf('E:\scilab\source\ctest.sci');
    -->mat = zeros(5, 5);
    -->matset(mat, 3, 3, 16.71)
     ans  = 
    !   0.    0.    0.       0.    0. !
    !   0.    0.    0.       0.    0. !
    !   0.    0.    16.71    0.    0. !
    !   0.    0.    0.       0.    0. !
    !   0.    0.    0.       0.    0. !    
           
    

6.2.2.7. Borland C 5.01 by Enrico Segre

These are the steps for creating a DLL with functions, which is callable from Scilab, using Borland® C 5.01 and to link them into Scilab. The process of creating DLL foo.dll from source foo.c, which defines function foo is also simple. The steps are:

  1. In BCW, create a new DLL project with File/New/Project/Target_type→DLL. Some relevant options are:


                Options/Project/16bitCompiler/entry-exit_code/Windows_DLL_all_functions
                Options/Project/32bitCompiler/callingConvention/C
            

  2. To this project add file foo.c. There is a button for that action in the icon bar.

  3. File foo.c must contain the following code.

    
#define STRICT
    #include <windows.h>
    
    BOOL WINAPI DllEntryPoint(HINSTANCE hinstdll,
                              DWORD fdwReason,
                              LPVOID lpvReserved)
    {
            return 1;
    }
           
    

    and, to define function foo as for example

    
void _export
    foo(const double *a, const double *b, double *c)
    {
            *c = *a + *b;
    }
           
    

    with the keyword _export in front of the function's head. File foo.c can contain more than one exported function, as well as other functions which are not defined with _export, and thus are not entry points for Scilab.

  4. Make the DLL (F9).

In Scilab, link the function with


link('foo.dll', '_foo', 'C')

Note the leading underscore! Execute the function with fort('_foo', ...) or call('_foo', ...), or even better, define a convenient wrapper function.


function foo(a, b, c)
     c = call('_foo',
              a, 1, 'd', b, 2, 'd',
              'out', [1, 1], 3, 'd')
endfunction

6.2.3. Pushing It Further

What? What are you doing in this section? Still not satisfied with your functions' performance?—Sorry, but there are no conventional ways to get more out of Scilab. Tinkering with the interface routines is not worth the effort. Some completely new approach is necessary.

6.2.3.1. Scilab as Prototyping Environment

If a problem is too tough, Scilab still can serve as a rapid prototyping environment. One sister program of Scilab, namely Tela has been written for exactly this purpose. Prototyping with an interpreted language is currently going through a big revival with C (and C++) developers discovering Python.

As whenever optimization is the final goal, an extensive test suite is the base for success. So one way to proceed could be to develop test routines and reference implementation completely in Scilab. The next step is rewriting the routines still in Scilab to match the signatures of for example BLAS/Lapack routines as closely as possible. The test suite can remain untouched in this step. The final step is to migrate the Scilab code to Fortran, C, or whatever, while making extensive use of BLAS/Lapack. Ideally the test suite remains under Scilab and can be used to exercise the new standalone code.