Last modified on 28 December 2009, at 20:16

Lush/Compiling

Lush is an interpreted language, but individual functions and class methods may be compiled, either to just speed it up, or to enable inline C/C++ code.

Compiling pure lush functionsEdit

Functions and class methods may be compiled, with a few modifications. Here's a simple function that we'd like to compile:

;; Prints the size, sum, and mean of a vector
(de print-mean (vec)
  (let* ((v-size (idx-dim vec 0))
         (v-sum (idx-sum vec)
         (v-mean (/ v-sum v-size))
    (printf "Sum = %f, size = %d, mean = %f\n" v-sum v-size v-mean)
    ())

Below we've added some type declarations make the function compilable, and followed it with a compilation command:

;; Prints the size, sum, and mean of a vector
(de print-mean (vec)
  ((-idx1- (-double-)) vec)
  (let* ((v-size (idx-dim vec 0))
         (v-sum (idx-sum vec)
         (v-mean (/ v-sum v-size))
    ((-int-) v-size)
    (printf "Sum = %f, size = %d, mean = %f\n" v-sum v-size v-mean)
    ())

; compile "print-mean"
(dhc-make () print-mean)

To be compilable, lush functions must:

  • Declare the types of any objects of ambiguous type.
  • Use only compilable functions and expressions.
  • List the function in the compilation statement at the end of the source file.

Declare typesEdit

In the above function, the lines immediately following the arguments declare the argument types. Note the function doesn't bother declaring the types of any other variable. This is because you only need to declare types for function arguments and non-double numeric types. This is because function argument types aren't evident at compile-time unless specified, and lush assumes that all numbers are doubles unless told otherwise.

All type declarations are of the form:

((type_name) var_name)

For example:

((-int-) my-int)

The following lists the type names of the compilable lush types, along with their C++ equivalents:

Lush type C++ type Description
-int- int 32-bit integer
-float- float 32-bit real
-double- double 64-bit real
-short- short 16-bit integer
-byte- char 8-bit integer
-ubyte- unsigned char 8-bit unsigned integer
-gptr- void* Type-indpenedent pointer (size is platform dependent).
-gptr- "std::string" std::string* Type-specific pointer (same size as -gptr-/void*)
-str- ? Lush string
-idx- (-double-) ? Lush tensor (-double- could be any basic type name.*)
-obj- (htable) ? Any other class (htable is the class name for Lush hash tables).

* Tensors can't contain type-specific pointers. In other words, the following is not allowed:

;; NOT ALLOWED: a tensor of char* pointers
((-idx1- (-gptr- "char")) my-char-pointer-tensor)

But this would be fine:

;; A tensor of void* pointers
((-idx2- (-gptr-)) my-pointer-tensor)

Only use compilable functionsEdit

Unfortunately, not all lush functions or expressions are compilable. Most functions that are not compilable are those that involve list manipulation. More generally, any particularly "lispy" feature of lush (lambda forms, hooks, code manipulation) will usually not be compilable. To see whether a particular function is compilable, use the function compilablep:

? (compilablep double-matrix)
= t
? (compilablep range)
= () 

Note that compilablep will only take single function and macro names, not complex lush expressions.

While list manipulation is generally non-compilable, an expression with a list in it is not necessarily non-compilable. For example, the function idx-transclone necessarily takes a list as one of its arguments, but is still compilable.

Add a compilation commandEdit

After making your functions compilation-friendly, you must call a compilation function at the end of the source file to actually compile them. Here's a function and two class methods, followed by the compilation function dhc-make:

(de some-func (...)
  ... )

(de my-class object
  ... )

(defmethod my-class method-1 (...)
  ... )

(defmethod my-class method-2 (...)
  ... )

(dhc-make ()
          (some-func)
          (my-class method-1
                    method-2))

The compilation function actually does 3 things:

  • Transcribes the functions/methods into equivalent C code. The C source files for the file src-dir/src-file.lsh will be in src-dir/C/src_file.c.
  • Compiles this C source file into an object file using gcc.
  • Links the resulting object file into memory.

The compilation function only runs if the C source file is older than the Lush source file.

Mixing in C/C++ codeEdit

You can insert C/C++ code into lush functions using the #{ ... #} construct:

(de 

You can interleave lush and native code pretty densely:

Accessing lush data from C/C++ codeEdit

The table below shows how to access different lush objects from C/C++ code:

Type lush name C/C++ name
basic type my-double $my_double
tensor type my-float-tensor float* raw_data = IDX_PTR( $my_float_tensor, float )
string my-str char* c_string = $my_str->data
object slots :my-obj:some-slot $my_obj->some_slot
object method (==> my-obj some-method arg1) $my_obj->vtbl->M_some_method($arg1)
global function (add-numbers num1 num2) C_add_numbers($num1, $num2)
global constant @MY_CONSTANT MY_CONSTANT

Different compilation functionsEdit

dhc-makeEdit

dhc-make-with-libsEdit

dhc-make-with-c++Edit

C code in the compilation functionEdit

You can stick preprocessor statements and other C code right into the compilation function:

#include and #defineEdit

C functions that call the Lush functionsEdit

If there are lots of them, you can put them in .h and .c files for better readability:

(dhc-make ()
          #{
          #include "c_funcs.h"
          #}
          
          lush-func-1
          lush-func-2
          ...
          
          #{
          #include "c_funcs.c"
          #})

Linking to librariesEdit

libload can return null; be careful.

Compiling and linking C/C++ source filesEdit

lushmake. Proper lushmake syntax? The way I do it, it doesn't pay attention to dependencies.

QuirksEdit

The problem of hidden argumentsEdit

One problem with lush functions compiled into C code is that any lush objects that you instantiate within the function become arguments in the C version of the function. These arguments that only show up in the C version of a lush function are called "hidden arguments." the lush function's signature (e.g. the number and types of arguments it takes) can change. This is because of

BooleansEdit

No void return typeEdit

Lush functions that return nil (a.k.a. ()) will return a char in its C incarnation. This can be a problem when you really need your function to not return anything, say when using it as a callback function for some C API that expects a void-returning function:

A nil-returning function in my-src.lsh:

(de my-callback ()
  ...
  ())

...becomes a char-returning function in C/my_src.c

char* C_my_callback(){
...
}

In this case, your best bet is to wrap the lush function in a C function that returns void. The question then is: how do I compile this wrapper function? The simplest way is to stick it right in the compilation function at the end of my-src.lsh:

(dhc-make ()
          #{ void callback_wrapper(); #}
          my-callback
          ... other lush functions & methods ...
          ...
          ...
          #{ void callback_wrapper(){ C_my_callback(); } #} )

Stack or Heap?:

No coutEdit

Dynamic allocation of wrapped C++ classes: the casting bug

Issues to consider before compilingEdit

There are several issues to consider before deciding whether or not you really want to compile your lush code.

to be compiled can be more of a hassle than writing the same code in c/c++, for a number of reasons:
  • Compilable Lush isn't as feature-rich or visually clean as C++
  • Lush compiler errors can be even more opaque than those of C/C++ compilers, especially if macros are involved.
  • Loss of debuggability