XQuery/Higher Order Functions

Motivation

You have a sequence of items and you would like to perform several sequential operations on each of the items in the sequence.

Method

You would like to use a single function where you pass a series of functions as parameters to that function.

Background on Functional Languages

Functional languages are languages that treat functions as first-order data types. They frequently have a function that let you pass a list of items and tell it what function to perform on each of those items. Just like XML transformations, functional languages are ideal when you have many small tasks to perform on a large number of items. Functional languages are excellent for these tasks since the actual order that the functions get executed on items does not have to be guaranteed. The developer does not have to be concerned about waiting for a transformation on item 1 to finish before the system starts on item 2.

The Google MapReduce algorithm is an example of functional systems. MapReduce allows a data set such as "all web sites" to be treated as a sequence of items. MapReduce then has different processors each receive small items of work that can be processes independently.

For more on functional languages see Functional programming on Wikipedia and Functional Programming on Wikibooks.

Because XQuery is also a functional language, you can also have the confidence that a large list of items passed off to an XQuery function can run independently on many processors without the concern of incorrect results if the items are processed out of order.

Simple example

In the following example we will declare two functions. We will then process a list of words by applying these functions to each of the items in the sequence. We will do this by passing the function name as an argument to another function.

NOTE: This only appears to work in eXist 1.3. eXist versions 1.2.X have the wrong data type associated with the QName() function.

The eXist system needs to turn each function into a function identifier. To do this it needs to call util:function(). util:function takes two arguments, the qualified name of the function (the prefix and the function name) as well as the arity of the function. In this case the arity of a function is the number of arguments that the function takes. The data type of the first argument must be of type QName. The data type of the arity, the second parameter, is an integer.

util:function($function as xs:QName, $arity as xs:integer) as function

declare namespace fw = "http://www.cems.uwe.ac.uk/xmlwiki/fw";

declare function fw:apply($words as xs:string*, $my-function function(item()) as item())  {
   for $word in $words
   return util:call($my-function,$word)
};

declare function fw:f1($string) {
   string-length($string)
};

declare function fw:f2($string) {
    substring($string,1,1)
};

let $f1 := util:function(QName("http://www.cems.uwe.ac.uk/xmlwiki/fw","fw:f1"),1)

let $mywords := ("red","green","purple")
return 
  <hofs>
     <data>{$mywords}</data>
     <hof>
         <task>length of each string</task>
         <result>{fw:apply($mywords,$f1)}</result>
      </hof>
      <hof>
        <task>Initial letter of each string</task>
        <result>{ fw:apply($mywords, util:function(QName("http://www.cems.uwe.ac.uk/xmlwiki/fw","fw:f2"),1) ) }</result>
      </hof>
  </hofs>