XQuery/URL Rewriting Basics

Motivation

You want to take simple, short, intuitive and well designed incoming URLs and map them to the appropriate structures in your database. You want to achieve the ideal of 'cool URLs' and make your XQuery apps portable within your database and to other databases.

Method

A typical URL in eXist has a format similar to the following:

http://www.example.com:8080/exist/rest/db/app/search.xq?q=apple

You want users to access this page through a cooler, less platform-dependent URL such as:

http://www.example.com/search?q=apple

In order to go transform your URLs into the latter cool form, you need to understand the fundamentals of URLs in eXist.

Parts of a URL

Fundamentally, eXist's URLs consist of 3 parts:

1. The Hostname and Port: In the example above the hostname is www.example.com and the port is 8080
2. The Web Application Context: In the example above the context is /exist
3. The Path: In the example above the path is /rest/db/app/search.xq?q=apple

Customizing an eXist URL can mean targeting 1 or more of the 3 parts.

Rewriting Primer

Some methods below make use of eXist's URL-rewriting facility, that conceptually will let your application follow a MVC (model-view-controller) design. eXist 1.5 comes preconfigured with a working setup that embodies these principles:

1. The collection that lives below /db/myapp/, which is exposed through the REST servlet via /exist/rest/db/myapp/, can at the same time be reached through URL-rewriting in the location /exist/apps/myapp/.
2. Placing a controller.xql inside of /db/myapp/ will determine how the data, a.k.a. model inside of this collection gets presented in the space created by URL-rewriting - so to say: it controls the view at the model.

Please read farther below on how to configure URL-rewriting in version 1.4.1 of eXist to get the same setup.

Customizing URLs

Changing the Port

The port for eXist's default web server (Jetty) is 8080, and it is set in $EXIST_HOME/tools/jetty/etc/jetty.xml line 51. You can modify this file, or you can set the port on startup by setting the -Djetty.port=80 flag upon startup.

Note that how you change the port is different based on how you start eXist. If you start eXist from the bin/startup using a UNIX or DOS shell you must change the startup.sh or startup.bat file. If you start eXist automatically using the UNIT tools/wrapper/exist.sh tools or the Windows Services you need to change the jetty.xml file.

Restart eXist. Now, with this change made, your URL will now look like:

http://www.example.com/exist/rest/db/app/search.xq?q=apple

instead of:

http://www.example.com:8080/exist/rest/db/app/search.xq?q=apple

On Unix (including Mac OS X) and Linux, you will need to run eXist as root in order to bind to port 80. Otherwise the server won't start.

Changing the Web Application Context

To trim your server's web application context from /exist to /, go to line 134 of the same $EXIST_HOME/tools/jetty/etc/jetty.xml file and change the following:

From:

<Arg>/exist</Arg>

To:

<Arg>/</Arg>

Restart eXist. Now, with this change made, your URL will now look like:

http://www.example.com/rest/db/app/search.xq?q=apple

instead of:

http://www.example.com/exist/rest/db/app/search.xq?q=apple

Customizing the Remainder of the URL

In customizing the remainder of the URL, eXist's URL Rewriting feature becomes both powerful and challenging. (See eXist Documentation on URL rewriting for complete documentation on this aspect of URLs in eXist.)

The heart of eXist's URL Rewriting is a file that controls the URLs for its portion of your site; this file is called controller.xql, and you place it at the root of your web application directory. It controls all of the URLs in its directory and in child directories (although child directories can contain their own controller.xql files - more on this later). If your web application is stored on the filesystem, you would likely place 'controller.xql' in the /webapp directory. If your web application is stored in the eXist database, you might put it in the /db collection. In our running example app, where would you store your controller.xql?

Current form: http://www.example.com/rest/db/app/search.xq?q=apple

Goal URL: http://www.example.com/search?q=apple

A natural location for the controller.xql would be the /db/app directory, because the search.xq file (and presumably the other .xq files) are stored in this directory or beneath.

Given this location for our app's root controller.xql, we need to tell eXist to look for the root controller.xql in the '/db/app' directory. We do this by editing the controller-config.xml file in the /webapp/WEB-INF folder. Comment out lines 27-28, and add the following:

<root pattern="/*" path="xmldb:exist:///db/app"/>

Then restart eXist. This new root pattern will forward all URL requests (/*) to the /db/app directory. Now, with this change made, your URL will now look like:

http://www.example.com/search.xq?q=apple

instead of:

http://www.example.com/rest/db/app/search.xq?q=apple

The final step to customizing the URL is to create a controller.xql file that will take a request for /search?q=apple and pass this request to the search.xq file along with the q parameter.

A basic controller.xql file that will accomplish this goal is as follows:

xquery version "1.0";

(:~
    Default controller XQuery.
    Forwards '/search' to search.xq in the same directory 
    and passes all other requests through.
:)

(: Root path: forward to search.xq in the same collection 
   (or directory) as the controller.xql :)
    if (starts-with($exist:path, '/search')) then
        let $query := request:get-parameter("q", ())
        return
            <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
                <forward url="search.xq"/>
                <set-attribute name="q" value="{$query}"/>
            </dispatch>

(: Let everything else pass through :)
else
    <ignore xmlns="http://exist.sourceforge.net/NS/exist">
        <cache-control cache="yes"/>
    </ignore>

Note that the $exist:path variable is a variable that eXist makes available to controller.xql files. The value of $exist:path is always equal to the portion of the requested URL that comes after the controller's root directory. A request to '/search' will cause $exist:path to be '/search'.

Save this query as controller.xql and place it in your /db/app directory. Congratulations! Our URL is now in the very cool form we had envisioned:

http://www.example.com/search?q=apple

instead of:

http://www.example.com/search.xq?q=apple

This $exist:path variable is one of 5 such variables available to controller.xql files. (See the full URL Rewriting documentation for more information on each.) These variables give you very fine control over the URLs requested as well as eXist's own internal paths to your app's resources.

Since you may wish to re-route a URL request based on the URL parameters (e.g. q=apple), you may wish to retrieve the URL parameter using the request:get-parameter() function, and then to explicitly pass this parameter to the target query using the <add-parameter> element, as in the example controller.xql file.

Thus, in customizing the "path" section of the URL, we have actually paid attention to 3 items:

1. The root pattern and path to its root controller directory (recall the <root> element inside the controller-config.xml file)
2. The remainder of the path after the controller directory
3. The URL parameters included as part of the URL

This simple example only touches the surface of what you can do with URL Rewriting. Using URL Rewriting not only gives your apps 'cool URLs', but it also allows your apps to be much more portable, both on your server and in getting your apps onto other servers.

Further considerations

Defining multiple 'roots'

If you want your main app to live in /db/app but you still want to access apps such as the admin app ('/webapp/admin') stored on the filesystem, add a <root> element to controller-config.xml declaring the root pattern you want to associate with the filesystem's /webapp directory. Replace your current root elements with the following:

<root pattern="/fs" path="/"/>
<root pattern="/*" path="xmldb:exist:///db/app"/>

This will pass all URL requests beginning with /fs to the filesystem's webapp directory. All other URLs will still go to the /db/app directory.

Using multiple controller.xql files

While you can get along fine with only one controller.xql (or even none!), eXist allows controller.xql files to be placed at any level of a root controller hierarchy, as defined in the controller-config.xml's <root> element(s). This allows the controller.xql files to be highly specific to the concerns of a given directory. eXist searches for the deepest controller.xql file that matches the deepest level of the URL request, working up toward the root controller.xql.

The importance of order in the controller.xql logic

Make sure that you arrange your conditional expressions in the proper order so that the rules are evaluated in that order, and no rules are inadverently evaluated first. In other words, if another rule matches URLs beginning with '/sea', the URL rewriter would always pass '/search' URLs to that rule instead of your '/search' rule.

Variable Standards

The code inside of controller.xql gets passed some variables in addition to the usual ones. Below controller.xql does not do any forwarding, but instead prints their values, and the path to the document requested, if there is one there…

xquery version "1.0";
declare namespace exist="http://exist.sourceforge.net/NS/exist";
import module namespace text="http://exist-db.org/xquery/text";

declare variable $exist:root external;
declare variable $exist:prefix external;
declare variable $exist:controller external;
declare variable $exist:path external;
declare variable $exist:resource external;

let $document := concat($exist:root, (: $exist:prefix, :) $exist:controller, $exist:path)

return
    <dummy>
        <exist-root>{$exist:root}</exist-root>
        <exist-prefix>{$exist:prefix}</exist-prefix>
        <exist-controller>{$exist:controller}</exist-controller>
        <exist-path>{$exist:path}</exist-path>
        <exist-resource>{$exist:resource}</exist-resource>
        <document>{$document}</document>
    </dummy>

Acknowledgments

Joe Wicentowski contributed the core of this article to the eXist-open mailing list on Mon, 19 Oct 2009. It was subsequently edited by Dan McCreary and Joe Wicentowski into its present form.