XQuery/Subversion

Motivation

You want to be able to access a Subversion (SVN) repository, including checking out the repository's files directly into the eXist database and committing changed files back to the repository, using XQuery.

Method

A subversion XQuery module has been added to the bleeding edge development version of eXist 1.5. You can use it to query remote subversion servers, and even to check out a remote repository to store the repository's contents in the database. (If you do check out a repository, note that the subversion repository's files, including its many ".svn" files, will be stored directly in your database.) As of May 2011, the subversion module can perform most, but not all, common subversion functions.

Installation Steps

Building the subversion extension

As with all eXist extensions that are not enabled by default, you need to instruct eXist's build process to include the extension.

You should first copy the file $EXIST_HOME/extensions/build.properties to a new file, called $EXIST_HOME/extensions/local.build.properties. This local file will be used by the build process, but it will be ignored by your subversion client so that you don't accidentally commit it to the eXist repository.

You should now locate the following line:

  #SVN extension
  include.feature.svn = false

Change false to true:

  include.feature.svn = true

Save the local.build.properties file. With these changes you must now rebuild (i.e. recompile) eXist so that the subversion extension is included in eXist's jar files.

Enable the subversion module in conf.xml

To ensure the module is available when you start eXist, un-comment the following lines in your $EXIST_HOME/conf.xml file

<module uri="http://exist-db.org/xquery/versioning/svn" class="org.exist.versioning.svn.xquery.SVNModule" />

Save conf.xml. Now you can start eXist, and the subversion module will now be ready for you to use. You can build the subversion function documentation at http://localhost:8080/exist/admin/admin.xql?panel=fundocs and then accessing http://localhost:8080/exist/functions/subversion.

You should now be able to test the subversion XQuery functions. This should look very similar to the function listings on the eXist demo site here: http://demo.exist-db.org/exist/xquery/functions.xql

Current Status

Subversion repositories can be accessed over HTTP and HTTPS, both anonymously and with username/password authentication.

The following functions have been tested to work:

subversion:checkout($repository-uri as xs:string, $database-path as xs:string) xs:long
subversion:checkout($repository-uri as xs:string, $database-path as xs:string, $login as xs:string, $password as xs:string) xs:long
subversion:get-latest-revision-number($repository-uri as xs:string, $login as xs:string, $password as xs:string) xs:long
subversion:info($database-path as xs:string) element()
subversion:list($repository-uri as xs:string) element()
subversion:log($repository-uri as xs:string, $login as xs:string, $password as xs:string, $start-revision as xs:integer?, $end-revision as xs:integer?) element()
subversion:status($database-path as xs:string) element()
subversion:update($database-path as xs:string) xs:long
subversion:update($database-path as xs:string, $login as xs:string, $passwrod as xs:string) xs:long
subversion:add($database-path as xs:string) empty()

The following works under some cases but has buffer errors for some sizes of commits:

subversion:commit($database-path as xs:string, $message as xs:string?, $login as xs:string, $password as xs:string) xs:long

The following functions are not yet confirmed to work and are still being tested:

subversion:clean-up($database-path as xs:string) empty()
subversion:lock($database-path as xs:string, $message as xs:string?) empty()
subversion:revert($database-path as xs:string) empty()
subversion:unlock($database-path as xs:string) empty()

Examples

Querying a Remote Repository

subversion:get-latest-revision-number()

The subversion:get-latest-revision-number() function queries the remote SVN repository, returning the latest revision number. For example:

xquery version "1.0";

import module namespace subversion = "http://exist-db.org/xquery/versioning/svn"; 
 
let $repository-uri := xs:anyURI('https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist/webapp/eXide')
let $username := ''
let $password := ''
return
  subversion:get-latest-revision-number($repository-uri, $username, $password)

This query returns the following result:

subversion:info()

Once you have done a checkout of a resource from subversion you can query that resource locally and find out more about it.

  subversion:info('/db/apps/faqs/data')

This will return:

<info uri="/db/cms/apps/faqs/data">
<info local-path="/db/apps/faqs/data" 
   URL="https://www.example.com/repo/trunk/db/apps/faq/data" 
   Repository-UUID="db6794ef-7b42-44a9-8912-f63d0efeae0f" 
   Revision="10" Node-Kind="dir" Schedule="normal"
   Last-Changed-Author="dmccreary" Last-Changed-Revision="8"
   Last-Changed-Date="Thu Sep 01 15:03:04 CDT 2011"/>

subversion:list()

The subversion:list() function lists the contents of a remote repository, returning the results as an XML node:

xquery version "1.0";

let $repository-uri := xs:anyURI('https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist/webapp/scripts/')
return
  subversion:list($repository-uri)

This script will return the following result:

<entries>
  <entry type="directory">edit_area</entry>
  <entry type="directory">jquery</entry>
  <entry type="directory">openid-selector</entry>
  <entry type="directory">syntax</entry>
  <entry type="directory">yui</entry>
  <entry type="file">fundocs.js</entry>
  <entry type="file">main.js</entry>
  <entry type="file">prototype.js</entry>
</entries>

subversion:log()

The subversion:log() function queries the remote SVN repository, returning the log of changes as an XML node. For example, this query will return show the log of changes between two arbitrary revision numbers (note that substituting empty nodes () for $start-revision and/or $end-revision will return a more open-ended log of revisions):

xquery version "1.0";

let $repository-uri := xs:anyURI('https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist/webapp/eXide')
let $username := ''
let $password := ''
let $start-revision := 14300
let $end-revision := 14350
return
  subversion:log($repository-uri, $username, $password, $start-revision, $end-revision)

The results of this query are as follows (note that the @revtype values are 'A' for item added, 'D' for item deleted, 'M' for item modified, and 'R' for item replaced):

<log uri="https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist/webapp/eXide" start="14300">
    <entry rev="14331" author="wolfgang_m" date="2011-04-29T07:00:54.297-04:00">
        <message>[feature] eXide - a web-based XQuery IDE for eXist. Features: fast syntax
            highlighting, ability to edit huge XQuery files, code completion for functions and
            variables, code templates, powerful navigation, on-the-fly compilation, generation of
            app skeletons, integration with app repository... This is the initial checkin of eXide.</message>
        <paths>
            <path revtype="A">/trunk/eXist/webapp/eXide/templates</path>
            <path revtype="A">/trunk/eXist/webapp/eXide/collections.xql</path>
            <path revtype="A">/trunk/eXist/webapp/eXide/session.xql</path>
           ....etc....
            <path revtype="A">/trunk/eXist/webapp/eXide/scripts/ace/cockpit.js</path>
            <path revtype="A">/trunk/eXist/webapp/eXide/index.html</path>
        </paths>
    </entry>
    <entry rev="14346" author="wolfgang_m" date="2011-04-30T08:35:23.395-04:00">
        <message>[website] eXide: fixed completion popup window (support mouse, extra "close" link
            if popup looses focus); improved auto-indent in editor after { and (.</message>
        <paths>
            <path revtype="M">/trunk/eXist/webapp/eXide/src/mode-xquery.js</path>
            <path revtype="M">/trunk/eXist/webapp/eXide/src/util.js</path>
            <path revtype="M">/trunk/eXist/webapp/eXide/eXide.css</path>
        </paths>
    </entry>
</log>

Getting the Last 10 Commit Messages

The log function can be combined with the get-latest-revision-number function to get the last 10 commit messages in the system.

let $latest-version := subversion:get-latest-revision-number($repo-url, $svn-account, $svn-password)

(: if we have more than 10 revisions then get them all, else start with one :)
let $start :=
   if ($latest-version gt 10)
      then $latest-version - 10
      else 1

return
<last-10-commit-messages>
  {subversion:log($repo-url, $svn-account, $svn-password, $start , $latest-version)//*:message}
</last-10-commit-messages>

Operating on a Local Working Copy

subversion:checkout()

The following example checks out eXist's "functions" app to the "/db/svn" collection:

xquery version "1.0";

let $repository-uri := xs:anyURI('https://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist/webapp/functions/')
let $destination-path := '/db/svn'
let $version := subversion:checkout($repository-uri, $destination-path) 
return 
    concat('Revision ', $version, ' successfully checked out to collection ', $destination-path)

This returns:

  Revision 14457 successfully checked out to collection /db/svn

The /db/svn collection will now contain the following files:

.svn (collection)
controller.xql
filter.xql
functions.xql

subversion:add()

After you have run a checkout you are now ready to do a subversion:commit() or an subversion:add().

The both of these functions take a single argument which is the database collection path you want to send to your subversion server.

subversion:update()

Assuming we have already checked out a repository to /db/svn, we can update the working copy to the latest revision using the subversion:update() function:

xquery version "1.0";

let $working-copy := '/db/svn'
let $update := subversion:update($working-copy) 
return 
    concat('Successfully updated to revision ', $update)