Contents

XQuery Examples CollectionEdit

Welcome to the XQuery Examples Collection Wikibook! XQuery is a World Wide Web Consortium recommendation for selecting data from documents and databases.

Current StatusEdit

We are in the process of testing many of these examples using the eXist 2.0 release. Please let us know if you find any problems. We are looking for volunteers to help us create XQuery 3.0 examples.

Recent Changes

New and Revised ArticlesEdit

About this ProjectEdit

This is a collaborative project and we encourage everyone who is using XQuery to contribute their XQuery examples. All example programs must conform to the creative-commons-2.5 share-alike with attribution license agreement [1].

Execution of examples use an eXist demo server.

  1. Instructors: please sign our Guest Registry if you are using this book for learning or teaching XQuery
  2. Contributors: please see our Naming Conventions to ensure your examples are consistent with the textbook
  3. Learners: If you are looking for an example of a specific XQuery language construct, technique or problem but can't find an example, please add a suggestion to the Examples Wanted section.

IntroductionEdit

  1. Background - A brief history and motivation for the XQuery standard.
  2. Benefits - Why use XQuery?
  3. Installing and Testing - How to install an XQuery server on your .
  4. Naming Conventions - Naming standards used throughout this book.

Example ScriptsEdit

Beginning ExamplesEdit

Examples that do not assume knowledge of functions and modules.

  1. HelloWorld - A simple test to see if XQuery is installed correctly.
  2. Loading data - A tour of several different ways to load data into a native XML database
  3. FLWOR Expression - A basic example of how XQuery FLWOR statements work.
  4. Sequences - Working with sequences is central to XQuery.
  5. Parsing CSV - Using FLWOR on sequences to parse CSV text.
  6. XPath examples - Sample XPath samples for people new to XML and XPath
  7. Regular Expressions - Regular expressions make it easy to parse text.
  8. Searching multiple collections - How to search multiple collections in a database.
  9. Getting URL Parameters - How to get parameters from the URL.
  10. Getting POST Data - How to get XML data posted to an XQuery.
  11. Checking for Required Parameters - How to check for a required parameter using if/then/else.
  12. Displaying Lists - How to take a list of values in an XML structure and return a comma separated list.
  13. Extracting data from XHTML files - How to use the doc() function to get data from XHTML pages.
  14. Displaying data in HTML Tables - How to display XML data in an HTML table.
  15. Limiting Result Sets - How to limit the number of records returned in an XQuery.
  16. Filtering Words - How to test to see if a word is on a list.
  17. Saving and Updating Data - How to have a single XQuery that saves new records or updates existing records.
  18. Quantified Expressions - Testing all the items in a sequence.
  19. Dates and Time - Sample expressions that work with date and time values
  20. Chaining Web Forms - Passing data from one web page to another using URL parameters, sessions or cookies

Intermediate ExamplesEdit

Assumes knowledge of functions and modules.

  1. Using XQuery Functions - How to read XQuery function documents and user XQuery functions
  2. Creating XQuery Functions - How to create your own local XQuery functions
  3. Returning the Longest String - A function to find the longest string from a list of strings
  4. Net Working Days - How to calculate the number of working days between two dates
  5. Tag Cloud - Counting and viewing the number of keywords
  6. String Analysis - Regular expression string analysis
  7. Manipulating URIs - How to get and manage URIs
  8. Parsing Query Strings - Parsing query strings using alternate delimiters.
  9. Splitting Files - Splitting a large XML files into many smaller files.
  10. Filling Portlets - How to fill regions of a web page with XQuery
  11. Filtering Nodes - How to use the identity transform to filter out nodes and add nodes
  12. Limiting Child Trees - You have a tree of information and you want to "prune" only at a specific level
  13. Higher Order Functions - Passing functions as arguments to functions
  14. Timing Fibonacci algorithms - A couple of Fibonacci algorithms and timing display
  15. Using Intermediate Documents - Analysis of a MusicXML file
  16. Formatting Numbers - using picture formats to format numbers
  17. Uploading Files - how to upload files using HTML forms
  18. TEI Concordance - How to build a TEI-based concordance
  19. Queries on Tables - How to extract data from tabular data
  20. Namespace Constructors - how to dynamically add namespaces when constructing documents

SearchEdit

  1. Introduction to XML Search - An overview of XML search terminology
  2. Basic Search - A simple search page
  3. Searching,Paging and Sorting - Searching and Viewing search results
  4. Keyword Search - full text search with Google-style results
  5. Employee Search - an Ajax example
  6. Incremental Search of the Chemical Elements - with Ajax
  7. Lucene Search - using eXist's Lucene-based fulltext search
  8. Incremental Searching - working with a JavaScript client to perform incremental search
  9. Advanced Search - creating complex searches using multiple search fields
  10. Open Search - creating an OpenSearch file to describe your search page
  11. Auto-generation of Index Config Files - scripts to automatically generate the index configuration file

InteractionEdit

  1. Adder - Creating a web service that adds two numbers.
  2. Simple XForms Examples
  3. Navigating Collections - an example of an AJAX browser
  4. Sending E-mail - How to send an e-mail message from within an XQuery


Creating Custom ViewsEdit

These examples use reflection on the structure of an XML document using name() to implement generic functions for XML transformations.

  1. HTML Table View - A generic HTML table representation
  2. Tree View - A generic HTML tree representation
  3. Grouping Items - how to group items in a report

Transforming Complex XML DocumentsEdit

XQuery has many features that allow you to transform XML and create full document-style transformation libraries. Unlike traditional "database" documents, complex XML documents have "complex content" that includes in-line elements in unpredictable order. This section provides a foundation for these transformations based on using the XQuery typeswitch functions. Typeswitch function transformations replace XSLT transforms but can also access indexes for very fast transforms of large collections.

  1. Transformation Styles overview of the three styles of transformation
  2. Typeswitch Transformations Using the typeswitch function for document-style transforms.
  3. Transformation idioms Handling transformation tasks
  4. Generating Skeleton Typeswitch Transformation Modules Using XQuery to generate a skeleton module for typeswitch-based document transformation
  5. Web XML Viewer Using the typeswitch function to transform an XML document to HTML

Paginated ReportsEdit

Unlike HTML pages, paginated reports use the concept of text flows between pages. These examples show you how to convert raw XML into high-quality PDF files suitable for printing. The examples use a markup standard called XSL-FO for "Formatted Objects"

  1. Installing_the_XSL-FO_module - update your 1.4 configuration to get the current software from the Apache web site
  2. Generating PDF from XSL-FO files - Converting XML-FO to PDF files
  3. XSL-FO Tables - Generating XSL-FO tables from XML files
  4. Converting HTML tables to XSL-FO tables - use a XQuery typeswitch transform to convert HTML tables to XSL-FO tables
  5. XSL-FO Images - Embedding images in generated (PDF) files

Content PublishingEdit

  1. Publishing Overview - How to transfer a document from an internal intranet server to a public web site
  2. Publishing to Subversion - How to transfer a document from an internal intranet to a public SVN server using SSL and digest authentication

XML Document Comparison and MergingEdit

  1. Compare two XML files - using the eXist compare() function to test to see if two XML files are exactly the same
  2. XML Differences - displaying the difference between two XML files
  3. Compare with XQuery - Using XQuery to Compare Lists
  4. Time Comparison with XQuery - Using XQuery to Compare Dated Items
  5. Synchronizing Remote Collections - Using lastModified time stamps to see what items have changed
  6. Finding Duplicate Documents - Using a hash function to find duplicate documents

Time Based QueriesEdit

  1. Time Based Queries - using dates and times to limit search results
  2. Timing a Query - profiling how long a query takes

XML document kindsEdit

TEI documentsEdit

Text Encoding Initiative.

  1. TEI Concordance - How to build a TEI-based concordance
  2. TEI Document Timeline - Using Simile Timeline to visualize a TEI document

DocBook DocumentsEdit

  1. DocBook to HTML
  2. DocBook to PDF
  3. DocBook to ePub
  4. DocBook to Microsoft Word

OpenOfficeEdit

  1. OpenOffice to HTML

Office Open XMLEdit

  1. Office Open XML

XML SchemasEdit

  1. XML Schema to Instance
  2. XML Schema to XForms
  3. XML Schema to SVG


Processing Special CharactersEdit

  1. Special Characters - dealing with newlines and other special characters.

XQuery and other languagesEdit

MusicXMLEdit

  1. Using Intermediate Documents Analysing MusicXML documents
  2. MusicXML to Arduino


Language ComparisonsEdit

PythonEdit

  1. XQuery and Python

SQLEdit

  1. XQuery SQL Module - Calling SQL from within your XQuery
  2. XQuery from SQL - Using XQuery to access a classic Relational database - Employee/Department/Salary

RDF/OWLEdit

  1. List OWL Classes - A simple XQuery script that will display all the OWL classes in an OWL file

Language combinationEdit

ExcelEdit

  1. Excel and XML

JavaScriptEdit

  1. Navigating Collections - basic AJAX
  2. Employee Search - basic AJAX
  3. Incremental Search of the Chemical Elements - AJAX
  4. DOJO data - basic JSON

SQLEdit

  1. XML to SQL
  2. Microsoft-Access

XHTML + VoiceEdit

  1. Simple RSS reader
  2. XHTML + Voice Twitter Radio for Opera

XSLTEdit

  1. XQuery and XSLT Executing an XSLT transform from within XQuery

Data MashupsEdit

AuthenticationEdit

  1. Basic Authentication - Logging in to a remote web server using HTTP Basic Authentication
  2. Digest Authentication - Logging in to a remote web server using HTTP Digest Authentication
  3. OAuth - A standard for protecting a set of user-owed data within a web service

Wikipedia interactionEdit

  1. Wikipedia Page scraping
  2. Wikipedia Lookup
  3. Wikipedia Events RSS
  4. Wiki weapons page

Wikibook applicationsEdit

  1. Wikibook index page
  2. Wikibook list of code links

FreebaseEdit

  1. Freebase - use XQuery to get data via JSON from Freebase

Google DocsEdit

  1. Google Docs - use XQuery to get data from Google Docs

VisualizationEdit

  1. Graph Visualization
  2. Graphing from RDF
  3. Dataflow diagrams
  4. Sequence Diagrams
  5. Example Sequencer - Step-by-step tutorial

Google ChartsEdit

Although the Google Charts functions only work when you are connected to the Internet, these examples show that XQuery is an ideal tool for converting XML data into charts.

  1. Google Charts Using XML and XQuery to generate Google Charts using REST
  2. Google Chart Sparkline - A demonstration of how to create a chart using the Google Charts API
  3. Google Chart Bullet Bar - A demonstration of how to a dashboard bullet bar using the Google Charts API
  4. Histogram of File Sizes - An XQuery report that generates a histogram of file sizes

There are also sample XForms that can be used to create front-ends in the XForms Tutorial and Cookbook

Digital DashboardsEdit

Digital dashboards are single screens that compress a great deal of information into a single web page. This section will leverage many of the Google Charts examples from the prior section.

  1. Dashboard Architecture - How to design dashboards that have fast response times

Page ScrapingEdit

Page scraping is the process of extracting well-formed XML data from any HTML web page. When creating mashup applications this is also known as the harvesting process.

  1. Overview of Page Scraping Techniques
  2. Page scraping and Yahoo Weather
  3. UK shipping forecast
  4. BBC Weather Forecast
  5. Page scraping and Mashup
  6. Simple RSS reader
  7. Multiple page scraping and Voting behaviour
  8. Link gathering
  9. REST interface definition
  10. Caching and indexes

MappingEdit

  1. Google Geocoding
  2. /String Analysis#Location_Mapping/ Mapping Car Registrations
  3. Flickr GoogleEarth
  4. Nationalgrid and Google Maps
  5. SMS tracker

TimelinesEdit

  1. Creating a Timeline - Creating a simple timeline view of events
  2. Timelines of Resource - Using creation and modification dates to create timelines
  3. TEI Document Timeline - Creating a timeline of all dates within a single TEI document

The Semantic WebEdit

  1. DBpedia with SPARQL - Football teams
  2. DBpedia with SPARQL and Simile Timeline - Album Chronology Creating a timeline of album releases using data from Wikipedia
  3. DBpedia with SPARQL - Stadium locations
  4. The Emp-Dept case study
    1. XML to RDF
    2. SPARQL Tutorial
    3. SPARQL interface
  5. Graphing Triples
  6. SPARQLing Country Calling Codes
  7. Southampton Pubs
  8. Alphabet Poster
  9. Simile Exhibit Browser visualizations using the Simile JavaScript libraries
  10. Latent Semantic Indexing Finding the semantic distance between documents


Development ToolsEdit

  1. Sitemap for Content Management System XQuery functions can easily perform many common web site content management functions
  2. Uptime monitor use XQuery to monitor a remote web service
  3. XQuery IDE - XQuery Integrated development environment
  4. Image Library - using an XQuery to preview your images
  5. XML Schema to Instance - XQuery function to generate a sample XML instance from an XML Schema file (.xsd)
  6. Lorum Ipsum text - generating sample text for inserting into test page layouts
  7. XQuery and XML Schema - Generating an XML instance document
  8. Generating XQDocs - Automating the generation of XQuery documentation for Modules and Functions
  9. XqUSEme - Firefox extension to allow XQueries including against the loaded document (even against originally non-XML (poorly formed) HTML).
  10. Call Graphs - Generating an call graph as an SVG file from module introspection
  11. System Properties - getting a list of all standard and java system properties and their values
  12. Environment Variables - getting a list of all the external environment variables and their values

ValidationEdit

  1. Validating a document - Validate a document with an XML Schema
  2. Validation using a Catalog - Using a Catalog file to validate documents
  3. Validating a hierarchy -
  4. Validation with Schematron - Apply specific rules to check a document

Path AnalysisEdit

  1. All Paths - A report of all paths in a document or collection
  2. All Leaf Paths - A report of all leaf paths in a document or collection

SecurityEdit

  1. Login and Logout - How to log users in and log them out
  2. URL Driven Authorization How to use URL rewriting to check for valid users
  3. Digital Signatures - How to use a custom module to use the XML Digital Signature standards
  4. Changing Permissions on Collections and Resources - how to change permissions on collections and resources

Unit Testing ToolsEdit

  1. XUnit Testing - what is XUnit testing and how to set them up
  2. XUnit Annotations - how to add annotations to your XQuery functions to drive your XUnit tests

Case StudiesEdit

  1. Fizzbuzz
  2. Project Euler
  3. Topological Sort
  4. Slideshow
  5. Sudoku
  6. Pachube feed
  7. World Temperature records - conversion of text data formats to XML, indexing and data presentation
  8. UWE StudentsOnline

eXist db specific Functions and ConfigurationEdit

ConfigurationEdit

Installing modulesEdit

  1. Installing the XSL-FO module

Setting HTTP HeadersEdit

  1. Setting HTTP Headers

ModulesEdit

dbutilEdit

  1. Database Utilities - a set of database utility functions for database administration

compressionEdit

Function Reference

  1. Get zipped XML file
  2. Unzipping an Office Open XML docx file - Uncompressing and storing a docx file

eXgitEdit

Module for interacting with any git revision control system

  1. Installing eXgit

EXPath File Transfer XQuery Extension ModuleEdit

This module does both FTP, SFTP (using the SSH protocol) using a standardized and consistent format.

  1. File Transfer Client

ftp clientEdit

This module allows you to interact with a remote FTP server on a remote system. It includes functions for listing, getting and putting files.

  1. FTP Client


httpclientEdit

Function Reference

  1. Digest Authentication
  2. UK shipping forecast

JSONEdit

  1. Convert XML to JSON

LuceneEdit

Function Reference Help

  1. Lucene_Search

mailEdit

Function Reference

  1. Sending E-mail
  2. Basic Feedback Form

mathEdit

  1. Using the Math Module

memcachedEdit

  1. Using the Memcached Module

processEdit

  1. Execute External Process - execute a shell command from within XQuery

requestEdit

Function Reference Function examples

  1. Getting URL Parameters
  2. Getting POST Data
  3. Checking for Required Parameters
  4. Manipulating URIs
  5. Parsing Query Strings
  6. Adder simple client-server interaction

schedulerEdit

Function Reference Help

  1. XQuery Batch Jobs

sequencesEdit

Function Reference

  1. Sequences Module - three additional functions (map, fold and filter)

sessionEdit

Function Reference

  1. Basic Session Management - the basics of session management including getting a setting session variables

subversionEdit

Function Reference

  1. Subversion - how to update a subversion repository from within XQuery using the subversion client

transformEdit

Function Reference

  1. String_Analysis

utilEdit

Function Reference

  1. Registered Modules : util:registered-modules()
  2. Registered Functions : util:registered-functions()
  3. Dynamic Module Loading : util:import-module(), util:eval()
  4. Higher Order Functions : util:function(), util:call()
  5. Timing Fibonacci algorithms : util:function(), util:call(), util:system-time()
  6. XMP data : util:binary-doc(), util:binary-to-string(), util:parse()
  7. Basic Authentication : util:string-to-binary(), httpclient:get()

validationEdit

Function Reference Help

  1. Validating a document

xmldbEdit

Function Reference

  1. Saving_and_Updating_Data
  2. Splitting_Files

xqdocEdit

Function Reference

  1. Generating xqDoc-based XQuery Documentation

xslfoEdit

XSL-FO (Formatted Objects) is a way of converting XML into PDF. Function Reference

  1. Installing the XSL-FO module - setting up your XSL-Module within eXist
  2. Generating PDF from XSL-FO files - generating PDF from a FO file
  3. XSL-FO Tables - adding tables to your PDF
  4. XSL-FO Images - adding images to your PDF
  5. XSL-FO SVG - adding SVG images to your PDF

TriggersEdit

  1. Using Triggers to Log Events - how to set up a trigger to log store, update and remove events on a collection
  2. Using Triggers to assign identifiers - how to use triggers to assign identifiers to new documents or new nodes
  3. Sending E-mail Email is one way to notify when a trigger has fired

XQuery UpdatesEdit

  1. Inserting and Updating Attributes
  2. Updates and Namespaces - How updates can change serialization

URL RewritingEdit

  1. URL Rewriting Basics How to make your URLs look nice

Apache Ant TasksEdit

  1. Reindex a Collection

ReplicationEdit

  1. eXist Replication

General guidanceEdit

AppendixesEdit

Systems that Support XQueryEdit

Using native and hybrid XML databases that implement XQuery

  1. BaseX - Native open source XML Database with visual frontend
  2. DataDirect XQuery - Java XQuery engine supporting relational, EDI, flat files and XML input/output
  3. eXist - Open source native XML database
  4. DB2 pureXML - DB2 9.1 includes the pureXML feature
  5. MarkLogic Server - MarkLogic Server commercial XML Content Server
  6. Microsoft SQL Server 2005 -
  7. NetKernel - NetKernel
  8. Oracle Berkeley DB XML - Open source embedded storage management
  9. Oracle XML DB - Oracle Server 11g includes the XML DB (XDB) feature
  10. Sedna - Open source native XML Database
  11. Stylus Studio - XQuery mapping/editing/debugging, ships with Saxon (and SA) and DataDirect XQuery
  12. EMC xDB - EMC Documentum xDB commercial native XML database
  13. XQilla - Open source XQuery library and command line utility
  14. Zorba - Open source XQuery engine C++ implementation with C, Java, Php, Python, Ruby library bindings and command line utility
  15. Qizx - Open source and pro XQuery engine Java implementation

Debugging XQueryEdit

  1. Gotchas - some pitfalls
  2. Ah-has - some ah-ha moments

Other sourcesEdit

Function LibrariesEdit

  1. FunctX XQuery Function Library by Priscilla Walmsley

Discussion GroupsEdit

  1. XQuery General


IndexesEdit

Key to symbols:

Wikibook Development Stages
Sparse text Developing text Maturing text Developed text Comprehensive text

indicates an XQuery/Best practice

Read in another language