Wikibooks

XQuery

XQuery Examples Collection

edit

Welcome to the XQuery Examples Collection Wikibook! XQuery is a World Wide Web Consortium recommendation for selecting data from documents and databases.

Current Status

edit

2019 refurbishment

edit

There is a Github project to track issues with this book and collaborate amongst eXist db folk at least to bring this resource up-to-date.

Main tasks in the current refurbishment are to:

  • get example code executable again
  • remove complex case studies which use obsolete sources
  • add XQuery 3.0 examples.

Recent Changes

edit

You may search the book here:



New and Revised Articles

edit

About this Project

edit

This is a collaborative project and we encourage everyone who is using XQuery to contribute their XQuery examples. All example programs must conform to the creative-commons-2.5 share-alike with attribution license agreement [1].

Execution of examples use an eXist demo server.

  1. Instructors: please sign our Guest Registry if you are using this book for learning or teaching XQuery
  2. Contributors: please see our Naming Conventions to ensure your examples are consistent with the textbook
  3. Learners: If you are looking for an example of a specific XQuery language construct, technique or problem but can't find an example, please add a suggestion to the Examples Wanted section.

Introduction

edit
  1. Background - A brief history and motivation for the XQuery standard.  
  2. Benefits - Why use XQuery?  
  3. Installing and Testing - How to install an XQuery server on your system.  
  4. Naming Conventions - Naming standards used throughout this book.  

Using XQuery

edit

Common tasks in using XQuery with XML

  1. Executing a script - Executing an XQuery script
  2. Loading data - A tour of several different ways to load data into a native XML database  

Example Scripts

edit

Beginning Examples

edit

Examples that do not assume knowledge of functions and modules.

  1. HelloWorld - A simple test to see if XQuery is installed correctly.  
  2. FLWOR Expression - A basic example of how XQuery FLWOR statements work.  
  3. Sequences - Working with sequences is central to XQuery.  
  4. XPath examples - Sample XPath samples for people new to XML and XPath  
  5. Regular Expressions - Regular expressions make it easy to parse text.  
  6. Searching multiple collections - How to search multiple collections in a database.  
  7. Displaying Lists - How to take a list of values in an XML structure and return a comma separated list.  
  8. Extracting data from XHTML files - How to use the doc() function to get data from XHTML pages.  
  9. Displaying data in HTML Tables - How to display XML data in an HTML table.  
  10. Limiting Result Sets - How to limit the number of records returned in an XQuery.  
  11. Filtering Words - How to test to see if a word is on a list.  
  12. Saving and Updating Data - How to have a single XQuery that saves new records or updates existing records.  
  13. Quantified Expressions - Testing all the items in a sequence.  
  14. Dates and Time - Sample expressions that work with date and time values  

Intermediate Examples

edit

Assumes knowledge of functions and modules.

  1. Using XQuery Functions - How to read XQuery function documents and user XQuery functions
  2. Creating XQuery Functions - How to create your own local XQuery functions
  3. Returning the Longest String - A function to find the longest string from a list of strings
  4. Net Working Days - How to calculate the number of working days between two dates  
  5. Tag Cloud - Counting and viewing the number of keywords  
  6. String Analysis - Regular expression string analysis
  7. Manipulating URIs - How to get and manage URIs
  8. Parsing Query Strings - Parsing query strings using alternate delimiters.  
  9. Splitting Files - Splitting a large XML files into many smaller files.  
  10. Filling Portlets - How to fill regions of a web page with XQuery
  11. Filtering Nodes - How to use the identity transform to filter out nodes and add nodes  
  12. Limiting Child Trees - You have a tree of information and you want to "prune" only at a specific level  
  13. Higher Order Functions - Passing functions as arguments to functions  
  14. Timing Fibonacci algorithms - A couple of Fibonacci algorithms and timing display
  15. Using Intermediate Documents - Analysis of a MusicXML file
  16. Formatting Numbers - using picture formats to format numbers  
  17. Uploading Files - how to upload files using HTML forms  
  18. TEI Concordance - How to build a TEI-based concordance  
  19. Queries on Tables - How to extract data from tabular data  
  20. Namespace Constructors - how to dynamically add namespaces when constructing documents  

Search

edit
  1. Introduction to XML Search - An overview of XML search terminology  
  2. Basic Search - A simple search page  
  3. Searching, Paging and Sorting - Searching and Viewing search results  
  4. Keyword Search - full text search with Google-style results  
  5. Employee Search - an Ajax example  
  6. Lucene Search - using eXist's Lucene-based fulltext search  
  7. Advanced Search - creating complex searches using multiple search fields  
  8. Open Search - creating an OpenSearch file to describe your search page  
  9. Auto-generation of Index Config Files - scripts to automatically generate the index configuration file  

Transformations

edit

Using XQuery to transform between XML and other representations

  1. Parsing CSV - Using FLWOR on sequences to parse CSV text.  


Interaction

edit
  1. Getting URL Parameters - How to get parameters from the URL.  
  2. Getting POST Data - How to get XML data posted to an XQuery.  
  3. Checking for Required Parameters - How to check for a required parameter using if/then/else.  
  4. Adder - Retaining sate in interactions :Creating a web service that adds two numbers.  
  5. Chaining Web Forms - Passing data from one web page to another using URL parameters, sessions or cookies  
  6. Simple XForms Examples
  7. Incremental Search of the Chemical Elements - AJAX  
  8. Sending E-mail - How to send an e-mail message from within an XQuery  

Visualization

edit
  1. Graph Visualization with Graphviz
  2. Google Chart Sparkline - Creating a Tufte Sparkline using the Google Charts API
  3. Google Chart Bullet Bar - Creating a bullet bar using the Google Charts API

Creating Custom Views

edit

These examples use reflection on the structure of an XML document using name() to implement generic functions for XML transformations.

  1. HTML Table View - A generic HTML table representation
  2. Tree View - A generic HTML tree representation
  3. Grouping Items - how to group items in a report

Transforming Complex XML Documents

edit

XQuery has many features that allow you to transform XML and create full document-style transformation libraries. Unlike traditional "database" documents, complex XML documents have "complex content" that includes in-line elements in unpredictable order. This section provides a foundation for these transformations based on using the XQuery typeswitch functions. Typeswitch function transformations replace XSLT transforms but can also access indexes for very fast transforms of large collections.  

  1. Transformation Styles overview of the three styles of transformation  
  2. Typeswitch Transformations Using the typeswitch function for document-style transforms.  
  3. Transformation idioms Handling transformation tasks  
  4. Generating Skeleton Typeswitch Transformation Modules Using XQuery to generate a skeleton module for typeswitch-based document transformation  
  5. Web XML Viewer Using the typeswitch function to transform an XML document to HTML  

Paginated Reports

edit

Unlike HTML pages, paginated reports use the concept of text flows between pages. These examples show you how to convert raw XML into high-quality PDF files suitable for printing. The examples use a markup standard called XSL-FO for "Formatted Objects"

  1. Installing the XSL-FO module - update your 1.4 configuration to get the current software from the Apache web site
  2. Generating PDF from XSL-FO files - Converting XML-FO to PDF files  
  3. XSL-FO Tables - Generating XSL-FO tables from XML files  
  4. Converting HTML tables to XSL-FO tables - use a XQuery typeswitch transform to convert HTML tables to XSL-FO tables  
  5. XSL-FO Images - Embedding images in generated (PDF) files

Content Publishing

edit
  1. Publishing Overview - How to transfer a document from an internal intranet server to a public web site  
  2. Publishing to Subversion - How to transfer a document from an internal intranet to a public SVN server using SSL and digest authentication  

XML Document Comparison and Merging

edit
  1. Compare two XML files - using the eXist compare() function to test to see if two XML files are exactly the same  
  2. XML Differences - displaying the difference between two XML files  
  3. Compare with XQuery - Using XQuery to Compare Lists  
  4. Time Comparison with XQuery - Using XQuery to Compare Dated Items  
  5. Synchronizing Remote Collections - Using lastModified time stamps to see what items have changed  
  6. Finding Duplicate Documents - Using a hash function to find duplicate documents  

Time Based Queries

edit
  1. Time Based Queries - using dates and times to limit search results  
  2. Timing a Query - profiling how long a query takes  

XML document kinds

edit

TEI documents

edit

Text Encoding Initiative.

  1. TEI Concordance - How to build a TEI-based concordance  
  2. TEI Document Timeline - Using Simile Timeline to visualize a TEI document

DocBook Documents

edit
  1. DocBook to HTML
  2. DocBook to PDF
  3. DocBook to ePub
  4. DocBook to Microsoft Word

OpenOffice

edit
  1. OpenOffice to HTML

Office Open XML

edit
  1. Office Open XML

XML Schemas

edit
  1. XML Schema to Instance
  2. XML Schema to XForms
  3. XML Schema to SVG


Processing Special Characters

edit
  1. Special Characters - dealing with newlines and other special characters.

XQuery and other languages

edit

MusicXML

edit
  1. Using Intermediate Documents Analysing MusicXML documents
  2. MusicXML to Arduino


Language Comparisons

edit

Python

edit
  1. XQuery and Python  

SQL

edit
  1. XQuery SQL Module - Calling SQL from within your XQuery
  2. XQuery from SQL - Using XQuery to access a classic Relational database - Employee/Department/Salary  

RDF/OWL

edit
  1. List OWL Classes - A simple XQuery script that will display all the OWL classes in an OWL file  

Language combination

edit

Excel

edit
  1. Excel and XML  

JavaScript

edit
  1. Navigating Collections - basic AJAX  
  2. Employee Search - basic AJAX  
  3. DOJO data - basic JSON  

SQL

edit
  1. XML to SQL  
  2. Microsoft-Access  

XHTML + Voice

edit
  1. Simple RSS reader  
  2. XHTML + Voice Twitter Radio for Opera  

XSLT

edit
  1. XQuery and XSLT Executing an XSLT transform from within XQuery  

Data Mashups

edit

Authentication

edit
  1. Basic Authentication - Logging in to a remote web server using HTTP Basic Authentication
  2. Digest Authentication - Logging in to a remote web server using HTTP Digest Authentication
  3. OAuth - A standard for protecting a set of user-owed data within a web service

Wikipedia interaction

edit
  1. Wikipedia Page scraping
  2. Wikipedia Lookup
  3. Wikipedia Events RSS
  4. Wiki weapons page

Wikibook applications

edit
  1. Wikibook index page
  2. Wikibook list of code links

Freebase

edit
  1. Freebase - use XQuery to get data via JSON from Freebase

Google Docs

edit
  1. Google Docs - use XQuery to get data from Google Docs

Visualization

edit
  1. Graph Visualization
  2. Graphing from RDF
  3. Dataflow diagrams
  4. Sequence Diagrams
  5. Example Sequencer - Step-by-step tutorial

Google Charts

edit

Although the Google Charts functions only work when you are connected to the Internet, these examples show that XQuery is an ideal tool for converting XML data into charts.

  1. Google Charts Using XML and XQuery to generate Google Charts using REST  
  2. Google Chart Sparkline - A demonstration of how to create a chart using the Google Charts API
  3. Google Chart Bullet Bar - A demonstration of how to a dashboard bullet bar using the Google Charts API
  4. Histogram of File Sizes - An XQuery report that generates a histogram of file sizes

There are also sample XForms that can be used to create front-ends in the XForms Tutorial and Cookbook

Digital Dashboards

edit

Digital dashboards are single screens that compress a great deal of information into a single web page. This section will leverage many of the Google Charts examples from the prior section.

  1. Dashboard Architecture - How to design dashboards that have fast response times  

Page Scraping

edit

Page scraping is the process of extracting well-formed XML data from any HTML web page. When creating mashup applications this is also known as the harvesting process.

  1. Overview of Page Scraping Techniques  
  2. Page scraping and Yahoo Weather
  3. UK shipping forecast
  4. BBC Weather Forecast
  5. Page scraping and Mashup
  6. Simple RSS reader
  7. Multiple page scraping and Voting behaviour
  8. Link gathering
  9. REST interface definition
  10. Caching and indexes

Mapping

edit
  1. Google Geocoding
  2. /String Analysis#Location Mapping/ Mapping Car Registrations
  3. Flickr GoogleEarth
  4. Nationalgrid and Google Maps
  5. SMS tracker

Timelines

edit
  1. Creating a Timeline - Creating a simple timeline view of events  
  2. Timelines of Resource - Using creation and modification dates to create timelines  
  3. TEI Document Timeline - Creating a timeline of all dates within a single TEI document  

The Semantic Web

edit
  1. DBpedia with SPARQL - Football teams
  2. DBpedia with SPARQL and Simile Timeline - Album Chronology Creating a timeline of album releases using data from Wikipedia  
  3. DBpedia with SPARQL - Stadium locations
  4. The Emp-Dept case study
    1. XML to RDF
    2. SPARQL Tutorial
    3. SPARQL interface
  5. Graphing Triples
  6. SPARQLing Country Calling Codes
  7. Southampton Pubs
  8. Alphabet Poster
  9. Simile Exhibit Browser visualizations using the Simile JavaScript libraries  
  10. Latent Semantic Indexing Finding the semantic distance between documents  


Development Tools

edit
  1. Sitemap for Content Management System XQuery functions can easily perform many common web site content management functions  
  2. Uptime monitor use XQuery to monitor a remote web service  
  3. XQuery IDE - XQuery Integrated development environment  
  4. Image Library - using an XQuery to preview your images  
  5. XML Schema to Instance - XQuery function to generate a sample XML instance from an XML Schema file (.xsd)  
  6. Lorum Ipsum text - generating sample text for inserting into test page layouts  
  7. XQuery and XML Schema - Generating an XML instance document  
  8. Generating XQDocs - Automating the generation of XQuery documentation for Modules and Functions  
  9. XqUSEme - Firefox extension to allow XQueries including against the loaded document (even against originally non-XML (poorly formed) HTML).
  10. Call Graphs - Generating an call graph as an SVG file from module introspection  
  11. System Properties - getting a list of all standard and java system properties and their values  
  12. Environment Variables - getting a list of all the external environment variables and their values  
  13. Generating xqDoc-based XQuery Documentation - generating module documentation using the xqDoc commenting standard

Validation

edit
  1. Validating a document - Validate a document with an XML Schema  
  2. Validation using a Catalog - Using a Catalog file to validate documents  
  3. Validating a hierarchy -  
  4. Validation with Schematron - Apply specific rules to check a document

Path Analysis

edit
  1. All Paths - A report of all paths in a document or collection  
  2. All Leaf Paths - A report of all leaf paths in a document or collection  

Security

edit
  1. Login and Logout - How to log users in and log them out
  2. URL Driven Authorization How to use URL rewriting to check for valid users
  3. Digital Signatures - How to use a custom module to use the XML Digital Signature standards  
  4. Changing Permissions on Collections and Resources - how to change permissions on collections and resources  

Unit Testing Tools

edit
  1. XUnit Testing - what is XUnit testing and how to set them up
  2. XUnit Annotations - how to add annotations to your XQuery functions to drive your XUnit tests  

Case Studies

edit
  1. Fizzbuzz
  2. Project Euler
  3. Topological Sort
  4. Slideshow
  5. Sudoku
  6. Pachube feed
  7. World Temperature records - conversion of text data formats to XML, indexing and data presentation
  8. UWE StudentsOnline

eXist db specific Functions and Configuration

edit

Configuration

edit

Installing modules

edit
  1. Installing the XSL-FO module

Setting HTTP Headers

edit
  1. Setting HTTP Headers

Modules

edit

dbutil

edit
  1. Database Utilities - a set of database utility functions for database administration

compression

edit

Function Reference

  1. Get zipped XML file
  2. Unzipping an Office Open XML docx file - Uncompressing and storing a docx file

eXgit

edit

Module for interacting with any git revision control system

  1. Installing eXgit

EXPath File Transfer XQuery Extension Module

edit

This module does both FTP, SFTP (using the SSH protocol) using a standardized and consistent format.

  1. File Transfer Client

ftp client

edit

This module allows you to interact with a remote FTP server on a remote system. It includes functions for listing, getting and putting files.

  1. FTP Client

httpclient

edit

Function Reference

  1. Digest Authentication
  2. UK shipping forecast

JSON

edit
  1. Convert XML to JSON

Lucene

edit

Function Reference Help

  1. Lucene Search

mail

edit

Function Reference

  1. Sending E-mail
  2. Basic Feedback Form

math

edit

Function Reference

  1. Using the Math Module

memcached

edit
  1. Using the Memcached Module

process

edit
  1. Execute External Process - execute a shell command from within XQuery

request

edit

Function Reference

  1. Getting URL Parameters
  2. Getting POST Data
  3. Checking for Required Parameters
  4. Manipulating URIs
  5. Parsing Query Strings
  6. Adder simple client-server interaction

scheduler

edit

Function Reference Help

  1. XQuery Batch Jobs

sequences

edit

Function Reference

  1. Sequences Module - three additional functions (map, fold and filter)

session

edit

Function Reference

  1. Basic Session Management - the basics of session management including getting a setting session variables  

subversion

edit

Function Reference

  1. Subversion - how to update a subversion repository from within XQuery using the subversion client  

transform

edit

Function Reference

  1. String Analysis

util

edit

Function Reference

  1. Registered Modules : util:registered-modules()
  2. Registered Functions : util:registered-functions()
  3. Dynamic Module Loading : util:import-module(), util:eval()
  4. Higher Order Functions : util:function(), util:call()
  5. Timing Fibonacci algorithms : util:function(), util:call(), util:system-time()
  6. XMP data : util:binary-doc(), util:binary-to-string(), util:parse()
  7. Basic Authentication : util:string-to-binary(), httpclient:get()

validation

edit

Function Reference Help

  1. Validating a document

xmldb

edit

Function Reference

  1. Saving and Updating Data
  2. Splitting Files

xqdoc

edit

Function Reference

  1. Generating xqDoc-based XQuery Documentation

xslfo

edit

XSL-FO (Formatted Objects) is a way of converting XML into PDF. Function Reference

  1. Installing the XSL-FO module - setting up your XSL-Module within eXist
  2. Generating PDF from XSL-FO files - generating PDF from a FO file
  3. XSL-FO Tables - adding tables to your PDF
  4. XSL-FO Images - adding images to your PDF
  5. XSL-FO SVG - adding SVG images to your PDF

Triggers

edit
  1. Using Triggers to Log Events - how to set up a trigger to log store, update and remove events on a collection  
  2. Using Triggers to assign identifiers - how to use triggers to assign identifiers to new documents or new nodes  
  3. Sending E-mail Email is one way to notify when a trigger has fired

XQuery Updates

edit
  1. Inserting and Updating Attributes
  2. Updates and Namespaces - How updates can change serialization  

URL Rewriting

edit
  1. URL Rewriting Basics How to make your URLs look nice  

Apache Ant Tasks

edit
  1. Reindex a Collection

Replication

edit
  1. eXist Replication

General guidance

edit

Appendixes

edit

Systems that Support XQuery

edit

Using native and hybrid XML databases that implement XQuery  

  1. BaseX - Native open source XML Database with visual frontend  
  2. DataDirect XQuery - Java XQuery engine supporting relational, EDI, flat files and XML input/output
  3. eXist - Open source native XML database  
  4. DB2 pureXML - DB2 9.1 includes the pureXML feature  
  5. MarkLogic Server - MarkLogic Server commercial XML Content Server  
  6. Microsoft SQL Server 2005 -  
  7. NetKernel - NetKernel  
  8. Oracle Berkeley DB XML - Open source embedded storage management  
  9. Oracle XML DB - Oracle Server 11g includes the XML DB (XDB) feature  
  10. Sedna - Open source native XML Database  
  11. Stylus Studio - XQuery mapping/editing/debugging, ships with Saxon (and SA) and DataDirect XQuery
  12. EMC xDB - EMC Documentum xDB commercial native XML database  
  13. XQilla - Open source XQuery library and command line utility  
  14. Zorba - Open source XQuery engine C++ implementation with C, Java, Php, Python, Ruby library bindings and command line utility
  15. Qizx - Open source and pro XQuery engine Java implementation

Debugging XQuery

edit
  1. Gotchas - some pitfalls
  2. Ah-has - some ah-ha moments

Other sources

edit

Function Libraries

edit
  1. FunctX XQuery Function Library by Priscilla Walmsley

Discussion Groups

edit
  1. XQuery General


Indexes

edit

Key to symbols:

Wikibook Development Stages
Sparse text   Developing text   Maturing text   Developed text   Comprehensive text  

  indicates an XQuery/Best practice

Retrieved from "https://en.wikibooks.org/w/index.php?title=XQuery&oldid=3857408"