SQLite/Printable version

SQLite

The current, editable version of this book is available in Wikibooks, the open-content textbooks collection, at
https://en.wikibooks.org/wiki/SQLite

Permission is granted to copy, distribute, and/or modify this document under the terms of the Creative Commons Attribution-ShareAlike 3.0 License.

Introduction

Design

Unlike client–server database management systems, the SQLite engine has no standalone processes with which the application program communicates. Instead, the SQLite library is linked in and thus becomes an integral part of the application program. The library can also be called dynamically. The application program uses SQLite's functionality through simple function calls, which reduce latency in database access: function calls within a single process are more efficient than inter-process communication. SQLite stores the entire database (definitions, tables, indices, and the data itself) as a single cross-platform file on a host machine. It implements this simple design by locking the entire database file during writing. SQLite read operations can be multitasked, though writes can only be performed sequentially.

Due to the server-less design, SQLite applications require less configuration than client-server databases. SQLite is called zero-conf because it does not require service management (such as startup scripts) or access control based on GRANT and passwords. Access control is handled by means of file system permissions given to the database file itself. Databases in client-server systems use file system permissions which give access to the database files only to the daemon process.

Another implication of the serverless design is that several processes may not be able to write to the database file. In server-based databases, several writers will all connect to the same daemon, which is able to handle its locks internally. SQLite on the other hand has to rely on file-system locks. It has less knowledge of the other processes that are accessing the database at the same time. Therefore, SQLite is not the preferred choice for write-intensive deployments. However, for simple queries with little concurrency, SQLite performance profits from avoiding the overhead of passing its data to another process.

SQLite uses PostgreSQL as a reference platform. “What would PostgreSQL do” is used to make sense of the SQL standard. One major deviation is that, with the exception of primary keys, SQLite does not enforce type checking; the type of a value is dynamic and not strictly constrained by the schema (although the schema will trigger a conversion when storing, if such a conversion is potentially reversible). SQLite strives to follow Postel's Rule.

History

D. Richard Hipp designed SQLite in the spring of 2000 while working for General Dynamics on contract with the United States Navy. Hipp was designing software used aboard guided missile destroyers, which originally used HP-UX with an IBM Informix database back-end. SQLite began as a Tcl extension.

The design goals of SQLite were to allow the program to be operated without installing a database management system or requiring a database administrator. Hipp based the syntax and semantics on those of PostgreSQL 6.5. In August 2000, version 1.0 of SQLite was released, with storage based on gdbm (GNU Database Manager). SQLite 2.0 replaced gdbm with a custom B-tree implementation, adding transaction capability. SQLite 3.0, partially funded by America Online, added internationalization, manifest typing, and other major improvements.

In 2011 Hipp announced his plans to add a NoSQL interface (managing documents expressed in JSON) to SQLite databases and to develop UnQLite, an embeddable document-oriented database.. UnQLite was released as an independent database.

Features

SQLite implements most of the SQL-92 standard for SQL but it lacks some features. For example, it partially provides triggers, and it cannot write to views (however it provides INSTEAD OF triggers that provide this functionality). While it provides complex queries, it still has limited ALTER TABLE function, as it cannot modify or delete columns.

SQLite uses an unusual type system for an SQL-compatible DBMS; instead of assigning a type to a column as in most SQL database systems, types are assigned to individual values; in language terms it is dynamically typed. Moreover, it is weakly typed in some of the same ways that Perl is: one can insert a string into an integer column (although SQLite will try to convert the string to an integer first, if the column's preferred type is integer). This adds flexibility to columns, especially when bound to a dynamically typed scripting language. However, the technique is not portable to other SQL products. A common criticism is that SQLite's type system lacks the data integrity mechanism provided by statically typed columns in other products. The SQLite web site describes a "strict affinity" mode, but this feature has not yet been added. However, it can be implemented with constraints like CHECK(typeof(x)='integer').

SQLite with full Unicode function is optional.

Several computer processes or threads may access the same database concurrently. Several read accesses can be satisfied in parallel. A write access can only be satisfied if no other accesses are currently being serviced. Otherwise, the write access fails with an error code (or can automatically be retried until a configurable timeout expires). This concurrent access situation would change when dealing with temporary tables. This restriction is relaxed in version 3.7 when write-ahead logging (WAL) is turned on enabling concurrent reads and writes.

SQLite version 3.7.4 first saw the addition of the FTS4 (full text search) module, which features enhancements over the older FTS3 module. FTS4 allows users to perform full text searches on documents similar to how search engines search webpages. Version 3.8.2 added support for creating tables without rowid, which may provide space and performance improvements. Common table expressions support was added to SQLite in version 3.8.3.

In 2015, with the json1 extension and new subtype interfaces, SQLite version 3.9 introduced JSON content managing.

Development and distribution

SQLite's code is hosted with Fossil, a distributed version control system that is itself built upon an SQLite database.

A standalone command-line program is provided in SQLite's distribution. It can be used to create a database, define tables, insert and change rows, run queries and manage an SQLite database file. It also serves as an example for writing applications that use the SQLite library.

SQLite uses automated regression testing prior to each release. Over 2 million tests are run as part of a release's verification. Starting with the August 10, 2009 release of SQLite 3.6.17, SQLite releases have 100% branch test coverage, one of the components of code coverage. The tests and test harnesses are partially public domain and partially proprietary.

Notable users

• The browsers Google Chrome, Opera, Safari and the Android Browser all allow for storing information in, and retrieving it from, a SQLite database within the browser, using the Web SQL Database technology, although this is rapidly becoming deprecated (namely superseded by IndexedDB).
• Mozilla Firefox and Mozilla Thunderbird store a variety of configuration data (bookmarks, cookies, contacts etc.) in internally managed SQLite databases. There is a third-party add-on that uses the code supporting this functionality to provide a user interface for managing arbitrary SQLite databases.
• Various web application frameworks
• Adobe Systems uses SQLite as its file format in Adobe Photoshop Lightroom, a standard database in Adobe AIR, and internally within Adobe Reader.
• Evernote uses SQLite to store its local database repository in Windows.
• Skype
• Apple adopted it as an option in macOS's Core Data API from the original implementation in Mac OS X 10.4 onwards, and also for administration of videos and songs, and in iOS for storage of text messages on the iPhone.
• Windows 10

Programming language support

A large number of programming languages provide bindings for SQLite, including:

• AutoIt
• various dialects of BASIC including FreeBASIC, PureBasic, RFO BASIC!, Visual Basic and Xojo
• C
• C#
• C++
• Clipper//Harbour
• Curl
• D
• Elixir
• Emacs Lisp
• F#
• Go
• Haskell
• Haxe
• Java (on JVM and DVM)
• JavaScript
• Julia
• distributions of Lisp such as Common Lisp, newLisp, OpenLisp
• Transcript (on LiveCode)
• LabVIEW
• Lua
• MATLAB
• Nim
• Objective-C (on macOS and iOS)
• OCaml
• several flavours of Pascal including Free Pascal, Component Pascal and Delphi
• Perl
• PHP
• Pike
• Python
• R
• Racket
• REBOL
• Ruby
• Scheme
• Smalltalk
• Swift (on macOS and iOS)
• Tcl
• Xojo

References

Sources

• Allen, Grant; Owens, Mike (November 5, 2010). The Definitive Guide to SQLite (2nd ed.). Apress. p. 368. ISBN 1-4302-3225-0.
• Kreibich, Jay A. (August 17, 2010). Using SQLite (1st ed.). O'Reilly Media. p. 528. ISBN 0-596-52118-9.
• van der Lans, Rick F. (September 7, 2009). The SQL Guide to SQLite (1st ed.). lulu.com. p. 542. ISBN 0-557-07676-5.
• Newman, Chris (November 9, 2004). SQLite (Developer's Library) (1st ed.). Sams. p. 336. ISBN 0-672-32685-X.

Downloading and using

Downloading

All tools as well as the C-sources and HTML-documentation you need to install sqlite3 on your computer are packed together in an archiveFile on SQLite's downloadPage.

In following files vv stands for version two digits following "3." uu=update (after ".vv.") and occasionally pp (usually 00) after ".uu."

As of now (12:31, 29 March 2020 (UTC)) vv=31, uu=01, pp=00

• Tools are stored in one or more files depending on your Operating System:

Android

sqlite-android-3vvuupp.aar

Linux

• sqlite-tools-linux-x86-3vvuupp.zip

Mac OS X (x86)

• sqlite-tools-osx-x86-3vvuupp.zip

Windows

• sqlite-tools-win32-x86-3vvuupp.zip
  • zipped 32-bit DLL (x86) for SQLite3vvuupp
  • zipped 64-bit DLL (x86) for SQLite3vvuupp

Universal Windows Platform

• VSIX package of SQLite3vvuupp for uwp

Windows Phone 8

• Phone 8.0 VSIX package with an extension SDK VS2012 for SQLite3vvuupp
• Phone 8.1 VSIX package with an extension SDK VS2013 for SQLite3vvuupp

Windows Runtime

• Visual Studio 2012 VSIX package with an extension SDK WinRT for SQLite3vvuupp
• Visual Studio 2013 VSIX package with an extension SDK WinRT 8.1 for SQLite3vvuupp

.NET

• source code and binaries of SQLite for .NET

Sources

• Amalgamation of SQLite3vvuupp
• Amalgamation of SQLite3vvuupp + TCL-configuration

Alternative sources

not recommended

• SourceCode of SQLite3vvuupp
• Preprocessed C-sources of SQLite3vvuupp

Beta sources

• Prerelease tarball of 3.32.0 on 26 March 2020

Documentation (HTML)

• Documentation as a bundle of static files to read in a webBrowser

Using

After unzipping and moving the content of the folder(s) under a directory in your %PATH% (or $PATH) you can use the executable sqlite3 from anywhere within your computer system.

Questions?

For questions, requests etcetera there is a forum.

SQL standard divergences

Types

SQLite is dynamically typed. This means that as you fill columns the affinity depends on the content. Four data types exist except NULL: BLOB, INTEGER, REAL (aka FLOAT) and TEXT.

Release3

Before reading this you're supposed to have downloaded and unpacked in your PATH the latest version of the executable command line interface sqlite3 for your operating system.

CommandLine

Usage: sqlite3 [OPTIONS] FILENAME [SQL]

• FILENAME is the name of an SQLite database. A new database is created if the file does not previously exist.
• SQL is a dot-, SQL-command or Pragma. OPTIONS include:

-A ARGS... = .archive ARGS and exit -append append database to end of file -ascii = .mode ascii -bail = .bail on -batch force batch I/O -box = .mode box -column = .mode column -cmd COMMAND run "COMMAND" before reading stdin -csv = .mode csv -deserialize open using sqlite3_deserialize() -echo = .echo on -init FILENAME read/process named file -[no]header = .header [off] or on -help show this message -html = .mode HTML -interactive force interactive I/O -json = .mode json -line = .mode line

-list = .mode list -lookaside SIZE N use N entries of SIZE bytes for lookaside memory -markdown = .mode markdown -memtrace trace all memory allocations and (de)allocations -mmap N default mmap size set to N -newline SEP = .separator "|" SEP. Default: "\n" -nofollow = refuse to open symbolic links to database files -nullvalue TEXT = .NULLvalue TEXT. Default "" -pagecache SIZE N use N slots of SIZE bytes each for page cache memory -quote = .mode quote -readonly open the database read-only -separator SEP = .separator SEP. Default: "|" -stats print memory stats before each finalize -table = .mode table -tabs = .mode tabs -version = .version -vfs NAME use NAME as the default VFS -zip open the file as a ZIP Archive

DotCommands

SQLite3.37 .helpScreen (a-g) .archive OPT... FILE... Manage SQL archives a .auth (ON,OFF) Show authorizer callbacks .backup ?DB? FILE Backup DB (default "main") to FILE .bail (on,off) Stop after hitting an error. Default OFF .binary (on,off) Turn binary output on or off. Default OFF .cd DIRECTORY Change the working DIRECTORY .changes (on,off) Show number of rows changed by SQL .check GLOB Fail if output since .testcase does not match .clone NEWDB Clone data into NEWDB from the existing database .connection (close #) Open/close auxiliary database .databases Show names and files of attached databases .dbconfig ?op? ?val? Show/change sqlite3_db_config() options .dbinfo ?DB? Show status information about the database .dump ?TABLE? ... Dump the database in an SQL text formatt .echo (on,off) Turn command echo on or off .eqp (on,off,trigger,full) En-/disable automatic EXPLAIN QUERY PLAN .excel Display the output of next command in a spreadsheet .exit ?CODE? Exit (=.q) this program (returning CODE) .expert EXPERIMENTAL. Indexes for spec. queries .explain (on,off,auto) Explain format def. auto .filectrl CMD ... Run sqlite3_file_control() operationsf .fullschema ?--indent? Show schema and the content of sqlite_stat tables

(h-q) .headers (on,off) Turn display of headers on or off .help ?REGEX? Show this message (or only REGEX)h .import OPT... FILE TABLE Import FILE > TABLE i .imposter INDEX TABLE Create imposter table TABLE on index INDEX .indexes ?TABLE? Show names of all indexest .limit ?LIMIT? ?VAL? Display or change the value of an SQLITE_LIMIT .lint OPTIONS Report potential schema issues. .load FILE ?ENTRY? Load an extension library .log (FILE,off) Turn logging on or off. FILE can be stderr/stdout .mode MODE ?TABLE? Set output modem for tablet .nullvalue STRING Use STRING in place of NULL values .nonce STRING disable safe mode for one command .once (-e,-x,FILE) Output for the next SQL command only to FILEe .open ?OPTIONS? ?FILE? Close existing database and reopen FILEo .output ?FILE? Send output to FILE or stdout .parameter CMD ... Manage SQL parameter bindingsp .print STRING... Print literal STRING .progress N Invoke progress handler after every N opcodes .prompt MAIN CONTINUE Replace the standard prompts .quit Quit (=.ex) this program ------ screenshot from: ------ https://en.wikibooks.org/wiki/SQLite/Release3#DotCommands

(r-z) .read FILENAME Execute SQL in FILENAME .recover Recover data from corrupt db .restore ?DB? FILE DB content (dflt "main") from FILE .save FILE Write in-memory database into FILE .scanstats (on,off) sqlite3_stmt_scanstatus() metrics .schema ?PATTERN? Show the CREATE statements matching PATTERNs1 .selftest ?--init? Run tests defined in the SELFTEST table .separator COL ?ROW? Change separator for column and optionally rows2 .sha3sum ?OPTIONS...? Compute a SHA3 hash of database content .shell CMD ARGS... Run CMD ARGS... in OS shell (=sy) .show Show the current values for various settings .stats (on,off) Show stats or turn stats on or off .system CMD ARGS... Run CMD ARGS... in OS shell (=.sh) .tables ?TABLE? List names of tablest1 .testcase NAME Begin redirecting output to 'testcase-out.txt' .testctrl CMD ... Run sqlite3_test_control() opst2 .timeout MS Try opening locked tables for MS milliseconds .timer (on,off) Turn SQL timer on or off .trace (FILE,off) Output each SQL statement as it is run .version Show detailed SQLite version info .vfsinfo ?AUX? Information about the top-level VFS .vfslist List all available VFSes .vfsname' ?AUX? Print the name of the VFS stack .width NUM1 ... Set col widths for "column" modew

D-notes

More info @sqlite.org/cli.html#special_commands_to_sqlite3_dot_commands_

a .ar [OPTION...] [FILE...] The .ar command manages sqlar archives. Examples: .ar -cf archive.sar foo bar # Create archive.sar from files foo and bar .ar -tf archive.sar # List members of archive.sar .ar -xvf archive.sar # Verbosely extract files from archive.sar Each command line must feature exactly one command option: -c, --create Create a new archive -u, --update Update or add files to an existing archive -t, --list List contents of archive -x, --extract Extract files from archive And zero or more optional options: -v, --verbose Print each filename as it is processed -f FILE, --file FILE Operate on archive FILE (default is current db) -a FILE, --append FILE Operate on FILE opened using the apndvfs VFS -C DIR, --directory DIR Change to directory DIR to read/extract files -n, --dryrun Show the SQL that would have occurred See also: sar-support e or invoke system text editor (-e) or spreadsheet (-x) on the output.

f CMD ... may be size_limit [LIMIT]; chunk_size SIZE; persist_wal [BOOLEAN]; psow [BOOLEAN]; tempfilename has_moved; lock_timeout MILLISEC h only show commands matching REGEX e.g. p*=starting with p i option to skip first line; --skip 1 l lintOptions: fkey-indexes Find missing foreign key indexes m mode: ascii delimited by 0x1F and 0x1E; csv Comma-separated values; column Left-aligned columns (See .width); html HTML <table> code; insert SQL insert statements for TABLE; line One value per line list delimited by "|"; quote Escape answers as for SQL tabs Tab-separated values; tcl TCL list elements o openOptions: --new option starts with an empty file Other options: --readonly --append --zip p paramCMDs: clear       Erase all bindings init          Initialize the TEMP table that holds bindings list          List the current parameter bindings set PARAMETER VALUE Given SQL parameter PARAMETER a value of VALUE      PARAMETER should start with '$', ':', '@', or '?' unset PARAMETER           Remove PARAMETER from the binding table s1 schemaOption: --indent for pretty-printing s2 separator for both the output mode and .import: t1 If TABLE specified, only dump tables matching LIKE pattern TABLE. t2 CMD ... may be always BOOLEAN; assert BOOLEAN; byteorder; imposter SCHEMA ON/OFF ROOTPAGE; internal_functions BOOLEAN; localtime_fault BOOLEAN never_corrupt BOOLEAN; optimizations DISABLE-MASK; pending_byte OFFSET; prng_reset; prng_restore; prng_save; reserve BYTES-OF-RESERVE w Negative values right-justify

KeyWords

If one wants to use one of following words as an identifier the word needs to be enclosed in two double quotes '"', backquotes '`' or '[' and ']'.

A-B ABORT ACTION ADD AFTER ALL ALTER ALWAYS ANALYZE AND AS ASC ATTACH AUTOINCREMENT BEFORE BEGIN BETWEEN BY

C-Def CASCADE CASE CAST CHECK COLLATE COLUMN COMMIT CONFLICT CONSTRAINT CREATE CROSS CURRENT CURRENT_DATE CURRENT_TIME CURRENT_TIMESTAMP DATABASE DEFAULT DEFERRABLE DEFERRED

Del-Fi DELETE DESC DETACH DISTINCT DO DROP EACH ELSE END ESCAPE EXCEPT EXCLUDE EXCLUSIVE EXISTS EXPLAIN FAIL FALSE FILTER FIRST

Fo-Inn FOLLOWING FOR FOREIGN FROM FULL GENERATED GLOB GROUP GROUPS HAVING IF IGNORE IMMEDIATE IN INDEX INDEXED INITIALLY INNER

Ins-N INSERT INSTEAD INTERSECT INTO IS ISNULL JOIN KEY LAST LEFT LIKE LIMIT MATCH NATURAL NO NOT NOTHING NOTNULL NULL NULLS

O-Ref OF OFFSET ON OR ORDER OTHERS OUTER OVER PARTITION PLAN PRAGMA PRECEDING PRIMARY QUERY RAISE RANGE RECURSIVE REFERENCES

Reg-Ti REGEXP REINDEX RELEASE RENAME REPLACE RESTRICT RIGHT ROLLBACK ROW ROWS SAVEPOINT SELECT SET TABLE TEMP TEMPORARY THEN TIES

To-Z TO TRANSACTION TRIGGER TRUE UNBOUNDED UNION UNIQUE UPDATE USING VACUUM VALUES VIEW VIRTUAL WHEN WHERE WINDOW WITH WITHOUT

K-notes and/or references

Pragmas

Pragmas are special commands to show or [change] behavior of open database[schema](s)

• Usage: PRAGMA [schema-name.]name [ = TEXT or INT ];

active pragmas (61)

analysis_limit application_id auto_vacuum automatic_index busy_timeout cache_size cache_spill case_sensitive_like cell_size_check checkpoint_fullfsync

collation_list compile_options data_version database_list defer_foreign_keys encoding foreign_key_check foreign_key_list foreign_keys freelist_count

fullfsync function_list hard_heap_limit ignore_check_constraints incremental_vacuum index_info index_list index_xinfo integrity_check journal_mode journal_size_limit

legacy_alter_table legacy_file_format locking_mode max_page_count mmap_size module_list optimize page_count page_size pragma_list

query_only quick_check read_uncommitted recursive_triggers reverse_unordered_selects schema_version secure_delete shrink_memory soft_heap_limit stats

synchronous table_info table_xinfo temp_store threads trusted_schema user_version wal_autocheckpoint wal_checkpoint writable_schema

deprecated pragmas (7)

count_changes

data_store_directory default_cache_size

empty_result_callbacks full_column_names

short_column_names temp_store_directory

pragmas with non-standard compile-options (5)

parser_trace

vdbe_addoptrace

vdbe_debug

vdbe_listing

vdbe_trace

• More info: List Of PRAGMAs @SQLite.org