WAL (Write Ahead Logging) files are files, where changed data values are stored in a binary format. This is additional information and in this respect it is redundant to the information in the database files. WAL files are a specific kind of 'diff' files.
Writing to WAL files is very fast as they are written always sequentially. In contrast to WAL files database files are organized in special structures like trees, which possibly must be reorganized during write operations or which points to blocks at far positions. Thus writes to database files are much slower.
For the mentioned performance reasons, when a client requests a write operation like
DELETE the modifications to the data are done in a special sequence and - in some parts - asynchronously to the client requests. First, data is written and flushed to WAL files. Second, it is stored in shared buffers in RAM. Finally, it is written from shared buffers to database files. The client doesn't wait until the end of all operations. After the first two very fast actions, he is informed that his request is completed. The third operation is performed asynchronously at an later (or prior) point in time.
WAL files are collected in the directory pg_wal (pg_xlog in PostgreSQL versions prior to version 10). Depending on the write activities on the database the total size of all WAL files may increase dramatically. Therefore the system must delete them when they are no longer needed. WAL files are available for deletion after the changes in the shared buffers (which correlate to the content of the WAL files) are flushed to the database files. As it is guarantied that this criterion is met after a
CHECKPOINT, there are some dependencies between WAL file delete operations and
- You can define a limit for the total size of all files in the directory:
max_wal_size. If it is reached, PostgreSQL performs an automatic
- You can define a
checkpoint_timeoutin seconds. No later than this number of seconds, PostgreSQL performs an automatic
In both cases the shared buffers gets written to disc, a checkpoint-record is written to the actual WAL file and all older WAL files are ready to be deleted.
The deletion of WAL files may be prevented by other criterion, especially by failing archive commands.
max_wal_size is a soft limit and may be exceeded in such situations.