Uniq

Uniq is a line-oriented Unix command-line tool dealing with adjacent identical lines, available on many platforms.

Options

Command-line options aka switches of uniq:

-u: (for unique) Output singleton blocks: output only lines that are not part of blocks of multiple adjacent identical lines.
-d: (for duplicate) Output collapsed multiline blocks: output a single line for each block of multiple adjacent identical lines, discarding lines for blocks of size one.
-c: (for count) Output a single line per a block of one or more adjacent identical lines, preceded by the number of lines in the block.

Command-line options aka switches of some versions of uniq:

Links

Examples of uniq use:

uniq file.txt
- Of adjacent one or more identical lines, outputs only one of them.
sort file.txt | uniq
- Sorts and then, of identical lines, outputs only one of them. Sorting ensures that originally non-adjacent lines that were identical become adjacent.
sort file.txt | uniq -u
- Outputs singleton blocks only.
sort file.txt | uniq -d
- Outputs a single line per multiline block, filtering out singletons.
sort file.txt | uniq -c
- Precedes each line with the block size.
sort file.txt | uniq -c | sort
- Precedes each line with the block size, and sorts the result by the block size. The sorting works well despite being alphabetical since uniq outputs the size indented by spaces to make it work.
sort file.txt | uniq -u -d
- In GNU uniq, outputs nothing, since each of the two options acts as a filter, and combined they filter out everything.

A version of GNU uniq for MS Windows is available from GnuWin32 project, as well as from Cygwin.