Anderson, Ross — Security Engineering, Wiley, advanced coverage of computer security issues, including cryptography, by one of its foremost practitioners, and most likely its best writer. Covers much more than merely cryptography. Brief on most topics due to the breadth of coverage. Exceptionally clearly written.
Bamford, James — The Puzzle Palace : A Report on America's Most Secret AgencyISBN 0140067485, and the more recent Body of Secrets. The best of a quite small group of books about the US Government's NSA. Most are inadequate, and untrustworthy, for various reasons, including an active reluctance to permit accurate information to be released in any form.
Ferguson, Niels and Bruce Schneier — Practical Cryptography, Wiley, 2003, ISBN 0471223573. Up to date cryptography reference. Covers both algorithms and protocols. This is an in depth consideration of one cryptographic problem, including paths not taken and some reasons why. Most of the material is not otherwise available in a single source. Some is not otherwise available. In a sense, a follow-up to 'Applied Cryptography'.
Kahn, David — The CodebreakersISBN 0684831309 The best available single volume source for cryptographic history, at least for events up to the mid '60s (i.e., to just before DES and the public release of asymmetric key cryptography). The added chapter on more recent developments (in the most recent edition) is regrettably far too thin. See also his other publications on cryptography, and cryptographic history, which have been uniformly excellent.
Katz, Jonathan and Yehuda Lindell - Introduction to Modern Cryptography, CRC Press, 2007. A textbook introduction to modern cryptography, aimed at undergraduate computer science/mathematics majors as well as the technically educated public.
A. J. Menezes, P. C. van Oorschot and S. A. Vanstone — Handbook of Applied CryptographyISBN 0849385237 (online version). Equivalent to Applied Cryptography in many ways, but seriously mathematical. For the technically inclined. Covers few meta-cryptographic topics, such as crypto system design.
Piper, Fred and Sean Murphy — Cryptography : A Very Short IntroductionISBN 0192803158 This book quickly sketches out the major goals, uses, methods, and developments in cryptography.
Schneier, Bruce — Applied Cryptography, 2 ed, Wiley, ISBN 0471117099. The best single volume available covering modern cryptographic practice and possibilities. About as comprehensive as a single volume could have been. Well written, not overly mathematical, and so accessible — mostly — to the non-technical.
Schneier, Bruce — Secrets and Lies, Wiley, ISBN 0471253111, a discussion of the context within which cryptography and cryptosystems work. Meta-cryptography, if you will. Required reading for would-be cryptographers, and nearly so for all cryptography users.
Singh, Simon — The Code BookISBN 1857028899. An anecdotal introduction to the history of cryptography, but much better than such an approach might be expected to produce. Covers more recent material than does Kahn's The Codebreakers. Well written. Sadly, the included cryptanalytic contest has been won and the prize awarded; the cyphers are still worth having a go at, however.