More libs exist than I previously realized. Here are some:
OpenSSL – Everybody knows this one.
GNU Multiple Precision Arithmetic Library (GMP) – Big, but fast.
XySSL MPI (Bignum) source code – Very portable, pretty fast.
MPI: Arbitrary Precision Integer Arithmetic
Arjen K Lenstra’s Long Integer Package
libgcrypt – based on GnuPG code
Also relevant is this page listing Free systems and libraries for doing number theory and computer algebra