reducing our reliance on MD5

There have been a few previous attempts to wean PostgreSQL off of MD5.
Back in 2012, Heikki proposed using SCRAM for our next-generation
authentication mechanism:

http://www.postgresql.org/message-id/507550BD.2030401@vmware.com

That seems likely to be a good idea, but nobody's come up with a
patch. Peter Bex *did* come up with a patch to replace MD5
authentication with "bcrypt", which I guess is based on Blowfish:

http://www.postgresql.org/message-id/20121227144638.GC610@frohike.homeunix.org

Although the patch was described as relatively easy to write, it never
went anywhere, because it *replaced* MD5 authentication with bcrypt,
which would be a big problem for existing clients. It seems clear
that we should add something new and not immediately kill off what
we've already got, so that people can transition smoothly. An idea I
just had today is to keep using basically the same system that we are
currently using for MD5, but with a stronger hash algorithm, like
SHA-1 or SHA-2 (which includes SHA-224, SHA-256, SHA-384, and
SHA-512). Those are slower, but my guess is that even SHA-512 is not
enough slower for anybody to care very much, and if they do, well
that's another reason to make use of the new stuff optional.

Maybe we could make the authentication handshake pluggable:

typedef (*password_encryption_fn)(const char *passwd, const char
*salt, size_t salt_len, char *buf); // like pg_md5_encrypt
extern void register_encrypted_authentication_type(char *name,
password_encryption_fn);

Then we could add a _PG_init() function to pgcrypto that would enable
the use of whatever pgcrypto has got to offer as authentication
handshake methods. This has a couple of advantages. First, we don't
have to suck pgcrypto into core, something we've been reluctant to do;
second, if MD5 or any successor algorithm is shown to be insecure,
users can just drop in a new one, either provided by us or one they
write themselves or something in an external library they have and can
integrate to.

There are a few complications:

1. Each encryption algorithm requires a wire-protocol constant to
identify it, and the client and the server have to agree on that
value. So we'd have to define constants for whatever algorithms we
want to provide via the core distribution, and possibly either
maintain a registry of other values or at least set aside a space
(values > 1e6, maybe) that is reserved for use by extensions.

2. We'd have to figure out how to get support for the new algorithms
into libpq. This actually seems a good bit harder than doing it on
the server-side, because I don't think libpq has any dynamic loading
facilities the way the server does.

Thoughts?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company