Connection time when using SSL

I am connecting to a Postgres instance using SSL and seeing fairly slow connect times. I would expect there would be some overhead but it's more than I anticipated. The connection is happening over a network. I am using a wildcard SSL certificate on the server side only.

Using one of these JDBC SSL connect strings takes on average: 1060 ms to connect to the database:
jdbc:postgresql://db01-dev.pointclickcare.com:5432/testdb?ssl=true&sslmode=require&sslfactory=org.postgresql.ssl.jdbc4.LibPQFactory
- or -
jdbc:postgresql://db01-dev.pointclickcare.com:5432/testdb?ssl=true&sslmode=require&sslfactory=org.postgresql.ssl.NonValidatingFactory

Using this JDBC non-SSL connect string takes on average: 190 ms to connect to the database:
jdbc:postgresql://db01-dev.pointclickcare.com:5432/testdb

Does this sound like a reasonable overhead that SSL would add to the connection time or does this seem high? (~870ms/~443% slower using SSL)

I am using this Postgres version:
PostgreSQL 9.4.1 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11), 64-bit

The Postgres JDBC driver I am using is:
postgresql-9.4-1201-jdbc41.jar

My pg_hba.conf is below. Not sure DNS names so DNS lookups shouldn't be a problem although performing an nslookup on my client IP does return very quickly. I've also tried connecting Postgres both using a DNS and IP directly.

# PostgreSQL Client Authentication Configuration File
# ===================================================
# TYPE DATABASE USER ADDRESS METHOD

local all postgres trust

local all all ident

host all all 127.0.0.1/32 md5

host all all ::1/128 md5

hostssl testdb all 0.0.0.0/0 md5

hostssl testdb all ::1/128 md5

# "local" is for Unix domain socket connections only
local all all peer

log_hostname in postgresql.conf is off.

I did a search on the forums and found some older posts. One suggested SSL compression is a culprit of slowdowns but I don't think that would apply to the connection time. Another says it could be the authentication that could be causing the slow down but changing md5 to either password or even trust made no difference to the connect time.