pg13 psql can't connect pg instance with ssl enabled after upgrading

Hello,
I got a strange issue with ssl connection after upgrading an instance from pg11 to pg13.
the instance didn't have ssl enabled, we upgraded it to pg13, then enabled ssl, however when we try to login with pg13 psql 13, psql always try ssl=off to login
public_postgresql_13_1/bin/psql "sslmode=require host=xxxxxx dbname=postgres"
psql: error: FATAL: no pg_hba.conf entry for host "xxxxx", user "xxxx", database "postgres", SSL off

however, if we use pg11 psql, it worked with the upgraded instance
public_postgresql_11_8/bin/psql "sslmode=require host=xxxxxx dbname=postgres"
psql (11.8, server 13.1)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)

but pg13 psql can work with other pg13 instance with ssl enabled(new installation), only the upgraded instance doesn't work

any thoughts?

thanks,

James