Re: Slow shutdowns sometimes on RDS Postgres

Yeah, I figured that. Unfortunately, every time it happens, I open a
support ticket with them, but they always just tell me that this is normal
behavior for postgres. Whether it's "normal" or not, I really would like
to get my db to shut down faster, and their suggestion of manually shutting
down all of my apps ahead of time is a real headache. Given that I haven't
gotten anywhere with their support, I figured I'd try asking on the mailing
list.

Thanks,
Chris

On Thu, Sep 13, 2018 at 3:17 PM Joshua D. Drake <jd(at)commandprompt(dot)com>
wrote:

> On 09/13/2018 03:04 PM, Chris Williams wrote:
> > Hi,
> >
> > I'm using AWS RDS Postgres (9.6.6) and have run into very slow
> > shutdowns (10+ minutes) a few times when making database modifications
> > (e.g. reboot, changing instance size, etc.). Other times, it shuts
> > down quickly (1 minute or so). I have not been able to figure out why
> > sometimes it takes a long time to shutdown.
>
> This is probably something you would have to talk to Amazon about. AWS
> RDS Postgres is a fork of PostgreSQL and not 100% compatible from an
> administrative perspective.
>
> JD
>
>
>
> >
> > When it happens, I see a bunch of lines in the postgres log like the
> > following over and over (almost every second or two) during this 10
> > minute shutdown period:
> > 2018-09-12 06:37:01
> > UTC:XXX.XXX.XXX.XXX(19712):my_user(at)my_db:[16495]:FATAL:
> > 2018-09-12 06:37:01
> > UTC:localhost(31368):rdsadmin(at)rdsadmin:[16488]:FATAL: the database
> > system is shutting down
> >
> > Once I start seeing these messages, I start manually shutting down all
> > of our applications that are connected to the db. I'm not sure if
> > shutting down the apps fixes it or if there's some timeout on the RDS
> > side, but it seems like once I start doing this, the database finally
> > shuts down.
> >
> > When it takes this long to shut down, it ends up causing a lot more
> > downtime than I would like. I've tried asking AWS's support why it
> > takes so long to shutdown sometimes, but they basically just told me
> > that's "how it works" and that I should try to shut down all of my
> > connections ahead of time before making database modifications.
> >
> > We just have a few ruby on rails applications connected to the
> > database, and don't really have any long running or heavy queries and
> > the db is under very light load, so I don't understand why it takes so
> > long to shutdown. We do have a sizeable number of connections though
> > (about 600) and there are two replicas connected to it. I also tried
> > setting idle_in_transaction_session_timeout to 300 seconds to see if
> > that would help, but it made no difference.
> >
> > I was wondering if anyone else had seen this behavior on their RDS
> > Postgres instances or had any suggestions on how I could shorten the
> > shutdown time?
> >
> > Thanks,
> > Chris
> >
>
> --
> Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc
> *** A fault and talent of mine is to tell it exactly how it is. ***
> PostgreSQL centered full stack support, consulting and development.
> Advocate: @amplifypostgres || Learn: https://postgresconf.org
> ***** Unless otherwise stated, opinions are my own. *****
>
>