Re: 13.4 on RDS, SSL SYSCALL EOF on restore

From: Wells Oliver <wells(dot)oliver(at)gmail(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: pgsql-admin <pgsql-admin(at)postgresql(dot)org>
Subject: Re: 13.4 on RDS, SSL SYSCALL EOF on restore
Date: 2021-10-08 22:09:40
Message-ID: CAOC+FBXr-nMm8u089OZXCNZo+bugi8fsczrTixkM5iuyVbJB5A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

The only thing I see in the RDS log are:

2021-10-06 16:36:05.359 GMT [5009] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-06 16:36:05.359 GMT [5009] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-06 16:36:05 UTC::@:[5009]:LOG: database system is shut down
Postgres Shared Memory Value: 4319797248 bytes
2021-10-06 16:36:05.619 GMT [5035] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-06 16:36:05.619 GMT [5035] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-06 16:36:05 UTC::@:[5035]:LOG: redirecting log output to logging
collector process
2021-10-06 16:36:05 UTC::@:[5035]:HINT: Future log output will appear in
directory "/rdsdbdata/log/error".
2021-10-07 21:52:21.343 GMT [12664] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-07 21:52:21.343 GMT [12664] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-07 21:52:21 UTC::@:[12664]:LOG: database system is shut down
Postgres Shared Memory Value: 4560945152 bytes
2021-10-07 21:52:21.542 GMT [12682] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-07 21:52:21.543 GMT [12682] LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"
2021-10-07 21:52:21 UTC::@:[12682]:LOG: redirecting log output to logging
collector process
2021-10-07 21:52:21 UTC::@:[12682]:HINT: Future log output will appear in
directory "/rdsdbdata/log/error".
2021-10-07 22:19:47 UTC::@:[12683]:LOG: skipping missing configuration
file "/rdsdbdata/config/recovery.conf"

Is there some parameter I can change to get more verbose logging in the
event of these failures?

On Fri, Oct 8, 2021 at 2:35 PM Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> On Fri, Oct 8, 2021 at 02:27:55PM -0700, Wells Oliver wrote:
> > Hi: I am restoring a ~100GB backup using 16 jobs from an EC2 instance to
> an RDS
> > instance (db.m6g.xlarge, which is 16GB RAM and 4 CPU) and it's dying
> midway
> > with the dreaded "SSL SYSCALL error: EOF detected" error.
> >
> > I did create a parameter group to hopefully speed the restoration
> process, it
> > includes:
> >
> > - wal_buffers 8192 (64MB)
> > - checkpoint_timeout 3600 (1h)
> > - min_wal_size 192 (192MB)
> > - max_wal_size 102400 (100GB)
> > - shared_buffers 524288 (4GB)
> > - synchronous_commit 0 (off)
> > - autovacuum 0 (off)
> > - maintenance_work_mem 2097152 (2GB)
> > - work_mem 32768 (32MB)
> >
> > I sourced these from a few different folks as well as some trial and
> error, but
> > now it's blowing up on me.
> >
> > If I revert the RDS instance back to default PG parameters, it restores,
> but it
> > takes 3x the time.
>
> Wow, that is weird. I see that error string happening when PG can't
> extend the receipt buffer on the client side:
>
> appendPQExpBufferStr(&conn->errorMessage,
> libpq_gettext("SSL SYSCALL error: EOF detected\n"));
>
> Can you check the server logs to see if there is any error there? If I
> had to take a guess, I would reduce maintenance_work_mem to 1GB and
> retest.
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
> EDB https://enterprisedb.com
>
> If only the physical world exists, free will is an illusion.
>
>

--
Wells Oliver
wells(dot)oliver(at)gmail(dot)com <wellsoliver(at)gmail(dot)com>

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Tom Lane 2021-10-08 22:16:15 Re: 13.4 on RDS, SSL SYSCALL EOF on restore
Previous Message Bruce Momjian 2021-10-08 21:35:35 Re: 13.4 on RDS, SSL SYSCALL EOF on restore