Re: speed up a logical replica setup

On Thu, Jan 4, 2024 at 12:22 PM Shlok Kyal <shlok(dot)kyal(dot)oss(at)gmail(dot)com> wrote:
>
> Hi,
> I was testing the patch with following test cases:
>
> Test 1 :
> - Create a 'primary' node
> - Setup physical replica using pg_basebackup "./pg_basebackup –h
> localhost –X stream –v –R –W –D ../standby "
> - Insert data before and after pg_basebackup
> - Run pg_subscriber and then insert some data to check logical
> replication "./pg_subscriber –D ../standby -S “host=localhost
> port=9000 dbname=postgres” -P “host=localhost port=9000
> dbname=postgres” -d postgres"
> - Also check pg_publication, pg_subscriber and pg_replication_slots tables.
>
> Observation:
> Data is not lost. Replication is happening correctly. Pg_subscriber is
> working as expected.
>
> Test 2:
> - Create a 'primary' node
> - Use normal pg_basebackup but don’t set up Physical replication
> "./pg_basebackup –h localhost –v –W –D ../standby"
> - Insert data before and after pg_basebackup
> - Run pg_subscriber
>
> Observation:
> Pg_subscriber command is not completing and is stuck with following
> log repeating:
> LOG: waiting for WAL to become available at 0/3000168
> LOG: invalid record length at 0/3000150: expected at least 24, got 0
>

I think probably the required WAL is not copied. Can you use the -X
option to stream WAL as well and then test? But I feel in this case
also, we should wait for some threshold time and then exit with
failure, removing new objects created, if any.

> Test 3:
> - Create a 'primary' node
> - Use normal pg_basebackup but don’t set up Physical replication
> "./pg_basebackup –h localhost –v –W –D ../standby"
> -Insert data before pg_basebackup but not after pg_basebackup
> -Run pg_subscriber
>
> Observation:
> Pg_subscriber command is not completing and is stuck with following
> log repeating:
> LOG: waiting for WAL to become available at 0/3000168
> LOG: invalid record length at 0/3000150: expected at least 24, got 0
>

This is similar to the previous test and you can try the same option
here as well.

--
With Regards,
Amit Kapila.