Re:

Extremely, sorry forgot to add attachment.

On Mon, Sep 16, 2019 at 11:11 AM nilsocket <nilsocket(at)gmail(dot)com> wrote:

> We want to export data from PG to Kafka,
> We can't rely on extension which we have written as there could be any
> problems which we are not aware of and PG might break.
> We don't want our master to go down because of the extension we have
> written.
>
> So, we are okay with having a new PG instance whose work is just to export
> data, as slave.
>
> What we thought of doing is to pause recovery (start-up) process, on any
> vacuum changes on system catalog tables and resume once, our
> logical-decoding has caught up.
> That way, we aren't bloating our master.
>
> Our problem is, we aren't able to exactly identify what WAL records are
> causing Vacuum changes, as far as our understanding goes, `HEAP_2 (rmgr),
> CLEAN` and `BTREE (rmgr), VACUUM` records.
>
> Inorder to see our understanding is right or not, IPC (inter-process
> communication) between WAL_SENDER and START_UP process, is not efficient
> enough.
>
> I have seen Latches, but I'm not sure exactly how to use them, as from
> comments, my understanding is START_UP process is not available in PGPROC
> array.
>
> For efficient inter-process communication, what would be ideal way to
> communicate between two processes.
>
> I'm new to PostgreSQL, and C - world.
>
> What we are trying to achieve is something similar to this:
>
> START_UP process goes to sleep, as soon as it sees any vacuum on catalog.
> WAL_SENDER process will resume recovery (wakeup), as soon as it caught up
> and goes to sleep.
> START_UP process will wake up when THERE is something for WAL_SENDER to
> send.
>
> Basically, IPC, between two processes, where one process generates work,
> and other consumes it.
> producer should go to sleep, until consumer caught up.
> consumer should signal producer of it's completion and wake up producer.
> cycle goes on...
>
> As I have indicated before, I new to C and PostgreSQL, I familiar with
> GoLang, and I have written a sample program in Go (attached below).
>
> Any suggestions and pointers would be greatly helpful.
>
> --
> Thank you
>

--
Thank you