Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

From: David Ventimiglia <davidaventimiglia(at)hasura(dot)io>
To: Jim Nasby <jim(dot)nasby(at)gmail(dot)com>
Cc: Ron Johnson <ronljohnsonjr(at)gmail(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?
Date: 2024-01-16 19:16:19
Message-ID: CADE7j6gJ8j+237N5xiB_zXdKUfqRSLLJ4-50AOOesh2X-PgXXg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Whoops! Wrong SO link. Here's the correct SO link:

https://stackoverflow.com/questions/77808615/how-to-use-logical-decoding-with-pg-recvlogical-to-pass-changes-through-a-non-tr

On Tue, Jan 16, 2024 at 1:15 PM David Ventimiglia <
davidaventimiglia(at)hasura(dot)io> wrote:

> Thanks for the reply, Jim. No, I'm afraid that's not the missing piece.
> I knew enough to use jq to transform the JSON output into SQL statements.
> What I didn't know enough was about jq. No, the missing piece turned out
> not to have anything to do with PostgreSQL or pg_recvlogical (I guessed
> incorrectly that it might), but rather with jq itself. I didn't realize
> that jq buffers its input and it turns out all I had to do was use its
> --unbuffered switch. The full chapter-and-verse is described in this
> Stack Overflow question and answer
> <https://stackoverflow.com/questions/75784345/how-to-pipe-pg-recvlogical-to-psql-for-logical-replication>
> .
>
> Cheers,
> David
>
> On Tue, Jan 16, 2024 at 12:57 PM Jim Nasby <jim(dot)nasby(at)gmail(dot)com> wrote:
>
>> On 1/13/24 3:34 PM, David Ventimiglia wrote:
>> > The business problem I'm trying to solve is:
>> >
>> > "How do I capture logical decoding events with the wal2json output
>> > encoder, filter them with jq, and pipe them to psql, using
>> pg_recvlogical?"
>>
>> I think the missing piece here is that you can't simply pipe JSON into
>> psql and expect anything useful to happen. Are you using jq to turn the
>> JSON into actual SQL statements? What does some of your jq output look
>> like?
>> --
>> Jim Nasby, Data Architect, Austin TX
>>
>>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2024-01-16 19:18:37 Re: pg_dump Running Slow
Previous Message David Ventimiglia 2024-01-16 19:15:20 Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?