Re: Add --include-table-data-where option to pg_dump, to export only a subset of table data

From: Carter Thaxton <carter(dot)thaxton(at)gmail(dot)com>
To: Stephen Frost <sfrost(at)snowman(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Add --include-table-data-where option to pg_dump, to export only a subset of table data
Date: 2018-05-21 21:48:09
Message-ID: CAGiT_HMZUUq4R4Y19C7DF6Bq7ZOkRs24U_ur65bqX=h1ccQkoQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

I've only taken a quick look but I don't see any regression tests, for
> starters, and it's not clear if this can be passed multiple times for
> one pg_dump run (I'd certainly hope that it could be...).
>

Yes, this will absolutely accept multiple options for one run, which is how
I'd imagine it would typically be used.

In fact, for each table_pattern:filter_clause you provide as an option, it
will apply a corresponding WHERE clause for *every* table that matches the
table_pattern.
So if you happened to use a wildcard in the table_pattern, you could
actually end up with multiple tables filtered by the same WHERE clause.

For example:
pg_dump --include-table-data-where="table_*:created_at >= '2018-05-01'"
--include-table-data-where="other_table:id < 100" db_name

This will filter every table named "table_*", e.g. ["table_0", "table_1",
"table_2", "table_associated"], each with "WHERE created_at >=
'2018-05-01'", and it will also filter "other_table" with "WHERE id < 100".

Not sure how useful the wildcard feature is, but it matches the behavior
of the other pg_dump options that specify tables, and came along for free
by reusing that implementation.

Also, if you haven't already, this should be registered on the
> commitfest app, so we don't lose track of it.
>

Done!
https://commitfest.postgresql.org/18/1644/

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2018-05-21 23:00:52 Re: Add --include-table-data-where option to pg_dump, to export only a subset of table data
Previous Message Jonathan S. Katz 2018-05-21 16:52:31 Re: Postgres 11 release notes