Re: logical changeset generation v6.1

Hi,

On 2013-10-01 10:07:19 -0400, Robert Haas wrote:
> - It seems that HeapSatisfiesHOTandKeyUpdate is now
> HeapSatisfiesHOTandKeyandCandidateKeyUpdate. Considering I think this
> was merely HeapSatisfiesHOTUpdate a year ago, it's hard not to be
> afraid that something unscalable is happening to this function. On a
> related node, any overhead added here costs broadly; I'm not sure if
> there's enough to worry about.

I haven't changed anything here - ISTM so far nobody had a better
suggestion.

> - RelationIsDoingTimetravel is still a crappy name. How about
> RelationRequiredForLogicalDecoding? And maybe the reloption
> treat_as_catalog_table can become required_for_logical_decoding.

Hm. I don't really like the name, required seems to imply that it's
necessary to turn this on to get data replicated in that relation. How
about "accessible_during_logical_decoding" or "user_catalog_table"? The
latter would allow us to use it to add checks for user relations used in
indexes which need a treatment similar to enums.

> All in all, it seems to me that we shouldn't try to punt. Maybe we
> should have something that works like ALTER TABLE name CLUSTER ON
> index_name to configure which index should be used for logical
> replication. Possibly this same syntax could be used as ALTER
> MATERIALIZED VIEW to set the candidate key for that case.

How about using the current logic by default but allow to tune it
additionally with an option like that?

So, attached is the new version:
Changes:
* Fix issues you noticed except the above
* Handle multixacts on system tables
* Logical slots now are checksummed and contain a version and length
* Improve logic for increasing the "restart lsn", the point where we
start to read WAL to decode from next time round
* Wait for xids in snapbuild, during the initial build
* s/RelationIsDoingTimetravel/RelationRequiredForLogicalDecoding/
* test_logical_decoding: confirm reception of changes at the end
* prohibit rewriting schema changes for treat_as_catalog_table relations
* add tests for dropping/adding primary/candidate keys
* PROCESS_INTERRUPTS whene reading wal for SQL SRF
* cleanup old serialized snapshots at check/restart points
* Add more isolationtester changes

Todo:
* rename treat_as_catalog_table, after agreeing on the new name
* rename remaining timetravel function names
* restrict SuspendDecodingSnapshots usage to RelationInitPhysicalAddr,
that ought to be enough.
* add InLogicalDecoding() function.
* throw away older data when reading xl_running_xacts records, to deal
with immediate shutdowns/crashes

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services