Re: Statistics Import and Export

On Mon, Mar 31, 2025 at 11:11:47AM -0400, Corey Huinker wrote:
> In light of v11-0001 being committed as 4694aedf63bf, I've rebased the
> remaining patches.

I spent the day preparing these for commit. A few notes:

* I've added a new prerequisite patch that skips the second WriteToc() call
for custom-format dumps that do not include data. After some testing and
code analysis, I haven't identified any examples where this produces
different output. This doesn't help much on its own, but it will become
rather important when we move the attribute statistics queries to happen
within WriteToc() in 0002.

* I was a little worried about the correctness of 0002 for dumps that run
the attribute statistics queries twice, but I couldn't identify any
problems here either.

* I removed a lot of miscellaneous refactoring that seemed unnecessary for
these patches. Let's move that to another patch set and keep these as
simple as possible.

* I made a small adjustment to the TOC scan restarting logic in
fetchAttributeStats(). Specifically, we now only allow the scan to
restart once for custom-format dumps that include data.

* While these patches help decrease pg_dump's memory footprint, I believe
pg_restore still reads the entire TOC into memory. That's not this patch
set's problem, but I think it's still an important consideration for the
bigger picture.

Regarding whether pg_dump should dump statistics by default, my current
thinking is that it shouldn't, but I think we _should_ have pg_upgrade
dump/restore statistics by default because that is arguably the most
important use-case. This is more a gut feeling than anything, so I reserve
the right to change my opinion.

My goal is to commit the attached patches on Friday morning, but of course
that is subject to change based on any feedback or objections that emerge
in the meantime.

--
nathan