From: | Maxim Orlov <orlovmg(at)gmail(dot)com> |
---|---|
To: | Heikki Linnakangas <hlinnaka(at)iki(dot)fi> |
Cc: | wenhui qiu <qiuwenhuifx(at)gmail(dot)com>, Alexander Korotkov <aekorotkov(at)gmail(dot)com>, Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: POC: make mxidoff 64 bits |
Date: | 2024-11-19 17:53:49 |
Message-ID: | CACG=ezajc_Pcqmy6fcq-N8+LzCRMzOzJzez2_BgHEu-6RVJtKQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Oops! Sorry for the noise. I've must have been overworking yesterday and
messed up the working branches. v7 was a correct set and v8 don't. Here is
the correction with extended Perl test.
The test itself is in src/bin/pg_upgrade/t/005_offset.pl It is rather heavy
and took about 45 minutes on my i5 with 2.7 Gb data generated. Basically,
each test here is creating a cluster and fill it with multixacts. Thus,
dozens of segments are created using two methods. One is with prepared
transactions, and it creates, roughly, the same amount of segments for
members and for offsets. The other one is based on Heikki's multixids.py
and creates more members than offsets. I've used both of these methods to
generate as much diverse data as possible.
Here is how I test this patch set:
1. You need two pg clusters: the "old" one, i.e. without patch set, and
the "new" with patch set v9 applied.
2. Apply v9-0005-TEST-initdb-option-to-initialize-cluster-with-non.patch.txt
to the "old" and "new" clusters. Note, this is only patch required for
"old" cluster. This will allow you to create a cluster with non-standard
initial multixact and multixact offset. Unfortunately, this patch was not
did not arouse public interest since it is assumed that there is similar
functionality to the pg_resetwal utility. But similar is not mean equal.
See, pg_resetwal must be used after cluster init, thus, we step into some
problems with vacuum and some SLRU segments must be filled with zeroes.
Also, template0 datminmxid must be manually updated. So, in me view,
using this patch is justified and very handy here.
3. Also, apply all the "TEST" (0006 and 0007) patches to the "new"
cluster.
4. Build "old" and "new" pg clusters.
5. Run the test with: PROVE_TESTS=t/005_offset.pl PG_TEST_NOCLEAN=1
oldinstall=/home/orlov/proj/OFFSET3/pgsql-old make check -s -C
src/bin/pg_upgrade/
6. In my case, it took around 45 minutes and generate roughly 2.7 Gb of
data.
"TEST" patches, of course, are for the test purposes and not to be
committed.
In src/bin/pg_upgrade/t/005_offset.pl I try to consider next cases:
- Basic sanity checks.
Here I test various initial multi and offset values (including
wraparound) and see how appropriate segments are generated.
- pg_upgarde tests.
Here is oldinstall ENV is for. Run pg_upgrade for old cluster with multi
and offset values just like in previous step. i.e. with various
combinations.
- Self pg_upgarde.
--
Best regards,
Maxim Orlov.
Attachment | Content-Type | Size |
---|---|---|
v9-0005-TEST-initdb-option-to-initialize-cluster-with-non.patch.txt | text/plain | 25.4 KB |
v9-0004-Get-rid-of-MultiXactMemberFreezeThreshold-call.patch | application/octet-stream | 8.8 KB |
v9-0002-Use-64-bit-multixact-offsets.patch | application/octet-stream | 12.9 KB |
v9-0001-Use-64-bit-format-output-for-multixact-offsets.patch | application/octet-stream | 9.0 KB |
v9-0003-Make-pg_upgrade-convert-multixact-offsets.patch | application/octet-stream | 17.9 KB |
v9-0007-TEST-bump-catver.patch.txt | text/plain | 1.1 KB |
v9-0006-TEST-add-src-bin-pg_upgrade-t-005_offset.pl.patch.txt | text/plain | 13.5 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Nathan Bossart | 2024-11-19 18:09:26 | Re: Statistics Import and Export |
Previous Message | Michel Pelletier | 2024-11-19 17:50:29 | Re: Using Expanded Objects other than Arrays from plpgsql |