AW: Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian

Hello Devrim,

Sorry for the long network outage!

>>This is what I do for the Fedora releases. I'm sure you've noticed that
>>in the past.

No, I didn't notice it and I missed it recently, so I made this proposal (see 1. below).

Here are some points concerning $topic:

1. Availability in beta phase of OS distribution

I recently installed a new server with Fedora 39 for production and another virtual machine for test/developing with Fedora 40beta to avoid the immediatly coming system upgrade from F39 to F40.

By using the standard installation procedure from the PGDG download site I wanted to download the repository for F40 for postgres, but this was not available:

sudo dnf install https://download.postgresql.org/pub/repos/yum/reporpms/F-40-x86_64/pgdg-fedora-repo-latest.noarch.rpm

As this is the start point of the standard installation I could not install the newest release from PGDG.

Because the beta software for Fedora silently upgrades to the final version when it becomes GA, this kind of installation guaranties no interruption later on.

Early installation helps to test the installation and possible changes concerning glibc, icu, jit, compiler or similar cases.

For the simplicity I propose to make the repositories available at the last regular minor version update before the normal beta phase of the operating system starts.
This frees you from hurriying on OS GA day to provide them at time. It is highly probable that in this period neither postgres nor the OS get significant changes so every user (being aware of a beta release of an OS) is quite safe.

The schedule for many OS (I only have experience with Fedora, but similar for ubuntu, debian and certainly others) is spring time or automn time.

So the solution is:

Provide the latest minor version packages at the regulary february and august dates for beta versions of upcoming OS versions and also advertise them on the website.

2. Fast propagation of minor versions to OS distributions

I really encourage you in emphasizing downstream to use always the latest minor releases of postgres with a new OS version!

Apart from possible security fixes I see a lot of complaints on general/hackers where users report problems with outdated versions.
Perhaps upstream is not aware of the fact that minor versions are only maintenance releases and users are urged to always use the newest version. (see the corresponding discussion on hackers)

I recently saw a chart (don't remember where) of the delays major distributions have for integrating minor versions. this mostly took over 100 days, sometimes more then 200!

looking at the example of Fedora 40 they provide PG 16.1 available on their repositories (not PGDG), more then 2 months after release of 16.2.
In contrast they integrated GCC 14.01 in F40 (still in beta until may), so no shy of early adoption of important changes.

It really would be usefull if they apply minor releases like any normal upstream patches in a timely fashion (e.g. google chrome, firefox, httpd ...).

3. Packaging and installation

For the normal user packaging and installation is not always obvious.
Postgres is split into different packages which can be installed separately. This is not documented well. There is no guide which packages should be installed to get a standard server like on windows or normal self compilation (including contrib, server, client, libs, libpq).

There is no information of how installing more then one major version on the same server (e.g. for pg_upgrade) interacts with each other (which libpq is in use?, which path is set?).

Removing an installation of an elder version (with more then one installed) leaves the path unset, so that the normal commands don't work as expected. (see my mail from february).

Further it is unclear if the installations bundled with the OS and the repositories provided by PGDG use the same packaging/tools and are fully interchangeable.

Is it possible to upgrade the pg version from OS distribution with the PGDG version on production without any hiccup?

4. Undocumented initialization behaviour

There are subtle differences in initializing the database.

I used to initialize the cluster with initdb and the appropriate flags, coming from windows installations.

Using the standard recommandation on Fedora:

PGSETUP_INITDB_OPTIONS=" --encoding=UTF8 --data-checksums --lc-messages=C --lc-collate=C"
export PGSETUP_INITDB_OPTIONS

sudo /usr/pgsql-16/bin/postgresql-16-setup initdb

I got a database without data checksum enabled!

It took me a long time to realize

-- no sudo, otherwise the options are not taken !!!

to succesfull get

/usr/pgsql-16/bin/postgresql-16-setup initdb

That sudo prohibits the options to be promoted so that postgresql-16-setup did not see them.

I think postgresql-xx-setup (and other changes to the source tree behaviour like pg_hba,conf,ownership of files, which tools run under which user (server, clients applications), priviliges for writing to the file system with server side copy, wal archive copying, firewall settings, service enabling, selinux implications etc.) should be fully documented and the user should be given a real example of its usage, since this is nowhere present in the source tree and unknown for peoble coming from e.g. windows.

5. Improving usability
All these points are from my real world experience of usablity of Postgres. I have managed to learn or circumvent specific aspects, but I think usability for every not so skilled user should be in the main focus.

Please see my suggestions this way!

Hans Buschmann

________________________________
Von: Devrim Gündüz <devrim(at)gunduz(dot)org>
Gesendet: Freitag, 3. Mai 2024 01:44
An: Hans Buschmann; pgsql-hackers(at)postgresql(dot)org
Betreff: Re: Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian

Hi,

On Wed, 2024-04-24 at 11:02 +0000, Hans Buschmann wrote:
> Today (24.4.2024) I upgraded my laptop to Fedora 40, but there where
> no repository available, so I ended with a mix of Fedora 40 and Fedora
> 39 installation.

This was caused by an unexpected and very long network outage that
blocked my access to the build instances. I released packages last
Thursday (2 days after F40 was released, which is 26 April 2024)

I sent emails to the RPM mailing list about the issue:

https://www.postgresql.org/message-id/aec36aec623741ae314692b318c890c646498ca6.camel%40gunduz.org
https://www.postgresql.org/message-id/1fe99b0def5d7539939421fa5b35db2c8f2a40ad.camel%40gunduz.org
https://www.postgresql.org/message-id/3a1b0f58673d35fae9979ed2b149972195c7b8bc.camel%40gunduz.org

> To mitigate this situation, I propose to provide these repositories
> much earlier in the beta phase of the distributions:

This is what I do for the Fedora releases. I'm sure you've noticed that
in the past.

Regards,

--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
Twitter: @DevrimGunduz , @DevrimGunduzTR