Use Python "Limited API" in PL/Python

From: Peter Eisentraut <peter(at)eisentraut(dot)org>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Use Python "Limited API" in PL/Python
Date: 2024-12-02 08:51:17
Message-ID: ee410de1-1e0b-4770-b125-eeefd4726a24@eisentraut.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

This patch changes PL/Python to use the Python "limited API". This API
has stronger ABI stability guarantees.[0] This means, you can build
PL/Python against any Python 3.x version and use any other Python 3.x
version at run time.

This is especially useful for binary packages where the operating system
does not come with a fixed suitable version of Python. For example,
Postgres.app (for macOS) would prefer to link against the Python version
supplied by python.org (Python.app). But that has a 3.x version that
changes over time. So instead they bundle a Python version inside
Postgres.app. The Windows installer used to also bundle Python but as of
PG17 you have to get it yourself, but you have to get a very specific
version [1], which is unsatisfactory. This patch fixes that: You can use
any Python version independent of what PL/Python was built against.
(There is a mechanism to say "at least 3.N", but for this patch, we
don't need that, we can stick with the current minimum of 3.2.)

(I have only tested the macOS side of this, not the Windows side. In
fact, the patch currently doesn't build on Windows on CI. I haven't
figured out why.)

For Linux-style packaging, I don't think this would have any benefit for
users right now, since the OS comes with a Python installation and all
the packages are built against that. But it could potentially be helpful
for packagers. For example, on Debian, this could detach the postgresql
packages from python version transitions. But AFAICT, the Python
packaging layout is not prepared for that. (There are only
libpython3.x.so libraries, no libpython3.so that one would have to link
against.)

Finally, I think this patch is part of a path toward making PL/Python
thread-safe. I don't think the patch by itself changes anything, but if
you read through [2], using heap types is part of the things mentioned
there.

[0]: https://docs.python.org/3/c-api/stable.html
[1]:
https://github.com/EnterpriseDB/edb-installers/blob/REL-17/server/resources/installation-notes.html#L34-L36
[2]: https://docs.python.org/3/howto/isolating-extensions.html

Attachment Content-Type Size
0001-Remove-obsolete-Python-version-check.patch text/plain 1.1 KB
0002-Use-Python-Limited-API-in-PL-Python.patch text/plain 15.9 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Yuya Watari 2024-12-02 08:51:56 Re: [PoC] Reducing planning time when tables have many partitions
Previous Message Bertrand Drouvot 2024-12-02 08:33:27 Drop back the redundant "Lock" suffix from LWLock wait event names