| From: | Peter Eisentraut <peter(at)eisentraut(dot)org> | 
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> | 
| Subject: | pure parsers and reentrant scanners | 
| Date: | 2024-12-02 09:46:00 | 
| Message-ID: | eb6faeac-2a8a-4b69-9189-c33c520e5b7b@eisentraut.org | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-hackers | 
This patch series changes several parsers in the backend and contrib 
modules to use bison pure parsers and flex reentrant scanners. This is 
ultimately toward thread-safety, but I think it's also just nicer in 
general, and it might also fix a few possible small memory leaks.
I organized this patch series into very small incremental changes so 
that it's easier to follow. The final commits should probably combined a 
bit more (e.g., one per module).
In this patch series I have so far dealt with
* contrib/cube/
* contrib/seg/
* src/backend/replication/repl_*
* src/backend/replication/syncrep_*
These four needed the whole treatment: pure parser, reentrant scanner, 
and updated memory handling.
Also:
* src/backend/utils/adt/jsonpath_scan.l
This one already had a pure parser and palloc-based memory handling, but 
not a reentrant scanner, so I just did that.
The above are all pretty similar, so it was relatively easy to work 
through them once I had the first one figured out.
A couple of things that are still missing in the above:
* For repl_scanner.l, I want to use yyextra to deal with the static 
variables marked /*FIXME*/, but somehow I made that buggy, I'll need to 
take another look later.
* For both the replication parser and the syncrep parser, get rid of the 
global variables to pass back the results. Again, here I need another 
look later. I confused either myself or the compiler on these.
cube, seg, and jsonpath are about as done as I would want them to be.
Not done yet:
* src/backend/utils/misc/guc-file.l
* src/pl/plpgsql/src/pl_gram.y
These have quite different structures and requirements, so I plan to 
deal with them separately.
Not relevant for backend thread-safety:
* src/backend/bootstrap/
It might make sense to eventually covert that one as well, just so that 
the APIs are kept similar. But that could be for later.
Note that the core scanner and parser are already reentrant+pure.
Also, there are various other scanners and parsers in frontends (psql, 
pgbench, ecpg) that are not relevant for this. (Again, it might make 
sense to convert some of them later, and some of them are already done.)
AFAICT, all the options and coding techniques used here are already in 
use elsewhere in the tree, so there shouldn't be any concerns about 
bison or flex version compatibility.
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Amit Kapila | 2024-12-02 09:59:56 | Re: Memory leak in WAL sender with pgoutput (v10~) | 
| Previous Message | Peter Eisentraut | 2024-12-02 09:09:59 | Re: Support POSITION with nondeterministic collations |