Re: pgsql: Refactor pg_get_line() to expose an alternative StringInfo-based

Hi

ne 6. 9. 2020 v 20:13 odesílatel Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> napsal:

> Refactor pg_get_line() to expose an alternative StringInfo-based API.
>
> Letting the caller provide a StringInfo to read into is helpful when
> the caller needs to merge lines or otherwise modify the data after
> it's been read. Notably, now the code added by commit 8f8154a50
> can use pg_get_line_append() instead of having its own copy of that
> logic. A follow-on commit will also make use of this.
>
> Also, since StringInfo buffers are a minimum of 1KB long, blindly
> using pg_get_line() in a loop can eat a lot more memory than one would
> expect. I discovered for instance that commit e0f05cd5b caused initdb
> to consume circa 10MB to read postgres.bki, even though that's under
> 1MB worth of data. A less memory-hungry alternative is to re-use the
> same StringInfo for all lines and pg_strdup the results.
>
>
I tried to reuse this new API in pg_dump.c, and I had a problem with
private struct StringInfo.

Inside initdb.c there is a access to this structure via #include
"access/xlog_internal.h" and it is little bit unreadable

maybe there should be included directly with #include "lib/stringinfo.h" ?

Regards

Pavel

> Discussion: https://postgr.es/m/1315832.1599345736@sss.pgh.pa.us
>
> Branch
> ------
> master
>
> Details
> -------
>
> https://git.postgresql.org/pg/commitdiff/8e3c58e6e459b285d37edb6129e412ed25cd90c1
>
> Modified Files
> --------------
> src/backend/libpq/hba.c | 40 +++++++++-----------------
> src/bin/initdb/initdb.c | 12 ++++++--
> src/common/pg_get_line.c | 70
> +++++++++++++++++++++++++++++++--------------
> src/include/common/string.h | 3 ++
> 4 files changed, 75 insertions(+), 50 deletions(-)
>
>