Re: Build issue with postgresql 17 undefined reference to `pg_encoding_to_char' and `pg_char_to_encoding'

Furthermore, the queries in this case are 10667 bytes in total, so should
fit inside ten-ish MTUs in an AWS VPC with the pg wire protocol overhead.
So I'm not too worried about the time spent sending the queries. Still, it
would be interesting to measure the impact of storing them in the DB and
pre-computing them as far as possible without authorization information.
We've planned further profiling and optimization of data access and
physical layout. No locks should affect this specific use case.

On Thu, Oct 10, 2024 at 11:07 PM Mikael Sand <msand(at)seaber(dot)io> wrote:

> One question,
>
> I wonder if it would be possible to link pgcommon_shlib and pgport_shlib
> statically into pq and do LTO already at that point, such that consumers of
> libpq could merely link against libpq.a and that would be enough?
>
> I haven't maintained any c or c++ library intended for public distribution
> and haven't needed this kind of functionality, but perhaps someone on the
> mailing list would already have the necessary expertise?
>
> Br Mikael
>
> On Thu, Oct 10, 2024 at 10:33 PM Mikael Sand <msand(at)seaber(dot)io> wrote:
>
>> Btw Aleksander
>>
>> To give some context, this is a component intended to solve our last
>> scaling issue / most demanding query, which happens at page load and thus
>> need to minimize latency over all else. We've used libpq in pipelined
>> binary mode, serialized the results it in a custom binary format,
>> compressed that using zstd, sent to the frontend, uncompressed by the
>> browser builtin support, and parsed from binary to javascript objects using
>> custom parser. This has improved performance more than 50x over the
>> existing graphql / postgraphile solution so far. It seems likely that doing
>> this work in a read replica would further improve performance
>> significantly, so as not to send the large amount of data from the database
>> to the backend, but this will be a later project. I can have measurements
>> regarding static + lto vs dynamic before that.
>>
>> Br Mikael
>>
>> On Thu, Oct 10, 2024 at 10:22 PM Mikael Sand <msand(at)seaber(dot)io> wrote:
>>
>>> Hi Aleksander
>>>
>>> Ok. So no actual benefit from using dynamic?
>>> Well, it seems postgresql and all dependencies already support it, no?
>>> Doesn't go do static linking by default / prefer it? Unless you use some
>>> part that uses CGO, in which case many go developers appear to disable CGO
>>> anyway and use the plain go implementation instead.
>>> Similarly, the rust community seems to have a preference / strong
>>> support for static, some with alpine, musl, etc.
>>> We're still developing this part of our system, and do intend to measure
>>> the difference, for this we need to be able to build the static version as
>>> well. I can post results once we have them. Have you measured the
>>> difference?
>>>
>>> Br Mikael
>>>
>>> On Thu, Oct 10, 2024 at 10:08 PM Aleksander Alekseev <
>>> aleksander(at)timescale(dot)com> wrote:
>>>
>>>> Hi Mikael,
>>>>
>>>> > We use static linking and link time optimization to squeeze the last
>>>> bits of performance out of the code in our most performance-critical
>>>> queries, and it simplifies our security audits to have a static binary
>>>> running inside chainguard/static as the data we handle is
>>>> sensitive/business critical.
>>>>
>>>> No comments about the security aspect. I'm not a security expert so I
>>>> leave this topic for somebody for whom this is an area of expertise.
>>>>
>>>> I don't buy the optimization part. The performance gain you get is
>>>> next to nothing compared to the time spent transferring SQL queries
>>>> over the network / UDP sockets, parsing and planning them, accessing
>>>> the disk, waiting for the locks, etc. Unless you actually measured it
>>>> in order to show the opposite which I seriously doubt you did. A
>>>> better time investment would be finding actual bottlenecks in the
>>>> given system in order to be able to eliminate them.
>>>>
>>>> > Aleksander, do you have something against static linking or am I
>>>> reading you wrong? I've seen this sentiment in several places but never
>>>> understood why anyone would hold this position. Could you elaborate?
>>>>
>>>> The question is whether it's practical for the PostgreSQL community to
>>>> put effort into supporting / testing static linking of libpq.
>>>> Personally I'm not convinced that demanding from every open-source
>>>> project in existence to support static linking is a sustainable idea
>>>> in the long run, given the fact that there are technologies like
>>>> Docker and programming languages like Go (with pgx client).
>>>>
>>>> --
>>>> Best regards,
>>>> Aleksander Alekseev
>>>>
>>>