Re: How to build statically on Windows

Daniele (and Jason),

One more comment - digging into this a bit is reassuring. I had worried
that by using psycopg2 windows builds and psycopg2-binary builds, we were
using builds against ancient Postgresql 9.x client libraries. By digging
into the builds process, I can see that as long as my users upgrade to
psycopg2>=2.9.1 and I build that on Linux with the version of PostgreSQL
libraries we install, then the Windows users will have a library built with
PostgreSQL 13.3. Even those application developers who have not
heeded me and are running psycopg2-binary will have libraries built with
PostgreSQL 13.3 (as long as they upgrade).

So, I think my digging in has helped me anyway, and you guys can consider
me helped :)

On Tue, Oct 5, 2021 at 5:04 PM Dan Davis <dansmood(at)gmail(dot)com> wrote:

>
> I am going to tell my team this is much, much harder than cx_Oracle, and
> we may have no choice but to coordinate updates to more recent copies. I
> would never ask you guys to go back and backfill, e.g. build 2.8.5 for
> Python 3.9 just for us.
>
> I decided to at least try the "easy button" and setup a fork of psycopg2
> with appveyor. I changed the files in my fork for the .appveyor
> sub-directory to only build for Python 3.9 and Python 3.6 64 bit. However,
> it seems that appveyor has too many settings for me to easily duplicate the
> environment - not enough is controlled simply from the files in .appveyor
> to make this easy button super easy. If it is possible for someone to
> document the settings, that probably would be of benefit for you guys as
> well.
>
> I know most people creating pull requests can simply rely on github.com
> and appveyor to test that everything works, however.
>
>
>
>
> On Tue, Oct 5, 2021 at 4:40 PM Dan Davis <dansmood(at)gmail(dot)com> wrote:
>
>> Jason,
>>
>> I tried to call the appveyor script, but it was looking for a number of
>> Environment variables I did not have. I got through PY_VER=39 and then
>> thought better of it.
>>
>> Do you think it would be easier to do it this way:
>> * fork psycopg2
>> * make my own appveyor account that builds the versions I need...
>>
>> Simulate the appveyor environment on my own workstation
>>
>> Wow, what a lot I've forgotten since before manylinux1 - I used to do
>> this for lxml, PyCrypto, and a lot of other packages :). Still, they were
>> all more or less one-offs - I never got as automated as modern DevOps
>> allows these days.
>>
>>
>>
>> On Tue, Oct 5, 2021 at 2:11 PM Jason Erickson <jerickso(at)stickpeople(dot)com>
>> wrote:
>>
>>> Hi Dan,
>>>
>>> Yeah, unfortunately --static-libpq doesn't do what it should anymore.
>>> We should remove that option.
>>>
>>> When building pyscopg2 from source, the binary Windows Postgres packages
>>> does not include the statically linked library (In fairness, I haven't
>>> checked the latest packages, but that is how it has been in the past). The
>>> library included with the binary packages was the DLL import library and it
>>> was named libpq.lib. This is an issue, because originally when you built
>>> from source on windows, the file libpq.lib was the static link library,
>>> whereas the DLL import library was named libpqdll.lib. So we have a name
>>> inconsistency and no way to link statically with the Postgres binary
>>> distribution.
>>>
>>> This got even more convoluted a few major versions back with the source
>>> of Postgres, as the Windows perl build scripts quit creating the build file
>>> for the static link library, only building the DLL import library AND
>>> naming it libpq.lib. With our Appveyor build script, I cheated and
>>> modified the perl script to build the library instead of the DLL at this
>>> line:
>>> file_replace('Mkvcbuild.pm', "'libpq', 'dll'", "'libpq', 'lib'")
>>>
>>> With that said, I am not happy with that solution and always intended to
>>> revisit the setup script portion for windows, but always had more questions
>>> than answers, some of them:
>>> * If static libraries are not part of the Postgres binary distribution
>>> (or even the build from source option by default), do we concern ourselves
>>> with them? Personally I prefer static libraries because I think it has less
>>> support issues, but???
>>> * If people are building psycopg from source, what libraries do we
>>> assume they have installed? pg_config probably would not be working, so
>>> include/library paths would have to be passed. What about other
>>> dependencies (ie openssl, still use the has_ssl flag?)? There are slightly
>>> different link libraries between some Postgres, so versions matter, too.
>>> * Do we try to differentiate between the DLL import library and the
>>> static library since the name can be the same between them (size?)?
>>> * Or do we say heck with it and just link against a file named
>>> libpq.lib, letting the builder point to the libpq they want to use?
>>>
>>> Maybe an option is to have setup.py on windows call the appveyor script
>>> (with some modifications) and build everything from scratch? It takes
>>> about 40 minutes to build everything, tho, and does require some storage
>>> space for the source and build files, so not the best solution.
>>>
>>> -jason
>>>
>>>
>>> On Mon, Oct 4, 2021 at 5:13 PM Daniele Varrazzo <
>>> daniele(dot)varrazzo(at)gmail(dot)com> wrote:
>>>
>>>> Hi Dan,
>>>>
>>>> On Tue, 5 Oct 2021 at 01:01, Dan Davis <dansmood(at)gmail(dot)com> wrote:
>>>> >
>>>> > Can anyone give me a solution to build psycopg2 statically on Windows?
>>>> >
>>>> > I have succeeded in building it, but when I run dumpbin /dependents
>>>> on the generated file (the PYD file), it still depends on libpq.dll even
>>>> when I pass --static-libpq.
>>>>
>>>> I haven't personally used --static-libpq in a long time, and neither
>>>> have I used windows for a while. As far as I know that part of the set
>>>> up might have bitrotten.
>>>>
>>>> If anyone can help Dan it would be appreciated.
>>>>
>>>> Dan, there is a ticket/MR in the tracker of which I've never been able
>>>> to make completely sense: https://github.com/psycopg/psycopg2/pull/758
>>>> Would you like to check if that's the issue that doesn't allow
>>>> building the lib statically and if so can you propose a MR or just
>>>> acknowledge that the proposed one works as expected?
>>>>
>>>> Thank you everyone
>>>>
>>>> -- Daniele
>>>>
>>>>
>>>>