Re: Psycopg2 Windows packaging

Hi Jason,

I'll take a look into the AppVeyor build environments and see if I can get
them working in the weekend.

It seems at least the VS 2008 and 2010 versions offer the C++ Express
environment and the VS 2015 is available, so it *should* be possible to
install the necessary libraries on the AppVeyor CI machines with some
effort, which could produce an immutable and SCM configurable build
environment.

As a side note I had the same idea of creating compiler VMs for the
binaries if AppVeyor doesn't indeed support different VS versions, but I
see you've got that covered.

BR,
Aleksi

On 8 November 2016 at 20:02, Jason Erickson <jerickso(at)stickpeople(dot)com>
wrote:

> Hi Aleksi,
>
> As Daniele noted, the latest release (2.6.2) does have windows whl files
> on the PyPI page. This is something new added in the past two releases.
> When we looked into distributing the whl files on PyPI, I considered
> including the .exes, but opted against it since wheels were looking to be
> the preferred method of distributing and the main reason for including them
> in PyPI was for PIP to be able to install them, ie the following works on
> windows with PIP and Python versions 2.6, 2.7, 3.2-3.5:
>
> pip install psycopg2
>
>
>
> As for building the binaries, distutils use the same compiler version that
> the python binaries were built with. This means you need the following
> compilers for the following Python Versions:
>
> Visual Studio 2015 (Visual C++ 14.0)
>
> Python 3.5
>
> Visual Studio 2010 (Visual C++ 10.0)
>
> Python 3.4
> Python 3.3
>
> Visual Studio 2008 (Visual C++ 9.0)
>
> Python 3.2
> Python 3.1
> Python 3.0
> Python 2.7
> Python 2.6
>
>
> Several years back, I had a problem that when multiple versions of VS were
> installed on a machine, at least one version of was sucking in libraries
> from another. My solution was to create a virtual machine for each version
> of Visual Studio to isolate them (Note, this isn't the same DLL hell that
> Daniele referred to).
>
> I am not familiar with AppVeyer, so I don't know its strengths or
> limitations. One requirement of a build environment is that it needs to
> support different versions of Visual C++ for compiling as noted above. The
> other requirement is that there are dependency libraries that need to be
> built with the same compiler version and available for linking. Psycopg2
> is linked against the following libraries:
>
> zlib
> openssl
> postgresql (libpq)
>
>
> If we can make AppVeyer work, I would be glad to move to it.
>
> Anything you can contribute would be appreciated. Feel free to shoot me
> any questions.
>
> -jason
>
> On Tue, Nov 8, 2016 at 6:44 AM, Aleksi Häkli <aleksi(dot)hakli(at)vincit(dot)com>
> wrote:
>
>> Hi, and thanks for the in-depth answer! I did not realize the package was
>> compiled and functional under Win :)
>>
>> In which way is it easiest to contact Jason and is there an IRC channel
>> or something alike besides this mailing list for this discussion?
>>
>> I'd be interested in helping out with the Windows distribution track if
>> there is need and space for new contributors on that front, but would like
>> to discuss with the current Windows packager and I do not have his contact
>> information.
>>
>> I have decent experience in Python packaging on Linux and think that the
>> Windows DLL hell is a thing that could be documented and sorted out in
>> itself. In addition to that, I saw that AppVeyor seems to support OSS
>> projects and Windows builds for free:
>>
>> https://www.appveyor.com/pricing/
>>
>> I am not aware of the current build configuration so forgive me if I'm
>> jumping ahead myself in this.
>>
>> BR,
>> Aleksi
>>
>> On 8 November 2016 at 15:34, Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com
>> > wrote:
>>
>>> On Tue, Nov 8, 2016 at 12:45 PM, Aleksi Häkli <aleksi(dot)hakli(at)vincit(dot)com>
>>> wrote:
>>> > Hello,
>>> >
>>> > I was wondering about the psycopg2 Windows package. Is there any
>>> particular
>>> > reason the package is not distributed for Windows as a precompiled
>>> binary
>>> > through pip? I always install the precompiled executables by
>>> Stickpeople or
>>> > the precompiled wheels by Gohlke, and have never had the actual need to
>>> > install the package on any Windows platform from source (since it is so
>>> > tedious).
>>>
>>> It is, relatively recently.
>>>
>>> https://pypi.python.org/pypi/psycopg2
>>>
>>> The why is: the original author (Federico) is not a Windows user. I
>>> was when I joined the project: I actually started contributing to
>>> psycopg2 with patches to compile with mingw, but Jason (correct me if
>>> I'm wrong) compiles libpq, psycopg, and other required libraries using
>>> the same MS compiler for the target Python version, guaranteeing I
>>> think to avoid to open the door of DLhell. My expertise in that field
>>> stops at Windows 2000.
>>>
>>> > Would it not be easier for everybody to distribute the package via
>>> PyPI as a
>>> > pip installable pre compiled signed binary instead of the current
>>> source
>>> > format which mostly makes the package uninstallable on the platform?
>>> >
>>> > I have ran into the installation issue about 5 years in row,
>>> developing with
>>> > both Windows and Linux, and write my own scripts to avert the issue,
>>> but I
>>> > think it causes unnecessary headaches for many junior developers. All
>>> senior
>>> > developers can probably compile the library themselves if need be.
>>>
>>> I haven't used Windows in the last 10 years, and I don't feel the urge
>>> to start doing it again. This is a free software project, made by
>>> volunteers with limited spare time in their hands and absolutely zero
>>> funding. Nobody has ever stepped ahead more than Jason has to support
>>> a platform he cares about, and while I make an effort to avoid
>>> breaking Windows build, if he wasn't around I wouldn't personally
>>> care. You are very welcome to do some work yourself to support the
>>> library the way you want, or to finance the development of the feature
>>> you care about. If you don't, you get what you pay for, which is what
>>> who pays with time or money needs.
>>>
>>> -- Daniele
>>>
>>
>>
>>
>> --
>> Aleksi Häkli
>> +358 400 357 013
>> aleksi(dot)hakli(at)vincit(dot)com
>>
>
>

--
Aleksi Häkli
+358 400 357 013
aleksi(dot)hakli(at)vincit(dot)com