Re: Meson far from ready on Windows

Hi

On Tue, 18 Jun 2024 at 17:08, Andres Freund <andres(at)anarazel(dot)de> wrote:

> > None of the dependencies include cmake files for distribution on Windows,
> > so there are no additional files to tell meson to search for. The same
> > applies to pkgconfig files, which is why the EDB team had to manually
> craft
> > them.
>
> Many of them do include at least cmake files on windows if you build them
> though?
>

The only one that does is libxml2 as far as I can see. And that doesn't
seem to work even if I use --cmake-prefix-path= as you suggested:

C:\Users\dpage\git\postgresql>meson setup --auto-features=disabled --wipe
-Dlibxml=enabled --cmake-prefix-path=C:\build64\lib\cmake\libxml2-2.11.8
build-libxml
The Meson build system
Version: 1.4.0
Source dir: C:\Users\dpage\git\postgresql
Build dir: C:\Users\dpage\git\postgresql\build-libxml
Build type: native build
Project name: postgresql
Project version: 17beta1
C compiler for the host machine: cl (msvc 19.39.33523 "Microsoft (R) C/C++
Optimizing Compiler Version 19.39.33523 for x64")
C linker for the host machine: link link 14.39.33523.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Run-time dependency threads found: YES
Library ws2_32 found: YES
Library secur32 found: YES
Program perl found: YES (C:\Strawberry\perl\bin\perl.EXE)
Program python found: YES (C:\Python312\python.EXE)
Program win_flex found: YES 2.6.4 2.6.4
(C:\ProgramData\chocolatey\bin\win_flex.EXE)
Program win_bison found: YES 3.7.4 3.7.4
(C:\ProgramData\chocolatey\bin\win_bison.EXE)
Program sed found: YES (C:\ProgramData\chocolatey\bin\sed.EXE)
Program prove found: YES (C:\Strawberry\perl\bin\prove.BAT)
Program tar found: YES (C:\Windows\system32\tar.EXE)
Program gzip found: YES (C:\ProgramData\chocolatey\bin\gzip.EXE)
Program lz4 found: NO
Program openssl found: YES (C:\build64\bin\openssl.EXE)
Program zstd found: NO
Program dtrace skipped: feature dtrace disabled
Program config/missing found: YES (sh
C:\Users\dpage\git\postgresql\config/missing)
Program cp found: YES (C:\Program Files (x86)\GnuWin32\bin\cp.EXE)
Program xmllint found: YES (C:\build64\bin\xmllint.EXE)
Program xsltproc found: YES (C:\build64\bin\xsltproc.EXE)
Program wget found: YES (C:\ProgramData\chocolatey\bin\wget.EXE)
Program C:\Python312\Scripts\meson found: YES
(C:\Python312\Scripts\meson.exe)
Check usable header "bsd_auth.h" skipped: feature bsd_auth disabled
Check usable header "dns_sd.h" skipped: feature bonjour disabled
Compiler for language cpp skipped: feature llvm disabled
Found pkg-config: YES (C:\ProgramData\chocolatey\bin\pkg-config.EXE) 0.28
Found CMake: C:\Program Files\Microsoft Visual
Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.EXE
(3.28.0)
Run-time dependency libxml-2.0 found: NO (tried pkgconfig and cmake)

meson.build:796:11: ERROR: Dependency "libxml-2.0" not found, tried
pkgconfig and cmake

A full log can be found at
C:\Users\dpage\git\postgresql\build-libxml\meson-logs\meson-log.txt

>
>
> Btw, I've been working with Bilal to add a few of the dependencies to the
> CI
> images so we can test those automatically. Using vcpkg. We got that nearly
> working, but he's on vacation this week... That does ensure both cmake and
> .pc files are generated, fwiw.
>
> Currently builds gettext, icu, libxml2, libxslt, lz4, openssl, pkgconf,
> python3, tcl, zlib, zstd.

That appears to be using Mingw/Msys, which is quite different from a VC++
build, in part because it's a full environment with its own package manager
and packages that people have put a lot of effort into making work as they
do on unix.

> I'm *NOT* sure that vcpkg is the way to go, fwiw. It does seem
> advantageous to
> use one of the toolkits thats commonly built for building dependencies on
> windows, which seems to mean vcpkg or conan.
>

I don't think requiring or expecting vcpkg or conan is reasonable at all,
for a number of reasons:

- Neither supports all the dependencies at present.
- There are real supply chain verification concerns for vendors.
- That would be a huge change from what we've required in the last 19
years, with no deprecation notices or warnings for packagers etc.

> > And that's why we really need to be able to locate headers and libraries
> > easily by passing paths to meson, as we can't rely on pkgconfig, cmake,
> or
> > things being in some standard directory on Windows.
>
> Except that that often causes hard to diagnose breakages, because that
> doesn't
> allow including the necessary compiler/linker flags [2]. It's a bad
> model, we shouldn't
> perpetuate it. If we want to forever make windows a complicated annoying
> stepchild, that's the way to go.
>

That is a good point, though I suspect it wouldn't solve your second
example of the Kerberos libraries, as you'll get both 32 and 64 bit libs if
you follow their standard process for building on Windows so you still need
to have code to pick the right ones.

>
> FWIW, at least libzstd, libxml [3], lz4, zlib can generate cmake dependency
> files on windows in their upstream code.
>

In the case of zstd, it does not if you build with VC++, the Makefile, or
Meson, at least in my testing. It looks like it would if you built it
with cmake, but I couldn't get that to work in 10 minutes or so of messing
around. And that's a perfect example of what I'm bleating about - there are
often many ways of building things on Windows and there are definitely many
ways of getting things on Windows, and they're not all equal. We've either
got to be extremely prescriptive in our docs, telling people precisely what
they need to download for each dependency, or how to build it themselves in
the way that will work with PostgreSQL, or the build system needs to be
flexible enough to handle different dependency variations, as the old VC++
build system was.

>
> I'm *not* against adding "hardcoded" dependency lookup stuff for libraries
> where other approaches aren't feasible, I just don't think it's a good
> idea to
> add fragile stuff that will barely be tested, when not necessary.
>
> Greetings,
>
> Andres Freund
>
>
> [1] Here's a build of PG with the dependencies installed, builds
> https://cirrus-ci.com/task/4953968097361920
>
> [2] E.g.
>
> https://github.com/postgres/postgres/blob/REL_16_STABLE/src/tools/msvc/Mkvcbuild.pm#L600
>
> https://github.com/postgres/postgres/blob/REL_16_STABLE/src/tools/msvc/Solution.pm#L1039
>
> [3] Actually, at least your libxml build actually *did* include both .pc
> and
> cmake files. So just pointing to the relevant path would do the trick.
>

--
Dave Page
pgAdmin: https://www.pgadmin.org
PostgreSQL: https://www.postgresql.org
EDB: https://www.enterprisedb.com