From: | Craig Ringer <craig(at)2ndquadrant(dot)com> |
---|---|
To: | pgsql-pkg-yum <pgsql-pkg-yum(at)postgresql(dot)org> |
Subject: | Packaging automation, testing and work-reduction |
Date: | 2014-08-19 08:45:38 |
Message-ID: | 53F30EB2.3000802@2ndquadrant.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-pkg-yum |
Hi all
I'm interested in getting more involved in the RPM packaging process for
PostgreSQL. I know I'm new to the scene, so the following might be
jumping the gun a bit, but I'm also aware that it's hard to find time
for PGDG RPM work and more contribution might be valued.
I've produced new spec files for PostgreSQL 9.4 that unify all of
EL-5/6/7 and Fedora 19/20/21 into a single spec that can be rebuilt for
each platform, and I think that's ready for wider testing now. Builds
for all platforms work correctly using 'mock' on a Fedora 20 build-host,
and they test-install into the target platform correctly, and
interoperate with existing PGDG RPMs.
I've fixed a number of packaging bugs in the process. I'll attach the
spec and the rest of the RPM sources in a followup mail to the "unifying
the spec files" thread.
I'm looking at future steps now, and I'd like to outline some ideas that
might make PGDG RPMs easier to maintain, more friendly toward
contribution from others, more reliable, easier to test, and more
inter-operable with other repositories.
My personal motivation - just so I'm upfront about this - is that what
I'm talking about would make it a *lot* easier for me to produce
"hotfix" RPMs for 2ndQuadrant customers who want fixes backported before
a new minor release, produce backport RPMs where they want something
backported into 9.2 or whatever, produce RPMs for customers with
perf/bug fixes that haven't been accepted by upstream yet, and make
packaging BDR a lot easier too.
From my understanding of how things currently work I think it'd also
make it easier to maintain PGDG, but of course I don't have much history
here or the experience with the current process to really back that.
I'd like to (short version):
* Depend on EPEL
(some packages already do, it's just not declared)
* Drop packages from PGDG that're already in EPEL
* Unify spec files for different dists
* Build with Mock
* Use Koji
* Test with Mock
* Let Koji handle createrepo etc
* Move package management to git
Details of what I'm suggesting:
- Depend on EPEL for RHEL/CentOS/SciLinux and remove all packages
provided by EPEL from PGDG to reduce duplication and potential for
conflicts. (This might involve applying for EPEL dev status
eventually, but shouldn't to start with).
As many packages would work w/o EPEL I doubt it'd have to be a hard
dependency. Just say that EPEL is recommended and will be required
for some packages.
- Progressively unify spec files for other packages to remove the need
for individual specs for each target OS, like I've already done
for postgresql94. (Done for Pg 9.4, not prior, but see next item).
- For PostgreSQL packages also make the major versions supported in the
same define, just by changing the macros. So packaging bug fixes are
easier. (I haven't done this, but it doesn't look too hard).
- Always build packages using mock, so issues with build-depends and
undeclared runtime dependencies like those I've recently reported are
identified at build time. (This works with the packages I've
rewritten).
Use mock target definition files that include EPEL repositories for
EL targets.
- Test-install packages in another mock chroot after building, and
run some basic tests - initdb, etc, at least. Trivial to automate,
just run a script within the mock chroot. (I've done part of this
and it's not hard to finish off).
- Automate package building with mock via Koji by setting up a Koji
server for PGDG. This is fairly well documented, and really just
needs a host with a pile of disk space, preferably running a new
Fedora release. Builds can then be submitted to Koji, either as SRPMs
or via pushes to git. The latter involves tarballs in git, but
that's what Fedora does.
Koji can pull the packages from Fedora and EPEL to use as a base
for builds, but (using Mock) will only install and enable those
declared for dependencies or build-depends in any individual build.
I haven't set a Koji up, but I'd very much like to. If PGDG
wants to go this way I'll be happy to do it for PGDG, rather than
creating a private Koji.
Check out:
http://koji.fedoraproject.org/koji/
https://fedorahosted.org/koji/
'mash' is used to manage koji -> yum repo work.
- Let Koji take care of running createrepo, repoview, etc as updates
are pushed, rather than running the manually;
- Test repositories with repoclosure and complain if we find
dependencies that can't be satisfied, e.g.
repoclosure -l fedora -l pgdg94
or
repoclosure -l el5 -l epel5 -l pgdg91
(with a corresponding yum configuration).
This can be done within 'mock', or with individual yum config files
rather than using the system ones.
Thoughts?
It probably makes sense to fire up a Koji instance first. Then start
progressively getting packages building with corrected build-depends and
depends, using mock, and where possible unified to build the same spec
on all dists, moving them into Koji-friendly git management as that happens.
As packages are updated, the copy in svn would want to be updated to
match so there's no drift, marking the spec in git as the authoritative one.
Once everything's building happily in Koji its repos could become the
authorative ones.
Crazy talk?
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | Craig Ringer | 2014-08-19 15:50:43 | RHEL5 postgresql94-contrib is broken, unsatisfied dependency on uuid-ossp |
Previous Message | Marco Nenciarini | 2014-08-18 16:20:44 | Missing packages for PostgreSQL version 9.1.14 RHEL5 x86_64 |