HELP -- what am I missing?

From: Larry Rosenman <ler(at)lerctr(dot)org>
To: Buildfarm Members <buildfarm-members(at)lists(dot)postgresql(dot)org>
Subject: HELP -- what am I missing?
Date: 2024-07-31 23:55:31
Message-ID: 39bf5851faeeb0e6c87b9e521cbba249@lerctr.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: buildfarm-members

Given the below config, why is this failing?

playbox% ./run_branches.pl --nosend --nostatus --verbose --run-all
syntax error at ./build-farm.conf line 405, near "# testset1 => { url =>
"git"
Global symbol "@somewhere" requires explicit package name (did you
forget to declare "my @somewhere"?) at ./build-farm.conf line 405.
Global symbol "@somewhere" requires explicit package name (did you
forget to declare "my @somewhere"?) at ./build-farm.conf line 408.
Unknown regexp modifier "/M" at ./build-farm.conf line 408, at end of
line
Unknown regexp modifier "/r" at ./build-farm.conf line 408, at end of
line
Unknown regexp modifier "/f" at ./build-farm.conf line 408, at end of
line
Unknown regexp modifier "/t" at ./build-farm.conf line 408, at end of
line
Unknown regexp modifier "/B" at ./build-farm.conf line 408, at end of
line
Regexp modifiers "/u" and "/l" are mutually exclusive at
./build-farm.conf line 408, at end of line
Regexp modifiers "/u" and "/d" are mutually exclusive at
./build-farm.conf line 408, at end of line
./build-farm.conf has too many errors.
Compilation failed in require at ./run_branches.pl line 86.
playbox%

# -*-perl-*- hey - emacs - this is a perl file

=comment

Copyright (c) 2003-2022, Andrew Dunstan

See accompanying License file for license details

=cut

package PGBuild; ## no critic (RequireFilenameMatchesPackage)

use strict;

use warnings FATAL => 'qw';

our (%conf);

# our($VERSION); $VERSION = 'REL_17';

my $branch;
{
no warnings 'once';
$branch = $main::branch;
}

# useful for making settings relative to the config file location
# all the clients should have used these two standard packages
my $confdir = File::Spec->rel2abs(File::Basename::dirname(__FILE__));

# to force from-source in the directory where this config file is
located,
# uncomment the line below. See also buildroot below
# $PGBuild::Options::from_source = $confdir;

%conf = (

# identity
animal => "anaconda",
secret => "<elided>",

# source code
scm => 'git', # or 'cvs'
git_keep_mirror => 1, # manage a git mirror in the build
root
git_ignore_mirror_failure => 1, # ignore failures in fetching to
mirror
git_gc_hours => 7 * 24, # garbage collect once a week, 0/undef to
disable

# use symlinked git repo from non-HEAD branches, like git-new-workdir
does
# Ony supported on Windows if the user has "Create Symbolic Links"
# privilege. See
# https://github.com/git-for-windows/git/wiki/Symbolic-Links
git_use_workdirs => 1,

# remove work trees between runs, saves disk space
rm_worktrees => 1,

# check if the default git branch name has changed
skip_git_default_check => undef,

scmrepo => undef, # default is community repo for either type
scm_url => undef, # webref for diffs on server - use default for
community

# git_reference => undef, # for --reference on git repo

# external commands and control
make => 'gmake', # or gmake if required. can include path if
necessary.
make_jobs => 2, # >1 for parallel "make" and "make check" steps
tar_log_cmd => undef, # default is "tar -z -cf runlogs.tgz *.log"
# replacement must have the same effect

# archive this many copies of the report summary
archive_reports => undef,

# if on, print the whole web request object if there is a web error
show_error_request => undef,

# Only works if the Unix::Uptime module is available
# inhibits a run if > 0 and the load average in last 1 minute or 5
minutes
# has been greater than this
max_load_avg => undef,

# max time in seconds allowed for a single branch run
# undef/0 means unlimited
wait_timeout => undef,

# if true run installcheck-parallel instead of installcheck
use_installcheck_parallel => undef,

# where and how to build
# must be absolute, can be either Unix or Windows style for MSVC
# undef means default, buildroot dir in script directory
# "$confdir/buildroot" means buildroot in the config file's directory,
# which is useful for auto from-source setups
build_root => undef, # or '/path/to/buildroot',
use_vpath => undef, # set true to do vpath builds

# valgrind settings - default is don't use valgrind
# It will use the .supp file in the source code so that's not in the
# options here.
use_valgrind => undef,
valgrind_options => join(
' ', qw{--quiet --trace-children=yes
--track-origins=yes --read-var-info=yes --num-callers=20
--leak-check=no --gen-suppressions=all --error-limit=no}
),

# if true run tests with debug_discard_caches=1, or the equivalent
# on older branches. Do not set CLOBBER_CACHE_ALWAYS if you use this.
use_discard_caches => undef,

# path to directory with auxiliary web script
# if relative, the must be relative to buildroot/branch
# Now only used on older Msys installations
# aux_path => "../..",

keep_error_builds => 0,
core_file_glob => "*core*", # should work for both Linux and *BSD

# where to report status
target => "https://buildfarm.postgresql.org/cgi-bin/pgstatus.pl",

# where to report change in OS version or compiler version
upgrade_target =>
"https://buildfarm.postgresql.org/cgi-bin/upgrade.pl",

# change this to a true value if using MSVC, in which case also
# see MSVC section below

using_msvc => undef,

# if force_every is a scalar it will be used on all branches, like this
# for legacy reasons:
# force_every => 336 , # max hours between builds, undef or 0 =
unforced
# we now prefer it to be a hash with branch names as the keys, like
this
#
# this setting should be kept conservatively high, or not used at all
-
# for the most part it's best to let the script decide if something
# has changed that requires a new run for the branch.
#
# an entry with a name of 'default' matches any branch not named
force_every => {

# HEAD => 48,
# default => 168,
},

# alerts are triggered if the server doesn't see a build on a branch
after
# this many hours, and then sent out every so often,

alerts => {

# HEAD => { alert_after => 72, alert_every => 24 },
# REL_10_STABLE => { alert_after => 240, alert_every => 48 },
},

# include / exclude patterns for files that trigger a build
# if both are specified then they are both applied as filters
# undef means don't ignore anything.
# exclude qr[^doc/|\.po$] to ignore changes to docs and po files
# (recommended)
# undef means null filter.
trigger_exclude => qr[^doc/|\.po$],
trigger_include => undef,

# settings for mail notices - default to notifying nobody
# these lists contain addresses to be notified
# must be complete email addresses, as the email is sent from the
server

mail_events => {
all => ["ler\(at)lerctr(dot)org"], # unconditional
fail => [], # if this build fails
change => [], # if this build causes a state change
green => [], # if this build causes a state change to/from OK
},

# if this flag is set and ccache is used, an unsuccessful run will
result
# in the removal of the ccache directory (and you need to make sure
that
# its parent is writable). The default is off - ccache should be able
to
# handle failures, although there have been suspicions in the past that
# it's not quite as reliable as we'd want, and thus we have this
option.

ccache_failure_remove => undef,

# set this if you want to use ccache with the default ccache directory
# location, effectively $buildroot/ccache-$animal.

use_default_ccache_dir => 1,

# set this to allow caching of the configure script's results

use_accache => 1,

# env settings to apply within build/report process
# these settings will be seen by all the processes, including the
# configure process.

build_env => {

# use a dedicated cache for the build farm. this should give us
# very high hit rates and slightly faster cache searching.
#
# only set this if you want to set your own path for the ccache
# directory
# CCACHE_DIR => "/path/to/your/ccache",

### use these settings for Cygwin - adjust dumper path to be correct
## see https://cygwin.org/cygwin-ug-net/using-cygwinenv.html
# CYGWIN => 'error_start=c:\cygwin\bin\dumper.exe -d %1 %2',

## Cygwin is known to choke on many concurrent connections, so
## reign back the regression test parallelism
# MAX_CONNECTIONS => '3',

### set this if you need a proxy setting for the
# outbound web transaction that reports the results
# BF_PROXY => 'http://my.proxy.server:portnum/',

# see below for MSVC settings

# possibly set this to something high if you get pg_ctl failures
# default is 120
# PGCTLTIMEOUT => '120',

# run extra TAP tests if listed here
# These are the ones omitted without the setting
# on a secure single user system it makes sense to enable these
# PG_TEST_EXTRA => "ssl ldap kerberos",

# Where to put port locks. Default is the buildroot
# PG_TEST_PORT_DIR => 'some path',
},

# env settings to pass to configure. These settings will only be seen
by
# configure.
config_env => {

# comment out if not using ccache
# ccache is known to cause issues sometimes on msys2
# don't set CC at all if using MSVC, especially with meson
CC => 'ccache cc',

# If using vpath builds, this makes it use true symbolic links
# on Msys2. Only use if the user has Create Symbolic Links privilege
# See https://github.com/git-for-windows/git/wiki/Symbolic-Links
# MSYS => 'winsymlinks:nativestrict',

# In the rare event that you need to tell configure where the prove
# program lives instead of letting it work it out you do it here.
# Msys is one such case.
# PROVE => '/bin/prove',

# CPPFLAGS can be set here and picked up by configure
# This example makes the regression tests run with extra
# checking of user names created by the tests
# CPPFLAGS => '-DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS',

# where to find a usable pkg-config or equivalent
# see for example https://github.com/pkgconf/pkgconf
# PKG_CONFIG => 'c:/path/to/pkg-config.exe',
# PATH-style list of extra directories to search for .pc files
# PKG_CONFIG_PATH => 'c:/path/to/libxml/pkgconfig',
},

# settings added to those in config_env if valgrind is being used
valgrind_config_env_extra => {

CFLAGS => "-fno-omit-frame-pointer -O0 -fPIC",
CPPFLAGS => "-DUSE_VALGRIND -DRELCACHE_FORCE_RELEASE",
},

# don't use --prefix or --with-pgport here
# they are set up by the script
# per-branch config can be done here or
# more simply by using the examples below.
# (default ldap flag is below because it's not supported in all
branches)

# see below for MSVC config

# set true if running with meson
# will be ignored for branches older than 16
using_meson => undef,

# similar to make_jobs above - how much parallelism do you want for
# meson's build and test stages. undef says use meson's default. (1?)
meson_jobs => undef,

# timeout multiplier for meson test jobs.
# default if value is undefined is 3, 0 means no timeout
meson_test_timeout => undef,

# options used with configure (and not for meson)
config_opts => [
qw(
--enable-cassert
--enable-debug
--enable-nls
--with-perl
--with-python
--with-tcl
--with-openssl
--with-ldap
--with-libxml
--with-libxslt
--with-includes=/usr/local/include
--with-libraries=/usr/local/lib
),

# could also add for example --enable-tap-tests
],

# Note, for meson we turn off all the auto options and then selectively
# re-enable them using these options, so zlib and readline have to be
# explicitly enabled here.
# see meson docs re interaction of buildtype, debug and
# optimization settings. debugoptimised = debug:on + optimisation:2
meson_opts => [
qw(
-Dcassert=true
-Dbuildtype=debugoptimized
-Dnls=enabled
-Dplperl=enabled
-Dplpython=enabled
-Dpltcl=enabled
-Dgssapi=enabled
-Dssl=openssl
-Dldap=enabled
-Dlibxml=enabled
-Dlibxslt=enabled
-Dzlib=enabled
-Dreadline=enabled
),

# could for example add -Dtap_tests=enabled
],

# per-branch contents of extra config for check stages.
# each branch has an array of setting lines (no \n required)
# a DEFAULT entry is used for all branches, before any
# branch-specific settings.
extra_config => {
DEFAULT => [
q(log_line_prefix = '%m [%p:%l] %q%a '),
"log_connections = 'true'",
"log_disconnections = 'true'",
"log_statement = 'all'",
"fsync = off"
],
HEAD => ['debug_parallel_query = regress'],
},

optional_steps => {

# which optional steps to run and when to run them
# valid keys are: branches, dow, min_hours_since, min_hour, max_hour
# find_typedefs => { branches => ['HEAD'], dow => [1,4],
# min_hours_since => 25 },
# build_docs => {min_hours_since => 24},
},

# string list of doc targets to build in addition to html
# could be "man postgres-US.pdf postgres-A4.pdf epub"
# see doc/src/sgml/Makefile for complete list
extra_doc_targets => undef,

# locales to test

locales => [qw( C )],

# port number actually used will be based on this param and the branch,
# so we ensure they don't collide

base_port => 5678,

modules => [qw(TestUpgrade TestDecoding)],

# settings used by run_branches.pl
global => {
branches_to_build => 'ALL',

# or 'HEAD_PLUS_LATEST' or 'HEAD_PLUS_LATEST2'
# or 'STABLE' (i.e. live branches other than HEAD)
# or [qw( HEAD RELx_y_STABLE etc )]

# set this if running multiple animals to have them coordinated
# default is this animal's buildroot.
global_lock_dir => "/home/pgbuildfarm/locks,

# settings for parallel runs
parallel_lockdir => undef, # default is global_lock_dir
parallel_stagger => undef, # default 60
max_parallel => undef, # default 10

},

my_tap_tests => {

# settings for module TestMyTap:
# branch setting can be missing altogether, in which case 'main' is
# used, or it can be a scalar, in which case that is used for all
# branches, or it can be a hashref, in which case the value used
# corresponds to the pg branch being used. 'default' is specified,
# is for any branch not otherwise specified. If PGBRANCH is mapped,
# then the Postgres branch is used.
#
# testset1 => { url => "git(at)somewhere(dot)com/testrepo.git",
# branch => 'mymainbranch' # default is 'main'
# },
# testset2 => { url => "git(at)somewhere-else(dot)com/othertestrepo.git",
# branch => { HEAD => 'main',
# default => 'PGBRANCH' },
# },
},

);

# MSVC setup

if ($conf{using_msvc})
{
# get the settings from the VisualStudio installation

require "PGBuild/VSenv.pm"; ## no critic (RequireBarewordIncludes)

# ancestor of vcvarsall.bat script we will use
my $VSdir = "c:/Program Files (x86)/Microsoft Visual Studio/2019";

# the more specific you are here the quicker the search will be
# e.g.
# $VSdir = "$VSdir/BuildTools/VC/Auxiliary/Build";

my $vsenv = PGBuild::VSenv::getenv($VSdir, 'x64');

# add other elements to the PATH, LIB and INCLUDE if needed
$vsenv->{PATH} =
join(';', 'c:/prog/depend/bin', 'c:/other/bin', $vsenv->{PATH});

%{ $conf{build_env} } = (%{ $conf{build_env} }, %$vsenv);

# MSVC non-meson needs a somewhat different style of config opts
(why??)
# What we write here will be literally (via Data::Dumper) put into
# the config.pl file for the MSVC build.

# This will be ignored for meson builds, which use meson_opts rather
# than config_opts. See above for meson_opts.

$conf{config_opts} = {
asserts => 1, # --enable-cassert
integer_datetimes => 1, # --enable-integer-datetimes
nls => undef, # --enable-nls=<path>
tcl => 'c:\tcl', # --with-tcl=<path>
perl => 'c:\perl', # --with-perl=<path>
python => 'c:\python25', # --with-python=<path>
krb5 => undef, # --with-krb5=<path>
ldap => 0, # --with-ldap
openssl => undef, # --with-ssl=<path>
xml => undef, # --with-libxml=<path>
xslt => undef, # --with-libxslt=<path>,
iconv => undef, # path to iconv library
zlib => undef, # --with-zlib=<path>
tap_tests => undef, # --enable-tap-tests
};

}

##################################
#
# examples of per branch processing
# tailor as required for your site.
#
##################################
if ($branch eq 'HEAD')
{

# push(@{$conf{config_opts}},"--enable-depend");

# Note that config_opts is a hashref for MSVC, rather than
# an arrayref like elsewhere. That means the right way to set
# semething there is like this:
#
# $conf{config_opts}->{openssl} = 'c:\path\to\openssl\installation';
}
elsif ($branch =~ /^REL7_/)
{

# push(@{$conf{config_opts}},"--without-tk");
}

1;

--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler(at)lerctr(dot)org
US Mail: 13425 Ranch Road 620 N, Apt 718, Austin, TX 78717-1010

Responses

Browse buildfarm-members by date

  From Date Subject
Next Message Michael Paquier 2024-08-01 06:49:18 Re: Build configured with injection points?
Previous Message Andrew Dunstan 2024-07-31 16:55:26 Re: Remind me...