Re: Move pgAdmin git repo to Github?

On Thu, May 21, 2020 at 8:27 PM Magnus Hagander <magnus(at)hagander(dot)net> wrote:

>
> On Thu, May 21, 2020 at 1:08 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>
>> We see a non-trivial amount of automated build failures caused by git
>> timeouts and Varnish cache meditation. This is only likely to get worse as
>> we've automated so many different build configurations, and the PostgreSQL
>> sysadmin team don't really want to get into the business of large-scale
>> repository hosting.
>>
>> I suggest we move our primary repo to Github. Any objections?
>>
>>
> Don't you already have a mirror there? Can't you just point the tests at
> that, regardless of where you keep your primary? (It is distributed after
> all)
>

I had discounted that idea as I thought the sync mechanism was done through
a cron job (thus making it a potential problem when doing releases if a
push doesn't hit the mirror immediately). However it does seem to be
synchronous, having done a test commit and remembered the garbage we get on
the output. How does that work exactly? I can't see any obvious hooks.

I'll try just shifting the build systems to use Github and see where that
takes us.

> That said you are of course free to change the primary for whatever
> reason, but this one doesn't seem like one.
>
> And AFAIK nobody has actually reported any such issues. But it is
> certainly true that a lot of the git serving stuff is terribly slow -- but
> I was under the impression that it was mostly gitweb, since thats what
> people tend to report issues with...
>
> But again, no actual objections to moving.
>

We get a lot of failures that look like the following. It doesn't seem to
be restricted to any particular servers in our buildfarm (which have a mix
of 1 and 10Gb/s network connections on a 40Gb/s backbone), and the upstream
network connection is stable (per monitoring) and has 500Mb/s bandwidth
which should be more than enough of course. My working theory is that it's
a dozen or more clones hitting gothos at once is just too much.

Started by upstream project "pgadmin4-rpm-build
<http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-rpm-build/>"
build number 12
<http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-rpm-build/12>
originally caused by:
Started by upstream project "pgadmin4-all-build
<http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-all-build/>"
build number 61
<http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-all-build/61>
originally caused by:
Started by user Dave Page
<http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/user/dpage>
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on pgabf-centos-7
<http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/computer/pgabf-centos-7>
(centos-7) in workspace
/home/jenkins/workspace/pgadmin4-rpm-build/label/centos-7
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] Done
No credentials specified
Cloning the remote Git repository
Cloning repository https://git.postgresql.org/git/pgadmin4.git
> git init /home/jenkins/workspace/pgadmin4-rpm-build/label/centos-7
# timeout=10
Fetching upstream changes from https://git.postgresql.org/git/pgadmin4.git
> git --version # timeout=10
> git fetch --tags --progress
https://git.postgresql.org/git/pgadmin4.git
+refs/heads/*:refs/remotes/origin/* # timeout=10
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress
https://git.postgresql.org/git/pgadmin4.git
+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: error: RPC failed; result=52, HTTP code = 0
fatal: The remote end hung up unexpectedly

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2430)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2044)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:81)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:569)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:798)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
at hudson.remoting.UserRequest.perform(UserRequest.java:211)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call
to pgabf-centos-7
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
at hudson.remoting.Channel.call(Channel.java:955)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor673.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy71.execute(Unknown Source)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1122)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167)
at hudson.scm.SCM.checkout(SCM.java:505)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1205)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1853)
at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
ERROR: Error cloning remote repo 'origin'
[Boolean condition] checking [true] against
[^(1|y|yes|t|true|on|run)$] (origin token: ${PUBLISH_FOR_QA})
Run condition [Boolean condition] enabling perform for step [[Send
build artifacts over SSH]]
SSH: Current build result is [FAILURE], not going to run.
Finished: FAILURE

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company