Re: Move pgAdmin git repo to Github?

FYI, switching the build system to use Github made a whole lot of red turn
blue. :-)

On Fri, May 22, 2020 at 9:21 AM Dave Page <dpage(at)pgadmin(dot)org> wrote:

>
>
> On Thu, May 21, 2020 at 8:27 PM Magnus Hagander <magnus(at)hagander(dot)net>
> wrote:
>
>>
>> On Thu, May 21, 2020 at 1:08 PM Dave Page <dpage(at)pgadmin(dot)org> wrote:
>>
>>> We see a non-trivial amount of automated build failures caused by git
>>> timeouts and Varnish cache meditation. This is only likely to get worse as
>>> we've automated so many different build configurations, and the PostgreSQL
>>> sysadmin team don't really want to get into the business of large-scale
>>> repository hosting.
>>>
>>> I suggest we move our primary repo to Github. Any objections?
>>>
>>>
>> Don't you already have a mirror there? Can't you just point the tests at
>> that, regardless of where you keep your primary? (It is distributed after
>> all)
>>
>
> I had discounted that idea as I thought the sync mechanism was done
> through a cron job (thus making it a potential problem when doing releases
> if a push doesn't hit the mirror immediately). However it does seem to be
> synchronous, having done a test commit and remembered the garbage we get on
> the output. How does that work exactly? I can't see any obvious hooks.
>
> I'll try just shifting the build systems to use Github and see where that
> takes us.
>
>
>> That said you are of course free to change the primary for whatever
>> reason, but this one doesn't seem like one.
>>
>> And AFAIK nobody has actually reported any such issues. But it is
>> certainly true that a lot of the git serving stuff is terribly slow -- but
>> I was under the impression that it was mostly gitweb, since thats what
>> people tend to report issues with...
>>
>> But again, no actual objections to moving.
>>
>
> We get a lot of failures that look like the following. It doesn't seem to
> be restricted to any particular servers in our buildfarm (which have a mix
> of 1 and 10Gb/s network connections on a 40Gb/s backbone), and the upstream
> network connection is stable (per monitoring) and has 500Mb/s bandwidth
> which should be more than enough of course. My working theory is that it's
> a dozen or more clones hitting gothos at once is just too much.
>
> Started by upstream project "pgadmin4-rpm-build <http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-rpm-build/>" build number 12 <http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-rpm-build/12>
> originally caused by:
> Started by upstream project "pgadmin4-all-build <http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-all-build/>" build number 61 <http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/job/pgadmin4-all-build/61>
> originally caused by:
> Started by user Dave Page <http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/user/dpage>
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on pgabf-centos-7 <http://pgabf-jenkins.ox.uk.enterprisedb.com:8080/computer/pgabf-centos-7> (centos-7) in workspace /home/jenkins/workspace/pgadmin4-rpm-build/label/centos-7
> [WS-CLEANUP] Deleting project workspace...
> [WS-CLEANUP] Deferred wipeout is used...
> [WS-CLEANUP] Done
> No credentials specified
> Cloning the remote Git repository
> Cloning repository https://git.postgresql.org/git/pgadmin4.git
> > git init /home/jenkins/workspace/pgadmin4-rpm-build/label/centos-7 # timeout=10
> Fetching upstream changes from https://git.postgresql.org/git/pgadmin4.git
> > git --version # timeout=10
> > git fetch --tags --progress https://git.postgresql.org/git/pgadmin4.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> ERROR: Error cloning remote repo 'origin'
> hudson.plugins.git.GitException: Command "git fetch --tags --progress https://git.postgresql.org/git/pgadmin4.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
> stdout:
> stderr: error: RPC failed; result=52, HTTP code = 0
> fatal: The remote end hung up unexpectedly
>
> at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2430)
> at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2044)
> at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:81)
> at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:569)
> at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:798)
> at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
> at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
> at hudson.remoting.UserRequest.perform(UserRequest.java:211)
> at hudson.remoting.UserRequest.perform(UserRequest.java:54)
> at hudson.remoting.Request$2.run(Request.java:369)
> at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to pgabf-centos-7
> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
> at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
> at hudson.remoting.Channel.call(Channel.java:955)
> at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
> at sun.reflect.GeneratedMethodAccessor673.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
> at com.sun.proxy.$Proxy71.execute(Unknown Source)
> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1122)
> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167)
> at hudson.scm.SCM.checkout(SCM.java:505)
> at hudson.model.AbstractProject.checkout(AbstractProject.java:1205)
> at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
> at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
> at hudson.model.Run.execute(Run.java:1853)
> at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
> at hudson.model.ResourceController.execute(ResourceController.java:97)
> at hudson.model.Executor.run(Executor.java:427)
> ERROR: Error cloning remote repo 'origin'
> [Boolean condition] checking [true] against [^(1|y|yes|t|true|on|run)$] (origin token: ${PUBLISH_FOR_QA})
> Run condition [Boolean condition] enabling perform for step [[Send build artifacts over SSH]]
> SSH: Current build result is [FAILURE], not going to run.
> Finished: FAILURE
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company