Re: More efficient build farm animal wakeup?

On Mon, Nov 21, 2022 at 9:51 PM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:

>
> On 2022-11-20 Su 17:32, Thomas Munro wrote:
> > On Sun, Nov 20, 2022 at 2:44 AM Andrew Dunstan <andrew(at)dunslane(dot)net>
> wrote:
> >> It might not suit your use case, but one of the things I do to reduce
> >> fetch load is to run a local mirror which runs
> >>
> >> git fetch -q --prune
> >>
> >> every 5 minutes. It also runs a git daemon, and several of my animals
> >> point at that.
> > Thanks. I understand now that my configuration without a local mirror
> > is super inefficient (it spends the first ~25s of each minute running
> > git commands). Still, even though that can be improved by me setting
> > up more stuff, I'd like something event-driven rather than short
> > polling-based for lower latency.
> >
> >> If there's a better git API I'll be happy to try to use it.
> > Cool. Seems like we just have to invent something first...
> >
> > FWIW I'm also trying to chase the short polling out of cfbot. It
> > regularly harasses the git servers at one end (could be fixed with
> > this approach), and wastes a percentage of our allotted CPU slots on
> > the other end by scheduling periodically (could be fixed with webhooks
> > from Cirrus).
>
>
>
> I think I have solved most of the actual issues without getting too
> complex.
>
> Here's how:
>
> The buildfarm server now creates a companion to branches_of_interest.txt
> called branches_of_interest.json which looks like this:
>
> [
> {
> "REL_11_STABLE" : "140c803723"
> },
> {
> "REL_12_STABLE" : "4cbcb7ed85"
> },
> {
> "REL_13_STABLE" : "c13667b518"
> },
> {
> "REL_14_STABLE" : "5cda142bb9"
> },
> {
> "REL_15_STABLE" : "ff9d27ee2b"
> },
> {
> "HEAD" : "51b5834cd5"
> }
> ]

Is there a reason this file is a list of hashes each hash with a single
value in it? Would it make more sense if it was:
{
"REL_11_STABLE": "140c803723",
"REL_12_STABLE": "4cbcb7ed85",
"REL_13_STABLE": "c13667b518",
"REL_14_STABLE": "5cda142bb9",
"REL_15_STABLE": "ff9d27ee2b",
"HEAD": "51b5834cd5"
}

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>