Re: More efficient build farm animal wakeup?

n Mon, Nov 21, 2022 at 9:58 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> > The buildfarm server now creates a companion to branches_of_interest.txt
> > called branches_of_interest.json which looks like this:
>
> ... okay ...
>

Yeah, it's not as efficient as something like long polling or web sockets,
but it is most definitely a lot simpler!

If we're going to have a lot of animals do pulls of this file every minute
or more, it's certainly a lot better to pull this small file than to make
multiple git calls.

It could trivially be made even more efficient by making the request with
either a If-None-Match or If-Modified-Since. While it's still small, that
cuts the size approximately in half, and would allow you to skip even more
processing if nothing has changed.

> It updates this every time it does a git fetch, currently every 5 minutes.
>
> That up-to-five-minute delay, on top of whatever cronjob delay one has
> on one's animals, seems kind of sad. I've gotten kind of spoiled maybe
> by seeing first buildfarm results typically within 15 minutes of a push.
> But if we're trying to improve matters in this area, this doesn't seem
> like quite the way to go.
>
> But it does seem like this eliminates one expense. Now that you have
> that bit, maybe we could arrange a webhook or something that allows
> branches_of_interest.json to get updated immediately after a push?
>

Webhooks are definitely a lot easier to implement in between our servers
yeah, so that shouldn't be too hard. We could use the same hooks that we
use for borka to build the docs, but have it just run whatever script it is
the buildfarm needs. I assume it's just something trivial to run there,
Andrew?

//Magnus