Re: gitweb is no longer a real-time view

On Mon, Mar 4, 2013 at 3:33 AM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> * Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
>> cron job? I was under the impression there was some sort of push
>> operation driven by a commit trigger. The web site has certainly
>> updated nearly immediately for as long as we've been using git.
>> Until this week, that is.
>
> Curiously, there's two cron jobs, apparently. There's a 'push' one and
> then another, independent, 'pull' one. I'll assume they're actually
> doing different things, but I wonder if the pull isn't just a
> hold-over.. In any case, the push-to-anon, which I hadn't seen

No, the pull one doesn't do the main repository - that one pulls
"third party" repositories in as mirrors, such as e.g. Bucardo. That
one was, btw, broken for several months and nobody noticed - so that's
clearly a less important one :)

> initially (looking at the pull side instead of the push side), does run
> once a minute, though it looks like there's a hook mechanism which
> would allow us to trigger the webserver to do a pull when a commit
> happens and would still be better than a once-a-minute cronjob.

The one that deals with the main repo mirroring runs once per minute.
The commit trigger only drops a file in the filesystem so the cronjob
knows there is something to do. So that we don't risk holding up the
actual commit in case there is a problem with the anonymous mirror
server.

> Even with that, however, the concern raised was that the gitweb perl
> script is quite expensive to run for every request, hence the reason for
> doing the cacheing. I've lowered the varnish cacheing to a 5m ttl and
> 15m grace and I'll keep an eye on it.

That's fixing the wrong problem. The cache is supposed to be
automatically purged whenever the push happens, to make sure that the
data never gets more than maybe a second stale. So clearly something
in that is not working - I didn't specifically test it with the
postgres.git repository, but I tested it with a couple of other ones
where it worked fine. But maybe we did something special with the main
one...

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/