Re: 403 - Forbidden on gitweb pages

On Feb 14, 2013 1:35 AM, "Dave Page" <dpage(at)pgadmin(dot)org> wrote:
>
>
>
> On Thursday, February 14, 2013, Alvaro Herrera wrote:
>>
>> Tom Lane wrote:
>> > Every so often (a few percent of the time, I think) I get "403 -
>> > Forbidden" from following a link on http://git.postgresql.org/gitweb/.
>> > So far, a reload has always worked first time, but there's definitely
>> > something fishy in there. I don't recall exactly when this started
>> > happening, maybe a month or so back; it hasn't been there all along.
>> >
>> > Anybody else see this, or have an idea what broke?
>>
>> Yes, I see this also. I reported it to Magnus, who muttered something
>> about Varnish possibly misbehaving, but that was the end of it.
>>
>
> He told me it was intentional rate limiting, applied because gitweb is so
inefficient.

Correct. And varnish is not involved at all. I think alvaro is confusing it
with me saying that varnish might be a way for us to actually fix the
problem.

The problem is basically that gitweb is a big perl cgi (yes, cgi only)
which means that for every request it had to reload all of perl and reparse
the whole script. And on top of that the actual implementation of several
features is fairly slow in itself.

This caused easy overload - for example the whole site went down when a
search engine spidered us...

The bandaid solution was to rate limit access per ip. The error you hit is
when you're rate limited.

Now, I was supposed to have on my to do list to make a better fix. But as
this one was implemented a long time ago (measured in years, really, I have
to admit I forgot... I'll put bit back on the list.

/Magnus