Re: no mailing list hits in google

On Wed, Aug 28, 2019 at 6:51 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Merlin Moncure <mmoncure(at)gmail(dot)com> writes:
> > [apologies if this is the incorrect list or is already discussed
> material]
>
> It's not the right list; redirecting to pgsql-www.
>
> > I've noticed that mailing list discussions in -hackers and other
> > mailing lists appear to not be indexed by google -- at all. We are
> > also not being tracked by any mailing list aggregators -- in contrast
> > to a decade ago where we had nabble and other systems to collect and
> > organize results (tbh, often better than we do) we are now at an
> > extreme disadvantage; mailing list activity was formerly and
> > absolutely fantastic research via google to find solutions to obscure
> > technical problems in the database. Limited access to this
> > information will directly lead to increased bug reports, lack of
> > solution confidence, etc.
>
> > My test case here is the query: pgsql-hackers ExecHashJoinNewBatch
> > I was searching out a link to recent bug report for copy/paste into
> > corporate email. In the old days this would fire right up but now
> > returns no hits even though the discussion is available in the
> > archives (which I had to find by looking up the specific day the
> > thread was active). Just a heads up.
>
> Hm. When I try googling that, the first thing I get is
>
> pgsql-hackers - PostgreSQL
>
> https://www.postgresql.org › list › pgsql-hackers
> No information is available for this page.
> Learn why
>
> and the "learn why" link says that "You are seeing this result because the
> page is blocked by a robots.txt file on your website."
>
> So somebody has blocked the archives from being indexed.
> Seems like a bad idea.
>

It blocks /list/ which has the subjects only. The actual emails in
/message-id/ are not blocked by robots.txt. I don't know why they stopped
appearing in the searches... Nothing has been changed around that for many
years from *our* side.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>