Re: Large index operation crashes postgres

Tom,

I'm pretty new to memory debugging, so please be patient if I'm not as
precise as you suppose me to be.
For the records I have run a valgrind postmaster session, starting my
initial indexing routine until it crashes postgres.
If you think this might be enlightning for you, I'll send you the
transcript. It's too long for the list.

I'm not sure, what you're thinking about generating a self-contained
test that exhibits similar bloat.
I have started an index creation using my data without calling postgis
functions. Just to make it busy:
<CREATE INDEX idx_placex_sector ON placex USING btree
(substring(geometry,1,100), rank_address, osm_type, osm_id);>
This is now running against the 50.000.000 rows in placex. I will
update you about the memory usage it takes.

The data itself isn't a secret. I need your experience to find and fix
the problem. For myself I 'll try all necessary steps, you suggest me
to do.

Kind regards
Frans

2010/3/26 Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
> Well, it's pretty clear that you've found a memory leak, but that was
> what we thought before; this data doesn't move us any closer to a fix.
> In particular it's not possible to guess whether the leak should be
> blamed on Postgres or Postgis code.  Even if we knew that, I'm not
> sure we could fix the leak without tracing through actual execution.
>
> Can you generate a self-contained test case that exhibits similar bloat?
> I would think it's probably not very dependent on the specific data in
> the column, so a simple script that constructs a lot of random data
> similar to yours might be enough, if you would rather not show us your
> real data.
>
>                        regards, tom lane
>