Re: [HACKERS] LONG varsize - how to go on

> Bruce Momjian wrote:
>
> > > I know that I can deal with a bunch of deferred patches,
> > > staying in sync with CURRENT and having new features only as
> > > patches. But that worx only as long as I have most catalog
> > > changes in CURRENT. One single concurrent catalog change can
> > > cost me days of work in the worst case otherwise.
> >
> > The Feb 1 date is not set in stone. If you would prefer March 1, we can
> > look at that option.
>
> Let's see how far I can get it in the next 3-4 weeks. Then it
> should have turned out if it's worth to delay the release for
> another couple of weeks or not.

Yes, let's see how it goes.

>
> Had an Idea just as I wrote the (now deleted) text that
> appeared here :-)
>
> The problem I wanted to write about are sections (possible,
> even if I don't know about one I haven't written myself :-)
> in the code, where a Datum is explicitly or implicitly
> casted, to get access to vl_len and vl_dat.

You will find only a few files that access vl_len/vl_dat, and the rest
all use macros. mkid or whatever indexing you use on the source tree
will do nicely. The bigger question is going to be handling the VARDATA
entries properly when the relate to system tables. The scope of how
long that data has to exist may be an issue, and textout() may be the
trick in all those cases. The only issue there is pfree'ing the string
once your are done with it.

>
> Well, I intend to redefine the varlena struct and rename any
> macro that deals with it. This way I'll catch any location in
> the code, that does anything with variable size attributes in
> a usual way.

Yes, but again, just using mkid or something else will find all of them
quickly.

Setting the compress bit to catch any unusual cases may be interesting,
though I can't see how any routine could get to the varlena data without
accessing the field name or macros.

>
> We wanted to use the top 2 bits of vl_len for flags, leaving
> us a theoretical maximum size of 1GB for one single extended
> attribute value. Since I want to leave out the compression
> part for now, I could set the compression info bit ALLWAYS.
> That would force any part of the code, where the above
> casting (abuse) occurs, to immediately CRASH at first hit
> (would allocate or access >=1G of memory and I think this is
> enough to trigger a crash somewhere). If such a setup passes
> BETA, I'll feel comfortable with it.

Yes, makes sense. Good thing user applications will never see our long
pointers. That would be very confusing for them.

I am concerned about your trying to continue development on a snapshot
while we release the 7.0 release. A single pgindent run will mess you
up terribly. I can prevent that, but other work will affect you.

I don't want a release to cause you any additional work. Marc is very
clear on that.

--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026