Re: 32TB relation size make mdnblocks overflow

From: Julien Rouhaud <rjuju123(at)gmail(dot)com>
To: 陈佳昕(步真) <buzhen(dot)cjx(at)alibaba-inc(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: 32TB relation size make mdnblocks overflow
Date: 2022-01-18 07:25:10
Message-ID: 20220118072510.uimhupioyhr7tc77@jrouhaud
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Tue, Jan 18, 2022 at 02:21:14PM +0800, 陈佳昕(步真) wrote:
>
> We know that PostgreSQL doesn't support a single relation size over 32TB,
> limited by the MaxBlockNumber. But if we just 'insert into' one relation over
> 32TB, it will get an error message 'unexpected data beyond EOF in block 0 of
> relation' in ReadBuffer_common. The '0 block' is from mdnblocks function
> where the segment number is over 256 and make segno * RELSEG_SIZE over
> uint32's max value. So is it necessary to make the error message more
> readable like 'The relation size is over max value ...' and elog in
> mdnblocks?

I didn't try it but this is supposed to be caught by mdextend():

/*
* If a relation manages to grow to 2^32-1 blocks, refuse to extend it any
* more --- we mustn't create a block whose number actually is
* InvalidBlockNumber. (Note that this failure should be unreachable
* because of upstream checks in bufmgr.c.)
*/
if (blocknum == InvalidBlockNumber)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("cannot extend file \"%s\" beyond %u blocks",
relpath(reln->smgr_rnode, forknum),
InvalidBlockNumber)));

Didn't you hit this?

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2022-01-18 07:44:48 Re: generic plans and "initial" pruning
Previous Message Julien Rouhaud 2022-01-18 07:00:47 Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)