Re: errors on restoring postgresql binary dump to glusterfs

On Mon, May 7, 2012 at 12:54 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Mon, May 7, 2012 at 5:02 PM, Liang Ma <ma(dot)satops(at)gmail(dot)com> wrote:
>> On Fri, May 4, 2012 at 3:58 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> On Mon, Apr 30, 2012 at 8:34 PM, Liang Ma <ma(dot)satops(at)gmail(dot)com> wrote:
>>>> Hi There,
>>>>
>>>> While trying to restore a ~700GM binary dump by command
>>>>
>>>> pg_restore -d dbdata < sampledbdata-20120327.pgdump
>>>>
>>>> I encountered following errors repeatedly
>>>>
>>>> pg_restore: [archiver (db)] Error from TOC entry 2882463; 2613
>>>> 10267347 BLOB 10267347 sdmcleod
>>>> pg_restore: [archiver (db)] could not execute query: ERROR:
>>>> unexpected data beyond EOF in block 500 of relation base/16386/11743
>>>> HINT:  This has been seen to occur with buggy kernels; consider
>>>> updating your system.
>>>
>>> Note the message right here...
>>>
>>> There may be further indications in the server log about what's wrong.
>>>
>>
>> The server's logs in message file were clean.
>
> Then your logging is incorrectly configured, because it should *at
> least* have the same message as the one that showed up in the client.
>

Oh, yes, the same error messages were logged in the postgresql log
file but no further information. I thought you implied that there may
be some indication in server's system logs, which I couldn't find any.

>
>>>> The server runs Ubuntu server 10.04 LTS with postgresql upgraded to
>>>> version 9.1.3-1~lucid. The postgresql data directory is located in a
>>>> glusterfs mounted directory to a replicated volume vol-2
>>>
>>> I assume you don't have more than one node actually *accessing* the
>>> data directory at the same time, right?
>>>
>>
>> Yes, you are right. I just set up this glusterfs and postgresql server
>> with two nodes for testing purpose. There was no other gluster
>> filesystem access activity at the time I tried to restore the
>> postgresql dump. Do you know if postgresql recommends any other
>> cluster filesystem, or it may not like cluster filesystem at all?
>
>
> Did you have PostgreSQL started on both nodes? That is *not*
> supported. If PostgreSQL only runs on one node at a time it should in
> theory work, provided the cluster filesystem provides all the services
> that a normal filesystem does, such as respecting fsync.
>

Postgresql are installed in both nodes, but only one node's postgresql
data directory points to glusterfs filesystem. Another one's data
directory is in its default location in the local ext4 filesystem.
This is the one I used to prove the dump file can be restored without
any problem when glusterfs is not involved.

According to its introduction and document, glusterfs is supposed to
appear as a normal filesystem when being mounted, although I don't
know how well it respects things like fsync.

> --
>  Magnus Hagander
>  Me: http://www.hagander.net/
>  Work: http://www.redpill-linpro.com/

Liang