RE: pg_upgrade

From: Загороднев Роман Евгеньевич <re(dot)zagorodnev(at)npff(dot)ru>
To: Juan José Santamaría Flecha <juanjo(dot)santamaria(at)gmail(dot)com>, Julien Rouhaud <rjuju123(at)gmail(dot)com>
Cc: "pgsql-bugs(at)postgresql(dot)org" <pgsql-bugs(at)postgresql(dot)org>
Subject: RE: pg_upgrade
Date: 2019-12-30 11:46:05
Message-ID: 813529b9f2b04074b00df9264fae0025@mexmbx01.bops.local
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Thanks for reply.
I performed some tests: fresh install windows server 2016 vm, postgres 9.4 and 12 install. One db on 9.4.
Upgrade command:
SET PGPASSWORD=postgres
C:\PostgreSQL\12\bin\pg_upgrade.exe -b "C:\PostgreSQL\9.4\bin" -B "C:\PostgreSQL\12\bin" -d "C:\PostgreSQL\9.4\data" -D "C:\PostgreSQL\12\data" -k -v -U postgres

Pg_upgrade output:
… linking "C:/PostgreSQL/9.4/data/base/16393/12092" to "C:/PostgreSQL/12/data/base/16410/2613"
linking "C:/PostgreSQL/9.4/data/base/16393/12092_fsm" to "C:/PostgreSQL/12/data/base/16410/2613_fsm"
linking "C:/PostgreSQL/9.4/data/base/16393/12094" to "C:/PostgreSQL/12/data/base/16410/2683"
linking "C:/PostgreSQL/9.4/data/base/16393/16394" to "C:/PostgreSQL/12/data/base/16410/16394"
linking "C:/PostgreSQL/9.4/data/base/16393/16394_fsm" to "C:/PostgreSQL/12/data/base/16410/16394_fsm"
linking "C:/PostgreSQL/9.4/data/base/16393/16401" to "C:/PostgreSQL/12/data/base/16410/16401"
linking "C:/PostgreSQL/9.4/data/base/16393/16401_fsm" to "C:/PostgreSQL/12/data/base/16410/16401_fsm"

Fsutil shows only one location:
C:\>fsutil hardlink list C:\PostgreSQL\12\data\base\16410\1249_vm
\PostgreSQL\12\data\base\16410\1249_vm

C:\>fsutil hardlink list C:\PostgreSQL\12\data\base\16410\113
\PostgreSQL\12\data\base\16410\113

In windows hard links created with mklink. Hardlink can be created only for files.
On the same server I create hardlink for file:
C:\Temp>mklink /H link_pg.cmd c:\Scripts\pg_upgrade.cmd
Hardlink created for link_pg.cmd <<===>> c:\Scripts\pg_upgrade.cmd

And fsutil shows two locations:
C:\Temp>fsutil hardlink list link_pg.cmd
\Scripts\pg_upgrade.cmd
\Temp\link_pg.cmd

In properties of file link_pg.cmd Size on disk are 0.
File properties in data directory:
[cid:image003(dot)jpg(at)01D5BF1F(dot)DC6314F0]

С уважением,
Роман Загороднев
From: Juan José Santamaría Flecha [mailto:juanjo(dot)santamaria(at)gmail(dot)com]
Sent: Friday, December 27, 2019 3:41 PM
To: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Cc: Загороднев Роман Евгеньевич <re(dot)zagorodnev(at)npff(dot)ru>; pgsql-bugs(at)postgresql(dot)org
Subject: Re: pg_upgrade

On Wed, Dec 25, 2019 at 5:03 PM Julien Rouhaud <rjuju123(at)gmail(dot)com<mailto:rjuju123(at)gmail(dot)com>> wrote:
On Wed, Dec 25, 2019 at 4:26 PM Загороднев Роман Евгеньевич
<re(dot)zagorodnev(at)npff(dot)ru<mailto:re(dot)zagorodnev(at)npff(dot)ru>> wrote:
>
> FS - NTFS.
> Used different version (10, 12) and result always the same. In log I see that links was created. But pg_upgrade copies all databases. In destination data folder I see files, not links, and if no free space on disk for copy all data, error appear while upgrade.

Unfortunately, I have no idea what encoding you used, so I can't read
the file. I tried some windows encoding but no luck, can you send the
file in utf8 or mention which encoding you're using? Could you also
specify the full pg_upgrade command line you're using?

Hard links are not easy to distinguish from regular files in Windows.
If you identify a file path from a relation in a user database, like so:

test=# select pg_relation_filepath(c.relname::text) from pg_class c
test-# join pg_namespace n on n.oid = c.relnamespace
test-# where n.nspname not in ('information_schema', 'pg_catalog') and c.relkind = 'r' limit 1;
pg_relation_filepath
----------------------
base/16384/16385
(1 row)

And then check the links of this file:

C:\> fsutil hardlink list %PGDATA%/base/16384/16385
You should see two locations. Can you please confirm this?

Regards,

Juan José Santamaría Flecha

In response to

  • Re: pg_upgrade at 2019-12-27 12:41:03 from Juan José Santamaría Flecha

Responses

  • Re: pg_upgrade at 2019-12-31 11:22:30 from Juan José Santamaría Flecha

Browse pgsql-bugs by date

  From Date Subject
Next Message PG Bug reporting form 2019-12-30 14:13:17 BUG #16180: s_lock botteneck when load pg_stat_statements and save=off|on
Previous Message Amit Kapila 2019-12-30 05:47:08 Re: Reorderbuffer crash during recovery