Re: Supporting huge pages on Windows

On Tue, Jan 10, 2017 at 8:49 AM, Tsunakawa, Takayuki
<tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote:
> Hello, Amit, Magnus,
>
> I'm sorry for my late reply. Yesterday was a national holiday in Japan.
>
>
> From: pgsql-hackers-owner(at)postgresql(dot)org
>> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Amit Kapila
>> PGSharedMemoryReAttach is called after the startup of new process whereas
>> pgwin32_ReserveSharedMemoryRegion is called before the new process could
>> actually start. Basically, pgwin32_ReserveSharedMemoryRegion is used to
>> reserve shared memory for each backend, so calling VirtualAlloc there should
>> follow spec for huge pages. If you have some reason for not using, then
>> it is not clear from your reply, can you try to explain in detail.
>
> OK. The processing takes place in the following order:
>
> 1. postmaster calls CreateProcess() to start a child postgres in a suspended state.
> 2. postmaster calls VirtualAlloc(MEM_RESERVE) in pgwin32_ReserveSharedMemoryRegion() to reserve the virtual address space in the child to secure space for the shared memory. This call just affects the virtual address space and does not allocate physical memory. So the large page is still irrelevant.

Okay, today again reading VirtualAlloc specs, I could see that
MEM_LARGE_PAGES is not not required to reserve the memory region. It
is only required during allocation.

>
> I succeeded by following the same procedure using secpol.msc on Win10, running 64-bit PostgreSQL. I started PostgreSQL as a Windows service because it's the normal way, with the service account being a regular Windows user account(i.e. not LocalSystem).
>
> But... I failed to start PostgreSQL by running "pg_ctl start" from a command prompt, receiving the same error message as you. On the other hand, I could start PostgreSQL on a command prompt with administrative privileges (right-click on "Command prompt" from the Start menu and select "Run as an administrator" in the menu.

Hmm. It doesn't work even on a command prompt with administrative
privileges. It gives below error:

waiting for server to start....2017-01-17 11:20:13.780 IST [4788] FATAL: could
not create shared memory segment: error code 1450
2017-01-17 11:20:13.780 IST [4788] DETAIL: Failed system call was CreateFileMap
ping(size=148897792, name=Global/PostgreSQL:E:/WorkSpace/PostgreSQL/master/Data)
.
2017-01-17 11:20:13.780 IST [4788] LOG: database system is shut down
stopped waiting
pg_ctl: could not start server
Examine the log output.

Now, error code 1450 can occur due to insufficient system resources,
so I have tried by increasing the size of shared memory (higher value
of shared_buffers) without your patch and it works. This indicates
some problem with the patch.

> It seems that Windows removes many privileges, including "Lock Pages in Memory", when starting the normal command prompt. As its evidence, you can use the attached priv.c to see what privileges are assigned and and enabled/disabled. Build it like "cl priv.c" and just run priv.exe on each command prompt. Those runs show different privileges.
>

This is bad.

> Should I need to do something, e.g. explain in the document that the user should use the command prompt with administrative privileges when he uses huge_pages?
>

I think it is better to document in some way if we decide to go-ahead
with the patch.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com