"WIP: Data at rest encryption" patch and, 2 phase commit.

Hi.

I applied the patch "WIP: Data at rest encryption" to PostgreSQL 11 - beta 2 and I'm working on it.

When this patch is applied, the following problem occurs.

* An error occurs when CHECKPOINT is executed during two-phase commit.
* After an error occurs, if you stop PostgreSQL, it will never start again.

(1) First, execute PREPARE TRANSACTION.

postgres=# BEGIN;
BEGIN
postgres=# PREPARE TRANSACTION 'foo';
PREPARE TRANSACTION
postgres=#

(2) Execute the CHECKPOINT command from another terminal.
CHEKPOINT command fails.

postgres=# CHECKPOINT;
ERROR: checkpoint request failed
HINT: Consult recent messages in the server log for details.
postgres=#

(3) ROLLBACK PREPARED command also fails.

postgres=# ROLLBACK PREPARED 'foo';
ERROR: could not read two-phase state from WAL at 0/167EBA0
postgres=#

(4) Shut down the PostgreSQL server.
During shutdown, a "could not read two-phase state from WAL" error occurs.

2018-07-23 14:49:08.924 JST [15821] LOG: received fast shutdown request
2018-07-23 14:49:08.925 JST [15821] LOG: aborting any active transactions
2018-07-23 14:49:08.925 JST [15831] FATAL: terminating connection due to administrator command
2018-07-23 14:49:08.928 JST [15821] LOG: background worker "logical replication launcher" (PID 15829) exited with exit code 1
2018-07-23 14:49:08.928 JST [15824] LOG: shutting down
2018-07-23 14:49:08.935 JST [15824] FATAL: could not read two-phase state from WAL at 0/167EBA0
2018-07-23 14:49:08.936 JST [15821] LOG: checkpointer process (PID 15824) exited with exit code 1
2018-07-23 14:49:08.936 JST [15821] LOG: terminating any other active server processes
2018-07-23 14:49:08.937 JST [15821] LOG: abnormal database system shutdown
2018-07-23 14:49:08.945 JST [15821] LOG: database system is shut down

(5) When restarting the PostgreSQL server, an error(could not read two-phase state from WAL) occurs
and the PostgreSQL server can not be started.

2018-07-23 14:49:42.489 JST [15864] LOG: listening on IPv6 address "::1", port 5432
2018-07-23 14:49:42.489 JST [15864] LOG: listening on IPv4 address "127.0.0.1", port 5432
2018-07-23 14:49:42.492 JST [15864] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2018-07-23 14:49:42.521 JST [15866] LOG: database system shutdown was interrupted; last known up at 2018-07-23 14:49:08 JST
2018-07-23 14:49:42.674 JST [15866] LOG: database system was not properly shut down; automatic recovery in progress
2018-07-23 14:49:42.676 JST [15866] LOG: redo starts at 0/167EB60
2018-07-23 14:49:42.676 JST [15866] LOG: invalid record length at 0/167EC70: wanted 24, got 0
2018-07-23 14:49:42.676 JST [15866] LOG: redo done at 0/167EC30
2018-07-23 14:49:42.677 JST [15866] FATAL: could not read two-phase state from WAL at 0/167EBA0
2018-07-23 14:49:42.678 JST [15864] LOG: startup process (PID 15866) exited with exit code 1
2018-07-23 14:49:42.678 JST [15864] LOG: aborting startup due to startup process failure
2018-07-23 14:49:42.682 JST [15864] LOG: database system is shut down

Regards.

----
Harada Toshi.
NTT TechnoCross Corporation

Antonin Houska <ah(at)cybertec(dot)at> wrote:
> Ants Aasma <ants(dot)aasma(at)gmail(dot)com> wrote:
>
> > Attached to this mail is a work in progress patch that adds an
> > extensible encryption mechanism. There are some loose ends left to tie
> > up, but the general concept and architecture is at a point where it's
> > ready for some feedback, fresh ideas and bikeshedding.
>
> Rebased patch is attached here, in case it helps to achieve (some of) the
> goals mentioned in the related thread [1].
>
> Besides encrypting table and WAL pages, it encrypts the temporary files
> (buffile.c), data stored during logical decoding (reorderbuffer.c) and
> statistics temporary files (pgstat.c). Unlike the previous version, SLRU files
> (e.g. CLOG) are not encrypted (it does not seem critical and the encryption
> makes torn page write quite difficult to handle).
>
> Another difference is that we use the OpenSSL of the (tweaked) AES XTS cipher
> now.
>
> Binary upgrade from unencrypted to encrypted cluster is not implemented yet.
>
>
> [1] https://www.postgresql.org/message-id/031401d3f41d$5c70ed90$1552c8b0$@lab.ntt.co.jp
>
> --
> Antonin Houska
> Cybertec Schonig & Schonig GmbH
> Grohrmuhlgasse 26, A-2700 Wiener Neustadt
> Web: https://www.cybertec-postgresql.com
>