From e08b49b39a572c7816a430a576e1a145a965d60a Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 10 Apr 2024 11:52:58 +0200 Subject: [PATCH v20] Automatic client-side column-level encryption This feature enables the automatic encryption and decryption of particular columns in the client. The data for those columns then only ever appears in ciphertext on the server, so it is protected from DBAs, sysadmins, cloud operators, etc. as well as accidental leakage to server logs, file-system backups, etc. The canonical use case for this feature is storing credit card numbers encrypted, in accordance with PCI DSS, as well as similar situations involving social security numbers etc. One can't do any computations with encrypted values on the server, but for these use cases, that is not necessary. This feature does support deterministic encryption as an alternative to the default randomized encryption, so in that mode one can do equality lookups, at the cost of some security. This functionality also exists in other database products, and the overall concepts were mostly adopted from there. (Note: This feature has nothing to do with any on-disk encryption feature. Both can exist independently.) You declare a column as encrypted in a CREATE TABLE statement. The column value is encrypted by a symmetric key called the column encryption key (CEK). The CEK is a catalog object. The CEK key material is in turn encrypted by an asymmetric key called the column master key (CMK). The CMK is not stored in the database but somewhere where the client can get to it, for example in a file or in a key management system. When a server sends rows containing encrypted column values to the client, it first sends the required CMK and CEK information (new protocol messages), which the client needs to record. Then, the client can use this information to automatically decrypt the incoming row data and forward it in plaintext to the application. For the CMKs, libpq has a new connection parameter "cmklookup" that specifies via a mini-language where to get the keys. Right now, you can use "file" to read it from a file, or "run" to run some program, which could get it from a KMS. The general idea would be for an application to have one CMK per area of secret stuff, for example, for credit card data. The CMK can be rotated: each CEK can be represented multiple times in the database, encrypted by a different CMK. (The CEK can't be rotated easily, since that would require reading out all the data from a table/column and reencrypting it. We could/should add some custom tooling for that, but it wouldn't be a routine operation.) Several encryption algorithms are provided. The CMK process uses RSAES_OAEP_SHA_1 or _256. The CEK process uses AEAD_AES_*_CBC_HMAC_SHA_* with several strengths. In the server, the encrypted datums are stored in types called pg_encrypted_rnd and pg_encrypted_det (for randomized and deterministic encryption). These are essentially cousins of bytea. For the rest of the database system below the protocol handling, there is nothing special about those. For example, pg_encrypted_rnd has no operators at all, pg_encrypted_det has only an equality operator. pg_attribute has a new column attrealtypid that stores the original type of the data in the column. This is only used for providing it to clients, so that higher-level clients can convert the decrypted value to their appropriate data types in their environments. The protocol extensions are guarded by a new protocol extension option "_pq_.column_encryption". If this is not set, nothing changes, the protocol stays the same, and no encryption or decryption happens. To get automatically encrypted data into the database (as opposed to reading it out), it is required to use protocol-level prepared statements (i.e., extended query). The client must first prepare a statement, then describe the statement to get parameter metadata, which indicates which parameters are to be encrypted and how. libpq's PQexecParams() does this internally. For the asynchronous interfaces, additional libpq functions are added to be able to pass the describe result back into the statement execution function. (Other client APIs that have a "statement handle" concept could do this more elegantly and probably without any API changes.) psql also supports this if the \bind command is used. Another challenge is that the parse analysis must check which underlying column a parameter corresponds to. This is similar to resorigtbl and resorigcol in the opposite direction. Discussion: https://www.postgresql.org/message-id/flat/89157929-c2b6-817b-6025-8e4b2d89d88f%40enterprisedb.com --- doc/src/sgml/acronyms.sgml | 18 + doc/src/sgml/catalogs.sgml | 317 +++++++ doc/src/sgml/charset.sgml | 10 + doc/src/sgml/datatype.sgml | 61 ++ doc/src/sgml/ddl.sgml | 444 +++++++++ doc/src/sgml/func.sgml | 60 ++ doc/src/sgml/glossary.sgml | 26 + doc/src/sgml/libpq.sgml | 322 +++++++ doc/src/sgml/protocol.sgml | 468 ++++++++++ doc/src/sgml/ref/allfiles.sgml | 6 + .../sgml/ref/alter_column_encryption_key.sgml | 197 ++++ doc/src/sgml/ref/alter_column_master_key.sgml | 134 +++ doc/src/sgml/ref/comment.sgml | 2 + doc/src/sgml/ref/copy.sgml | 10 + .../ref/create_column_encryption_key.sgml | 173 ++++ .../sgml/ref/create_column_master_key.sgml | 107 +++ doc/src/sgml/ref/create_table.sgml | 55 +- doc/src/sgml/ref/discard.sgml | 14 +- .../sgml/ref/drop_column_encryption_key.sgml | 112 +++ doc/src/sgml/ref/drop_column_master_key.sgml | 112 +++ doc/src/sgml/ref/grant.sgml | 12 +- doc/src/sgml/ref/pg_dump.sgml | 42 + doc/src/sgml/ref/pg_dumpall.sgml | 27 + doc/src/sgml/ref/psql-ref.sgml | 39 + doc/src/sgml/reference.sgml | 6 + src/backend/access/common/printsimple.c | 8 + src/backend/access/common/printtup.c | 236 ++++- src/backend/access/hash/hashvalidate.c | 2 +- src/backend/bootstrap/bootparse.y | 1 + src/backend/catalog/aclchk.c | 60 ++ src/backend/catalog/dependency.c | 6 + src/backend/catalog/heap.c | 61 +- src/backend/catalog/index.c | 4 + src/backend/catalog/namespace.c | 272 ++++++ src/backend/catalog/objectaddress.c | 287 ++++++ src/backend/catalog/toasting.c | 1 + src/backend/commands/Makefile | 1 + src/backend/commands/alter.c | 14 + src/backend/commands/cluster.c | 1 + src/backend/commands/colenccmds.c | 449 ++++++++++ src/backend/commands/createas.c | 30 + src/backend/commands/discard.c | 8 +- src/backend/commands/dropcmds.c | 15 + src/backend/commands/event_trigger.c | 6 + src/backend/commands/meson.build | 1 + src/backend/commands/seclabel.c | 3 + src/backend/commands/tablecmds.c | 239 ++++- src/backend/commands/variable.c | 7 +- src/backend/commands/view.c | 22 +- src/backend/nodes/nodeFuncs.c | 2 + src/backend/parser/gram.y | 194 +++- src/backend/parser/parse_param.c | 157 ++++ src/backend/parser/parse_relation.c | 2 +- src/backend/parser/parse_utilcmd.c | 14 + src/backend/tcop/backend_startup.c | 19 +- src/backend/tcop/postgres.c | 64 ++ src/backend/tcop/utility.c | 56 ++ src/backend/utils/adt/acl.c | 398 +++++++++ src/backend/utils/adt/varlena.c | 107 +++ src/backend/utils/cache/lsyscache.c | 83 ++ src/backend/utils/cache/plancache.c | 4 +- src/backend/utils/mb/mbutils.c | 18 +- src/bin/pg_dump/common.c | 44 + src/bin/pg_dump/dumputils.c | 4 + src/bin/pg_dump/pg_backup.h | 1 + src/bin/pg_dump/pg_backup_archiver.c | 2 + src/bin/pg_dump/pg_backup_db.c | 9 +- src/bin/pg_dump/pg_dump.c | 416 ++++++++- src/bin/pg_dump/pg_dump.h | 35 + src/bin/pg_dump/pg_dump_sort.c | 14 + src/bin/pg_dump/pg_dumpall.c | 5 + src/bin/pg_dump/t/002_pg_dump.pl | 109 +++ src/bin/psql/command.c | 6 +- src/bin/psql/describe.c | 191 +++- src/bin/psql/describe.h | 6 + src/bin/psql/help.c | 4 + src/bin/psql/settings.h | 1 + src/bin/psql/startup.c | 10 + src/bin/psql/tab-complete.c | 72 +- src/common/Makefile | 1 + src/common/colenc.c | 107 +++ src/common/meson.build | 1 + src/include/access/printtup.h | 4 + src/include/catalog/Makefile | 5 +- src/include/catalog/heap.h | 4 +- src/include/catalog/meson.build | 3 + src/include/catalog/namespace.h | 6 + src/include/catalog/pg_amop.dat | 5 + src/include/catalog/pg_amproc.dat | 6 + src/include/catalog/pg_attribute.h | 14 +- src/include/catalog/pg_colenckey.h | 49 + src/include/catalog/pg_colenckeydata.h | 49 + src/include/catalog/pg_colmasterkey.h | 50 ++ src/include/catalog/pg_opclass.dat | 2 + src/include/catalog/pg_operator.dat | 15 + src/include/catalog/pg_opfamily.dat | 2 + src/include/catalog/pg_proc.dat | 100 +++ src/include/catalog/pg_type.dat | 13 + src/include/catalog/pg_type.h | 1 + src/include/commands/colenccmds.h | 26 + src/include/commands/tablecmds.h | 4 +- src/include/common/colenc.h | 51 ++ src/include/libpq/libpq-be.h | 1 + src/include/libpq/protocol.h | 2 + src/include/nodes/parsenodes.h | 41 +- src/include/parser/kwlist.h | 2 + src/include/parser/parse_param.h | 1 + src/include/tcop/cmdtaglist.h | 7 + src/include/utils/acl.h | 2 + src/include/utils/lsyscache.h | 4 + src/include/utils/plancache.h | 3 + src/interfaces/libpq/Makefile | 1 + src/interfaces/libpq/exports.txt | 4 + src/interfaces/libpq/fe-connect.c | 46 + src/interfaces/libpq/fe-encrypt-openssl.c | 840 ++++++++++++++++++ src/interfaces/libpq/fe-encrypt.h | 33 + src/interfaces/libpq/fe-exec.c | 674 +++++++++++++- src/interfaces/libpq/fe-protocol3.c | 157 +++- src/interfaces/libpq/fe-trace.c | 56 +- src/interfaces/libpq/libpq-fe.h | 20 + src/interfaces/libpq/libpq-int.h | 37 + src/interfaces/libpq/meson.build | 2 + src/interfaces/libpq/nls.mk | 1 + src/interfaces/libpq/t/010_encrypt.pl | 72 ++ src/interfaces/libpq/test/.gitignore | 1 + src/interfaces/libpq/test/Makefile | 7 + src/interfaces/libpq/test/meson.build | 23 + src/test/Makefile | 4 +- src/test/column_encryption/.gitignore | 3 + src/test/column_encryption/Makefile | 31 + src/test/column_encryption/meson.build | 23 + .../t/001_column_encryption.pl | 307 +++++++ .../column_encryption/t/002_cmk_rotation.pl | 125 +++ src/test/column_encryption/test_client.c | 161 ++++ .../column_encryption/test_run_decrypt.pl | 61 ++ src/test/meson.build | 1 + .../regress/expected/column_encryption.out | 541 +++++++++++ src/test/regress/expected/object_address.out | 35 + src/test/regress/expected/oidjoins.out | 8 + src/test/regress/expected/opr_sanity.out | 12 +- src/test/regress/expected/type_sanity.out | 6 +- src/test/regress/parallel_schedule | 2 +- src/test/regress/pg_regress_main.c | 2 +- src/test/regress/sql/column_encryption.sql | 371 ++++++++ src/test/regress/sql/object_address.sql | 11 + src/test/regress/sql/type_sanity.sql | 2 + src/tools/pgindent/typedefs.list | 9 + 147 files changed, 10710 insertions(+), 115 deletions(-) create mode 100644 doc/src/sgml/ref/alter_column_encryption_key.sgml create mode 100644 doc/src/sgml/ref/alter_column_master_key.sgml create mode 100644 doc/src/sgml/ref/create_column_encryption_key.sgml create mode 100644 doc/src/sgml/ref/create_column_master_key.sgml create mode 100644 doc/src/sgml/ref/drop_column_encryption_key.sgml create mode 100644 doc/src/sgml/ref/drop_column_master_key.sgml create mode 100644 src/backend/commands/colenccmds.c create mode 100644 src/common/colenc.c create mode 100644 src/include/catalog/pg_colenckey.h create mode 100644 src/include/catalog/pg_colenckeydata.h create mode 100644 src/include/catalog/pg_colmasterkey.h create mode 100644 src/include/commands/colenccmds.h create mode 100644 src/include/common/colenc.h create mode 100644 src/interfaces/libpq/fe-encrypt-openssl.c create mode 100644 src/interfaces/libpq/fe-encrypt.h create mode 100644 src/interfaces/libpq/t/010_encrypt.pl create mode 100644 src/test/column_encryption/.gitignore create mode 100644 src/test/column_encryption/Makefile create mode 100644 src/test/column_encryption/meson.build create mode 100644 src/test/column_encryption/t/001_column_encryption.pl create mode 100644 src/test/column_encryption/t/002_cmk_rotation.pl create mode 100644 src/test/column_encryption/test_client.c create mode 100755 src/test/column_encryption/test_run_decrypt.pl create mode 100644 src/test/regress/expected/column_encryption.out create mode 100644 src/test/regress/sql/column_encryption.sql diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml index 817d062c7e6..ecdd0cc602a 100644 --- a/doc/src/sgml/acronyms.sgml +++ b/doc/src/sgml/acronyms.sgml @@ -65,6 +65,15 @@ Acronyms + + CEK + + + Column Encryption Key; see + + + + CIDR @@ -76,6 +85,15 @@ Acronyms + + CMK + + + Column Master Key; see + + + + CPAN diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 096ddab481c..f03fc63f764 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -105,6 +105,21 @@ System Catalogs collations (locale information) + + pg_colenckey + column encryption keys + + + + pg_colenckeydata + column encryption key data + + + + pg_colmasterkey + column master keys + + pg_constraint check constraints, unique constraints, primary key constraints, foreign key constraints @@ -1410,6 +1425,44 @@ <structname>pg_attribute</structname> Columns + + + attcek oid + (references pg_colenckey.oid) + + + If the column is encrypted, a reference to the column encryption key, else null. + + + + + + attusertypid oid + (references pg_type.oid) + + + If the column is encrypted, then this column indicates the type of the + encrypted data that is reported to the client. If the column is not + encrypted, then null. For encrypted columns, the field + atttypid is either + pg_encrypted_det or pg_encrypted_rnd. + + + + + + attusertypmod int4 + + + If the column is encrypted, then this column indicates the type + modifier (analogous to atttypmod) that is + reported to the client. If the column is not encrypted, then null. For + encrypted columns, the field atttypmod) + contains the identifier of the encryption algorithm; see for possible values. + + + attmissingval anyarray @@ -2494,6 +2547,270 @@ <structname>pg_collation</structname> Columns + + <structname>pg_colenckey</structname> + + + pg_colenckey + + + + The catalog pg_colenckey contains information + about the column encryption keys in the database. The actual key material + of the column encryption keys is in the catalog pg_colenckeydata. + + + + <structname>pg_colenckey</structname> Columns + + + + + Column Type + + + Description + + + + + + + + oid oid + + + Row identifier + + + + + + cekname name + + + Column encryption key name + + + + + + ceknamespace oid + (references pg_namespace.oid) + + + The OID of the namespace that contains this column encryption key + + + + + + cekowner oid + (references pg_authid.oid) + + + Owner of the column encryption key + + + + + + cekacl aclitem[] + + + Access privileges; see for details + + + + +
+ + + + <structname>pg_colenckeydata</structname> + + + pg_colenckeydata + + + + The catalog pg_colenckeydata contains the key + material of column encryption keys. Each column encryption key object can + contain several versions of the key material, each encrypted with a + different column master key. That allows the gradual rotation of the + column master keys. Thus, (ckdcekid, ckdcmkid) is a + unique key of this table. + + + + The key material of column encryption keys should never be decrypted inside + the database instance. It is meant to be sent as-is to the client, where + it is decrypted using the associated column master key, and then used to + encrypt or decrypt column values. + + + + <structname>pg_colenckeydata</structname> Columns + + + + + Column Type + + + Description + + + + + + + + oid oid + + + Row identifier + + + + + + ckdcekid oid + (references pg_colenckey.oid) + + + The column encryption key this entry belongs to + + + + + + ckdcmkid oid + (references pg_colmasterkey.oid) + + + The column master key that the key material is encrypted with + + + + + + ckdcmkalg int4 + + + The encryption algorithm used for encrypting the key material; see + for possible values. + + + + + + ckdencval bytea + + + The key material of this column encryption key, encrypted using the + referenced column master key + + + + +
+ + + + <structname>pg_colmasterkey</structname> + + + pg_colmasterkey + + + + The catalog pg_colmasterkey contains information + about column master keys. The keys themselves are not stored in the + database. The catalog entry only contains information that is used by + clients to locate the keys, for example in a file or in a key management + system. + + + + <structname>pg_colmasterkey</structname> Columns + + + + + Column Type + + + Description + + + + + + + + oid oid + + + Row identifier + + + + + + cmkname name + + + Column master key name + + + + + + cmknamespace oid + (references pg_namespace.oid) + + + The OID of the namespace that contains this column master key + + + + + + cmkowner oid + (references pg_authid.oid) + + + Owner of the column master key + + + + + + cmkrealm text + + + A realm associated with this column master key. This is + a freely chosen string that is used by clients to determine how to look + up the key. A typical configuration would put all CMKs that are looked + up in the same way into the same realm. + + + + + + cmkacl aclitem[] + + + Access privileges; see for details + + + + +
+ + <structname>pg_constraint</structname> diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml index 55bbb20dacc..021b2fffc7b 100644 --- a/doc/src/sgml/charset.sgml +++ b/doc/src/sgml/charset.sgml @@ -2397,6 +2397,16 @@ Automatic Character Set Conversion Between Server and Client Just as for the server, use of SQL_ASCII is unwise unless you are working with all-ASCII data. + + + When automatic client-side column-level encryption is used, then no + encoding conversion is possible. (The encoding conversion happens on the + server, and the server cannot look inside any encrypted column values.) + If automatic client-side column-level encryption is enabled for a + session, then the server enforces that the client encoding matches the + server encoding, and any attempts to change the client encoding will be + rejected by the server. + diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml index 73e51b0b114..49203a86806 100644 --- a/doc/src/sgml/datatype.sgml +++ b/doc/src/sgml/datatype.sgml @@ -5390,4 +5390,65 @@ Pseudo-Types + + Types Related to Encryption + + + An encrypted column value (see ) is + internally stored using the types + pg_encrypted_rnd (for randomized encryption) or + pg_encrypted_det (for deterministic encryption); see . Most of the database system treats + these as normal types. For example, the type pg_encrypted_det has + an equals operator that allows lookup of encrypted values. It is, + however, not allowed to create a table using one of these types directly + as a column type. + + + + The external representation of these types is the string + encrypted$ followed by hexadecimal byte values, for + example + encrypted$3aacd063d2d3a1a04119df76874e0b9785ea466177f18fe9c0a1a313eaf09c98. + Clients that don't support automatic client-side column-level encryption + or have disabled it will see the encrypted values in this format. Clients + that support automatic client-side column-level encryption will not see + these types in result sets, as the protocol layer will translate them back + to the declared underlying type in the table definition. + + + + Types Related to Encryption + + + + + + + Name + Storage Size + Description + + + + + pg_encrypted_det + 1 or 4 bytes plus the actual binary string + encrypted column value, deterministic encryption + + + pg_encrypted_rnd + 1 or 4 bytes plus the actual binary string + encrypted column value, randomized encryption + + + +
+ + + For encrypted columns, the type modifier (atttypmod) + contains the identifier of the encryption algorithm; see for possible values. + + diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml index 00f44f56faf..2182c6c7865 100644 --- a/doc/src/sgml/ddl.sgml +++ b/doc/src/sgml/ddl.sgml @@ -1375,6 +1375,440 @@ Exclusion Constraints + + Automatic Client-side Column-level Encryption + + + With automatic client-side column-level encryption, + columns can be stored encrypted in the database. The encryption and + decryption happens automatically on the client, so that the plaintext value + is never seen in the database instance or on the server hosting the + database. The drawback is that most operations, such as function calls or + sorting, are not possible on encrypted values. + + + + Using Automatic Client-side Column-level Encryption + + + Automatic client-side column-level encryption uses two levels of + cryptographic keys. The actual column value is encrypted using a symmetric + algorithm, such as AES, using a column encryption + key (CEK). The column encryption key is in + turn encrypted using an asymmetric algorithm, such as RSA, using a + column master key (CMK). The + encrypted CEK is stored in the database system. The CMK is not stored in + the database system; it is stored on the client or somewhere where the + client can access it, such as in a local file or in a key management + system. The database system only records where the CMK is stored and + provides this information to the client. When rows containing encrypted + columns are sent to the client, the server first sends any necessary CMK + information, followed by any required CEK. The client then looks up the + CMK and uses that to decrypt the CEK. Then it decrypts incoming row data + using the CEK and provides the decrypted row data to the application. + + + + Here is an example declaring a column as encrypted: + +CREATE TABLE customers ( + id int PRIMARY KEY, + name text NOT NULL, + ... + creditcard_num text ENCRYPTED WITH (column_encryption_key = cek1) +); + + + + + Column encryption supports randomized + (also known as probabilistic) and + deterministic encryption. The above example uses + randomized encryption, which is the default. Randomized encryption uses a + random initialization vector for each encryption, so that even if the + plaintext of two rows is equal, the encrypted values will be different. + This prevents someone with direct access to the database server from making + computations such as distinct counts on the encrypted values. + Deterministic encryption uses a fixed initialization vector. This reduces + security, but it allows equality searches on encrypted values. The + following example declares a column with deterministic encryption: + +CREATE TABLE employees ( + id int PRIMARY KEY, + name text NOT NULL, + ... + ssn text ENCRYPTED WITH ( + column_encryption_key = cek1, encryption_type = deterministic) +); + + + + + Null values are not encrypted by automatic client-side column-level + encryption; null values sent by the client are visible as null values in + the database. If the fact that a value is null needs to be hidden from the + server, this information needs to be encoded into a nonnull value in the + client somehow. + + + + + Reading and Writing Encrypted Columns + + + Reading and writing encrypted columns is meant to be handled automatically + by the client library/driver and should be mostly transparent to the + application code, if certain prerequisites are fulfilled: + + + + + The client library needs to support automatic client-side column-level + encryption. Not all client libraries do. Furthermore, the client + library might require that automatic client-side column-level + encryption is explicitly enabled at connection time. See the + documentation of the client library for details. + + + + + + Column master keys and column encryption keys have been set up, and the + client library has been configured to be able to look up column master + keys from the key store or key management system. + + + + + + + Reading from encrypted columns will then work automatically. For example, + using the above example, + +SELECT ssn FROM employees WHERE id = 5; + + would return the unencrypted value for the ssn column + in any rows found. + + + + Writing to encrypted columns requires that the extended query protocol + (protocol-level prepared statements) be used, so that the values to be + encrypted are supplied separately from the SQL command. For example, + using, say, psql or libpq, the following would not work: + +-- WRONG! +INSERT INTO employees (id, name, ssn) VALUES (1, 'Someone', '12345'); + + This would leak the unencrypted value 12345 to the + server, thus defeating the point of client-side column-level encryption. + (And even ignoring that, it could not work because the server does not + have access to the keys to perform the encryption.) Note that using + server-side prepared statements using the SQL commands + PREPARE and EXECUTE is equally + incorrect, since that would also leak the parameters provided to + EXECUTE to the server. + + + + This shows a correct invocation in libpq (without error checking): + +PGresult *res; +const char *values[] = {"1", "Someone", "12345"}; + +res = PQexecParams(conn, "INSERT INTO employees (id, name, ssn) VALUES ($1, $2, $3)", + 3, NULL, values, NULL, NULL, 0); + + Higher-level client libraries might use the protocol-level prepared + statements automatically and thus won't require any code changes. + + + + psql provides the command + \bind to run statements with parameters like this: + +INSERT INTO employees (id, name, ssn) VALUES ($1, $2, $3) \bind '1' 'Someone', '12345' \g + + + + + Similarly, if deterministic encryption is used, parameters need to be used + in search conditions using encrypted columns: + +SELECT * FROM employees WHERE ssn = $1 \bind '12345' \g + + + + + + Setting up Automatic Client-side Column-level Encryption + + + The steps to set up automatic client-side column-level encryption for a + database are: + + + + + Create the key material for the CMK, for example, using a cryptographic + library or toolkit, or a key management system. Secure access to the + key as appropriate, using access control, passwords, etc. + + + + + + Register the CMK in the database using the SQL command . + + + + + + Create the (unencrypted) key material for the CEK in a temporary + location. (It will be encrypted in the next step. Depending on the + available tools, it might be possible and sensible to combine these two + steps.) + + + + + + Encrypt the created CEK key material using the CMK (created earlier). + (The unencrypted version of the CEK key material can now be disposed + of.) + + + + + + Register the CEK in the database using the SQL command . This command + uploads the encrypted CEK key material created in the + previous step to the database server. The local copy of the CEK key + material can then be removed. + + + + + + Create encrypted columns using the created CEK. + + + + + + Configure the client library/driver to be able to look up the CMK + created earlier. + + + + + Once this is done, values can be written to and read from the encrypted + columns in a transparent way. + + + + Note that these steps should not be run on the database server, but on some + client machine. Neither the CMK nor the unencrypted CEK should ever appear + on the database server host. + + + + The specific details of this setup depend on the desired CMK storage + mechanism/key management system as well as the client libraries to be used. + The following example uses the openssl command-line tool + to set up the keys. + + + + + Create the key material for the CMK and write it to a file: + +openssl genpkey -algorithm rsa -out cmk1.pem + + + + + + + Register the CMK in the database: + +psql ... -c "CREATE COLUMN MASTER KEY cmk1" + + + + + + + Create the unencrypted CEK key material in a file: + +openssl rand -out cek1.bin 48 + + (See for required key lengths.) + + + + + + Encrypt the created CEK key material: + +openssl pkeyutl -encrypt -inkey cmk1.pem -pkeyopt rsa_padding_mode:oaep -in cek1.bin -out cek1.bin.enc +rm cek1.bin + + + + + + + Register the CEK in the database: + +# convert file contents to hex encoding; this is just one possible way +cekenchex=$(perl -0ne 'print unpack "H*"' cek1.bin.enc) +psql ... -c "CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES (column_master_key = cmk1, algorithm = 'RSAES_OAEP_SHA_1', encrypted_value = '\\x${cekenchex}')" +rm cek1.bin.enc + + + + + + + Create encrypted columns as shown in the examples above. + + + + + + Configure the libpq for CMK lookup (see also ): + +PGCMKLOOKUP="*=file:$PWD/%k.pem" +export PGCMKLOOKUP + + + + + Additionally, libpq requires that the connection parameter be set in order to activate + the automatic client-side column-level encryption functionality. This + should be done in the connection parameters of the application, but an + environment variable (PGCOLUMNENCRYPTION) is also + available. + + + + + + + + Guidance on Using Automatic Client-side Column-level Encryption + + + This section contains some information on when it is or is not appropriate + to use automatic client-side column-level encryption, and what precautions + need to be taken to maintain its security. + + + + In general, column encryption is never a replacement for additional + security and encryption techniques such as transport encryption + (SSL/TLS), storage encryption, strong access control, and password + security. Column encryption only targets specific use cases and should be + used in conjunction with additional security measures. + + + + A typical use case for column encryption is to encrypt specific values + with additional security requirements, for example credit card numbers. + This allows you to store that security-sensitive data together with the + rest of your data (thus getting various benefits, such as referential + integrity, consistent backups), while giving access to that data only to + specific clients and preventing accidental leakage on the server side + (server logs, file system backups, etc.). + + + + When using parameters to provide values to insert or search by, care must + be taken that values meant to be encrypted are not accidentally leaked to + the server. The server will tell the client which parameters to encrypt, + based on the schema definition on the server. But if the query or client + application is faulty, values meant to be encrypted might accidentally be + associated with parameters that the server does not think need to be + encrypted. Additional robustness can be achieved by forcing encryption of + certain parameters in the client library (see its documentation; for + libpq, see ). + + + + Column encryption cannot hide the existence or absence of data, it can + only disguise the particular data that is known to exist. For example, + storing a cleartext person name and an encrypted credit card number + indicates that the person has a credit card. That might not reveal too + much if the database is for an online store and there is other data nearby + that shows that the person has recently made purchases. But in another + example, storing a cleartext person name and an encrypted diagnosis in a + medical database probably indicates that the person has a medical issue. + Depending on the circumstances, that might not by itself be sufficient + security. + + + + Encryption cannot completely hide the length of values. The encryption + methods will pad values to multiples of the underlying cipher's block size + (usually 16 bytes), so some length differences will be unified this way. + There is no concern if all values are of the same length, but if there are + signficant length differences between valid values and that length + information is security-sensitive, then application-specific workarounds + such as padding would need to be applied. How to do that securely is + beyond the scope of this manual. Note that column encryption is applied + to the text representation of the stored value, so length differences can + be leaked even for fixed-length column types (e.g. bigint, + whose largest decimal representation is longer than 16 bytes). + + + + Column encryption provides only partial protection against a malicious + user with write access to the table. Once encrypted, any modifications to + a stored value on the server side will cause a decryption failure on the + client. However, a user with write access can still freely swap encrypted + values between rows or columns (or even separate database clusters) as + long as they were encrypted with the same key. Attackers can also remove + values by replacing them with nulls, and users with ownership over the + table schema can replace encryption keys or strip encryption from the + columns entirely. All of this is to say: Proper access control is still + of vital importance when using this feature. + + + + + One might be inclined to think of the client-side column-level encryption + feature as a mechanism for application writers and users to protect + themselves against an evil DBA, but that is not the + intended purpose. Rather, it is (also) a tool for the DBA to control + which data they do not want (in plaintext) on the server. + + + + + When using asymmetric CMK algorithms to encrypt CEKs, the + public half of the CMK can be used to replace existing + column encryption keys with keys of an attacker's choosing, compromising + confidentiality and authenticity for values encrypted under that CMK. For + this reason, it's important to keep both the private + and public halves of the CMK key pair confidential. + + + + + Storing data such credit card data, medical data, and so on is usually + subject to government or industry regulations. This section is not meant + to provide complete instructions on how to do this correctly. Please + seek additional advice when engaging in such projects. + + + + + System Columns @@ -2136,6 +2570,14 @@ Privileges server. Grantees may also create, alter, or drop their own user mappings associated with that server. + + For column master keys, allows the creation of column encryption keys + using the master key. + + + For column encryption keys, allows the use of the key in the creation + of table columns. + @@ -2302,6 +2744,8 @@ ACL Privilege Abbreviations USAGE U + COLUMN ENCRYPTION KEY, + COLUMN MASTER KEY, DOMAIN, FOREIGN DATA WRAPPER, FOREIGN SERVER, diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index bf13216e477..77e7ee0bbd0 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -24896,6 +24896,40 @@ Access Privilege Inquiry Functions + + + + has_column_encryption_key_privilege + + has_column_encryption_key_privilege ( + user name or oid, + cek text or oid, + privilege text ) + boolean + + + Does user have privilege for column encryption key? + The only allowable privilege type is USAGE. + + + + + + + has_column_master_key_privilege + + has_column_master_key_privilege ( + user name or oid, + cmk text or oid, + privilege text ) + boolean + + + Does user have privilege for column master key? + The only allowable privilege type is USAGE. + + + @@ -25386,6 +25420,32 @@ Schema Visibility Inquiry Functions + + + + pg_cek_is_visible + + pg_cek_is_visible ( cek oid ) + boolean + + + Is column encryption key visible in search path? + + + + + + + pg_cmk_is_visible + + pg_cmk_is_visible ( cmk oid ) + boolean + + + Is column master key visible in search path? + + + diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml index a81c17a8690..89e67b6b5c5 100644 --- a/doc/src/sgml/glossary.sgml +++ b/doc/src/sgml/glossary.sgml @@ -433,6 +433,32 @@ Glossary + + Column encryption key + + + A cryptographic key used to encrypt column values when using automatic + client-side column-level encryption (). Column encryption keys are stored in + the database encrypted by another key, the column master key. + + + + + + Column master key + + + A cryptographic key used to encrypt column encryption + keys. (So the column master key is a key + encryption key.) Column master keys are stored outside the + database system, for example in a key management system. + + + + Commit diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 35fb346ed97..9f8f2c3c13b 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -2262,6 +2262,141 @@ Parameter Key Words + + column_encryption + + + If set to on, true, or + 1, this enables automatic client-side column-level + encryption for the connection. If encrypted columns are queried and + this is not enabled, the encrypted values are returned. See for more information about this + feature. + + + + + + cmklookup + + + This specifies how libpq should look up column master keys (CMKs) in + order to decrypt the column encryption keys (CEKs). + The value is a list of key=value entries separated + by semicolons. Each key is the name of a key realm, or + * to match all realms. The value is a + scheme:data specification. The scheme specifies + the method to look up the key, the remaining data is specific to the + scheme. Placeholders are replaced in the remaining data as follows: + + + + %a + + + The CMK algorithm name (see ) + + + + + + %j + + + The CMK algorithm name in JSON Web Algorithms format (see ). This is useful for interfacing + with some key management systems that use these names. + + + + + + %k + + + The CMK key name + + + + + + %p + + + The name of a temporary file with the encrypted CEK data (only for + the run scheme) + + + + + + %r + + + The realm name + + + + + + + + Available schemes are: + + + file + + + Load the key material from a file. The remaining data is the file + name. Use this if the CMKs are kept in a file on the file system. + + + + The file scheme does not support the CMK algorithm + unspecified. + + + + + + run + + + Run the specified command to decrypt the CEK. The remaining data + is a shell command. Use this with key management systems that + perform the decryption themselves. The command must print the + decrypted plaintext on the standard output. + + + + + + + + The default value is empty. + + + + Example: + +cmklookup="r1=file:/some/where/secrets/%k.pem;*=file:/else/where/%r/%k.pem" + + This specification says, for keys in realm r1, load + them from the specified file, replacing %k by the + key name. For keys in other realms, load them from the file, + replacing realm and key names as specified. + + + + An example for interacting with a (hypothetical) key management + system: + +cmklookup="*=run:acmekms decrypt --key %k --alg %a --infile '%p'" + + + + + load_balance_hosts @@ -3252,6 +3387,32 @@ Main Functions src/backend/utils/adt/numeric.c::numeric_send() and src/backend/utils/adt/numeric.c::numeric_recv(). + + + When column encryption is enabled, the second-least-significant + half-byte of this parameter specifies whether encryption should be + forced for a parameter. Set this half-byte to one to force + encryption. For example, use the C code literal + 0x10 to specify text format with forced + encryption. If the array pointer is null then encryption is not + forced for any parameter. + + + + Parameters corresponding to encrypted columns must be passed in + text format. Specifying binary format for such a parameter will + result in an error. + + + + If encryption is forced for a parameter but the parameter does not + correspond to an encrypted column on the server, then the call + will fail and the parameter will not be sent. This can be used + for additional security against a compromised server. (The + drawback is that application code then needs to be kept up to date + with knowledge about which columns are encrypted rather than + letting the server specify this.) + @@ -3264,6 +3425,13 @@ Main Functions to obtain different result columns in different formats, although that is possible in the underlying protocol.) + + + If column encryption is used, then encrypted columns will be + returned in text format independent of this setting. Applications + can check the format of each result column with before accessing it. + @@ -3413,6 +3581,44 @@ Main Functions + + PQexecPreparedDescribedPQexecPreparedDescribed + + + + Sends a request to execute a prepared statement with given + parameters, and waits for the result, with support for encrypted columns. + +PGresult *PQexecPreparedDescribed(PGconn *conn, + const char *stmtName, + int nParams, + const char * const *paramValues, + const int *paramLengths, + const int *paramFormats, + int resultFormat, + PGresult *paramDesc); + + + + + is like with additional support for encrypted + columns. The parameter paramDesc must be a + result set obtained from on + the same prepared statement. + + + + This function must be used if a statement parameter corresponds to an + underlying encrypted column. In that situation, the prepared + statement needs to be described first so that libpq can obtain the + necessary key and other information from the server. When that is + done, the parameters corresponding to encrypted columns are + automatically encrypted appropriately before being sent to the server. + + + + PQdescribePreparedPQdescribePrepared @@ -4341,6 +4547,28 @@ Retrieving Query Result Information + + PQfisencryptedPQfisencrypted + + + + Returns whether the value for the given column came from an encrypted + column. Column numbers start at 0. + +int PQfisencrypted(const PGresult *res, + int column_number); + + + + + Encrypted column values are automatically decrypted, so this function + is not necessary to access the column value. It can be used for extra + security to check whether the value was stored encrypted when one + thought it should be. + + + + PQfsizePQfsize @@ -4522,6 +4750,31 @@ Retrieving Query Result Information + + PQparamisencryptedPQparamisencrypted + + + + Returns whether the value for the given parameter is destined for an + encrypted column. Parameter numbers start at 0. + +int PQparamisencrypted(const PGresult *res, int param_number); + + + + + Values for parameters destined for encrypted columns are automatically + encrypted, so this function is not necessary to prepare the parameter + value. It can be used for extra security to check whether the value + will be stored encrypted when one thought it should be. (But see also + at for another way to do that.) + This function is only useful when inspecting the result of . For other types of results it + will return false. + + + + PQprintPQprint @@ -5047,6 +5300,7 @@ Asynchronous Command Processing , , , + , , , , and @@ -5056,6 +5310,7 @@ Asynchronous Command Processing , , , + , , , and @@ -5114,6 +5369,13 @@ Asynchronous Command Processing , it allows only one command in the query string. + + + If column encryption is enabled, then this function is not + asynchronous. To get asynchronous behavior, followed by should be called individually. + @@ -5168,6 +5430,45 @@ Asynchronous Command Processing + + PQsendQueryPreparedDescribedPQsendQueryPreparedDescribed + + + + Sends a request to execute a prepared statement with given + parameters, without waiting for the result(s), with support for encrypted columns. + +int PQsendQueryPreparedDescribed(PGconn *conn, + const char *stmtName, + int nParams, + const char * const *paramValues, + const int *paramLengths, + const int *paramFormats, + int resultFormat, + PGresult *paramDesc); + + + + + is like with additional support for encrypted + columns. The parameter paramDesc must be a + result set obtained from on + the same prepared statement. + + + + This function must be used if a statement parameter corresponds to an + underlying encrypted column. In that situation, the prepared + statement needs to be described first so that libpq can obtain the + necessary key and other information from the server. When that is + done, the parameters corresponding to encrypted columns are + automatically encrypted appropriately before being sent to the server. + See also under . + + + + PQsendDescribePreparedPQsendDescribePrepared @@ -5258,6 +5559,7 @@ Asynchronous Command Processing , , , + , , , , @@ -8848,6 +9150,26 @@ Environment Variables + + + + PGCMKLOOKUP + + PGCMKLOOKUP behaves the same as the connection parameter. + + + + + + + PGCOLUMNENCRYPTION + + PGCOLUMNENCRYPTION behaves the same as the connection parameter. + + + diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index a8ec72c27f4..e88af6f31f1 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -1091,6 +1091,76 @@ Pipelining + + Automatic Client-side Column-level Encryption + + + Automatic client-side column-level encryption is enabled by sending the + parameter _pq_.column_encryption with a value of + 1 in the StartupMessage. This is a protocol extension + that enables a few additional protocol messages and adds additional fields + to existing protocol messages. Client drivers should only activate this + protocol extension when requested by the user, for example through a + connection parameter. + + + + When automatic client-side column-level encryption is enabled, the + messages ColumnMasterKey and ColumnEncryptionKey can appear before + RowDescription and ParameterDescription messages. Clients should collect + the information in these messages and keep them for the duration of the + connection. A server is not required to resend the key information for + each statement cycle if it was already sent during this connection. If a + server resends a key that the client has already stored (that is, a key + having an ID equal to one already stored), the new information should + replace the old. (This could happen, for example, if the key was altered + by server-side DDL commands.) + + + + A client supporting automatic column-level encryption should automatically + decrypt the column value fields of DataRow messages corresponding to + encrypted columns, and it should automatically encrypt the parameter value + fields of Bind messages corresponding to encrypted columns. + + + + When column encryption is used, format specifications (text/binary) in the + various protocol messages apply to the ciphertext. The plaintext inside + the ciphertext is always in text format, but this is invisible to the + protocol. Even though the ciphertext could in theory be sent in either + text or binary format, the server will always send it in binary if the + column-level encryption protocol option is enabled. That way, a client + library only needs to support decrypting data sent in binary and does not + have to support decoding the text format of the encryption-related types + (see ). + + + + When deterministic encryption is used, clients need to take care to + represent plaintext to be encrypted in a consistent form. For example, + encrypting an integer represented by the string 100 and + an integer represented by the string +100 would result + in two different ciphertexts, thus defeating the main point of + deterministic encryption. This protocol specification requires the + plaintext to be in canonical form, which is the form that + is produced by the server when it outputs a particular value in text + format. + + + + When automatic client-side column-level encryption is enabled, the client + encoding must match the server encoding. This ensures that all values + encrypted or decrypted by the client match the server encoding. + + + + The cryptographic operations used for automatic client-side column-level + encryption are described in . + + + Function Call @@ -3994,6 +4064,16 @@ Message Formats The parameter format codes. Each must presently be zero (text) or one (binary). + + + If the protocol extension _pq_.column_encryption + is enabled (see ), + then the second-least-significant half-byte is set to one if the + parameter was encrypted by the client. (So, for example, to send an + encrypted value in binary, the field is set to 0x11 in total.) This + is used by the server to check that a parameter that was required to + be encrypted was actually encrypted. + @@ -4214,6 +4294,140 @@ Message Formats + + ColumnEncryptionKey (B) + + + This message can only appear if the protocol extension + _pq_.column_encryption is enabled. (See .) + + + + + Byte1('Y') + + + Identifies the message as a column encryption key message. + + + + + + Int32 + + + Length of message contents in bytes, including self. + + + + + + Int32 + + + The session-specific identifier of the key. + + + + + + Int32 + + + The identifier of the master key used to encrypt this key. + + + + + + Int32 + + + The identifier of the algorithm used to encrypt this key. + + + + + + Int32 + + + The length of the following key material. + + + + + + Byten + + + The key material, encrypted with the master key referenced above. + + + + + + + + + ColumnMasterKey (B) + + + This message can only appear if the protocol extension + _pq_.column_encryption is enabled. (See .) + + + + + Byte1('y') + + + Identifies the message as a column master key message. + + + + + + Int32 + + + Length of message contents in bytes, including self. + + + + + + Int32 + + + The session-specific identifier of the key. + + + + + + String + + + The name of the key. + + + + + + String + + + The key's realm. + + + + + + + CommandComplete (B) @@ -5317,6 +5531,45 @@ Message Formats + + + If the protocol extension _pq_.column_encryption is + enabled (see ), then + there is also the following for each parameter: + + + + + Int32 + + + If this parameter is to be encrypted, this specifies the + identifier of the column encryption key to use, else zero. + + + + + + Int32 + + + If this parameter is to be encrypted, this specifies the + identifier of the encryption algorithm, else zero. + + + + + + Int16 + + + This is used as a bit field of flags. If the parameter is to be + encrypted and bit 0x0001 is set, the column underlying the parameter + uses deterministic encryption, otherwise randomized encryption. + + + + @@ -5705,6 +5958,50 @@ Message Formats + + + If the protocol extension _pq_.column_encryption is + enabled (see ), then + there is also the following for each field: + + + + + Int32 + + + If the field is encrypted, this specifies the identifier of the + column encryption key to use, else zero. + + + + + + Int32 + + + If the field is encrypted, this specifies the identifier of the + encryption algorithm, else zero. + + + + + + Int16 + + + This is used as a bit field of flags. If the field is encrypted and + bit 0x0001 is set, the field uses deterministic encryption, otherwise + randomized encryption. + + + + + @@ -7530,6 +7827,177 @@ Logical Replication Message Formats + + Automatic Client-side Column-level Encryption Cryptography + + + This section describes the cryptographic operations used by the automatic + client-side column-level encryption functionality. A client that supports + this functionality needs to implement these operations as specified here in + order to be able to interoperate with other clients. + + + + Column encryption key algorithms and column master key algorithms are + identified by integers in the protocol messages and the system catalogs. + Additional algorithms may be added to this protocol specification without a + change in the protocol version number. Clients should implement support + for all the algorithms specified here. If a client encounters an algorithm + identifier it does not recognize or does not support, it must raise an + error. A suitable error message should be provided to the application or + user. + + + + Column Master Keys + + + The currently defined algorithms for column master keys are listed in + . + + + + + + Column Master Key Algorithms + + + + PostgreSQL ID + Name + JWA (RFC 7518) name + Description + + + + + 1 + unspecified + (none) + interpreted by client + + + 2 + RSAES_OAEP_SHA_1 + RSA-OAEP + RSAES OAEP using default parameters (RFC + 8017/PKCS #1) + + + 3 + RSAES_OAEP_SHA_256 + RSA-OAEP-256 + RSAES OAEP using SHA-256 and MGF1 with SHA-256 (RFC + 8017/PKCS #1) + + + +
+ + + + Column Encryption Keys + + + The currently defined algorithms for column encryption keys are listed in + . + + + + + + The key material of a column encryption key consists of three components, + concatenated in this order: the MAC key, the encryption key, and the IV + key. shows the total length that a + key generated for each algorithm is required to have. The MAC key and the + encryption key are used by the referenced encryption algorithms; see there + for details. The IV key is used for computing the static initialization + vector for deterministic encryption; it is unused for randomized + encryption. + + + + + Column Encryption Key Algorithms + + + + PostgreSQL ID + Name + Description + MAC key length (octets) + Encryption key length (octets) + IV key length (octets) + Total key length (octets) + + + + + 32768 + AEAD_AES_128_CBC_HMAC_SHA_256 + + 16 + 16 + 16 + 48 + + + 32769 + AEAD_AES_192_CBC_HMAC_SHA_384 + + 24 + 24 + 24 + 72 + + + 32770 + AEAD_AES_256_CBC_HMAC_SHA_384 + + 24 + 32 + 24 + 90 + + + 32771 + AEAD_AES_256_CBC_HMAC_SHA_512 + + 32 + 32 + 32 + 96 + + + +
+ + + The associated data in these algorithms consists of 4 + bytes: The ASCII letters P and G + (byte values 80 and 71), followed by the version number as a 16-bit + unsigned integer in network byte order. The version number is currently + always 1. (This is intended to allow for possible incompatible changes or + extensions in the future.) + + + + The length of the initialization vector is 16 octets for all CEK algorithm + variants. For randomized encryption, the initialization vector should be + (cryptographically strong) random bytes. For deterministic encryption, + the initialization vector is constructed as + +SUBSTRING(HMAC(K, P) FOR IVLEN) + + where HMAC is the HMAC function associated with + the algorithm, K is the IV key, and + P is the plaintext to be encrypted. + + + + Summary of Changes since Protocol 2.0 diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml index f5be638867a..bf598832d16 100644 --- a/doc/src/sgml/ref/allfiles.sgml +++ b/doc/src/sgml/ref/allfiles.sgml @@ -8,6 +8,8 @@ + + @@ -62,6 +64,8 @@ + + @@ -109,6 +113,8 @@ + + diff --git a/doc/src/sgml/ref/alter_column_encryption_key.sgml b/doc/src/sgml/ref/alter_column_encryption_key.sgml new file mode 100644 index 00000000000..655e1e00d8d --- /dev/null +++ b/doc/src/sgml/ref/alter_column_encryption_key.sgml @@ -0,0 +1,197 @@ + + + + + ALTER COLUMN ENCRYPTION KEY + + + + ALTER COLUMN ENCRYPTION KEY + 7 + SQL - Language Statements + + + + ALTER COLUMN ENCRYPTION KEY + change the definition of a column encryption key + + + + +ALTER COLUMN ENCRYPTION KEY name ADD VALUE ( + COLUMN_MASTER_KEY = cmk, + [ ALGORITHM = algorithm, ] + ENCRYPTED_VALUE = encval +) + +ALTER COLUMN ENCRYPTION KEY name DROP VALUE ( + COLUMN_MASTER_KEY = cmk +) + +ALTER COLUMN ENCRYPTION KEY name RENAME TO new_name +ALTER COLUMN ENCRYPTION KEY name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER } +ALTER COLUMN ENCRYPTION KEY name SET SCHEMA new_schema + + + + + Description + + + ALTER COLUMN ENCRYPTION KEY changes the definition of a + column encryption key. + + + + The first form adds new encrypted key data to a column encryption key, + which must be encrypted with a different column master key than the + existing key data. The second form removes a key data entry for a given + column master key. Together, these forms can be used for column master key + rotation. + + + + You must own the column encryption key to use ALTER COLUMN + ENCRYPTION KEY. To alter the owner, you must also be a direct or + indirect member of the new owning role, and that role must have + CREATE privilege on the column encryption key's + schema. (These restrictions enforce that altering the owner doesn't do + anything you couldn't do by dropping and recreating the column encryption + key. However, a superuser can alter ownership of any column encryption key + anyway.) + + + + + Parameters + + + + name + + + The name (optionally schema-qualified) of an existing column encryption + key. + + + + + + cmk + + + + The name of the column master key that was used to encrypt this column + encryption key. + + + + + + algorithm + + + + The encryption algorithm that was used to encrypt the key material of + this column encryption key. See for details + + + + + + encval + + + + The key material of this column encryption key, encrypted with the + specified column master key using the specified algorithm. The value + must be a bytea-compatible literal. + + + + + + new_name + + + The new name of the column encryption key. + + + + + + new_owner + + + The new owner of the column encryption key. + + + + + + new_schema + + + The new schema for the column encryption key. + + + + + + + + Examples + + + To rotate the master keys used to encrypt a given column encryption key, + use a command sequence like this: + +ALTER COLUMN ENCRYPTION KEY cek1 ADD VALUE ( + COLUMN_MASTER_KEY = cmk2, + ENCRYPTED_VALUE = '\x01020204...' +); + +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE ( + COLUMN_MASTER_KEY = cmk1 +); + + + + + To rename the column encryption key cek1 to + cek2: + +ALTER COLUMN ENCRYPTION KEY cek1 RENAME TO cek2; + + + + + To change the owner of the column encryption key cek1 to + joe: + +ALTER COLUMN ENCRYPTION KEY cek1 OWNER TO joe; + + + + + Compatibility + + + There is no ALTER COLUMN ENCRYPTION KEY statement in the + SQL standard. + + + + + See Also + + + + + + + diff --git a/doc/src/sgml/ref/alter_column_master_key.sgml b/doc/src/sgml/ref/alter_column_master_key.sgml new file mode 100644 index 00000000000..7f0e656ef06 --- /dev/null +++ b/doc/src/sgml/ref/alter_column_master_key.sgml @@ -0,0 +1,134 @@ + + + + + ALTER COLUMN MASTER KEY + + + + ALTER COLUMN MASTER KEY + 7 + SQL - Language Statements + + + + ALTER COLUMN MASTER KEY + change the definition of a column master key + + + + +ALTER COLUMN MASTER KEY name ( REALM = realm ) + +ALTER COLUMN MASTER KEY name RENAME TO new_name +ALTER COLUMN MASTER KEY name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER } +ALTER COLUMN MASTER KEY name SET SCHEMA new_schema + + + + + Description + + + ALTER COLUMN MASTER KEY changes the definition of a + column master key. + + + + The first form changes the parameters of a column master key. See for details. + + + + You must own the column master key to use ALTER COLUMN MASTER + KEY. To alter the owner, you must also be a direct or indirect + member of the new owning role, and that role must have + CREATE privilege on the column master key's schema. + (These restrictions enforce that altering the owner doesn't do anything you + couldn't do by dropping and recreating the column master key. However, a + superuser can alter ownership of any column master key anyway.) + + + + + Parameters + + + + name + + + The name (optionally schema-qualified) of an existing column master key. + + + + + + new_name + + + The new name of the column master key. + + + + + + new_owner + + + The new owner of the column master key. + + + + + + new_schema + + + The new schema for the column master key. + + + + + + + + Examples + + + To rename the column master key cmk1 to + cmk2: + +ALTER COLUMN MASTER KEY cmk1 RENAME TO cmk2; + + + + + To change the owner of the column master key cmk1 to + joe: + +ALTER COLUMN MASTER KEY cmk1 OWNER TO joe; + + + + + Compatibility + + + There is no ALTER COLUMN MASTER KEY statement in the + SQL standard. + + + + + See Also + + + + + + + diff --git a/doc/src/sgml/ref/comment.sgml b/doc/src/sgml/ref/comment.sgml index 5b43c56b133..1caf9bfa569 100644 --- a/doc/src/sgml/ref/comment.sgml +++ b/doc/src/sgml/ref/comment.sgml @@ -28,6 +28,8 @@ CAST (source_type AS target_type) | COLLATION object_name | COLUMN relation_name.column_name | + COLUMN ENCRYPTION KEY object_name | + COLUMN MASTER KEY object_name | CONSTRAINT constraint_name ON table_name | CONSTRAINT constraint_name ON DOMAIN domain_name | CONVERSION object_name | diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgml index 33ce7c4ea6c..213c8e8ed20 100644 --- a/doc/src/sgml/ref/copy.sgml +++ b/doc/src/sgml/ref/copy.sgml @@ -625,6 +625,16 @@ Notes null strings to null values and unquoted null strings to empty strings. + + COPY does not support automatic client-side + column-level encryption or decryption; its input or output data will + always be the ciphertext. This is usually suitable for backups (see also + ). If automatic client-side encryption or + decryption is wanted, INSERT and + SELECT need to be used instead to write and read the + data. + + diff --git a/doc/src/sgml/ref/create_column_encryption_key.sgml b/doc/src/sgml/ref/create_column_encryption_key.sgml new file mode 100644 index 00000000000..65534fb03f3 --- /dev/null +++ b/doc/src/sgml/ref/create_column_encryption_key.sgml @@ -0,0 +1,173 @@ + + + + + CREATE COLUMN ENCRYPTION KEY + + + + CREATE COLUMN ENCRYPTION KEY + 7 + SQL - Language Statements + + + + CREATE COLUMN ENCRYPTION KEY + define a new column encryption key + + + + +CREATE COLUMN ENCRYPTION KEY name WITH VALUES ( + COLUMN_MASTER_KEY = cmk, + ALGORITHM = algorithm, + ENCRYPTED_VALUE = encval +) +[ , ... ] + + + + + Description + + + CREATE COLUMN ENCRYPTION KEY defines a new column + encryption key. A column encryption key is used for client-side encryption + of table columns that have been defined as encrypted. The key material of + a column encryption key is stored in the database's system catalogs, + encrypted (wrapped) by a column master key (which in turn is only + accessible to the client, not the database server). + + + + A column encryption key can be associated with more than one column master + key. To specify that, specify more than one parenthesized definition (see + also the examples). + + + + + Parameters + + + + name + + + + The name of the new column encryption key. The name can be + schema-qualified. + + + + + + cmk + + + + The name of the column master key that was used to encrypt this column + encryption key. You must have USAGE privilege on the + column master key. + + + + + + algorithm + + + + The encryption algorithm that was used to encrypt the key material of + this column encryption key. Supported algorithms are: + + + unspecified + + + RSAES_OAEP_SHA_1 + + + RSAES_OAEP_SHA_256 + + + + + + This is informational only. The specified value is provided to the + client, which may use it for decrypting the column encryption key on the + client side. But a client is also free to ignore this information and + figure out how to arrange the decryption in some other way. In that + case, specifying the algorithm as unspecified would be + appropriate. + + + + + + encval + + + + The key material of this column encryption key, encrypted with the + specified column master key using the specified algorithm. The value + must be a bytea-compatible literal. + + + + + + + + Examples + + + +CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES ( + COLUMN_MASTER_KEY = cmk1, + ALGORITHM = 'RSAES_OAEP_SHA_1', + ENCRYPTED_VALUE = '\x01020204...' +); + + + + + To specify more than one associated column master key: + +CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES ( + COLUMN_MASTER_KEY = cmk1, + ALGORITHM = 'RSAES_OAEP_SHA_1', + ENCRYPTED_VALUE = '\x01020204...' +), +( + COLUMN_MASTER_KEY = cmk2, + ALGORITHM = 'RSAES_OAEP_SHA_1', + ENCRYPTED_VALUE = '\xF1F2F2F4...' +); + + + + + + Compatibility + + + There is no CREATE COLUMN ENCRYPTION KEY statement in + the SQL standard. + + + + + See Also + + + + + + + + + diff --git a/doc/src/sgml/ref/create_column_master_key.sgml b/doc/src/sgml/ref/create_column_master_key.sgml new file mode 100644 index 00000000000..6aaa1088d19 --- /dev/null +++ b/doc/src/sgml/ref/create_column_master_key.sgml @@ -0,0 +1,107 @@ + + + + + CREATE COLUMN MASTER KEY + + + + CREATE COLUMN MASTER KEY + 7 + SQL - Language Statements + + + + CREATE COLUMN MASTER KEY + define a new column master key + + + + +CREATE COLUMN MASTER KEY name [ WITH ( + [ REALM = realm ] +) ] + + + + + Description + + + CREATE COLUMN MASTER KEY defines a new column master + key. A column master key is used to encrypt column encryption keys, which + are the keys that actually encrypt the column data. The key material of + the column master key is not stored in the database. The definition of a + column master key records information that will allow a client to locate + the key material, for example in a file or in a key management system. + + + + + Parameters + + + + name + + + + The name of the new column master key. The name can be + schema-qualified. + + + + + + realm + + + + This is an optional string that can be used to organize column master + keys into groups for lookup by clients. The intent is that all column + master keys that are stored in the same system (file system location, + key management system, etc.) should be in the same realm. A client + would then be configured to look up all keys in a given realm in a + certain way. See the documentation of the respective client library for + further usage instructions. + + + + The default is the empty string. + + + + + + + + Examples + + +CREATE COLUMN MASTER KEY cmk1 (realm = 'myrealm'); + + + + + Compatibility + + + There is no CREATE COLUMN MASTER KEY statement in + the SQL standard. + + + + + See Also + + + + + + + + + diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index 02f31d2d6fd..e9b301ca2e2 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -22,7 +22,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] table_name ( [ - { column_name data_type [ STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT } ] [ COMPRESSION compression_method ] [ COLLATE collation ] [ column_constraint [ ... ] ] + { column_name data_type [ ENCRYPTED WITH ( encryption_options ) ] [ STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT } ] [ COMPRESSION compression_method ] [ COLLATE collation ] [ column_constraint [ ... ] ] | table_constraint | LIKE source_table [ like_option ... ] } [, ... ] @@ -88,7 +88,7 @@ and like_option is: -{ INCLUDING | EXCLUDING } { COMMENTS | COMPRESSION | CONSTRAINTS | DEFAULTS | GENERATED | IDENTITY | INDEXES | STATISTICS | STORAGE | ALL } +{ INCLUDING | EXCLUDING } { COMMENTS | COMPRESSION | CONSTRAINTS | DEFAULTS | ENCRYPTED | GENERATED | IDENTITY | INDEXES | STATISTICS | STORAGE | ALL } and partition_bound_spec is: @@ -352,6 +352,47 @@ Parameters + + ENCRYPTED WITH ( encryption_options ) + + + Enables automatic client-side column-level encryption for the column. + encryption_options are comma-separated + key=value specifications. The following options are + available: + + + column_encryption_key + + + Specifies the name of the column encryption key to use. Specifying + this is mandatory. You must have USAGE privilege + on the column encryption key. + + + + + encryption_type + + + randomized (the default) or deterministic + + + + + algorithm + + + The encryption algorithm to use. The default is + AEAD_AES_128_CBC_HMAC_SHA_256. + + + + + + + + INHERITS ( parent_table [, ... ] ) @@ -717,6 +758,16 @@ Parameters + + INCLUDING ENCRYPTED + + + Column encryption specifications for the copied column definitions + will be copied. By default, new columns will be unencrypted. + + + + INCLUDING GENERATED diff --git a/doc/src/sgml/ref/discard.sgml b/doc/src/sgml/ref/discard.sgml index bf44c523cac..6a94706ef79 100644 --- a/doc/src/sgml/ref/discard.sgml +++ b/doc/src/sgml/ref/discard.sgml @@ -21,7 +21,7 @@ -DISCARD { ALL | PLANS | SEQUENCES | TEMPORARY | TEMP } +DISCARD { ALL | COLUMN ENCRYPTION KEYS | PLANS | SEQUENCES | TEMPORARY | TEMP } @@ -42,6 +42,17 @@ Parameters + + COLUMN ENCRYPTION KEYS + + + Discards knowledge about which column encryption keys and column master + keys have been sent to the client in this session. (They will + subsequently be re-sent as required.) + + + + PLANS @@ -93,6 +104,7 @@ Parameters DISCARD PLANS; DISCARD TEMP; DISCARD SEQUENCES; +DISCARD COLUMN ENCRYPTION KEYS; diff --git a/doc/src/sgml/ref/drop_column_encryption_key.sgml b/doc/src/sgml/ref/drop_column_encryption_key.sgml new file mode 100644 index 00000000000..f2ac1beb084 --- /dev/null +++ b/doc/src/sgml/ref/drop_column_encryption_key.sgml @@ -0,0 +1,112 @@ + + + + + DROP COLUMN ENCRYPTION KEY + + + + DROP COLUMN ENCRYPTION KEY + 7 + SQL - Language Statements + + + + DROP COLUMN ENCRYPTION KEY + remove a column encryption key + + + + +DROP COLUMN ENCRYPTION KEY [ IF EXISTS ] name [ CASCADE | RESTRICT ] + + + + + Description + + + DROP COLUMN ENCRYPTION KEY removes a previously defined + column encryption key. To be able to drop a column encryption key, you + must be its owner. + + + + + Parameters + + + + IF EXISTS + + + Do not throw an error if the column encryption key does not exist. + A notice is issued in this case. + + + + + + name + + + + The name (optionally schema-qualified) of the column encryption key. + + + + + + CASCADE + + + Automatically drop objects that depend on the column encryption key, + and in turn all objects that depend on those objects + (see ). + + + + + + RESTRICT + + + Refuse to drop the column encryption key if any objects depend on it. This + is the default. + + + + + + + + Examples + + + +DROP COLUMN ENCRYPTION KEY cek1; + + + + + Compatibility + + + There is no DROP COLUMN ENCRYPTION KEY statement in + the SQL standard. + + + + + See Also + + + + + + + + diff --git a/doc/src/sgml/ref/drop_column_master_key.sgml b/doc/src/sgml/ref/drop_column_master_key.sgml new file mode 100644 index 00000000000..fae95e09d19 --- /dev/null +++ b/doc/src/sgml/ref/drop_column_master_key.sgml @@ -0,0 +1,112 @@ + + + + + DROP COLUMN MASTER KEY + + + + DROP COLUMN MASTER KEY + 7 + SQL - Language Statements + + + + DROP COLUMN MASTER KEY + remove a column master key + + + + +DROP COLUMN MASTER KEY [ IF EXISTS ] name [ CASCADE | RESTRICT ] + + + + + Description + + + DROP COLUMN MASTER KEY removes a previously defined + column master key. To be able to drop a column master key, you + must be its owner. + + + + + Parameters + + + + IF EXISTS + + + Do not throw an error if the column master key does not exist. + A notice is issued in this case. + + + + + + name + + + + The name (optionally schema-qualified) of the column master key. + + + + + + CASCADE + + + Automatically drop objects that depend on the column master key, + and in turn all objects that depend on those objects + (see ). + + + + + + RESTRICT + + + Refuse to drop the column master key if any objects depend on it. This + is the default. + + + + + + + + Examples + + + +DROP COLUMN MASTER KEY cek1; + + + + + Compatibility + + + There is no DROP COLUMN MASTER KEY statement in + the SQL standard. + + + + + See Also + + + + + + + + diff --git a/doc/src/sgml/ref/grant.sgml b/doc/src/sgml/ref/grant.sgml index 65b1fe77119..480b5bda676 100644 --- a/doc/src/sgml/ref/grant.sgml +++ b/doc/src/sgml/ref/grant.sgml @@ -46,6 +46,16 @@ TO role_specification [, ...] [ WITH GRANT OPTION ] [ GRANTED BY role_specification ] +GRANT { USAGE | ALL [ PRIVILEGES ] } + ON COLUMN ENCRYPTION KEY cek_name [, ...] + TO role_specification [, ...] [ WITH GRANT OPTION ] + [ GRANTED BY role_specification ] + +GRANT { USAGE | ALL [ PRIVILEGES ] } + ON COLUMN MASTER KEY cmk_name [, ...] + TO role_specification [, ...] [ WITH GRANT OPTION ] + [ GRANTED BY role_specification ] + GRANT { USAGE | ALL [ PRIVILEGES ] } ON DOMAIN domain_name [, ...] TO role_specification [, ...] [ WITH GRANT OPTION ] @@ -518,7 +528,7 @@ Compatibility - Privileges on databases, tablespaces, schemas, languages, and + Privileges on databases, tablespaces, schemas, keys, languages, and configuration parameters are PostgreSQL extensions. diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml index b99793e4148..1b40fc66d47 100644 --- a/doc/src/sgml/ref/pg_dump.sgml +++ b/doc/src/sgml/ref/pg_dump.sgml @@ -744,6 +744,48 @@ Options + + + + + This option causes the values of all encrypted columns to be decrypted + and written to the output in plaintext. By default, the values of + encrypted columns are written to the dump in ciphertext (that is, they + are not decrypted). + + + + This option turns on the column encryption connection option in + libpq (see ). Column master key + lookup must be configured by the user, either through a connection + option or an environment setting (see ). + + + + This option requires that , + or + is also specified. + (COPY does not support column decryption.) + + + + For routine backups, the default behavior is appropriate and most + efficient. This option is suitable if the data is meant to be + inspected or exported for other purposes. (But then it is recommended + to not do this on the same host as the server, to avoid exposing + unencrypted data that is meant to be kept encrypted on the server.) + Note that a dump created with this option cannot be restored into a + database with column encryption. + + + + + diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml index 4d7c0464687..8e848c337a3 100644 --- a/doc/src/sgml/ref/pg_dumpall.sgml +++ b/doc/src/sgml/ref/pg_dumpall.sgml @@ -293,6 +293,33 @@ Options + + + + + This option causes the values of all encrypted columns to be decrypted + and written to the output in plaintext. By default, the values of + encrypted columns are written to the dump in ciphertext (that is, they + are not decrypted). + + + + This option requires that , + or + is also specified. + (COPY does not support column decryption.) + + + + For routine backups, the default behavior is appropriate and most + efficient. This option is suitable if the data is meant to be + inspected or exported for other purposes. Note that a dump created + with this option cannot be restored into a database with column + encryption. + + + + diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index 539748ffc29..e56b19024fb 100644 --- a/doc/src/sgml/ref/psql-ref.sgml +++ b/doc/src/sgml/ref/psql-ref.sgml @@ -1424,6 +1424,34 @@ Meta-Commands + + \dcek[+] [ pattern ] + + + Lists column encryption keys. If pattern is specified, only column + encryption keys whose names match the pattern are listed. If + + is appended to the command name, each object is + listed with its associated description. + + + + + + + \dcmk[+] [ pattern ] + + + Lists column master keys. If pattern is specified, only column + master keys whose names match the pattern are listed. If + + is appended to the command name, each object is + listed with its associated description. + + + + + \dconfig[+] [ pattern ] @@ -4053,6 +4081,17 @@ Variables + + HIDE_COLUMN_ENCRYPTION + + + If this variable is set to true, column encryption + details are not displayed. This is mainly useful for regression + tests. + + + + HIDE_TOAST_COMPRESSION diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml index ff85ace83fc..4e40bc41434 100644 --- a/doc/src/sgml/reference.sgml +++ b/doc/src/sgml/reference.sgml @@ -36,6 +36,8 @@ SQL Commands &abort; &alterAggregate; &alterCollation; + &alterColumnEncryptionKey; + &alterColumnMasterKey; &alterConversion; &alterDatabase; &alterDefaultPrivileges; @@ -90,6 +92,8 @@ SQL Commands &createAggregate; &createCast; &createCollation; + &createColumnEncryptionKey; + &createColumnMasterKey; &createConversion; &createDatabase; &createDomain; @@ -137,6 +141,8 @@ SQL Commands &dropAggregate; &dropCast; &dropCollation; + &dropColumnEncryptionKey; + &dropColumnMasterKey; &dropConversion; &dropDatabase; &dropDomain; diff --git a/src/backend/access/common/printsimple.c b/src/backend/access/common/printsimple.c index 4c5eedcdc17..8a4a18b7ff5 100644 --- a/src/backend/access/common/printsimple.c +++ b/src/backend/access/common/printsimple.c @@ -20,8 +20,10 @@ #include "access/printsimple.h" #include "catalog/pg_type.h" +#include "libpq/libpq-be.h" #include "libpq/pqformat.h" #include "libpq/protocol.h" +#include "miscadmin.h" #include "utils/builtins.h" /* @@ -47,6 +49,12 @@ printsimple_startup(DestReceiver *self, int operation, TupleDesc tupdesc) pq_sendint16(&buf, attr->attlen); pq_sendint32(&buf, attr->atttypmod); pq_sendint16(&buf, 0); /* format code */ + if (MyProcPort->column_encryption_enabled) + { + pq_sendint32(&buf, 0); /* CEK */ + pq_sendint32(&buf, 0); /* CEK alg */ + pq_sendint16(&buf, 0); /* flags */ + } } pq_endmessage(&buf); diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index f2d5ca14fee..f9b788e64a1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -15,12 +15,27 @@ */ #include "postgres.h" +#include "access/genam.h" #include "access/printtup.h" +#include "access/skey.h" +#include "access/table.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colenckeydata.h" +#include "catalog/pg_colmasterkey.h" +#include "libpq/libpq-be.h" #include "libpq/pqformat.h" +#include "miscadmin.h" #include "tcop/pquery.h" +#include "utils/array.h" +#include "utils/arrayaccess.h" +#include "utils/builtins.h" +#include "utils/fmgroids.h" +#include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memdebug.h" #include "utils/memutils.h" +#include "utils/rel.h" +#include "utils/syscache.h" static void printtup_startup(DestReceiver *self, int operation, @@ -150,6 +165,164 @@ printtup_startup(DestReceiver *self, int operation, TupleDesc typeinfo) */ } +/* + * Send ColumnMasterKey message, unless it's already been sent in this session + * for this key. + */ +List *cmk_sent = NIL; + +static void +cmk_change_cb(Datum arg, int cacheid, uint32 hashvalue) +{ + list_free(cmk_sent); + cmk_sent = NIL; +} + +static void +MaybeSendColumnMasterKeyMessage(Oid cmkid) +{ + HeapTuple tuple; + Form_pg_colmasterkey cmkform; + Datum datum; + StringInfoData buf; + static bool registered_inval = false; + MemoryContext oldcontext; + + Assert(MyProcPort->column_encryption_enabled); + + if (list_member_oid(cmk_sent, cmkid)) + return; + + tuple = SearchSysCache1(CMKOID, ObjectIdGetDatum(cmkid)); + if (!HeapTupleIsValid(tuple)) + elog(ERROR, "cache lookup failed for column master key %u", cmkid); + cmkform = (Form_pg_colmasterkey) GETSTRUCT(tuple); + + pq_beginmessage(&buf, 'y'); /* ColumnMasterKey */ + pq_sendint32(&buf, cmkform->oid); + pq_sendstring(&buf, NameStr(cmkform->cmkname)); + datum = SysCacheGetAttrNotNull(CMKOID, tuple, Anum_pg_colmasterkey_cmkrealm); + pq_sendstring(&buf, TextDatumGetCString(datum)); + pq_endmessage(&buf); + + ReleaseSysCache(tuple); + + if (!registered_inval) + { + CacheRegisterSyscacheCallback(CMKOID, cmk_change_cb, (Datum) 0); + registered_inval = true; + } + + oldcontext = MemoryContextSwitchTo(TopMemoryContext); + cmk_sent = lappend_oid(cmk_sent, cmkid); + MemoryContextSwitchTo(oldcontext); +} + +/* + * Send ColumnEncryptionKey message, unless it's already been sent in this + * session for this key. + */ +List *cek_sent = NIL; + +static void +cek_change_cb(Datum arg, int cacheid, uint32 hashvalue) +{ + list_free(cek_sent); + cek_sent = NIL; +} + +void +MaybeSendColumnEncryptionKeyMessage(Oid attcek) +{ + HeapTuple tuple; + ScanKeyData skey[1]; + SysScanDesc sd; + Relation rel; + bool found = false; + static bool registered_inval = false; + MemoryContext oldcontext; + + Assert(MyProcPort->column_encryption_enabled); + + if (list_member_oid(cek_sent, attcek)) + return; + + /* + * We really only need data from pg_colenckeydata, but before we scan + * that, let's check that an entry exists in pg_colenckey, so that if + * there are catalog inconsistencies, we can locate them better. + */ + if (!SearchSysCacheExists1(CEKOID, ObjectIdGetDatum(attcek))) + elog(ERROR, "cache lookup failed for column encryption key %u", attcek); + + /* + * Now scan pg_colenckeydata. + */ + ScanKeyInit(&skey[0], + Anum_pg_colenckeydata_ckdcekid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(attcek)); + rel = table_open(ColumnEncKeyDataRelationId, AccessShareLock); + sd = systable_beginscan(rel, ColumnEncKeyCekidCmkidIndexId, true, NULL, 1, skey); + + while ((tuple = systable_getnext(sd))) + { + Form_pg_colenckeydata ckdform = (Form_pg_colenckeydata) GETSTRUCT(tuple); + Datum datum; + bool isnull; + bytea *ba; + StringInfoData buf; + + MaybeSendColumnMasterKeyMessage(ckdform->ckdcmkid); + + datum = heap_getattr(tuple, Anum_pg_colenckeydata_ckdencval, RelationGetDescr(rel), &isnull); + Assert(!isnull); + ba = pg_detoast_datum_packed((bytea *) DatumGetPointer(datum)); + + pq_beginmessage(&buf, 'Y'); /* ColumnEncryptionKey */ + pq_sendint32(&buf, ckdform->ckdcekid); + pq_sendint32(&buf, ckdform->ckdcmkid); + pq_sendint32(&buf, ckdform->ckdcmkalg); + pq_sendint32(&buf, VARSIZE_ANY_EXHDR(ba)); + pq_sendbytes(&buf, VARDATA_ANY(ba), VARSIZE_ANY_EXHDR(ba)); + pq_endmessage(&buf); + + found = true; + } + + /* + * This is a user-facing message, because with ALTER it is possible to + * delete all data entries for a CEK. + */ + if (!found) + ereport(ERROR, + errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("no data for column encryption key \"%s\"", get_cek_name(attcek, false))); + + systable_endscan(sd); + table_close(rel, NoLock); + + if (!registered_inval) + { + CacheRegisterSyscacheCallback(CEKDATAOID, cek_change_cb, (Datum) 0); + registered_inval = true; + } + + oldcontext = MemoryContextSwitchTo(TopMemoryContext); + cek_sent = lappend_oid(cek_sent, attcek); + MemoryContextSwitchTo(oldcontext); +} + +void +DiscardColumnEncryptionKeys(void) +{ + list_free(cmk_sent); + cmk_sent = NIL; + + list_free(cek_sent); + cek_sent = NIL; +} + /* * SendRowDescriptionMessage --- send a RowDescription message to the frontend * @@ -166,6 +339,7 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, List *targetlist, int16 *formats) { int natts = typeinfo->natts; + size_t sz; int i; ListCell *tlist_item = list_head(targetlist); @@ -182,14 +356,18 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, * Have to overestimate the size of the column-names, to account for * character set overhead. */ - enlargeStringInfo(buf, (NAMEDATALEN * MAX_CONVERSION_GROWTH /* attname */ - + sizeof(Oid) /* resorigtbl */ - + sizeof(AttrNumber) /* resorigcol */ - + sizeof(Oid) /* atttypid */ - + sizeof(int16) /* attlen */ - + sizeof(int32) /* attypmod */ - + sizeof(int16) /* format */ - ) * natts); + sz = (NAMEDATALEN * MAX_CONVERSION_GROWTH /* attname */ + + sizeof(Oid) /* resorigtbl */ + + sizeof(AttrNumber) /* resorigcol */ + + sizeof(Oid) /* atttypid */ + + sizeof(int16) /* attlen */ + + sizeof(int32) /* attypmod */ + + sizeof(int16)); /* format */ + if (MyProcPort->column_encryption_enabled) + sz += (sizeof(int32) /* attcekid */ + + sizeof(int32) /* attencalg */ + + sizeof(int16)); /* flags */ + enlargeStringInfo(buf, sz * natts); for (i = 0; i < natts; ++i) { @@ -199,6 +377,9 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, Oid resorigtbl; AttrNumber resorigcol; int16 format; + Oid attcekid = InvalidOid; + int32 attencalg = 0; + int16 flags = 0; /* * If column is a domain, send the base type and typmod instead. @@ -230,6 +411,32 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, else format = 0; + if (MyProcPort->column_encryption_enabled && type_is_encrypted(atttypid)) + { + HeapTuple tp; + Form_pg_attribute orig_att; + + tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(resorigtbl), Int16GetDatum(resorigcol)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for attribute %d of relation %u", resorigcol, resorigtbl); + orig_att = (Form_pg_attribute) GETSTRUCT(tp); + attcekid = DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attcek)); + atttypid = DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypid)); + atttypmod = DatumGetInt32(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypmod)); + attencalg = orig_att->atttypmod; + if (orig_att->atttypid == PG_ENCRYPTED_DETOID) + flags |= 0x0001; + ReleaseSysCache(tp); + + MaybeSendColumnEncryptionKeyMessage(attcekid); + + /* + * Encrypted types are always sent in binary when column + * encryption is enabled. + */ + format = 1; + } + pq_writestring(buf, NameStr(att->attname)); pq_writeint32(buf, resorigtbl); pq_writeint16(buf, resorigcol); @@ -237,6 +444,12 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, pq_writeint16(buf, att->attlen); pq_writeint32(buf, atttypmod); pq_writeint16(buf, format); + if (MyProcPort->column_encryption_enabled) + { + pq_writeint32(buf, attcekid); + pq_writeint32(buf, attencalg); + pq_writeint16(buf, flags); + } } pq_endmessage_reuse(buf); @@ -270,6 +483,13 @@ printtup_prepare_info(DR_printtup *myState, TupleDesc typeinfo, int numAttrs) int16 format = (formats ? formats[i] : 0); Form_pg_attribute attr = TupleDescAttr(typeinfo, i); + /* + * Encrypted types are always sent in binary when column encryption is + * enabled. + */ + if (MyProcPort->column_encryption_enabled && type_is_encrypted(attr->atttypid)) + format = 1; + thisState->format = format; if (format == 0) { diff --git a/src/backend/access/hash/hashvalidate.c b/src/backend/access/hash/hashvalidate.c index 40164e2ea2b..a6646748706 100644 --- a/src/backend/access/hash/hashvalidate.c +++ b/src/backend/access/hash/hashvalidate.c @@ -331,7 +331,7 @@ check_hash_func_signature(Oid funcid, int16 amprocnum, Oid argtype) argtype == BOOLOID) /* okay, allowed use of hashchar() */ ; else if ((funcid == F_HASHVARLENA || funcid == F_HASHVARLENAEXTENDED) && - argtype == BYTEAOID) + (argtype == BYTEAOID || argtype == PG_ENCRYPTED_DETOID)) /* okay, allowed use of hashvarlena() */ ; else result = false; diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y index 3c9c1da0216..438ed6e1cfe 100644 --- a/src/backend/bootstrap/bootparse.y +++ b/src/backend/bootstrap/bootparse.y @@ -226,6 +226,7 @@ Boot_CreateStmt: BOOTSTRAP_SUPERUSERID, HEAP_TABLE_AM_OID, tupdesc, + NULL, NIL, RELKIND_RELATION, RELPERSISTENCE_PERMANENT, diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index 7abf3c2a74a..c4fa1b48e9b 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -51,6 +51,8 @@ #include "catalog/objectaccess.h" #include "catalog/pg_authid.h" #include "catalog/pg_class.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_database.h" #include "catalog/pg_default_acl.h" #include "catalog/pg_foreign_data_wrapper.h" @@ -256,6 +258,12 @@ restrict_and_check_grant(bool is_grant, AclMode avail_goptions, bool all_privs, case OBJECT_SEQUENCE: whole_mask = ACL_ALL_RIGHTS_SEQUENCE; break; + case OBJECT_CEK: + whole_mask = ACL_ALL_RIGHTS_CEK; + break; + case OBJECT_CMK: + whole_mask = ACL_ALL_RIGHTS_CMK; + break; case OBJECT_DATABASE: whole_mask = ACL_ALL_RIGHTS_DATABASE; break; @@ -482,6 +490,14 @@ ExecuteGrantStmt(GrantStmt *stmt) all_privileges = ACL_ALL_RIGHTS_SEQUENCE; errormsg = gettext_noop("invalid privilege type %s for sequence"); break; + case OBJECT_CEK: + all_privileges = ACL_ALL_RIGHTS_CEK; + errormsg = gettext_noop("invalid privilege type %s for column encryption key"); + break; + case OBJECT_CMK: + all_privileges = ACL_ALL_RIGHTS_CMK; + errormsg = gettext_noop("invalid privilege type %s for column master key"); + break; case OBJECT_DATABASE: all_privileges = ACL_ALL_RIGHTS_DATABASE; errormsg = gettext_noop("invalid privilege type %s for database"); @@ -606,6 +622,12 @@ ExecGrantStmt_oids(InternalGrant *istmt) case OBJECT_SEQUENCE: ExecGrant_Relation(istmt); break; + case OBJECT_CEK: + ExecGrant_common(istmt, ColumnEncKeyRelationId, ACL_ALL_RIGHTS_CEK, NULL); + break; + case OBJECT_CMK: + ExecGrant_common(istmt, ColumnMasterKeyRelationId, ACL_ALL_RIGHTS_CMK, NULL); + break; case OBJECT_DATABASE: ExecGrant_common(istmt, DatabaseRelationId, ACL_ALL_RIGHTS_DATABASE, NULL); break; @@ -685,6 +707,26 @@ objectNamesToOids(ObjectType objtype, List *objnames, bool is_grant) objects = lappend_oid(objects, relOid); } break; + case OBJECT_CEK: + foreach(cell, objnames) + { + List *cekname = (List *) lfirst(cell); + Oid oid; + + oid = get_cek_oid(cekname, false); + objects = lappend_oid(objects, oid); + } + break; + case OBJECT_CMK: + foreach(cell, objnames) + { + List *cmkname = (List *) lfirst(cell); + Oid oid; + + oid = get_cmk_oid(cmkname, false); + objects = lappend_oid(objects, oid); + } + break; case OBJECT_DATABASE: foreach(cell, objnames) { @@ -2702,6 +2744,12 @@ aclcheck_error(AclResult aclerr, ObjectType objtype, case OBJECT_AGGREGATE: msg = gettext_noop("permission denied for aggregate %s"); break; + case OBJECT_CEK: + msg = gettext_noop("permission denied for column encryption key %s"); + break; + case OBJECT_CMK: + msg = gettext_noop("permission denied for column master key %s"); + break; case OBJECT_COLLATION: msg = gettext_noop("permission denied for collation %s"); break; @@ -2807,6 +2855,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype, case OBJECT_AMPROC: case OBJECT_ATTRIBUTE: case OBJECT_CAST: + case OBJECT_CEKDATA: case OBJECT_DEFAULT: case OBJECT_DEFACL: case OBJECT_DOMCONSTRAINT: @@ -2837,6 +2886,12 @@ aclcheck_error(AclResult aclerr, ObjectType objtype, case OBJECT_AGGREGATE: msg = gettext_noop("must be owner of aggregate %s"); break; + case OBJECT_CEK: + msg = gettext_noop("must be owner of column encryption key %s"); + break; + case OBJECT_CMK: + msg = gettext_noop("must be owner of column master key %s"); + break; case OBJECT_COLLATION: msg = gettext_noop("must be owner of collation %s"); break; @@ -2947,6 +3002,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype, case OBJECT_AMPROC: case OBJECT_ATTRIBUTE: case OBJECT_CAST: + case OBJECT_CEKDATA: case OBJECT_DEFAULT: case OBJECT_DEFACL: case OBJECT_DOMCONSTRAINT: @@ -3028,6 +3084,10 @@ pg_aclmask(ObjectType objtype, Oid object_oid, AttrNumber attnum, Oid roleid, case OBJECT_TABLE: case OBJECT_SEQUENCE: return pg_class_aclmask(object_oid, roleid, mask, how); + case OBJECT_CEK: + return object_aclmask(ColumnEncKeyRelationId, object_oid, roleid, mask, how); + case OBJECT_CMK: + return object_aclmask(ColumnMasterKeyRelationId, object_oid, roleid, mask, how); case OBJECT_DATABASE: return object_aclmask(DatabaseRelationId, object_oid, roleid, mask, how); case OBJECT_FUNCTION: diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c index 6e8f6a57051..ed2a01a08da 100644 --- a/src/backend/catalog/dependency.c +++ b/src/backend/catalog/dependency.c @@ -30,7 +30,10 @@ #include "catalog/pg_authid.h" #include "catalog/pg_auth_members.h" #include "catalog/pg_cast.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colenckeydata.h" #include "catalog/pg_collation.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_constraint.h" #include "catalog/pg_conversion.h" #include "catalog/pg_database.h" @@ -1444,6 +1447,9 @@ doDeletion(const ObjectAddress *object, int flags) case CastRelationId: case CollationRelationId: + case ColumnEncKeyRelationId: + case ColumnEncKeyDataRelationId: + case ColumnMasterKeyRelationId: case ConversionRelationId: case LanguageRelationId: case OperatorClassRelationId: diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c index cc31909012d..bc68fc9bf3d 100644 --- a/src/backend/catalog/heap.c +++ b/src/backend/catalog/heap.c @@ -42,6 +42,7 @@ #include "catalog/partition.h" #include "catalog/pg_am.h" #include "catalog/pg_attrdef.h" +#include "catalog/pg_colenckey.h" #include "catalog/pg_collation.h" #include "catalog/pg_constraint.h" #include "catalog/pg_foreign_table.h" @@ -453,7 +454,7 @@ heap_create(const char *relname, * -------------------------------- */ void -CheckAttributeNamesTypes(TupleDesc tupdesc, char relkind, +CheckAttributeNamesTypes(TupleDesc tupdesc, const FormExtraData_pg_attribute tupdesc_extra[], char relkind, int flags) { int i; @@ -512,7 +513,14 @@ CheckAttributeNamesTypes(TupleDesc tupdesc, char relkind, TupleDescAttr(tupdesc, i)->atttypid, TupleDescAttr(tupdesc, i)->attcollation, NIL, /* assume we're creating a new rowtype */ - flags); + flags | + + /* + * Allow encrypted types if CEK has been provided, which means this + * type has been internally generated. We don't want to allow + * explicitly using these types. + */ + (tupdesc_extra && !tupdesc_extra[i].attcek.isnull ? CHKATYPE_ENCRYPTED : 0)); } } @@ -657,6 +665,21 @@ CheckAttributeType(const char *attname, flags); } + /* + * Encrypted types are not allowed explictly as column types. Most + * callers run this check before transforming the column definition to use + * the encrypted types. Some callers call it again after; those should + * set the CHKATYPE_ENCRYPTED to let this pass. + */ + if (type_is_encrypted(atttypid) && !(flags & CHKATYPE_ENCRYPTED)) + { + ereport(ERROR, + (errcode(ERRCODE_INVALID_TABLE_DEFINITION), + errbacktrace(), + errmsg("column \"%s\" has internal type %s", + attname, format_type_be(atttypid)))); + } + /* * This might not be strictly invalid per SQL standard, but it is pretty * useless, and it cannot be dumped, so we must disallow it. @@ -763,11 +786,23 @@ InsertPgAttributeTuples(Relation pg_attribute_rel, slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attrs_extra->attoptions.value; slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = attrs_extra->attoptions.isnull; + + slot[slotCount]->tts_values[Anum_pg_attribute_attcek - 1] = attrs_extra->attcek.value; + slot[slotCount]->tts_isnull[Anum_pg_attribute_attcek - 1] = attrs_extra->attcek.isnull; + + slot[slotCount]->tts_values[Anum_pg_attribute_attusertypid - 1] = attrs_extra->attusertypid.value; + slot[slotCount]->tts_isnull[Anum_pg_attribute_attusertypid - 1] = attrs_extra->attusertypid.isnull; + + slot[slotCount]->tts_values[Anum_pg_attribute_attusertypmod - 1] = attrs_extra->attusertypmod.value; + slot[slotCount]->tts_isnull[Anum_pg_attribute_attusertypmod - 1] = attrs_extra->attusertypmod.isnull; } else { slot[slotCount]->tts_isnull[Anum_pg_attribute_attstattarget - 1] = true; slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true; + slot[slotCount]->tts_isnull[Anum_pg_attribute_attcek - 1] = true; + slot[slotCount]->tts_isnull[Anum_pg_attribute_attusertypid - 1] = true; + slot[slotCount]->tts_isnull[Anum_pg_attribute_attusertypmod - 1] = true; } /* @@ -819,6 +854,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel, static void AddNewAttributeTuples(Oid new_rel_oid, TupleDesc tupdesc, + const FormExtraData_pg_attribute tupdesc_extra[], char relkind) { Relation rel; @@ -834,7 +870,7 @@ AddNewAttributeTuples(Oid new_rel_oid, indstate = CatalogOpenIndexes(rel); - InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate); + InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, tupdesc_extra, indstate); /* add dependencies on their datatypes and collations */ for (int i = 0; i < natts; i++) @@ -853,6 +889,20 @@ AddNewAttributeTuples(Oid new_rel_oid, tupdesc->attrs[i].attcollation); recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); } + + if (tupdesc_extra && !tupdesc_extra[i].attcek.isnull) + { + ObjectAddressSet(referenced, ColumnEncKeyRelationId, + DatumGetObjectId(tupdesc_extra[i].attcek.value)); + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + } + + if (tupdesc_extra && !tupdesc_extra[i].attusertypid.isnull) + { + ObjectAddressSet(referenced, TypeRelationId, + DatumGetObjectId(tupdesc_extra[i].attusertypid.value)); + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + } } /* @@ -1110,6 +1160,7 @@ heap_create_with_catalog(const char *relname, Oid ownerid, Oid accessmtd, TupleDesc tupdesc, + const FormExtraData_pg_attribute tupdesc_extra[], List *cooked_constraints, char relkind, char relpersistence, @@ -1147,7 +1198,7 @@ heap_create_with_catalog(const char *relname, * allow_system_table_mods is on, allow ANYARRAY to be used; this is a * hack to allow creating pg_statistic and cloning it during VACUUM FULL. */ - CheckAttributeNamesTypes(tupdesc, relkind, + CheckAttributeNamesTypes(tupdesc, tupdesc_extra, relkind, allow_system_table_mods ? CHKATYPE_ANYARRAY : 0); /* @@ -1414,7 +1465,7 @@ heap_create_with_catalog(const char *relname, /* * now add tuples to pg_attribute for the attributes in our new relation. */ - AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind); + AddNewAttributeTuples(relid, new_rel_desc->rd_att, tupdesc_extra, relkind); /* * Make a dependency link to force the relation to be deleted if its diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 9b7ef71d6fe..9a03328e5d1 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -529,6 +529,10 @@ AppendAttributeTuples(Relation indexRelation, const Datum *attopts, const Nullab attrs_extra[i].attstattarget = stattargets[i]; else attrs_extra[i].attstattarget.isnull = true; + + attrs_extra[i].attcek.isnull = true; + attrs_extra[i].attusertypid.isnull = true; + attrs_extra[i].attusertypmod.isnull = true; } } diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c index 4548a917234..81018f74f7a 100644 --- a/src/backend/catalog/namespace.c +++ b/src/backend/catalog/namespace.c @@ -27,6 +27,8 @@ #include "catalog/namespace.h" #include "catalog/objectaccess.h" #include "catalog/pg_authid.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_collation.h" #include "catalog/pg_conversion.h" #include "catalog/pg_database.h" @@ -2298,6 +2300,254 @@ OpfamilyIsVisibleExt(Oid opfid, bool *is_missing) return visible; } +/* + * get_cek_oid - find a CEK by possibly qualified name + */ +Oid +get_cek_oid(List *names, bool missing_ok) +{ + char *schemaname; + char *cekname; + Oid namespaceId; + Oid cekoid = InvalidOid; + ListCell *l; + + /* deconstruct the name list */ + DeconstructQualifiedName(names, &schemaname, &cekname); + + if (schemaname) + { + /* use exact schema given */ + namespaceId = LookupExplicitNamespace(schemaname, missing_ok); + if (missing_ok && !OidIsValid(namespaceId)) + cekoid = InvalidOid; + else + cekoid = GetSysCacheOid2(CEKNAMENSP, Anum_pg_colenckey_oid, + PointerGetDatum(cekname), + ObjectIdGetDatum(namespaceId)); + } + else + { + /* search for it in search path */ + recomputeNamespacePath(); + + foreach(l, activeSearchPath) + { + namespaceId = lfirst_oid(l); + + if (namespaceId == myTempNamespace) + continue; /* do not look in temp namespace */ + + cekoid = GetSysCacheOid2(CEKNAMENSP, Anum_pg_colenckey_oid, + PointerGetDatum(cekname), + ObjectIdGetDatum(namespaceId)); + if (OidIsValid(cekoid)) + return cekoid; + } + } + + /* Not found in path */ + if (!OidIsValid(cekoid) && !missing_ok) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("column encryption key \"%s\" does not exist", + NameListToString(names)))); + return cekoid; +} + +/* + * CEKIsVisible + * Determine whether a CEK (identified by OID) is visible in the + * current search path. Visible means "would be found by searching + * for the unqualified CEK name". + */ +bool +CEKIsVisible(Oid cekid) +{ + HeapTuple tup; + Form_pg_colenckey form; + Oid namespace; + bool visible; + + tup = SearchSysCache1(CEKOID, ObjectIdGetDatum(cekid)); + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for column encryption key %u", cekid); + form = (Form_pg_colenckey) GETSTRUCT(tup); + + recomputeNamespacePath(); + + /* + * Quick check: if it ain't in the path at all, it ain't visible. Items in + * the system namespace are surely in the path and so we needn't even do + * list_member_oid() for them. + */ + namespace = form->ceknamespace; + if (namespace != PG_CATALOG_NAMESPACE && + !list_member_oid(activeSearchPath, namespace)) + visible = false; + else + { + /* + * If it is in the path, it might still not be visible; it could be + * hidden by another parser of the same name earlier in the path. So + * we must do a slow check for conflicting CEKs. + */ + char *name = NameStr(form->cekname); + ListCell *l; + + visible = false; + foreach(l, activeSearchPath) + { + Oid namespaceId = lfirst_oid(l); + + if (namespaceId == myTempNamespace) + continue; /* do not look in temp namespace */ + + if (namespaceId == namespace) + { + /* Found it first in path */ + visible = true; + break; + } + if (SearchSysCacheExists2(CEKNAMENSP, + PointerGetDatum(name), + ObjectIdGetDatum(namespaceId))) + { + /* Found something else first in path */ + break; + } + } + } + + ReleaseSysCache(tup); + + return visible; +} + +/* + * get_cmk_oid - find a CMK by possibly qualified name + */ +Oid +get_cmk_oid(List *names, bool missing_ok) +{ + char *schemaname; + char *cmkname; + Oid namespaceId; + Oid cmkoid = InvalidOid; + ListCell *l; + + /* deconstruct the name list */ + DeconstructQualifiedName(names, &schemaname, &cmkname); + + if (schemaname) + { + /* use exact schema given */ + namespaceId = LookupExplicitNamespace(schemaname, missing_ok); + if (missing_ok && !OidIsValid(namespaceId)) + cmkoid = InvalidOid; + else + cmkoid = GetSysCacheOid2(CMKNAMENSP, Anum_pg_colmasterkey_oid, + PointerGetDatum(cmkname), + ObjectIdGetDatum(namespaceId)); + } + else + { + /* search for it in search path */ + recomputeNamespacePath(); + + foreach(l, activeSearchPath) + { + namespaceId = lfirst_oid(l); + + if (namespaceId == myTempNamespace) + continue; /* do not look in temp namespace */ + + cmkoid = GetSysCacheOid2(CMKNAMENSP, Anum_pg_colmasterkey_oid, + PointerGetDatum(cmkname), + ObjectIdGetDatum(namespaceId)); + if (OidIsValid(cmkoid)) + return cmkoid; + } + } + + /* Not found in path */ + if (!OidIsValid(cmkoid) && !missing_ok) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("column master key \"%s\" does not exist", + NameListToString(names)))); + return cmkoid; +} + +/* + * CMKIsVisible + * Determine whether a CMK (identified by OID) is visible in the + * current search path. Visible means "would be found by searching + * for the unqualified CMK name". + */ +bool +CMKIsVisible(Oid cmkid) +{ + HeapTuple tup; + Form_pg_colmasterkey form; + Oid namespace; + bool visible; + + tup = SearchSysCache1(CMKOID, ObjectIdGetDatum(cmkid)); + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for column master key %u", cmkid); + form = (Form_pg_colmasterkey) GETSTRUCT(tup); + + recomputeNamespacePath(); + + /* + * Quick check: if it ain't in the path at all, it ain't visible. Items in + * the system namespace are surely in the path and so we needn't even do + * list_member_oid() for them. + */ + namespace = form->cmknamespace; + if (namespace != PG_CATALOG_NAMESPACE && + !list_member_oid(activeSearchPath, namespace)) + visible = false; + else + { + /* + * If it is in the path, it might still not be visible; it could be + * hidden by another parser of the same name earlier in the path. So + * we must do a slow check for conflicting CMKs. + */ + char *name = NameStr(form->cmkname); + ListCell *l; + + visible = false; + foreach(l, activeSearchPath) + { + Oid namespaceId = lfirst_oid(l); + + if (namespaceId == myTempNamespace) + continue; /* do not look in temp namespace */ + + if (namespaceId == namespace) + { + /* Found it first in path */ + visible = true; + break; + } + if (SearchSysCacheExists2(CMKNAMENSP, + PointerGetDatum(name), + ObjectIdGetDatum(namespaceId))) + { + /* Found something else first in path */ + break; + } + } + } + + ReleaseSysCache(tup); + + return visible; +} + /* * lookup_collation * If there's a collation of the given name/namespace, and it works @@ -4950,6 +5200,28 @@ pg_opfamily_is_visible(PG_FUNCTION_ARGS) PG_RETURN_BOOL(result); } +Datum +pg_cek_is_visible(PG_FUNCTION_ARGS) +{ + Oid oid = PG_GETARG_OID(0); + + if (!SearchSysCacheExists1(CEKOID, ObjectIdGetDatum(oid))) + PG_RETURN_NULL(); + + PG_RETURN_BOOL(CEKIsVisible(oid)); +} + +Datum +pg_cmk_is_visible(PG_FUNCTION_ARGS) +{ + Oid oid = PG_GETARG_OID(0); + + if (!SearchSysCacheExists1(CMKOID, ObjectIdGetDatum(oid))) + PG_RETURN_NULL(); + + PG_RETURN_BOOL(CMKIsVisible(oid)); +} + Datum pg_collation_is_visible(PG_FUNCTION_ARGS) { diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c index 7b536ac6fde..dc3baa304cc 100644 --- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -28,7 +28,10 @@ #include "catalog/pg_authid.h" #include "catalog/pg_auth_members.h" #include "catalog/pg_cast.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colenckeydata.h" #include "catalog/pg_collation.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_constraint.h" #include "catalog/pg_conversion.h" #include "catalog/pg_database.h" @@ -188,6 +191,48 @@ static const ObjectPropertyType ObjectProperty[] = OBJECT_COLLATION, true }, + { + "column encryption key", + ColumnEncKeyRelationId, + ColumnEncKeyOidIndexId, + CEKOID, + CEKNAMENSP, + Anum_pg_colenckey_oid, + Anum_pg_colenckey_cekname, + Anum_pg_colenckey_ceknamespace, + Anum_pg_colenckey_cekowner, + Anum_pg_colenckey_cekacl, + OBJECT_CEK, + true + }, + { + "column encryption key data", + ColumnEncKeyDataRelationId, + ColumnEncKeyDataOidIndexId, + CEKDATAOID, + -1, + Anum_pg_colenckeydata_oid, + InvalidAttrNumber, + InvalidAttrNumber, + InvalidAttrNumber, + InvalidAttrNumber, + -1, + false + }, + { + "column master key", + ColumnMasterKeyRelationId, + ColumnMasterKeyOidIndexId, + CMKOID, + CMKNAMENSP, + Anum_pg_colmasterkey_oid, + Anum_pg_colmasterkey_cmkname, + Anum_pg_colmasterkey_cmknamespace, + Anum_pg_colmasterkey_cmkowner, + Anum_pg_colmasterkey_cmkacl, + OBJECT_CMK, + true + }, { "constraint", ConstraintRelationId, @@ -721,6 +766,15 @@ static const struct object_type_map { "collation", OBJECT_COLLATION }, + { + "column encryption key", OBJECT_CEK + }, + { + "column encryption key data", OBJECT_CEKDATA + }, + { + "column master key", OBJECT_CMK + }, { "table constraint", OBJECT_TABCONSTRAINT }, @@ -991,6 +1045,16 @@ get_object_address(ObjectType objtype, Node *object, address.objectSubId = 0; } break; + case OBJECT_CEK: + address.classId = ColumnEncKeyRelationId; + address.objectId = get_cek_oid(castNode(List, object), missing_ok); + address.objectSubId = 0; + break; + case OBJECT_CMK: + address.classId = ColumnMasterKeyRelationId; + address.objectId = get_cmk_oid(castNode(List, object), missing_ok); + address.objectSubId = 0; + break; case OBJECT_DATABASE: case OBJECT_EXTENSION: case OBJECT_TABLESPACE: @@ -1070,6 +1134,21 @@ get_object_address(ObjectType objtype, Node *object, address.objectSubId = 0; } break; + case OBJECT_CEKDATA: + { + List *cekname = linitial_node(List, castNode(List, object)); + List *cmkname = lsecond_node(List, castNode(List, object)); + Oid cekid; + Oid cmkid; + + cekid = get_cek_oid(cekname, missing_ok); + cmkid = get_cmk_oid(cmkname, missing_ok); + + address.classId = ColumnEncKeyDataRelationId; + address.objectId = get_cekdata_oid(cekid, cmkid, missing_ok); + address.objectSubId = 0; + } + break; case OBJECT_TRANSFORM: { TypeName *typename = linitial_node(TypeName, castNode(List, object)); @@ -2273,6 +2352,8 @@ pg_get_object_address(PG_FUNCTION_ARGS) case OBJECT_FOREIGN_TABLE: case OBJECT_COLUMN: case OBJECT_ATTRIBUTE: + case OBJECT_CEK: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_STATISTIC_EXT: @@ -2329,6 +2410,7 @@ pg_get_object_address(PG_FUNCTION_ARGS) break; case OBJECT_AMOP: case OBJECT_AMPROC: + case OBJECT_CEKDATA: objnode = (Node *) list_make2(name, args); break; case OBJECT_FUNCTION: @@ -2451,6 +2533,8 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address, aclcheck_error(ACLCHECK_NOT_OWNER, objtype, strVal(object)); break; + case OBJECT_CEK: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_OPCLASS: @@ -2543,6 +2627,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address, break; case OBJECT_AMOP: case OBJECT_AMPROC: + case OBJECT_CEKDATA: case OBJECT_DEFAULT: case OBJECT_DEFACL: case OBJECT_PUBLICATION_NAMESPACE: @@ -3005,6 +3090,94 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok) break; } + case ColumnEncKeyRelationId: + { + HeapTuple tup; + Form_pg_colenckey form; + char *nspname; + + tup = SearchSysCache1(CEKOID, + ObjectIdGetDatum(object->objectId)); + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column encryption key %u", + object->objectId); + break; + } + + form = (Form_pg_colenckey) GETSTRUCT(tup); + + /* Qualify the name if not visible in search path */ + if (CEKIsVisible(object->objectId)) + nspname = NULL; + else + nspname = get_namespace_name(form->ceknamespace); + + appendStringInfo(&buffer, _("column encryption key %s"), + quote_qualified_identifier(nspname, + NameStr(form->cekname))); + ReleaseSysCache(tup); + break; + } + + case ColumnEncKeyDataRelationId: + { + HeapTuple tup; + Form_pg_colenckeydata cekdata; + ObjectAddress cekaddr, cmkaddr; + + tup = SearchSysCache1(CEKDATAOID, ObjectIdGetDatum(object->objectId)); + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column encryption key data %u", + object->objectId); + break; + } + + cekdata = (Form_pg_colenckeydata) GETSTRUCT(tup); + + ObjectAddressSet(cekaddr, ColumnEncKeyRelationId, cekdata->ckdcekid); + ObjectAddressSet(cmkaddr, ColumnMasterKeyRelationId, cekdata->ckdcmkid); + appendStringInfo(&buffer, _("column encryption key data of %s for %s"), + getObjectDescription(&cekaddr, false), getObjectDescription(&cmkaddr, false)); + + ReleaseSysCache(tup); + break; + } + + case ColumnMasterKeyRelationId: + { + HeapTuple tup; + Form_pg_colmasterkey form; + char *nspname; + + tup = SearchSysCache1(CMKOID, + ObjectIdGetDatum(object->objectId)); + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column encryption key %u", + object->objectId); + break; + } + + form = (Form_pg_colmasterkey) GETSTRUCT(tup); + + /* Qualify the name if not visible in search path */ + if (CMKIsVisible(object->objectId)) + nspname = NULL; + else + nspname = get_namespace_name(form->cmknamespace); + + appendStringInfo(&buffer, _("column master key %s"), + quote_qualified_identifier(nspname, + NameStr(form->cmkname))); + ReleaseSysCache(tup); + break; + } + case ConstraintRelationId: { HeapTuple conTup; @@ -4400,6 +4573,18 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok) appendStringInfoString(&buffer, "collation"); break; + case ColumnEncKeyRelationId: + appendStringInfoString(&buffer, "column encryption key"); + break; + + case ColumnEncKeyDataRelationId: + appendStringInfoString(&buffer, "column encryption key data"); + break; + + case ColumnMasterKeyRelationId: + appendStringInfoString(&buffer, "column master key"); + break; + case ConstraintRelationId: getConstraintTypeDescription(&buffer, object->objectId, missing_ok); @@ -4863,6 +5048,108 @@ getObjectIdentityParts(const ObjectAddress *object, break; } + case ColumnEncKeyRelationId: + { + HeapTuple tup; + Form_pg_colenckey form; + char *schema; + + tup = SearchSysCache1(CEKOID, ObjectIdGetDatum(object->objectId)); + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column encryption key %u", + object->objectId); + break; + } + form = (Form_pg_colenckey) GETSTRUCT(tup); + schema = get_namespace_name_or_temp(form->ceknamespace); + appendStringInfoString(&buffer, + quote_qualified_identifier(schema, + NameStr(form->cekname))); + if (objname) + *objname = list_make2(schema, pstrdup(NameStr(form->cekname))); + ReleaseSysCache(tup); + break; + } + + case ColumnEncKeyDataRelationId: + { + HeapTuple tup; + Form_pg_colenckeydata form; + + tup = SearchSysCache1(CEKDATAOID, ObjectIdGetDatum(object->objectId)); + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column encryption key data %u", + object->objectId); + break; + } + form = (Form_pg_colenckeydata) GETSTRUCT(tup); + appendStringInfo(&buffer, + "of %s for %s", + getObjectIdentityParts(&(ObjectAddress){ColumnEncKeyRelationId, form->ckdcekid}, NULL, NULL, false), + getObjectIdentityParts(&(ObjectAddress){ColumnMasterKeyRelationId, form->ckdcmkid}, NULL, NULL, false)); + + if (objname) + { + HeapTuple tup2; + Form_pg_colenckey form2; + char *schema; + + tup2 = SearchSysCache1(CEKOID, ObjectIdGetDatum(form->ckdcekid)); + if (!HeapTupleIsValid(tup2)) + elog(ERROR, "cache lookup failed for column encryption key %u", form->ckdcekid); + form2 = (Form_pg_colenckey) GETSTRUCT(tup2); + schema = get_namespace_name_or_temp(form2->ceknamespace); + *objname = list_make2(schema, pstrdup(NameStr(form2->cekname))); + ReleaseSysCache(tup2); + } + if (objargs) + { + HeapTuple tup2; + Form_pg_colmasterkey form2; + char *schema; + + tup2 = SearchSysCache1(CMKOID, ObjectIdGetDatum(form->ckdcmkid)); + if (!HeapTupleIsValid(tup2)) + elog(ERROR, "cache lookup failed for column master key %u", form->ckdcmkid); + form2 = (Form_pg_colmasterkey) GETSTRUCT(tup2); + schema = get_namespace_name_or_temp(form2->cmknamespace); + if (objargs) + *objargs = list_make2(schema, pstrdup(NameStr(form2->cmkname))); + ReleaseSysCache(tup2); + } + ReleaseSysCache(tup); + break; + } + + case ColumnMasterKeyRelationId: + { + HeapTuple tup; + Form_pg_colmasterkey form; + char *schema; + + tup = SearchSysCache1(CMKOID, ObjectIdGetDatum(object->objectId)); + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column master key %u", + object->objectId); + break; + } + form = (Form_pg_colmasterkey) GETSTRUCT(tup); + schema = get_namespace_name_or_temp(form->cmknamespace); + appendStringInfoString(&buffer, + quote_qualified_identifier(schema, + NameStr(form->cmkname))); + if (objname) + *objname = list_make2(schema, pstrdup(NameStr(form->cmkname))); + ReleaseSysCache(tup); + break; + } + case ConstraintRelationId: { HeapTuple conTup; diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c index 738bc46ae82..09f81e739fd 100644 --- a/src/backend/catalog/toasting.c +++ b/src/backend/catalog/toasting.c @@ -252,6 +252,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid, rel->rd_rel->relowner, table_relation_toast_am(rel), tupdesc, + NULL, NIL, RELKIND_TOASTVALUE, rel->rd_rel->relpersistence, diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile index cede90c3b98..cf7a533e0fe 100644 --- a/src/backend/commands/Makefile +++ b/src/backend/commands/Makefile @@ -19,6 +19,7 @@ OBJS = \ analyze.o \ async.o \ cluster.o \ + colenccmds.o \ collationcmds.o \ comment.o \ constraint.o \ diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c index 12802b9d3fd..cfea5929972 100644 --- a/src/backend/commands/alter.c +++ b/src/backend/commands/alter.c @@ -21,7 +21,9 @@ #include "catalog/indexing.h" #include "catalog/namespace.h" #include "catalog/objectaccess.h" +#include "catalog/pg_colenckey.h" #include "catalog/pg_collation.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_conversion.h" #include "catalog/pg_database_d.h" #include "catalog/pg_event_trigger.h" @@ -115,6 +117,12 @@ report_namespace_conflict(Oid classId, const char *name, Oid nspOid) switch (classId) { + case ColumnEncKeyRelationId: + msgfmt = gettext_noop("column encryption key \"%s\" already exists in schema \"%s\""); + break; + case ColumnMasterKeyRelationId: + msgfmt = gettext_noop("column master key \"%s\" already exists in schema \"%s\""); + break; case ConversionRelationId: Assert(OidIsValid(nspOid)); msgfmt = gettext_noop("conversion \"%s\" already exists in schema \"%s\""); @@ -400,6 +408,8 @@ ExecRenameStmt(RenameStmt *stmt) return RenameType(stmt); case OBJECT_AGGREGATE: + case OBJECT_CEK: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_EVENT_TRIGGER: @@ -548,6 +558,8 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt, /* generic code path */ case OBJECT_AGGREGATE: + case OBJECT_CEK: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_FUNCTION: @@ -861,6 +873,8 @@ ExecAlterOwnerStmt(AlterOwnerStmt *stmt) /* Generic cases */ case OBJECT_AGGREGATE: + case OBJECT_CEK: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_FUNCTION: diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c index c04886c4090..2c6a31dd3be 100644 --- a/src/backend/commands/cluster.c +++ b/src/backend/commands/cluster.c @@ -747,6 +747,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, Oid NewAccessMethod, OldHeap->rd_rel->relowner, NewAccessMethod, OldHeapDesc, + NULL, NIL, RELKIND_RELATION, relpersistence, diff --git a/src/backend/commands/colenccmds.c b/src/backend/commands/colenccmds.c new file mode 100644 index 00000000000..2a59aa4e1bf --- /dev/null +++ b/src/backend/commands/colenccmds.c @@ -0,0 +1,449 @@ +/*------------------------------------------------------------------------- + * + * colenccmds.c + * column-encryption-related commands support code + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/commands/colenccmds.c + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "access/htup_details.h" +#include "access/table.h" +#include "catalog/catalog.h" +#include "catalog/dependency.h" +#include "catalog/indexing.h" +#include "catalog/namespace.h" +#include "catalog/objectaccess.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colenckeydata.h" +#include "catalog/pg_colmasterkey.h" +#include "catalog/pg_namespace.h" +#include "commands/colenccmds.h" +#include "commands/dbcommands.h" +#include "commands/defrem.h" +#include "common/colenc.h" +#include "miscadmin.h" +#include "utils/acl.h" +#include "utils/builtins.h" +#include "utils/lsyscache.h" +#include "utils/rel.h" +#include "utils/syscache.h" + +static void +parse_cek_attributes(ParseState *pstate, List *definition, Oid *cmkoid_p, int *alg_p, char **encval_p) +{ + ListCell *lc; + DefElem *cmkEl = NULL; + DefElem *algEl = NULL; + DefElem *encvalEl = NULL; + Oid cmkoid = InvalidOid; + int alg = 0; + char *encval = NULL; + + Assert(cmkoid_p); + + foreach(lc, definition) + { + DefElem *defel = lfirst_node(DefElem, lc); + DefElem **defelp; + + if (strcmp(defel->defname, "column_master_key") == 0) + defelp = &cmkEl; + else if (strcmp(defel->defname, "algorithm") == 0) + defelp = &algEl; + else if (strcmp(defel->defname, "encrypted_value") == 0) + defelp = &encvalEl; + else + { + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("column encryption key attribute \"%s\" not recognized", + defel->defname), + parser_errposition(pstate, defel->location))); + } + if (*defelp != NULL) + errorConflictingDefElem(defel, pstate); + *defelp = defel; + } + + if (cmkEl) + { + List *val = defGetQualifiedName(cmkEl); + + cmkoid = get_cmk_oid(val, false); + } + else + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("attribute \"%s\" must be specified", + "column_master_key"))); + + if (algEl) + { + char *val = defGetString(algEl); + + if (!alg_p) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("attribute \"%s\" must not be specified", + "algorithm"))); + + alg = get_cmkalg_num(val); + if (!alg) + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("unrecognized encryption algorithm: %s", val), + parser_errposition(pstate, algEl->location)); + } + else + { + if (alg_p) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("attribute \"%s\" must be specified", + "algorithm"))); + } + + if (encvalEl) + { + if (!encval_p) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("attribute \"%s\" must not be specified", + "encrypted_value"))); + + encval = defGetString(encvalEl); + } + else + { + if (encval_p) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("attribute \"%s\" must be specified", + "encrypted_value"))); + } + + *cmkoid_p = cmkoid; + if (alg_p) + *alg_p = alg; + if (encval_p) + *encval_p = encval; +} + +static void +insert_cekdata_record(Oid cekoid, Oid cmkoid, int alg, char *encval) +{ + Oid cekdataoid; + Relation rel; + Datum values[Natts_pg_colenckeydata] = {0}; + bool nulls[Natts_pg_colenckeydata] = {0}; + HeapTuple tup; + ObjectAddress myself; + ObjectAddress other; + + rel = table_open(ColumnEncKeyDataRelationId, RowExclusiveLock); + + cekdataoid = GetNewOidWithIndex(rel, ColumnEncKeyDataOidIndexId, Anum_pg_colenckeydata_oid); + values[Anum_pg_colenckeydata_oid - 1] = ObjectIdGetDatum(cekdataoid); + values[Anum_pg_colenckeydata_ckdcekid - 1] = ObjectIdGetDatum(cekoid); + values[Anum_pg_colenckeydata_ckdcmkid - 1] = ObjectIdGetDatum(cmkoid); + values[Anum_pg_colenckeydata_ckdcmkalg - 1] = Int32GetDatum(alg); + values[Anum_pg_colenckeydata_ckdencval - 1] = DirectFunctionCall1(byteain, CStringGetDatum(encval)); + + tup = heap_form_tuple(RelationGetDescr(rel), values, nulls); + CatalogTupleInsert(rel, tup); + heap_freetuple(tup); + + ObjectAddressSet(myself, ColumnEncKeyDataRelationId, cekdataoid); + + /* dependency cekdata -> cek */ + ObjectAddressSet(other, ColumnEncKeyRelationId, cekoid); + recordDependencyOn(&myself, &other, DEPENDENCY_AUTO); + + /* dependency cekdata -> cmk */ + ObjectAddressSet(other, ColumnMasterKeyRelationId, cmkoid); + recordDependencyOn(&myself, &other, DEPENDENCY_NORMAL); + + table_close(rel, NoLock); +} + +ObjectAddress +CreateCEK(ParseState *pstate, DefineStmt *stmt) +{ + Oid namespaceId; + char *ceknamestr; + AclResult aclresult; + Relation rel; + ObjectAddress myself; + ObjectAddress referenced; + Oid cekoid; + ListCell *lc; + NameData cekname; + Datum values[Natts_pg_colenckey] = {0}; + bool nulls[Natts_pg_colenckey] = {0}; + HeapTuple tup; + + namespaceId = QualifiedNameGetCreationNamespace(stmt->defnames, &ceknamestr); + + aclresult = object_aclcheck(NamespaceRelationId, namespaceId, GetUserId(), ACL_CREATE); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_SCHEMA, get_namespace_name(namespaceId)); + + rel = table_open(ColumnEncKeyRelationId, RowExclusiveLock); + + if (SearchSysCacheExists2(CEKNAMENSP, PointerGetDatum(ceknamestr), ObjectIdGetDatum(namespaceId))) + ereport(ERROR, + errcode(ERRCODE_DUPLICATE_OBJECT), + errmsg("column encryption key \"%s\" already exists", ceknamestr)); + + cekoid = GetNewOidWithIndex(rel, ColumnEncKeyOidIndexId, Anum_pg_colenckey_oid); + + foreach(lc, stmt->definition) + { + List *definition = lfirst_node(List, lc); + Oid cmkoid = 0; + int alg; + char *encval; + + parse_cek_attributes(pstate, definition, &cmkoid, &alg, &encval); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkoid, GetUserId(), ACL_USAGE); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_CMK, get_cmk_name(cmkoid, false)); + + /* pg_colenckeydata */ + insert_cekdata_record(cekoid, cmkoid, alg, encval); + } + + /* pg_colenckey */ + namestrcpy(&cekname, ceknamestr); + values[Anum_pg_colenckey_oid - 1] = ObjectIdGetDatum(cekoid); + values[Anum_pg_colenckey_cekname - 1] = NameGetDatum(&cekname); + values[Anum_pg_colenckey_ceknamespace - 1] = ObjectIdGetDatum(namespaceId); + values[Anum_pg_colenckey_cekowner - 1] = ObjectIdGetDatum(GetUserId()); + nulls[Anum_pg_colenckey_cekacl - 1] = true; + + tup = heap_form_tuple(RelationGetDescr(rel), values, nulls); + CatalogTupleInsert(rel, tup); + heap_freetuple(tup); + + ObjectAddressSet(myself, ColumnEncKeyRelationId, cekoid); + + ObjectAddressSet(referenced, NamespaceRelationId, namespaceId); + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + + recordDependencyOnOwner(ColumnEncKeyRelationId, cekoid, GetUserId()); + + table_close(rel, RowExclusiveLock); + + InvokeObjectPostCreateHook(ColumnEncKeyRelationId, cekoid, 0); + + return myself; +} + +ObjectAddress +AlterColumnEncryptionKey(ParseState *pstate, AlterColumnEncryptionKeyStmt *stmt) +{ + Oid cekoid; + ObjectAddress address; + + cekoid = get_cek_oid(stmt->cekname, false); + + if (!object_ownercheck(ColumnEncKeyRelationId, cekoid, GetUserId())) + aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_CEK, NameListToString(stmt->cekname)); + + if (stmt->isDrop) + { + Oid cmkoid = 0; + Oid cekdataoid; + ObjectAddress obj; + + parse_cek_attributes(pstate, stmt->definition, &cmkoid, NULL, NULL); + cekdataoid = get_cekdata_oid(cekoid, cmkoid, false); + ObjectAddressSet(obj, ColumnEncKeyDataRelationId, cekdataoid); + performDeletion(&obj, DROP_CASCADE, 0); + } + else + { + Oid cmkoid = 0; + int alg; + char *encval; + AclResult aclresult; + + parse_cek_attributes(pstate, stmt->definition, &cmkoid, &alg, &encval); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkoid, GetUserId(), ACL_USAGE); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_CMK, get_cmk_name(cmkoid, false)); + + if (get_cekdata_oid(cekoid, cmkoid, true)) + ereport(ERROR, + errcode(ERRCODE_DUPLICATE_OBJECT), + errmsg("column encryption key \"%s\" already has data for master key \"%s\"", + NameListToString(stmt->cekname), get_cmk_name(cmkoid, false))); + insert_cekdata_record(cekoid, cmkoid, alg, encval); + } + + InvokeObjectPostAlterHook(ColumnEncKeyRelationId, cekoid, 0); + ObjectAddressSet(address, ColumnEncKeyRelationId, cekoid); + + return address; +} + +ObjectAddress +CreateCMK(ParseState *pstate, DefineStmt *stmt) +{ + Oid namespaceId; + char *cmknamestr; + AclResult aclresult; + Relation rel; + ObjectAddress myself; + ObjectAddress referenced; + Oid cmkoid; + ListCell *lc; + DefElem *realmEl = NULL; + char *realm; + NameData cmkname; + Datum values[Natts_pg_colmasterkey] = {0}; + bool nulls[Natts_pg_colmasterkey] = {0}; + HeapTuple tup; + + namespaceId = QualifiedNameGetCreationNamespace(stmt->defnames, &cmknamestr); + + aclresult = object_aclcheck(NamespaceRelationId, namespaceId, GetUserId(), ACL_CREATE); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_SCHEMA, get_namespace_name(namespaceId)); + + rel = table_open(ColumnMasterKeyRelationId, RowExclusiveLock); + + if (SearchSysCacheExists2(CMKNAMENSP, PointerGetDatum(cmknamestr), ObjectIdGetDatum(namespaceId))) + ereport(ERROR, + errcode(ERRCODE_DUPLICATE_OBJECT), + errmsg("column master key \"%s\" already exists", cmknamestr)); + + foreach(lc, stmt->definition) + { + DefElem *defel = lfirst_node(DefElem, lc); + DefElem **defelp; + + if (strcmp(defel->defname, "realm") == 0) + defelp = &realmEl; + else + { + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("column master key attribute \"%s\" not recognized", + defel->defname), + parser_errposition(pstate, defel->location))); + } + if (*defelp != NULL) + errorConflictingDefElem(defel, pstate); + *defelp = defel; + } + + if (realmEl) + realm = defGetString(realmEl); + else + realm = ""; + + cmkoid = GetNewOidWithIndex(rel, ColumnMasterKeyOidIndexId, Anum_pg_colmasterkey_oid); + namestrcpy(&cmkname, cmknamestr); + values[Anum_pg_colmasterkey_oid - 1] = ObjectIdGetDatum(cmkoid); + values[Anum_pg_colmasterkey_cmkname - 1] = NameGetDatum(&cmkname); + values[Anum_pg_colmasterkey_cmknamespace - 1] = ObjectIdGetDatum(namespaceId); + values[Anum_pg_colmasterkey_cmkowner - 1] = ObjectIdGetDatum(GetUserId()); + values[Anum_pg_colmasterkey_cmkrealm - 1] = CStringGetTextDatum(realm); + nulls[Anum_pg_colmasterkey_cmkacl - 1] = true; + + tup = heap_form_tuple(RelationGetDescr(rel), values, nulls); + CatalogTupleInsert(rel, tup); + heap_freetuple(tup); + + ObjectAddressSet(myself, ColumnMasterKeyRelationId, cmkoid); + + ObjectAddressSet(referenced, NamespaceRelationId, namespaceId); + recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL); + + recordDependencyOnOwner(ColumnMasterKeyRelationId, cmkoid, GetUserId()); + + table_close(rel, RowExclusiveLock); + + InvokeObjectPostCreateHook(ColumnMasterKeyRelationId, cmkoid, 0); + + return myself; +} + +ObjectAddress +AlterColumnMasterKey(ParseState *pstate, AlterColumnMasterKeyStmt *stmt) +{ + Oid cmkoid; + Relation rel; + HeapTuple tup; + HeapTuple newtup; + ObjectAddress address; + ListCell *lc; + DefElem *realmEl = NULL; + Datum values[Natts_pg_colmasterkey] = {0}; + bool nulls[Natts_pg_colmasterkey] = {0}; + bool replaces[Natts_pg_colmasterkey] = {0}; + + cmkoid = get_cmk_oid(stmt->cmkname, false); + + rel = table_open(ColumnMasterKeyRelationId, RowExclusiveLock); + + tup = SearchSysCache1(CMKOID, ObjectIdGetDatum(cmkoid)); + + if (!HeapTupleIsValid(tup)) + elog(ERROR, "cache lookup failed for column master key %u", cmkoid); + + if (!object_ownercheck(ColumnMasterKeyRelationId, cmkoid, GetUserId())) + aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_CMK, NameListToString(stmt->cmkname)); + + foreach(lc, stmt->definition) + { + DefElem *defel = lfirst_node(DefElem, lc); + DefElem **defelp; + + if (strcmp(defel->defname, "realm") == 0) + defelp = &realmEl; + else + { + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("column master key attribute \"%s\" not recognized", + defel->defname), + parser_errposition(pstate, defel->location))); + } + if (*defelp != NULL) + errorConflictingDefElem(defel, pstate); + *defelp = defel; + } + + if (realmEl) + { + values[Anum_pg_colmasterkey_cmkrealm - 1] = CStringGetTextDatum(defGetString(realmEl)); + replaces[Anum_pg_colmasterkey_cmkrealm - 1] = true; + } + + newtup = heap_modify_tuple(tup, RelationGetDescr(rel), values, nulls, replaces); + + CatalogTupleUpdate(rel, &tup->t_self, newtup); + + InvokeObjectPostAlterHook(ColumnMasterKeyRelationId, cmkoid, 0); + + ObjectAddressSet(address, ColumnMasterKeyRelationId, cmkoid); + + heap_freetuple(newtup); + ReleaseSysCache(tup); + + table_close(rel, RowExclusiveLock); + + return address; +} diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c index c5df96e374a..9f11a26ff5f 100644 --- a/src/backend/commands/createas.c +++ b/src/backend/commands/createas.c @@ -45,6 +45,7 @@ #include "utils/rel.h" #include "utils/rls.h" #include "utils/snapmgr.h" +#include "utils/syscache.h" typedef struct { @@ -211,6 +212,24 @@ create_ctas_nodata(List *tlist, IntoClause *into) format_type_be(col->typeName->typeOid)), errhint("Use the COLLATE clause to set the collation explicitly."))); + if (type_is_encrypted(exprType((Node *) tle->expr))) + { + HeapTuple tp; + + if (!tle->resorigtbl || !tle->resorigcol) + ereport(ERROR, + errcode(ERRCODE_INVALID_OBJECT_DEFINITION), + errmsg("underlying table and column could not be determined for encrypted table column")); + + tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(tle->resorigtbl), Int16GetDatum(tle->resorigcol)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for attribute %d of relation %u", tle->resorigcol, tle->resorigtbl); + col->typeName = makeTypeNameFromOid(DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypid)), + DatumGetInt32(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypmod))); + col->encryption = makeColumnEncryption(tp); + ReleaseSysCache(tp); + } + attrList = lappend(attrList, col); } } @@ -520,6 +539,17 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo) format_type_be(col->typeName->typeOid)), errhint("Use the COLLATE clause to set the collation explicitly."))); + if (type_is_encrypted(attribute->atttypid)) + { + /* + * We don't have the required information available here, so + * prevent it for now. + */ + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("encrypted columns not yet implemented for this command"))); + } + attrList = lappend(attrList, col); } diff --git a/src/backend/commands/discard.c b/src/backend/commands/discard.c index 92d983ac748..da2e54d3548 100644 --- a/src/backend/commands/discard.c +++ b/src/backend/commands/discard.c @@ -13,6 +13,7 @@ */ #include "postgres.h" +#include "access/printtup.h" #include "access/xact.h" #include "catalog/namespace.h" #include "commands/async.h" @@ -25,7 +26,7 @@ static void DiscardAll(bool isTopLevel); /* - * DISCARD { ALL | SEQUENCES | TEMP | PLANS } + * DISCARD */ void DiscardCommand(DiscardStmt *stmt, bool isTopLevel) @@ -36,6 +37,10 @@ DiscardCommand(DiscardStmt *stmt, bool isTopLevel) DiscardAll(isTopLevel); break; + case DISCARD_COLUMN_ENCRYPTION_KEYS: + DiscardColumnEncryptionKeys(); + break; + case DISCARD_PLANS: ResetPlanCache(); break; @@ -75,4 +80,5 @@ DiscardAll(bool isTopLevel) ResetPlanCache(); ResetTempTableNamespace(); ResetSequenceCaches(); + DiscardColumnEncryptionKeys(); } diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c index 85eec7e3947..0205c03b132 100644 --- a/src/backend/commands/dropcmds.c +++ b/src/backend/commands/dropcmds.c @@ -271,6 +271,20 @@ does_not_exist_skipping(ObjectType objtype, Node *object) name = NameListToString(castNode(List, object)); } break; + case OBJECT_CEK: + if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name)) + { + msg = gettext_noop("column encryption key \"%s\" does not exist, skipping"); + name = NameListToString(castNode(List, object)); + } + break; + case OBJECT_CMK: + if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name)) + { + msg = gettext_noop("column master key \"%s\" does not exist, skipping"); + name = NameListToString(castNode(List, object)); + } + break; case OBJECT_CONVERSION: if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name)) { @@ -499,6 +513,7 @@ does_not_exist_skipping(ObjectType objtype, Node *object) case OBJECT_AMOP: case OBJECT_AMPROC: case OBJECT_ATTRIBUTE: + case OBJECT_CEKDATA: case OBJECT_DEFAULT: case OBJECT_DEFACL: case OBJECT_DOMCONSTRAINT: diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c index 0d3214df9ca..605299e564d 100644 --- a/src/backend/commands/event_trigger.c +++ b/src/backend/commands/event_trigger.c @@ -2163,6 +2163,9 @@ stringify_grant_objtype(ObjectType objtype) case OBJECT_AMPROC: case OBJECT_ATTRIBUTE: case OBJECT_CAST: + case OBJECT_CEK: + case OBJECT_CEKDATA: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_DEFAULT: @@ -2246,6 +2249,9 @@ stringify_adefprivs_objtype(ObjectType objtype) case OBJECT_AMPROC: case OBJECT_ATTRIBUTE: case OBJECT_CAST: + case OBJECT_CEK: + case OBJECT_CEKDATA: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_DEFAULT: diff --git a/src/backend/commands/meson.build b/src/backend/commands/meson.build index 7549be5dc3b..008ed999daf 100644 --- a/src/backend/commands/meson.build +++ b/src/backend/commands/meson.build @@ -7,6 +7,7 @@ backend_sources += files( 'analyze.c', 'async.c', 'cluster.c', + 'colenccmds.c', 'collationcmds.c', 'comment.c', 'constraint.c', diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c index 5607273bf9f..feecb704e50 100644 --- a/src/backend/commands/seclabel.c +++ b/src/backend/commands/seclabel.c @@ -66,6 +66,9 @@ SecLabelSupportsObjectType(ObjectType objtype) case OBJECT_AMPROC: case OBJECT_ATTRIBUTE: case OBJECT_CAST: + case OBJECT_CEK: + case OBJECT_CEKDATA: + case OBJECT_CMK: case OBJECT_COLLATION: case OBJECT_CONVERSION: case OBJECT_DEFAULT: diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 8a98a0af482..ede9d335be0 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -36,6 +36,7 @@ #include "catalog/partition.h" #include "catalog/pg_am.h" #include "catalog/pg_attrdef.h" +#include "catalog/pg_colenckey.h" #include "catalog/pg_collation.h" #include "catalog/pg_constraint.h" #include "catalog/pg_depend.h" @@ -64,6 +65,7 @@ #include "commands/typecmds.h" #include "commands/user.h" #include "commands/vacuum.h" +#include "common/colenc.h" #include "executor/executor.h" #include "foreign/fdwapi.h" #include "foreign/foreign.h" @@ -669,6 +671,8 @@ static List *GetParentedForeignKeyRefs(Relation partition); static void ATDetachCheckNoForeignKeyRefs(Relation partition); static char GetAttributeCompression(Oid atttypid, const char *compression); static char GetAttributeStorage(Oid atttypid, const char *storagemode); +static void GetColumnEncryption(ParseState *pstate, const List *coldefencryption, + FormData_pg_attribute *attr, FormExtraData_pg_attribute *attr_extra); static void ATExecSplitPartition(List **wqueue, AlteredTableInfo *tab, Relation rel, PartitionCmd *cmd, @@ -705,6 +709,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, Oid tablespaceId; Relation rel; TupleDesc descriptor; + FormExtraData_pg_attribute *desc_extra; List *inheritOids; List *old_constraints; List *old_notnulls; @@ -937,7 +942,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, * not default values, NOT NULL or CHECK constraints; we handle those * below. */ - descriptor = BuildDescForRelation(stmt->tableElts); + descriptor = BuildDescForRelation(stmt->tableElts, &desc_extra); /* * Find columns with default values and prepare for insertion of the @@ -1010,6 +1015,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, ownerId, accessMethodId, descriptor, + desc_extra, list_concat(cookedDefaults, old_constraints), relkind, @@ -1315,12 +1321,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, * Note: tdtypeid will need to be filled in later on. */ TupleDesc -BuildDescForRelation(const List *columns) +BuildDescForRelation(const List *columns, FormExtraData_pg_attribute **tupdesc_extra_p) { int natts; AttrNumber attnum; ListCell *l; TupleDesc desc; + FormExtraData_pg_attribute *desc_extra; bool has_not_null; char *attname; Oid atttypid; @@ -1333,6 +1340,7 @@ BuildDescForRelation(const List *columns) */ natts = list_length(columns); desc = CreateTemplateTupleDesc(natts); + desc_extra = palloc_array(FormExtraData_pg_attribute, natts); has_not_null = false; attnum = 0; @@ -1341,7 +1349,8 @@ BuildDescForRelation(const List *columns) { ColumnDef *entry = lfirst(l); AclResult aclresult; - Form_pg_attribute att; + FormData_pg_attribute *att; + FormExtraData_pg_attribute *att_extra; /* * for each entry in the list, get the name and type information from @@ -1373,10 +1382,17 @@ BuildDescForRelation(const List *columns) TupleDescInitEntry(desc, attnum, attname, atttypid, atttypmod, attdim); att = TupleDescAttr(desc, attnum - 1); + att_extra = &desc_extra[attnum - 1]; /* Override TupleDescInitEntry's settings as requested */ TupleDescInitEntryCollation(desc, attnum, attcollation); + att_extra->attstattarget.isnull = true; + att_extra->attoptions.isnull = true; + att_extra->attcek.isnull = true; + att_extra->attusertypid.isnull = true; + att_extra->attusertypmod.isnull = true; + /* Fill in additional stuff not handled by TupleDescInitEntry */ att->attnotnull = entry->is_not_null; has_not_null |= entry->is_not_null; @@ -1389,6 +1405,15 @@ BuildDescForRelation(const List *columns) att->attstorage = entry->storage; else if (entry->storage_name) att->attstorage = GetAttributeStorage(att->atttypid, entry->storage_name); + + if (entry->encryption) + { + GetColumnEncryption(NULL, entry->encryption, att, att_extra); + Assert(!att_extra->attcek.isnull); + aclresult = object_aclcheck(ColumnEncKeyRelationId, DatumGetObjectId(att_extra->attcek.value), GetUserId(), ACL_USAGE); + if (aclresult != ACLCHECK_OK) + aclcheck_error(aclresult, OBJECT_CEK, get_cek_name(DatumGetObjectId(att_extra->attcek.value), false)); + } } if (has_not_null) @@ -1409,6 +1434,7 @@ BuildDescForRelation(const List *columns) desc->constr = NULL; } + *tupdesc_extra_p = desc_extra; return desc; } @@ -2742,6 +2768,18 @@ MergeAttributes(List *columns, const List *supers, char relpersistence, */ newdef = makeColumnDef(attributeName, attribute->atttypid, attribute->atttypmod, attribute->attcollation); + if (type_is_encrypted(attribute->atttypid)) + { + HeapTuple tp; + + tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(attribute->attrelid), Int16GetDatum(attribute->attnum)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for attribute %d of relation %u", attribute->attnum, attribute->attrelid); + newdef->typeName = makeTypeNameFromOid(DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypid)), + DatumGetInt32(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypmod))); + newdef->encryption = makeColumnEncryption(tp); + ReleaseSysCache(tp); + } newdef->storage = attribute->attstorage; newdef->generated = attribute->attgenerated; if (CompressionMethodIsValid(attribute->attcompression)) @@ -3300,6 +3338,32 @@ MergeChildAttribute(List *inh_columns, int exist_attno, int newcol_attno, const errdetail("%s versus %s", inhdef->compression, newdef->compression))); } + /* + * Check encryption parameter. All parents and children must have the + * same encryption settings for a column. + */ + if ((inhdef->encryption && !newdef->encryption) || (!inhdef->encryption && newdef->encryption)) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("column \"%s\" has an encryption specification conflict", + attributeName))); + else if (inhdef->encryption && newdef->encryption) + { + FormData_pg_attribute inha, + newa; + FormExtraData_pg_attribute inhax, + newax; + + GetColumnEncryption(NULL, inhdef->encryption, &inha, &inhax); + GetColumnEncryption(NULL, newdef->encryption, &newa, &newax); + + if (inha.atttypid != newa.atttypid || inha.atttypmod != newa.atttypmod || inhax.attcek.value != newax.attcek.value) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("column \"%s\" has an encryption specification conflict", + attributeName))); + } + /* * Merge of not-null constraints = OR 'em together */ @@ -3394,6 +3458,29 @@ MergeInheritedAttribute(List *inh_columns, attributeName))); prevdef = list_nth_node(ColumnDef, inh_columns, exist_attno - 1); + /* + * Check encryption parameter. All parents must have the same encryption + * settings for a column. + */ + if ((prevdef->encryption && !newdef->encryption) || (!prevdef->encryption && newdef->encryption)) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("column \"%s\" has an encryption specification conflict", + attributeName))); + else if (prevdef->encryption && newdef->encryption) + { + /* + * Merging the encryption properties of two encrypted parent columns + * is not yet implemented. Right now, this would confuse the checks + * of the type etc. below (we must check the physical and the real + * types against each other, respectively), which might require a + * larger restructuring. For now, just give up here. + */ + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("multiple inheritance of encrypted columns is not implemented"))); + } + /* * Must have the same type and typmod */ @@ -7113,6 +7200,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel, AlterTableCmd *childcmd; ObjectAddress address; TupleDesc tupdesc; + FormExtraData_pg_attribute *tupdesc_extra; /* since this function recurses, it could be driven to stack overflow */ check_stack_depth(); @@ -7250,7 +7338,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel, /* * Construct new attribute's pg_attribute entry. */ - tupdesc = BuildDescForRelation(list_make1(colDef)); + tupdesc = BuildDescForRelation(list_make1(colDef), &tupdesc_extra); attribute = TupleDescAttr(tupdesc, 0); @@ -7260,9 +7348,10 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel, /* make sure datatype is legal for a column */ CheckAttributeType(NameStr(attribute->attname), attribute->atttypid, attribute->attcollation, list_make1_oid(rel->rd_rel->reltype), - 0); + !tupdesc_extra[0].attcek.isnull ? CHKATYPE_ENCRYPTED : 0); + - InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL); + InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, tupdesc_extra, NULL); table_close(attrdesc, RowExclusiveLock); @@ -20887,6 +20976,144 @@ GetAttributeStorage(Oid atttypid, const char *storagemode) return cstorage; } +/* + * resolve column encryption specification + */ +static void +GetColumnEncryption(ParseState *pstate, const List *coldefencryption, + FormData_pg_attribute *attr, FormExtraData_pg_attribute *attr_extra) +{ + ListCell *lc; + DefElem *cekEl = NULL; + DefElem *encdetEl = NULL; + DefElem *algEl = NULL; + Oid cekoid; + bool encdet; + int alg; + + foreach(lc, coldefencryption) + { + DefElem *defel = lfirst_node(DefElem, lc); + DefElem **defelp; + + if (strcmp(defel->defname, "column_encryption_key") == 0) + defelp = &cekEl; + else if (strcmp(defel->defname, "encryption_type") == 0) + defelp = &encdetEl; + else if (strcmp(defel->defname, "algorithm") == 0) + defelp = &algEl; + else + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("unrecognized column encryption parameter: %s", defel->defname), + parser_errposition(pstate, defel->location)); + if (*defelp != NULL) + errorConflictingDefElem(defel, pstate); + *defelp = defel; + } + + if (cekEl) + { + List *val = defGetQualifiedName(cekEl); + + cekoid = get_cek_oid(val, false); + } + else + ereport(ERROR, + errcode(ERRCODE_INVALID_COLUMN_DEFINITION), + errmsg("column encryption key must be specified")); + + if (encdetEl) + { + char *val = strVal(linitial(castNode(TypeName, encdetEl->arg)->names)); + + if (strcmp(val, "deterministic") == 0) + encdet = true; + else if (strcmp(val, "randomized") == 0) + encdet = false; + else + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("unrecognized encryption type: %s", val), + parser_errposition(pstate, encdetEl->location)); + } + else + encdet = false; + + if (algEl) + { + char *val = strVal(algEl->arg); + + alg = get_cekalg_num(val); + + if (!alg) + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("unrecognized encryption algorithm: %s", val), + parser_errposition(pstate, algEl->location)); + } + else + alg = PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256; + + attr_extra->attcek.value = ObjectIdGetDatum(cekoid); + attr_extra->attusertypid.value = ObjectIdGetDatum(attr->atttypid); + attr_extra->attusertypmod.value = Int32GetDatum(attr->atttypmod); + attr_extra->attcek.isnull = attr_extra->attusertypid.isnull = attr_extra->attusertypmod.isnull = false; + + /* override physical type */ + if (encdet) + attr->atttypid = PG_ENCRYPTED_DETOID; + else + attr->atttypid = PG_ENCRYPTED_RNDOID; + get_typlenbyvalalign(attr->atttypid, + &attr->attlen, &attr->attbyval, &attr->attalign); + attr->attstorage = get_typstorage(attr->atttypid); + attr->attcollation = InvalidOid; + + attr->atttypmod = alg; +} + +/* + * Construct input to GetColumnEncryption(), for synthesizing a column + * definition. + */ +List * +makeColumnEncryption(HeapTuple attrtup) +{ + List *result; + HeapTuple cektup; + Form_pg_colenckey cekform; + char *nspname; + Form_pg_attribute attr; + Oid attcek; + + attr = (Form_pg_attribute) GETSTRUCT(attrtup); + + attcek = DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, attrtup, Anum_pg_attribute_attcek)); + + cektup = SearchSysCache1(CEKOID, ObjectIdGetDatum(attcek)); + if (!HeapTupleIsValid(cektup)) + elog(ERROR, "cache lookup failed for column encryption key %u", attcek); + + cekform = (Form_pg_colenckey) GETSTRUCT(cektup); + nspname = get_namespace_name(cekform->ceknamespace); + + result = list_make3(makeDefElem("column_encryption_key", + (Node *) list_make2(makeString(nspname), makeString(pstrdup(NameStr(cekform->cekname)))), + -1), + makeDefElem("encryption_type", + (Node *) makeTypeName(attr->atttypid == PG_ENCRYPTED_DETOID ? + "deterministic" : "randomized"), + -1), + makeDefElem("algorithm", + (Node *) makeString(pstrdup(get_cekalg_name(attr->atttypmod))), + -1)); + + ReleaseSysCache(cektup); + + return result; +} + /* * Struct with context of new partition for insert rows from splited partition */ diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c index 01151ca2b5a..19362e34168 100644 --- a/src/backend/commands/variable.c +++ b/src/backend/commands/variable.c @@ -25,6 +25,7 @@ #include "access/xlogprefetcher.h" #include "catalog/pg_authid.h" #include "common/string.h" +#include "libpq/libpq-be.h" #include "mb/pg_wchar.h" #include "miscadmin.h" #include "postmaster/postmaster.h" @@ -706,7 +707,11 @@ check_client_encoding(char **newval, void **extra, GucSource source) */ if (PrepareClientEncoding(encoding) < 0) { - if (IsTransactionState()) + if (MyProcPort->column_encryption_enabled) + GUC_check_errdetail("Conversion between %s and %s is not possible when column encryption is enabled.", + canonical_name, + GetDatabaseEncodingName()); + else if (IsTransactionState()) { /* Must be a genuine no-such-conversion problem */ GUC_check_errcode(ERRCODE_FEATURE_NOT_SUPPORTED); diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c index fdad8338324..86f006d7c27 100644 --- a/src/backend/commands/view.c +++ b/src/backend/commands/view.c @@ -29,6 +29,7 @@ #include "utils/builtins.h" #include "utils/lsyscache.h" #include "utils/rel.h" +#include "utils/syscache.h" static void checkViewColumns(TupleDesc newdesc, TupleDesc olddesc); @@ -83,6 +84,24 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace, else Assert(!OidIsValid(def->collOid)); + if (type_is_encrypted(exprType((Node *) tle->expr))) + { + HeapTuple tp; + + if (!tle->resorigtbl || !tle->resorigcol) + ereport(ERROR, + errcode(ERRCODE_INVALID_OBJECT_DEFINITION), + errmsg("underlying table and column could not be determined for encrypted view column")); + + tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(tle->resorigtbl), Int16GetDatum(tle->resorigcol)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for attribute %d of relation %u", tle->resorigcol, tle->resorigtbl); + def->typeName = makeTypeNameFromOid(DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypid)), + DatumGetInt32(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypmod))); + def->encryption = makeColumnEncryption(tp); + ReleaseSysCache(tp); + } + attrList = lappend(attrList, def); } } @@ -100,6 +119,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace, { Relation rel; TupleDesc descriptor; + FormExtraData_pg_attribute *desc_extra; List *atcmds = NIL; AlterTableCmd *atcmd; ObjectAddress address; @@ -129,7 +149,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace, * verify that the old column list is an initial prefix of the new * column list. */ - descriptor = BuildDescForRelation(attrList); + descriptor = BuildDescForRelation(attrList, &desc_extra); checkViewColumns(descriptor, rel->rd_att); /* diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c index e1df1894b69..18b9bfc2deb 100644 --- a/src/backend/nodes/nodeFuncs.c +++ b/src/backend/nodes/nodeFuncs.c @@ -4512,6 +4512,8 @@ raw_expression_tree_walker_impl(Node *node, if (WALK(coldef->typeName)) return true; + if (WALK(coldef->encryption)) + return true; if (WALK(coldef->raw_default)) return true; if (WALK(coldef->collClause)) diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0523f7e891e..2b5537d020f 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -283,6 +283,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %type stmt toplevel_stmt schema_stmt routine_body_stmt AlterEventTrigStmt AlterCollationStmt + AlterColumnEncryptionKeyStmt AlterColumnMasterKeyStmt AlterDatabaseStmt AlterDatabaseSetStmt AlterDomainStmt AlterEnumStmt AlterFdwStmt AlterForeignServerStmt AlterGroupStmt AlterObjectDependsStmt AlterObjectSchemaStmt AlterOwnerStmt @@ -424,6 +425,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %type parse_toplevel stmtmulti routine_body_stmt_list OptTableElementList TableElementList OptInherit definition + list_of_definitions OptTypedTableElementList TypedTableElementList reloptions opt_reloptions OptWith opt_definition func_args func_args_list @@ -599,6 +601,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %type TableConstraint TableLikeClause %type TableLikeOptionList TableLikeOption %type column_compression opt_column_compression column_storage opt_column_storage +%type opt_column_encryption %type ColQualList %type ColConstraint ColConstraintElem ConstraintAttr %type key_match @@ -727,7 +730,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); DETACH DICTIONARY DISABLE_P DISCARD DISTINCT DO DOCUMENT_P DOMAIN_P DOUBLE_P DROP - EACH ELSE EMPTY_P ENABLE_P ENCODING ENCRYPTED END_P ENUM_P ERROR_P ESCAPE + EACH ELSE EMPTY_P ENABLE_P ENCODING ENCRYPTED ENCRYPTION END_P ENUM_P ERROR_P ESCAPE EVENT EXCEPT EXCLUDE EXCLUDING EXCLUSIVE EXECUTE EXISTS EXPLAIN EXPRESSION EXTENSION EXTERNAL EXTRACT @@ -752,7 +755,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); LEADING LEAKPROOF LEAST LEFT LEVEL LIKE LIMIT LISTEN LOAD LOCAL LOCALTIME LOCALTIMESTAMP LOCATION LOCK_P LOCKED LOGGED - MAPPING MATCH MATCHED MATERIALIZED MAXVALUE MERGE MERGE_ACTION METHOD + MAPPING MASTER MATCH MATCHED MATERIALIZED MAXVALUE MERGE MERGE_ACTION METHOD MINUTE_P MINVALUE MODE MONTH_P MOVE NAME_P NAMES NATIONAL NATURAL NCHAR NESTED NEW NEXT NFC NFD NFKC NFKD NO @@ -1005,6 +1008,8 @@ toplevel_stmt: stmt: AlterEventTrigStmt | AlterCollationStmt + | AlterColumnEncryptionKeyStmt + | AlterColumnMasterKeyStmt | AlterDatabaseStmt | AlterDatabaseSetStmt | AlterDefaultPrivilegesStmt @@ -2031,7 +2036,7 @@ CheckPointStmt: /***************************************************************************** * - * DISCARD { ALL | TEMP | PLANS | SEQUENCES } + * DISCARD * *****************************************************************************/ @@ -2057,6 +2062,13 @@ DiscardStmt: n->target = DISCARD_TEMP; $$ = (Node *) n; } + | DISCARD COLUMN ENCRYPTION KEYS + { + DiscardStmt *n = makeNode(DiscardStmt); + + n->target = DISCARD_COLUMN_ENCRYPTION_KEYS; + $$ = (Node *) n; + } | DISCARD PLANS { DiscardStmt *n = makeNode(DiscardStmt); @@ -3821,14 +3833,15 @@ TypedTableElement: | TableConstraint { $$ = $1; } ; -columnDef: ColId Typename opt_column_storage opt_column_compression create_generic_options ColQualList +columnDef: ColId Typename opt_column_encryption opt_column_storage opt_column_compression create_generic_options ColQualList { ColumnDef *n = makeNode(ColumnDef); n->colname = $1; n->typeName = $2; - n->storage_name = $3; - n->compression = $4; + n->encryption = $3; + n->storage_name = $4; + n->compression = $5; n->inhcount = 0; n->is_local = true; n->is_not_null = false; @@ -3837,8 +3850,8 @@ columnDef: ColId Typename opt_column_storage opt_column_compression create_gener n->raw_default = NULL; n->cooked_default = NULL; n->collOid = InvalidOid; - n->fdwoptions = $5; - SplitColQualList($6, &n->constraints, &n->collClause, + n->fdwoptions = $6; + SplitColQualList($7, &n->constraints, &n->collClause, yyscanner); n->location = @1; $$ = (Node *) n; @@ -3895,6 +3908,11 @@ opt_column_compression: | /*EMPTY*/ { $$ = NULL; } ; +opt_column_encryption: + ENCRYPTED WITH '(' def_list ')' { $$ = $4; } + | /*EMPTY*/ { $$ = NULL; } + ; + column_storage: STORAGE ColId { $$ = $2; } | STORAGE DEFAULT { $$ = pstrdup("default"); } @@ -4158,6 +4176,7 @@ TableLikeOption: | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; } | CONSTRAINTS { $$ = CREATE_TABLE_LIKE_CONSTRAINTS; } | DEFAULTS { $$ = CREATE_TABLE_LIKE_DEFAULTS; } + | ENCRYPTED { $$ = CREATE_TABLE_LIKE_ENCRYPTED; } | IDENTITY_P { $$ = CREATE_TABLE_LIKE_IDENTITY; } | GENERATED { $$ = CREATE_TABLE_LIKE_GENERATED; } | INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; } @@ -6435,6 +6454,33 @@ DefineStmt: n->if_not_exists = true; $$ = (Node *) n; } + | CREATE COLUMN ENCRYPTION KEY any_name WITH VALUES list_of_definitions + { + DefineStmt *n = makeNode(DefineStmt); + + n->kind = OBJECT_CEK; + n->defnames = $5; + n->definition = $8; + $$ = (Node *) n; + } + | CREATE COLUMN MASTER KEY any_name + { + DefineStmt *n = makeNode(DefineStmt); + + n->kind = OBJECT_CMK; + n->defnames = $5; + n->definition = NIL; + $$ = (Node *) n; + } + | CREATE COLUMN MASTER KEY any_name WITH definition + { + DefineStmt *n = makeNode(DefineStmt); + + n->kind = OBJECT_CMK; + n->defnames = $5; + n->definition = $7; + $$ = (Node *) n; + } ; definition: '(' def_list ')' { $$ = $2; } @@ -6454,6 +6500,10 @@ def_elem: ColLabel '=' def_arg } ; +list_of_definitions: definition { $$ = list_make1($1); } + | list_of_definitions ',' definition { $$ = lappend($1, $3); } + ; + /* Note: any simple identifier will be returned as a type name! */ def_arg: func_type { $$ = (Node *) $1; } | reserved_keyword { $$ = (Node *) makeString(pstrdup($1)); } @@ -6992,6 +7042,8 @@ object_type_any_name: | INDEX { $$ = OBJECT_INDEX; } | FOREIGN TABLE { $$ = OBJECT_FOREIGN_TABLE; } | COLLATION { $$ = OBJECT_COLLATION; } + | COLUMN ENCRYPTION KEY { $$ = OBJECT_CEK; } + | COLUMN MASTER KEY { $$ = OBJECT_CMK; } | CONVERSION_P { $$ = OBJECT_CONVERSION; } | STATISTICS { $$ = OBJECT_STATISTIC_EXT; } | TEXT_P SEARCH PARSER { $$ = OBJECT_TSPARSER; } @@ -7803,6 +7855,24 @@ privilege_target: n->objs = $2; $$ = n; } + | COLUMN ENCRYPTION KEY any_name_list + { + PrivTarget *n = (PrivTarget *) palloc(sizeof(PrivTarget)); + + n->targtype = ACL_TARGET_OBJECT; + n->objtype = OBJECT_CEK; + n->objs = $4; + $$ = n; + } + | COLUMN MASTER KEY any_name_list + { + PrivTarget *n = (PrivTarget *) palloc(sizeof(PrivTarget)); + + n->targtype = ACL_TARGET_OBJECT; + n->objtype = OBJECT_CMK; + n->objs = $4; + $$ = n; + } | DATABASE name_list { PrivTarget *n = (PrivTarget *) palloc(sizeof(PrivTarget)); @@ -9332,6 +9402,26 @@ RenameStmt: ALTER AGGREGATE aggregate_with_argtypes RENAME TO name n->missing_ok = false; $$ = (Node *) n; } + | ALTER COLUMN ENCRYPTION KEY any_name RENAME TO name + { + RenameStmt *n = makeNode(RenameStmt); + + n->renameType = OBJECT_CEK; + n->object = (Node *) $5; + n->newname = $8; + n->missing_ok = false; + $$ = (Node *) n; + } + | ALTER COLUMN MASTER KEY any_name RENAME TO name + { + RenameStmt *n = makeNode(RenameStmt); + + n->renameType = OBJECT_CMK; + n->object = (Node *) $5; + n->newname = $8; + n->missing_ok = false; + $$ = (Node *) n; + } | ALTER CONVERSION_P any_name RENAME TO name { RenameStmt *n = makeNode(RenameStmt); @@ -10009,6 +10099,26 @@ AlterObjectSchemaStmt: n->missing_ok = false; $$ = (Node *) n; } + | ALTER COLUMN ENCRYPTION KEY any_name SET SCHEMA name + { + AlterObjectSchemaStmt *n = makeNode(AlterObjectSchemaStmt); + + n->objectType = OBJECT_CEK; + n->object = (Node *) $5; + n->newschema = $8; + n->missing_ok = false; + $$ = (Node *) n; + } + | ALTER COLUMN MASTER KEY any_name SET SCHEMA name + { + AlterObjectSchemaStmt *n = makeNode(AlterObjectSchemaStmt); + + n->objectType = OBJECT_CMK; + n->object = (Node *) $5; + n->newschema = $8; + n->missing_ok = false; + $$ = (Node *) n; + } | ALTER CONVERSION_P any_name SET SCHEMA name { AlterObjectSchemaStmt *n = makeNode(AlterObjectSchemaStmt); @@ -10342,6 +10452,24 @@ AlterOwnerStmt: ALTER AGGREGATE aggregate_with_argtypes OWNER TO RoleSpec n->newowner = $6; $$ = (Node *) n; } + | ALTER COLUMN ENCRYPTION KEY any_name OWNER TO RoleSpec + { + AlterOwnerStmt *n = makeNode(AlterOwnerStmt); + + n->objectType = OBJECT_CEK; + n->object = (Node *) $5; + n->newowner = $8; + $$ = (Node *) n; + } + | ALTER COLUMN MASTER KEY any_name OWNER TO RoleSpec + { + AlterOwnerStmt *n = makeNode(AlterOwnerStmt); + + n->objectType = OBJECT_CMK; + n->object = (Node *) $5; + n->newowner = $8; + $$ = (Node *) n; + } | ALTER CONVERSION_P any_name OWNER TO RoleSpec { AlterOwnerStmt *n = makeNode(AlterOwnerStmt); @@ -11512,6 +11640,52 @@ AlterCollationStmt: ALTER COLLATION any_name REFRESH VERSION_P ; +/***************************************************************************** + * + * ALTER COLUMN ENCRYPTION KEY + * + *****************************************************************************/ + +AlterColumnEncryptionKeyStmt: + ALTER COLUMN ENCRYPTION KEY any_name ADD_P VALUE_P definition + { + AlterColumnEncryptionKeyStmt *n = makeNode(AlterColumnEncryptionKeyStmt); + + n->cekname = $5; + n->isDrop = false; + n->definition = $8; + $$ = (Node *) n; + } + | ALTER COLUMN ENCRYPTION KEY any_name DROP VALUE_P definition + { + AlterColumnEncryptionKeyStmt *n = makeNode(AlterColumnEncryptionKeyStmt); + + n->cekname = $5; + n->isDrop = true; + n->definition = $8; + $$ = (Node *) n; + } + ; + + +/***************************************************************************** + * + * ALTER COLUMN MASTER KEY + * + *****************************************************************************/ + +AlterColumnMasterKeyStmt: + ALTER COLUMN MASTER KEY any_name definition + { + AlterColumnMasterKeyStmt *n = makeNode(AlterColumnMasterKeyStmt); + + n->cmkname = $5; + n->definition = $6; + $$ = (Node *) n; + } + ; + + /***************************************************************************** * * ALTER SYSTEM @@ -17639,6 +17813,7 @@ unreserved_keyword: | ENABLE_P | ENCODING | ENCRYPTED + | ENCRYPTION | ENUM_P | ERROR_P | ESCAPE @@ -17707,6 +17882,7 @@ unreserved_keyword: | LOCKED | LOGGED | MAPPING + | MASTER | MATCH | MATCHED | MATERIALIZED @@ -18218,6 +18394,7 @@ bare_label_keyword: | ENABLE_P | ENCODING | ENCRYPTED + | ENCRYPTION | END_P | ENUM_P | ERROR_P @@ -18322,6 +18499,7 @@ bare_label_keyword: | LOCKED | LOGGED | MAPPING + | MASTER | MATCH | MATCHED | MATERIALIZED diff --git a/src/backend/parser/parse_param.c b/src/backend/parser/parse_param.c index dbf1a7dff08..a0b6a905bd3 100644 --- a/src/backend/parser/parse_param.c +++ b/src/backend/parser/parse_param.c @@ -29,6 +29,7 @@ #include "catalog/pg_type.h" #include "nodes/nodeFuncs.h" #include "parser/parse_param.h" +#include "parser/parsetree.h" #include "utils/builtins.h" #include "utils/lsyscache.h" @@ -357,3 +358,159 @@ query_contains_extern_params_walker(Node *node, void *context) return expression_tree_walker(node, query_contains_extern_params_walker, context); } + +/* + * Walk a query tree and find out what tables and columns a parameter is + * associated with. + * + * We need to find 1) parameters written directly into a table column, and 2) + * binary predicates relating a parameter to a table column. + * + * We just need to find Var and Param nodes in appropriate places. We don't + * need to do harder things like looking through casts, since this is used for + * column encryption, and encrypted columns can't be usefully cast to + * anything. + */ + +struct find_param_origs_context +{ + const Query *query; + Oid *param_orig_tbls; + AttrNumber *param_orig_cols; +}; + +static bool +find_param_origs_walker(Node *node, struct find_param_origs_context *context) +{ + if (node == NULL) + return false; + + if (IsA(node, OpExpr) || IsA(node, DistinctExpr) || IsA(node, NullIfExpr)) + { + OpExpr *opexpr = (OpExpr *) node; + + if (list_length(opexpr->args) == 2) + { + Node *lexpr = linitial(opexpr->args); + Node *rexpr = lsecond(opexpr->args); + Var *v = NULL; + Param *p = NULL; + + if (IsA(lexpr, Var) && IsA(rexpr, Param)) + { + v = castNode(Var, lexpr); + p = castNode(Param, rexpr); + } + else if (IsA(rexpr, Var) && IsA(lexpr, Param)) + { + v = castNode(Var, rexpr); + p = castNode(Param, lexpr); + } + + if (v && p) + { + RangeTblEntry *rte; + + rte = rt_fetch(v->varno, context->query->rtable); + if (rte->rtekind == RTE_RELATION) + { + context->param_orig_tbls[p->paramid - 1] = rte->relid; + context->param_orig_cols[p->paramid - 1] = v->varattno; + } + } + } + return false; + } + + /* + * TargetEntry in a query with a result relation + */ + if (IsA(node, TargetEntry) && context->query->resultRelation > 0) + { + TargetEntry *te = (TargetEntry *) node; + RangeTblEntry *resrte; + + resrte = rt_fetch(context->query->resultRelation, context->query->rtable); + if (resrte->rtekind == RTE_RELATION) + { + Expr *expr = te->expr; + + /* + * If it's a RelabelType, look inside. (For encrypted columns, + * this would typically be a typmod adjustment.) + */ + if (IsA(expr, RelabelType)) + expr = castNode(RelabelType, expr)->arg; + + /* + * Param directly in a target list + */ + if (IsA(expr, Param)) + { + Param *p = (Param *) expr; + + context->param_orig_tbls[p->paramid - 1] = resrte->relid; + context->param_orig_cols[p->paramid - 1] = te->resno; + } + + /* + * If it's a Var, check whether it corresponds to a VALUES list + * with top-level parameters. This covers multi-row INSERTS. + */ + else if (IsA(expr, Var)) + { + Var *v = (Var *) expr; + RangeTblEntry *srcrte; + + srcrte = rt_fetch(v->varno, context->query->rtable); + if (srcrte->rtekind == RTE_VALUES) + { + ListCell *lc; + + foreach(lc, srcrte->values_lists) + { + List *values_list = lfirst_node(List, lc); + Expr *value = list_nth(values_list, v->varattno - 1); + + if (IsA(value, RelabelType)) + value = castNode(RelabelType, value)->arg; + + if (IsA(value, Param)) + { + Param *p = (Param *) value; + + context->param_orig_tbls[p->paramid - 1] = resrte->relid; + context->param_orig_cols[p->paramid - 1] = te->resno; + } + } + } + } + } + return false; + } + + if (IsA(node, Query)) + { + return query_tree_walker((Query *) node, find_param_origs_walker, context, 0); + } + + return expression_tree_walker(node, find_param_origs_walker, context); +} + +void +find_param_origs(List *query_list, Oid **param_orig_tbls, AttrNumber **param_orig_cols) +{ + struct find_param_origs_context context; + ListCell *lc; + + context.param_orig_tbls = *param_orig_tbls; + context.param_orig_cols = *param_orig_cols; + + foreach(lc, query_list) + { + Query *query = lfirst_node(Query, lc); + + context.query = query; + query_tree_walker(query, find_param_origs_walker, &context, 0); + } +} diff --git a/src/backend/parser/parse_relation.c b/src/backend/parser/parse_relation.c index 7ca793a369f..0da07f0e68e 100644 --- a/src/backend/parser/parse_relation.c +++ b/src/backend/parser/parse_relation.c @@ -1945,7 +1945,7 @@ addRangeTableEntryForFunction(ParseState *pstate, * coldeflist doesn't represent anything that will be visible to * other sessions. */ - CheckAttributeNamesTypes(tupdesc, RELKIND_COMPOSITE_TYPE, + CheckAttributeNamesTypes(tupdesc, NULL, RELKIND_COMPOSITE_TYPE, CHKATYPE_ANYRECORD); } else diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c index ceba0699050..2680d6d06b6 100644 --- a/src/backend/parser/parse_utilcmd.c +++ b/src/backend/parser/parse_utilcmd.c @@ -1092,6 +1092,20 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla def = makeColumnDef(NameStr(attribute->attname), attribute->atttypid, attribute->atttypmod, attribute->attcollation); + if (type_is_encrypted(attribute->atttypid)) + { + HeapTuple tp; + + tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(attribute->attrelid), Int16GetDatum(attribute->attnum)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for attribute %d of relation %u", attribute->attnum, attribute->attrelid); + def->typeName = makeTypeNameFromOid(DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypid)), + DatumGetInt32(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypmod))); + if (table_like_clause->options & CREATE_TABLE_LIKE_ENCRYPTED) + def->encryption = makeColumnEncryption(tp); + ReleaseSysCache(tp); + } + /* * For constraints, ONLY the not-null constraint is inherited by the * new column definition per SQL99; however we cannot do that diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c index ee73d01e16c..33031795bd2 100644 --- a/src/backend/tcop/backend_startup.c +++ b/src/backend/tcop/backend_startup.c @@ -743,12 +743,27 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done) valptr), errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\"."))); } + else if (strcmp(nameptr, "_pq_.column_encryption") == 0) + { + /* + * Right now, the only accepted value is "1". This gives room + * to expand this into a version number, for example. + */ + if (strcmp(valptr, "1") == 0) + port->column_encryption_enabled = true; + else + ereport(FATAL, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("invalid value for parameter \"%s\": \"%s\"", + "column_encryption", + valptr), + errhint("Valid values are: 1."))); + } else if (strncmp(nameptr, "_pq_.", 5) == 0) { /* * Any option beginning with _pq_. is reserved for use as a - * protocol-level option, but at present no such options are - * defined. + * protocol-level option. */ unrecognized_protocol_options = lappend(unrecognized_protocol_options, pstrdup(nameptr)); diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 76f48b13d20..4bef29e0c44 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -49,6 +49,7 @@ #include "nodes/print.h" #include "optimizer/optimizer.h" #include "parser/analyze.h" +#include "parser/parse_param.h" #include "parser/parser.h" #include "pg_getopt.h" #include "pg_trace.h" @@ -77,6 +78,7 @@ #include "utils/memutils.h" #include "utils/ps_status.h" #include "utils/snapmgr.h" +#include "utils/syscache.h" #include "utils/timeout.h" #include "utils/timestamp.h" @@ -1845,6 +1847,16 @@ exec_bind_message(StringInfo input_message) else pformat = 0; /* default = text */ + if (type_is_encrypted(ptype)) + { + if (pformat & 0xF0) + pformat &= ~0xF0; + else + ereport(ERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("parameter $%d corresponds to an encrypted column, but the parameter value was not encrypted", paramno + 1))); + } + if (pformat == 0) /* text mode */ { Oid typinput; @@ -2597,6 +2609,8 @@ static void exec_describe_statement_message(const char *stmt_name) { CachedPlanSource *psrc; + Oid *param_orig_tbls; + AttrNumber *param_orig_cols; /* * Start up a transaction command. (Note that this will normally change @@ -2655,11 +2669,61 @@ exec_describe_statement_message(const char *stmt_name) * message type */ pq_sendint16(&row_description_buf, psrc->num_params); + /* + * If column encryption is enabled, find the associated tables and columns + * for any parameters, so that we can determine encryption information for + * them. + */ + if (MyProcPort->column_encryption_enabled && psrc->num_params) + { + param_orig_tbls = palloc0_array(Oid, psrc->num_params); + param_orig_cols = palloc0_array(AttrNumber, psrc->num_params); + + RevalidateCachedQuery(psrc, NULL); + find_param_origs(psrc->query_list, ¶m_orig_tbls, ¶m_orig_cols); + } + for (int i = 0; i < psrc->num_params; i++) { Oid ptype = psrc->param_types[i]; + Oid pcekid = InvalidOid; + int pcekalg = 0; + int16 pflags = 0; + + if (MyProcPort->column_encryption_enabled && type_is_encrypted(ptype)) + { + Oid porigtbl = param_orig_tbls[i]; + AttrNumber porigcol = param_orig_cols[i]; + HeapTuple tp; + Form_pg_attribute orig_att; + + if (porigtbl == InvalidOid || porigcol == InvalidAttrNumber) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("parameter $%d corresponds to an encrypted column, but an underlying table and column could not be determined", i + 1))); + + tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(porigtbl), Int16GetDatum(porigcol)); + if (!HeapTupleIsValid(tp)) + elog(ERROR, "cache lookup failed for attribute %d of relation %u", porigcol, porigtbl); + orig_att = (Form_pg_attribute) GETSTRUCT(tp); + ptype = DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attusertypid)); + pcekid = DatumGetObjectId(SysCacheGetAttrNotNull(ATTNUM, tp, Anum_pg_attribute_attcek)); + pcekalg = orig_att->atttypmod; + ReleaseSysCache(tp); + + if (psrc->param_types[i] == PG_ENCRYPTED_DETOID) + pflags |= 0x0001; + + MaybeSendColumnEncryptionKeyMessage(pcekid); + } pq_sendint32(&row_description_buf, (int) ptype); + if (MyProcPort->column_encryption_enabled) + { + pq_sendint32(&row_description_buf, (int) pcekid); + pq_sendint32(&row_description_buf, pcekalg); + pq_sendint16(&row_description_buf, pflags); + } } pq_endmessage_reuse(&row_description_buf); diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c index c6bb3e45da4..dc3551fa0b9 100644 --- a/src/backend/tcop/utility.c +++ b/src/backend/tcop/utility.c @@ -27,6 +27,7 @@ #include "commands/alter.h" #include "commands/async.h" #include "commands/cluster.h" +#include "commands/colenccmds.h" #include "commands/collationcmds.h" #include "commands/comment.h" #include "commands/conversioncmds.h" @@ -131,6 +132,8 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree) switch (nodeTag(parsetree)) { case T_AlterCollationStmt: + case T_AlterColumnEncryptionKeyStmt: + case T_AlterColumnMasterKeyStmt: case T_AlterDatabaseRefreshCollStmt: case T_AlterDatabaseSetStmt: case T_AlterDatabaseStmt: @@ -1456,6 +1459,14 @@ ProcessUtilitySlow(ParseState *pstate, stmt->definition, &secondaryObject); break; + case OBJECT_CEK: + Assert(stmt->args == NIL); + address = CreateCEK(pstate, stmt); + break; + case OBJECT_CMK: + Assert(stmt->args == NIL); + address = CreateCMK(pstate, stmt); + break; case OBJECT_COLLATION: Assert(stmt->args == NIL); address = DefineCollation(pstate, @@ -1932,6 +1943,14 @@ ProcessUtilitySlow(ParseState *pstate, address = AlterCollation((AlterCollationStmt *) parsetree); break; + case T_AlterColumnEncryptionKeyStmt: + address = AlterColumnEncryptionKey(pstate, (AlterColumnEncryptionKeyStmt *) parsetree); + break; + + case T_AlterColumnMasterKeyStmt: + address = AlterColumnMasterKey(pstate, (AlterColumnMasterKeyStmt *) parsetree); + break; + default: elog(ERROR, "unrecognized node type: %d", (int) nodeTag(parsetree)); @@ -2253,6 +2272,12 @@ AlterObjectTypeCommandTag(ObjectType objtype) case OBJECT_COLUMN: tag = CMDTAG_ALTER_TABLE; break; + case OBJECT_CEK: + tag = CMDTAG_ALTER_COLUMN_ENCRYPTION_KEY; + break; + case OBJECT_CMK: + tag = CMDTAG_ALTER_COLUMN_MASTER_KEY; + break; case OBJECT_CONVERSION: tag = CMDTAG_ALTER_CONVERSION; break; @@ -2668,6 +2693,12 @@ CreateCommandTag(Node *parsetree) case OBJECT_STATISTIC_EXT: tag = CMDTAG_DROP_STATISTICS; break; + case OBJECT_CEK: + tag = CMDTAG_DROP_COLUMN_ENCRYPTION_KEY; + break; + case OBJECT_CMK: + tag = CMDTAG_DROP_COLUMN_MASTER_KEY; + break; default: tag = CMDTAG_UNKNOWN; } @@ -2788,6 +2819,12 @@ CreateCommandTag(Node *parsetree) case OBJECT_COLLATION: tag = CMDTAG_CREATE_COLLATION; break; + case OBJECT_CEK: + tag = CMDTAG_CREATE_COLUMN_ENCRYPTION_KEY; + break; + case OBJECT_CMK: + tag = CMDTAG_CREATE_COLUMN_MASTER_KEY; + break; case OBJECT_ACCESS_METHOD: tag = CMDTAG_CREATE_ACCESS_METHOD; break; @@ -2945,6 +2982,9 @@ CreateCommandTag(Node *parsetree) case DISCARD_ALL: tag = CMDTAG_DISCARD_ALL; break; + case DISCARD_COLUMN_ENCRYPTION_KEYS: + tag = CMDTAG_DISCARD_COLUMN_ENCRYPTION_KEYS; + break; case DISCARD_PLANS: tag = CMDTAG_DISCARD_PLANS; break; @@ -3091,6 +3131,14 @@ CreateCommandTag(Node *parsetree) tag = CMDTAG_ALTER_COLLATION; break; + case T_AlterColumnEncryptionKeyStmt: + tag = CMDTAG_ALTER_COLUMN_ENCRYPTION_KEY; + break; + + case T_AlterColumnMasterKeyStmt: + tag = CMDTAG_ALTER_COLUMN_MASTER_KEY; + break; + case T_PrepareStmt: tag = CMDTAG_PREPARE; break; @@ -3716,6 +3764,14 @@ GetCommandLogLevel(Node *parsetree) lev = LOGSTMT_DDL; break; + case T_AlterColumnEncryptionKeyStmt: + lev = LOGSTMT_DDL; + break; + + case T_AlterColumnMasterKeyStmt: + lev = LOGSTMT_DDL; + break; + /* already-planned queries */ case T_PlannedStmt: { diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c index dc10b4a4839..07bcdea79aa 100644 --- a/src/backend/utils/adt/acl.c +++ b/src/backend/utils/adt/acl.c @@ -22,6 +22,8 @@ #include "catalog/pg_auth_members.h" #include "catalog/pg_authid.h" #include "catalog/pg_class.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_database.h" #include "catalog/pg_foreign_data_wrapper.h" #include "catalog/pg_foreign_server.h" @@ -107,6 +109,10 @@ static AclMode convert_table_priv_string(text *priv_type_text); static AclMode convert_sequence_priv_string(text *priv_type_text); static AttrNumber convert_column_name(Oid tableoid, text *column); static AclMode convert_column_priv_string(text *priv_type_text); +static Oid convert_column_encryption_key_name(text *cekname); +static AclMode convert_column_encryption_key_priv_string(text *priv_type_text); +static Oid convert_column_master_key_name(text *cmkname); +static AclMode convert_column_master_key_priv_string(text *priv_type_text); static Oid convert_database_name(text *databasename); static AclMode convert_database_priv_string(text *priv_type_text); static Oid convert_foreign_data_wrapper_name(text *fdwname); @@ -806,6 +812,14 @@ acldefault(ObjectType objtype, Oid ownerId) world_default = ACL_NO_RIGHTS; owner_default = ACL_ALL_RIGHTS_SEQUENCE; break; + case OBJECT_CEK: + world_default = ACL_NO_RIGHTS; + owner_default = ACL_ALL_RIGHTS_CEK; + break; + case OBJECT_CMK: + world_default = ACL_NO_RIGHTS; + owner_default = ACL_ALL_RIGHTS_CMK; + break; case OBJECT_DATABASE: /* for backwards compatibility, grant some rights by default */ world_default = ACL_CREATE_TEMP | ACL_CONNECT; @@ -917,6 +931,12 @@ acldefault_sql(PG_FUNCTION_ARGS) case 's': objtype = OBJECT_SEQUENCE; break; + case 'Y': + objtype = OBJECT_CEK; + break; + case 'y': + objtype = OBJECT_CMK; + break; case 'd': objtype = OBJECT_DATABASE; break; @@ -2948,6 +2968,384 @@ convert_column_priv_string(text *priv_type_text) } +/* + * has_column_encryption_key_privilege variants + * These are all named "has_column_encryption_key_privilege" at the SQL level. + * They take various combinations of column encryption key name, + * cek OID, user name, user OID, or implicit user = current_user. + * + * The result is a boolean value: true if user has the indicated + * privilege, false if not. + */ + +/* + * has_column_encryption_key_privilege_name_name + * Check user privileges on a column encryption key given + * name username, text cekname, and text priv name. + */ +Datum +has_column_encryption_key_privilege_name_name(PG_FUNCTION_ARGS) +{ + Name username = PG_GETARG_NAME(0); + text *cekname = PG_GETARG_TEXT_PP(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + Oid roleid; + Oid cekid; + AclMode mode; + AclResult aclresult; + + roleid = get_role_oid_or_public(NameStr(*username)); + cekid = convert_column_encryption_key_name(cekname); + mode = convert_column_encryption_key_priv_string(priv_type_text); + + aclresult = object_aclcheck(ColumnEncKeyRelationId, cekid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_encryption_key_privilege_name + * Check user privileges on a column encryption key given + * text cekname and text priv name. + * current_user is assumed + */ +Datum +has_column_encryption_key_privilege_name(PG_FUNCTION_ARGS) +{ + text *cekname = PG_GETARG_TEXT_PP(0); + text *priv_type_text = PG_GETARG_TEXT_PP(1); + Oid roleid; + Oid cekid; + AclMode mode; + AclResult aclresult; + + roleid = GetUserId(); + cekid = convert_column_encryption_key_name(cekname); + mode = convert_column_encryption_key_priv_string(priv_type_text); + + aclresult = object_aclcheck(ColumnEncKeyRelationId, cekid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_encryption_key_privilege_name_id + * Check user privileges on a column encryption key given + * name usename, column encryption key oid, and text priv name. + */ +Datum +has_column_encryption_key_privilege_name_id(PG_FUNCTION_ARGS) +{ + Name username = PG_GETARG_NAME(0); + Oid cekid = PG_GETARG_OID(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + Oid roleid; + AclMode mode; + AclResult aclresult; + + roleid = get_role_oid_or_public(NameStr(*username)); + mode = convert_column_encryption_key_priv_string(priv_type_text); + + if (!SearchSysCacheExists1(CEKOID, ObjectIdGetDatum(cekid))) + PG_RETURN_NULL(); + + aclresult = object_aclcheck(ColumnEncKeyRelationId, cekid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_encryption_key_privilege_id + * Check user privileges on a column encryption key given + * column encryption key oid, and text priv name. + * current_user is assumed + */ +Datum +has_column_encryption_key_privilege_id(PG_FUNCTION_ARGS) +{ + Oid cekid = PG_GETARG_OID(0); + text *priv_type_text = PG_GETARG_TEXT_PP(1); + Oid roleid; + AclMode mode; + AclResult aclresult; + + roleid = GetUserId(); + mode = convert_column_encryption_key_priv_string(priv_type_text); + + if (!SearchSysCacheExists1(CEKOID, ObjectIdGetDatum(cekid))) + PG_RETURN_NULL(); + + aclresult = object_aclcheck(ColumnEncKeyRelationId, cekid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_encryption_key_privilege_id_name + * Check user privileges on a column encryption key given + * roleid, text cekname, and text priv name. + */ +Datum +has_column_encryption_key_privilege_id_name(PG_FUNCTION_ARGS) +{ + Oid roleid = PG_GETARG_OID(0); + text *cekname = PG_GETARG_TEXT_PP(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + Oid cekid; + AclMode mode; + AclResult aclresult; + + cekid = convert_column_encryption_key_name(cekname); + mode = convert_column_encryption_key_priv_string(priv_type_text); + + aclresult = object_aclcheck(ColumnEncKeyRelationId, cekid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_encryption_key_privilege_id_id + * Check user privileges on a column encryption key given + * roleid, cek oid, and text priv name. + */ +Datum +has_column_encryption_key_privilege_id_id(PG_FUNCTION_ARGS) +{ + Oid roleid = PG_GETARG_OID(0); + Oid cekid = PG_GETARG_OID(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + AclMode mode; + AclResult aclresult; + + mode = convert_column_encryption_key_priv_string(priv_type_text); + + if (!SearchSysCacheExists1(CEKOID, ObjectIdGetDatum(cekid))) + PG_RETURN_NULL(); + + aclresult = object_aclcheck(ColumnEncKeyRelationId, cekid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * Support routines for has_column_encryption_key_privilege family. + */ + +/* + * Given a CEK name expressed as a string, look it up and return Oid + */ +static Oid +convert_column_encryption_key_name(text *cekname) +{ + return get_cek_oid(textToQualifiedNameList(cekname), false); +} + +/* + * convert_column_encryption_key_priv_string + * Convert text string to AclMode value. + */ +static AclMode +convert_column_encryption_key_priv_string(text *priv_type_text) +{ + static const priv_map column_encryption_key_priv_map[] = { + {"USAGE", ACL_USAGE}, + {"USAGE WITH GRANT OPTION", ACL_GRANT_OPTION_FOR(ACL_USAGE)}, + {NULL, 0} + }; + + return convert_any_priv_string(priv_type_text, column_encryption_key_priv_map); +} + + +/* + * has_column_master_key_privilege variants + * These are all named "has_column_master_key_privilege" at the SQL level. + * They take various combinations of column master key name, + * cmk OID, user name, user OID, or implicit user = current_user. + * + * The result is a boolean value: true if user has the indicated + * privilege, false if not. + */ + +/* + * has_column_master_key_privilege_name_name + * Check user privileges on a column master key given + * name username, text cmkname, and text priv name. + */ +Datum +has_column_master_key_privilege_name_name(PG_FUNCTION_ARGS) +{ + Name username = PG_GETARG_NAME(0); + text *cmkname = PG_GETARG_TEXT_PP(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + Oid roleid; + Oid cmkid; + AclMode mode; + AclResult aclresult; + + roleid = get_role_oid_or_public(NameStr(*username)); + cmkid = convert_column_master_key_name(cmkname); + mode = convert_column_master_key_priv_string(priv_type_text); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_master_key_privilege_name + * Check user privileges on a column master key given + * text cmkname and text priv name. + * current_user is assumed + */ +Datum +has_column_master_key_privilege_name(PG_FUNCTION_ARGS) +{ + text *cmkname = PG_GETARG_TEXT_PP(0); + text *priv_type_text = PG_GETARG_TEXT_PP(1); + Oid roleid; + Oid cmkid; + AclMode mode; + AclResult aclresult; + + roleid = GetUserId(); + cmkid = convert_column_master_key_name(cmkname); + mode = convert_column_master_key_priv_string(priv_type_text); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_master_key_privilege_name_id + * Check user privileges on a column master key given + * name usename, column master key oid, and text priv name. + */ +Datum +has_column_master_key_privilege_name_id(PG_FUNCTION_ARGS) +{ + Name username = PG_GETARG_NAME(0); + Oid cmkid = PG_GETARG_OID(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + Oid roleid; + AclMode mode; + AclResult aclresult; + + roleid = get_role_oid_or_public(NameStr(*username)); + mode = convert_column_master_key_priv_string(priv_type_text); + + if (!SearchSysCacheExists1(CMKOID, ObjectIdGetDatum(cmkid))) + PG_RETURN_NULL(); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_master_key_privilege_id + * Check user privileges on a column master key given + * column master key oid, and text priv name. + * current_user is assumed + */ +Datum +has_column_master_key_privilege_id(PG_FUNCTION_ARGS) +{ + Oid cmkid = PG_GETARG_OID(0); + text *priv_type_text = PG_GETARG_TEXT_PP(1); + Oid roleid; + AclMode mode; + AclResult aclresult; + + roleid = GetUserId(); + mode = convert_column_master_key_priv_string(priv_type_text); + + if (!SearchSysCacheExists1(CMKOID, ObjectIdGetDatum(cmkid))) + PG_RETURN_NULL(); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_master_key_privilege_id_name + * Check user privileges on a column master key given + * roleid, text cmkname, and text priv name. + */ +Datum +has_column_master_key_privilege_id_name(PG_FUNCTION_ARGS) +{ + Oid roleid = PG_GETARG_OID(0); + text *cmkname = PG_GETARG_TEXT_PP(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + Oid cmkid; + AclMode mode; + AclResult aclresult; + + cmkid = convert_column_master_key_name(cmkname); + mode = convert_column_master_key_priv_string(priv_type_text); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * has_column_master_key_privilege_id_id + * Check user privileges on a column master key given + * roleid, cmk oid, and text priv name. + */ +Datum +has_column_master_key_privilege_id_id(PG_FUNCTION_ARGS) +{ + Oid roleid = PG_GETARG_OID(0); + Oid cmkid = PG_GETARG_OID(1); + text *priv_type_text = PG_GETARG_TEXT_PP(2); + AclMode mode; + AclResult aclresult; + + mode = convert_column_master_key_priv_string(priv_type_text); + + if (!SearchSysCacheExists1(CMKOID, ObjectIdGetDatum(cmkid))) + PG_RETURN_NULL(); + + aclresult = object_aclcheck(ColumnMasterKeyRelationId, cmkid, roleid, mode); + + PG_RETURN_BOOL(aclresult == ACLCHECK_OK); +} + +/* + * Support routines for has_column_master_key_privilege family. + */ + +/* + * Given a CMK name expressed as a string, look it up and return Oid + */ +static Oid +convert_column_master_key_name(text *cmkname) +{ + return get_cmk_oid(textToQualifiedNameList(cmkname), false); +} + +/* + * convert_column_master_key_priv_string + * Convert text string to AclMode value. + */ +static AclMode +convert_column_master_key_priv_string(text *priv_type_text) +{ + static const priv_map column_master_key_priv_map[] = { + {"USAGE", ACL_USAGE}, + {"USAGE WITH GRANT OPTION", ACL_GRANT_OPTION_FOR(ACL_USAGE)}, + {NULL, 0} + }; + + return convert_any_priv_string(priv_type_text, column_master_key_priv_map); +} + + /* * has_database_privilege variants * These are all named "has_database_privilege" at the SQL level. diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index d1b09dedfd4..8be950e6114 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -681,6 +681,113 @@ unknownsend(PG_FUNCTION_ARGS) PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); } +/* + * pg_encrypted_in - + * + * Input function for pg_encrypted_* types. + * + * The format and additional checks ensure that one cannot easily insert a + * value directly into an encrypted column by accident. (That's why we don't + * just use the bytea format, for example.) But we still have to support + * direct inserts into encrypted columns, for example for restoring backups + * made by pg_dump. + */ +Datum +pg_encrypted_in(PG_FUNCTION_ARGS) +{ + char *inputText = PG_GETARG_CSTRING(0); + Node *escontext = fcinfo->context; + char *ip; + size_t hexlen; + int bc; + bytea *result; + + if (strncmp(inputText, "encrypted$", 10) != 0) + ereturn(escontext, (Datum) 0, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("invalid input value for encrypted column value: \"%s\"", + inputText)); + + ip = inputText + 10; + hexlen = strlen(ip); + + /* sanity check to catch obvious mistakes */ + if (hexlen / 2 < 32 || (hexlen / 2) % 16 != 0) + ereturn(escontext, (Datum) 0, + errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), + errmsg("invalid input value for encrypted column value: \"%s\"", + inputText)); + + bc = hexlen / 2 + VARHDRSZ; /* maximum possible length */ + result = palloc(bc); + bc = hex_decode(ip, hexlen, VARDATA(result)); + SET_VARSIZE(result, bc + VARHDRSZ); /* actual length */ + + PG_RETURN_BYTEA_P(result); +} + +/* + * pg_encrypted_out - + * + * Output function for pg_encrypted_* types. + * + * This output is seen when reading an encrypted column without column + * encryption mode enabled. Therefore, the output format is chosen so that it + * is easily recognizable. + */ +Datum +pg_encrypted_out(PG_FUNCTION_ARGS) +{ + bytea *vlena = PG_GETARG_BYTEA_PP(0); + char *result; + char *rp; + + rp = result = palloc(VARSIZE_ANY_EXHDR(vlena) * 2 + 10 + 1); + memcpy(rp, "encrypted$", 10); + rp += 10; + rp += hex_encode(VARDATA_ANY(vlena), VARSIZE_ANY_EXHDR(vlena), rp); + *rp = '\0'; + PG_RETURN_CSTRING(result); +} + +/* + * pg_encrypted_recv - + * + * Receive function for pg_encrypted_* types. + */ +Datum +pg_encrypted_recv(PG_FUNCTION_ARGS) +{ + StringInfo buf = (StringInfo) PG_GETARG_POINTER(0); + bytea *result; + int nbytes; + + nbytes = buf->len - buf->cursor; + /* sanity check to catch obvious mistakes */ + if (nbytes < 32) + ereport(ERROR, + errcode(ERRCODE_INVALID_BINARY_REPRESENTATION), + errmsg("invalid binary input value for encrypted column value")); + result = (bytea *) palloc(nbytes + VARHDRSZ); + SET_VARSIZE(result, nbytes + VARHDRSZ); + pq_copymsgbytes(buf, VARDATA(result), nbytes); + PG_RETURN_BYTEA_P(result); +} + +/* + * pg_encrypted_send - + * + * Send function for pg_encrypted_* types. + */ +Datum +pg_encrypted_send(PG_FUNCTION_ARGS) +{ + bytea *vlena = PG_GETARG_BYTEA_P_COPY(0); + + /* just return input */ + PG_RETURN_BYTEA_P(vlena); +} + /* ========== PUBLIC ROUTINES ========== */ diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c index 26368ffcc97..9247475f58b 100644 --- a/src/backend/utils/cache/lsyscache.c +++ b/src/backend/utils/cache/lsyscache.c @@ -24,7 +24,10 @@ #include "catalog/pg_amproc.h" #include "catalog/pg_cast.h" #include "catalog/pg_class.h" +#include "catalog/pg_colenckey.h" +#include "catalog/pg_colenckeydata.h" #include "catalog/pg_collation.h" +#include "catalog/pg_colmasterkey.h" #include "catalog/pg_constraint.h" #include "catalog/pg_index.h" #include "catalog/pg_language.h" @@ -2678,6 +2681,25 @@ type_is_multirange(Oid typid) return (get_typtype(typid) == TYPTYPE_MULTIRANGE); } +bool +type_is_encrypted(Oid typid) +{ + HeapTuple tp; + + tp = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid)); + if (HeapTupleIsValid(tp)) + { + Form_pg_type typtup = (Form_pg_type) GETSTRUCT(tp); + bool result; + + result = (typtup->typcategory == TYPCATEGORY_ENCRYPTED); + ReleaseSysCache(tp); + return result; + } + else + return false; +} + /* * get_type_category_preferred * @@ -3692,3 +3714,64 @@ get_subscription_name(Oid subid, bool missing_ok) return subname; } + +char * +get_cek_name(Oid cekid, bool missing_ok) +{ + HeapTuple tup; + char *cekname; + + tup = SearchSysCache1(CEKOID, ObjectIdGetDatum(cekid)); + + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column encryption key %u", cekid); + return NULL; + } + + cekname = pstrdup(NameStr(((Form_pg_colenckey) GETSTRUCT(tup))->cekname)); + + ReleaseSysCache(tup); + + return cekname; +} + +Oid +get_cekdata_oid(Oid cekid, Oid cmkid, bool missing_ok) +{ + Oid cekdataid; + + cekdataid = GetSysCacheOid2(CEKDATACEKCMK, Anum_pg_colenckeydata_oid, + ObjectIdGetDatum(cekid), + ObjectIdGetDatum(cmkid)); + if (!OidIsValid(cekdataid) && !missing_ok) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("column encryption key \"%s\" has no data for master key \"%s\"", + get_cek_name(cekid, false), get_cmk_name(cmkid, false)))); + + return cekdataid; +} + +char * +get_cmk_name(Oid cmkid, bool missing_ok) +{ + HeapTuple tup; + char *cmkname; + + tup = SearchSysCache1(CMKOID, ObjectIdGetDatum(cmkid)); + + if (!HeapTupleIsValid(tup)) + { + if (!missing_ok) + elog(ERROR, "cache lookup failed for column master key %u", cmkid); + return NULL; + } + + cmkname = pstrdup(NameStr(((Form_pg_colmasterkey) GETSTRUCT(tup))->cmkname)); + + ReleaseSysCache(tup); + + return cmkname; +} diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c index 5af1a168ec2..eb0c6645f07 100644 --- a/src/backend/utils/cache/plancache.c +++ b/src/backend/utils/cache/plancache.c @@ -100,8 +100,6 @@ static dlist_head saved_plan_list = DLIST_STATIC_INIT(saved_plan_list); static dlist_head cached_expression_list = DLIST_STATIC_INIT(cached_expression_list); static void ReleaseGenericPlan(CachedPlanSource *plansource); -static List *RevalidateCachedQuery(CachedPlanSource *plansource, - QueryEnvironment *queryEnv); static bool CheckCachedPlan(CachedPlanSource *plansource); static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist, ParamListInfo boundParams, QueryEnvironment *queryEnv); @@ -579,7 +577,7 @@ ReleaseGenericPlan(CachedPlanSource *plansource) * had to do re-analysis, and NIL otherwise. (This is returned just to save * a tree copying step in a subsequent BuildCachedPlan call.) */ -static List * +List * RevalidateCachedQuery(CachedPlanSource *plansource, QueryEnvironment *queryEnv) { diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c index 62777b14c9b..e8b9296830d 100644 --- a/src/backend/utils/mb/mbutils.c +++ b/src/backend/utils/mb/mbutils.c @@ -36,7 +36,9 @@ #include "access/xact.h" #include "catalog/namespace.h" +#include "libpq/libpq-be.h" #include "mb/pg_wchar.h" +#include "miscadmin.h" #include "utils/fmgrprotos.h" #include "utils/memutils.h" #include "varatt.h" @@ -129,6 +131,12 @@ PrepareClientEncoding(int encoding) encoding == PG_SQL_ASCII) return 0; + /* + * Cannot do conversion when column encryption is enabled. + */ + if (MyProcPort->column_encryption_enabled) + return -1; + if (IsTransactionState()) { /* @@ -236,6 +244,12 @@ SetClientEncoding(int encoding) return 0; } + /* + * Cannot do conversion when column encryption is enabled. + */ + if (MyProcPort->column_encryption_enabled) + return -1; + /* * Search the cache for the entry previously prepared by * PrepareClientEncoding; if there isn't one, we lose. While at it, @@ -296,7 +310,9 @@ InitializeClientEncoding(void) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("conversion between %s and %s is not supported", pg_enc2name_tbl[pending_client_encoding].name, - GetDatabaseEncodingName()))); + GetDatabaseEncodingName()), + (MyProcPort->column_encryption_enabled) ? + errdetail("Encoding conversion is not possible when column encryption is enabled.") : 0)); } /* diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c index c7dd0b11fd2..7a9082ee6ca 100644 --- a/src/bin/pg_dump/common.c +++ b/src/bin/pg_dump/common.c @@ -18,7 +18,9 @@ #include #include "catalog/pg_class_d.h" +#include "catalog/pg_colenckey_d.h" #include "catalog/pg_collation_d.h" +#include "catalog/pg_colmasterkey_d.h" #include "catalog/pg_extension_d.h" #include "catalog/pg_namespace_d.h" #include "catalog/pg_operator_d.h" @@ -203,6 +205,12 @@ getSchemaData(Archive *fout, int *numTablesPtr) pg_log_info("reading user-defined collations"); (void) getCollations(fout, &numCollations); + pg_log_info("reading column master keys"); + getColumnMasterKeys(fout); + + pg_log_info("reading column encryption keys"); + getColumnEncryptionKeys(fout); + pg_log_info("reading user-defined conversions"); getConversions(fout, &numConversions); @@ -936,6 +944,42 @@ findOprByOid(Oid oid) return (OprInfo *) dobj; } +/* + * findCekByOid + * finds the DumpableObject for the CEK with the given oid + * returns NULL if not found + */ +CekInfo * +findCekByOid(Oid oid) +{ + CatalogId catId; + DumpableObject *dobj; + + catId.tableoid = ColumnEncKeyRelationId; + catId.oid = oid; + dobj = findObjectByCatalogId(catId); + Assert(dobj == NULL || dobj->objType == DO_CEK); + return (CekInfo *) dobj; +} + +/* + * findCmkByOid + * finds the DumpableObject for the CMK with the given oid + * returns NULL if not found + */ +CmkInfo * +findCmkByOid(Oid oid) +{ + CatalogId catId; + DumpableObject *dobj; + + catId.tableoid = ColumnMasterKeyRelationId; + catId.oid = oid; + dobj = findObjectByCatalogId(catId); + Assert(dobj == NULL || dobj->objType == DO_CMK); + return (CmkInfo *) dobj; +} + /* * findCollationByOid * finds the DumpableObject for the collation with the given oid diff --git a/src/bin/pg_dump/dumputils.c b/src/bin/pg_dump/dumputils.c index 5649859aa1e..1ca0d33fa3d 100644 --- a/src/bin/pg_dump/dumputils.c +++ b/src/bin/pg_dump/dumputils.c @@ -484,6 +484,10 @@ do { \ CONVERT_PRIV('C', "CREATE"); CONVERT_PRIV('U', "USAGE"); } + else if (strcmp(type, "COLUMN ENCRYPTION KEY") == 0) + CONVERT_PRIV('U', "USAGE"); + else if (strcmp(type, "COLUMN MASTER KEY") == 0) + CONVERT_PRIV('U', "USAGE"); else if (strcmp(type, "DATABASE") == 0) { CONVERT_PRIV('C', "CREATE"); diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h index fbf5f1c515e..f8d8d5a97dc 100644 --- a/src/bin/pg_dump/pg_backup.h +++ b/src/bin/pg_dump/pg_backup.h @@ -86,6 +86,7 @@ typedef struct _connParams char *pghost; char *username; trivalue promptPassword; + int column_encryption; /* If not NULL, this overrides the dbname obtained from command line */ /* (but *only* the DB name, not anything else in the connstring) */ char *override_dbname; diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c index 465e9ce777f..cd2c554336a 100644 --- a/src/bin/pg_dump/pg_backup_archiver.c +++ b/src/bin/pg_dump/pg_backup_archiver.c @@ -3581,6 +3581,8 @@ _getObjectDescription(PQExpBuffer buf, const TocEntry *te) /* objects that don't require special decoration */ if (strcmp(type, "COLLATION") == 0 || + strcmp(type, "COLUMN ENCRYPTION KEY") == 0 || + strcmp(type, "COLUMN MASTER KEY") == 0 || strcmp(type, "CONVERSION") == 0 || strcmp(type, "DOMAIN") == 0 || strcmp(type, "FOREIGN TABLE") == 0 || diff --git a/src/bin/pg_dump/pg_backup_db.c b/src/bin/pg_dump/pg_backup_db.c index a02841c4050..47cfa5bab2c 100644 --- a/src/bin/pg_dump/pg_backup_db.c +++ b/src/bin/pg_dump/pg_backup_db.c @@ -133,8 +133,8 @@ ConnectDatabase(Archive *AHX, */ do { - const char *keywords[8]; - const char *values[8]; + const char *keywords[9]; + const char *values[9]; int i = 0; /* @@ -159,6 +159,11 @@ ConnectDatabase(Archive *AHX, } keywords[i] = "fallback_application_name"; values[i++] = progname; + if (cparams->column_encryption) + { + keywords[i] = "column_encryption"; + values[i++] = "1"; + } keywords[i] = NULL; values[i++] = NULL; Assert(i <= lengthof(keywords)); diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index c52e961b309..1ae914182f3 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -54,6 +54,7 @@ #include "catalog/pg_subscription.h" #include "catalog/pg_trigger_d.h" #include "catalog/pg_type_d.h" +#include "common/colenc.h" #include "common/connect.h" #include "common/relpath.h" #include "compress_io.h" @@ -245,6 +246,8 @@ static void dumpAccessMethod(Archive *fout, const AccessMethodInfo *aminfo); static void dumpOpclass(Archive *fout, const OpclassInfo *opcinfo); static void dumpOpfamily(Archive *fout, const OpfamilyInfo *opfinfo); static void dumpCollation(Archive *fout, const CollInfo *collinfo); +static void dumpColumnEncryptionKey(Archive *fout, const CekInfo *cekinfo); +static void dumpColumnMasterKey(Archive *fout, const CmkInfo *cekinfo); static void dumpConversion(Archive *fout, const ConvInfo *convinfo); static void dumpRule(Archive *fout, const RuleInfo *rinfo); static void dumpAgg(Archive *fout, const AggInfo *agginfo); @@ -414,6 +417,7 @@ main(int argc, char **argv) {"attribute-inserts", no_argument, &dopt.column_inserts, 1}, {"binary-upgrade", no_argument, &dopt.binary_upgrade, 1}, {"column-inserts", no_argument, &dopt.column_inserts, 1}, + {"decrypt-encrypted-columns", no_argument, &dopt.cparams.column_encryption, 1}, {"disable-dollar-quoting", no_argument, &dopt.disable_dollar_quoting, 1}, {"disable-triggers", no_argument, &dopt.disable_triggers, 1}, {"enable-row-security", no_argument, &dopt.enable_row_security, 1}, @@ -742,6 +746,9 @@ main(int argc, char **argv) * --inserts are already implied above if --column-inserts or * --rows-per-insert were specified. */ + if (dopt.cparams.column_encryption && dopt.dump_inserts == 0) + pg_fatal("option --decrypt-encrypted-columns requires option --inserts, --rows-per-insert, or --column-inserts"); + if (dopt.do_nothing && dopt.dump_inserts == 0) pg_fatal("option --on-conflict-do-nothing requires option --inserts, --rows-per-insert, or --column-inserts"); @@ -1128,6 +1135,7 @@ help(const char *progname) printf(_(" -x, --no-privileges do not dump privileges (grant/revoke)\n")); printf(_(" --binary-upgrade for use by upgrade utilities only\n")); printf(_(" --column-inserts dump data as INSERT commands with column names\n")); + printf(_(" --decrypt-encrypted-columns decrypt encrypted columns in the output\n")); printf(_(" --disable-dollar-quoting disable dollar quoting, use SQL standard quoting\n")); printf(_(" --disable-triggers disable triggers during data-only restore\n")); printf(_(" --enable-row-security enable row security (dump only content user has\n" @@ -6071,6 +6079,164 @@ getCollations(Archive *fout, int *numCollations) return collinfo; } +/* + * getColumnEncryptionKeys + * get information about column encryption keys + */ +void +getColumnEncryptionKeys(Archive *fout) +{ + PQExpBuffer query; + PGresult *res; + int ntups; + CekInfo *cekinfo; + int i_tableoid; + int i_oid; + int i_cekname; + int i_ceknamespace; + int i_cekowner; + int i_cekacl; + int i_acldefault; + + if (fout->remoteVersion < 170000) + return; + + query = createPQExpBuffer(); + + appendPQExpBuffer(query, + "SELECT cek.tableoid, cek.oid, cek.cekname, cek.ceknamespace, cek.cekowner, cek.cekacl, acldefault('Y', cek.cekowner) AS acldefault\n" + "FROM pg_colenckey cek"); + + res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + + ntups = PQntuples(res); + + i_tableoid = PQfnumber(res, "tableoid"); + i_oid = PQfnumber(res, "oid"); + i_cekname = PQfnumber(res, "cekname"); + i_ceknamespace = PQfnumber(res, "ceknamespace"); + i_cekowner = PQfnumber(res, "cekowner"); + i_cekacl = PQfnumber(res, "cekacl"); + i_acldefault = PQfnumber(res, "acldefault"); + + cekinfo = pg_malloc(ntups * sizeof(CekInfo)); + + for (int i = 0; i < ntups; i++) + { + PGresult *res2; + int ntups2; + + cekinfo[i].dobj.objType = DO_CEK; + cekinfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid)); + cekinfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid)); + AssignDumpId(&cekinfo[i].dobj); + cekinfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_cekname)); + cekinfo[i].dobj.namespace = findNamespace(atooid(PQgetvalue(res, i, i_ceknamespace))); + cekinfo[i].dacl.acl = pg_strdup(PQgetvalue(res, i, i_cekacl)); + cekinfo[i].dacl.acldefault = pg_strdup(PQgetvalue(res, i, i_acldefault)); + cekinfo[i].dacl.privtype = 0; + cekinfo[i].dacl.initprivs = NULL; + cekinfo[i].rolname = getRoleName(PQgetvalue(res, i, i_cekowner)); + + resetPQExpBuffer(query); + appendPQExpBuffer(query, + "SELECT ckdcmkid, ckdcmkalg, ckdencval\n" + "FROM pg_catalog.pg_colenckeydata\n" + "WHERE ckdcekid = %u", cekinfo[i].dobj.catId.oid); + res2 = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + ntups2 = PQntuples(res2); + cekinfo[i].numdata = ntups2; + cekinfo[i].cekcmks = pg_malloc(sizeof(CmkInfo *) * ntups2); + cekinfo[i].cekcmkalgs = pg_malloc(sizeof(int) * ntups2); + cekinfo[i].cekencvals = pg_malloc(sizeof(char *) * ntups2); + for (int j = 0; j < ntups2; j++) + { + Oid ckdcmkid; + + ckdcmkid = atooid(PQgetvalue(res2, j, PQfnumber(res2, "ckdcmkid"))); + cekinfo[i].cekcmks[j] = findCmkByOid(ckdcmkid); + cekinfo[i].cekcmkalgs[j] = atoi(PQgetvalue(res2, j, PQfnumber(res2, "ckdcmkalg"))); + cekinfo[i].cekencvals[j] = pg_strdup(PQgetvalue(res2, j, PQfnumber(res2, "ckdencval"))); + } + PQclear(res2); + + selectDumpableObject(&(cekinfo[i].dobj), fout); + if (!PQgetisnull(res, i, i_cekacl)) + cekinfo[i].dobj.components |= DUMP_COMPONENT_ACL; + } + PQclear(res); + + destroyPQExpBuffer(query); +} + +/* + * getColumnMasterKeys + * get information about column master keys + */ +void +getColumnMasterKeys(Archive *fout) +{ + PQExpBuffer query; + PGresult *res; + int ntups; + CmkInfo *cmkinfo; + int i_tableoid; + int i_oid; + int i_cmkname; + int i_cmknamespace; + int i_cmkowner; + int i_cmkrealm; + int i_cmkacl; + int i_acldefault; + + if (fout->remoteVersion < 170000) + return; + + query = createPQExpBuffer(); + + appendPQExpBuffer(query, + "SELECT cmk.tableoid, cmk.oid, cmk.cmkname, cmk.cmknamespace, cmk.cmkowner, cmk.cmkrealm, cmk.cmkacl, acldefault('y', cmk.cmkowner) AS acldefault\n" + "FROM pg_colmasterkey cmk"); + + res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); + + ntups = PQntuples(res); + + i_tableoid = PQfnumber(res, "tableoid"); + i_oid = PQfnumber(res, "oid"); + i_cmkname = PQfnumber(res, "cmkname"); + i_cmknamespace = PQfnumber(res, "cmknamespace"); + i_cmkowner = PQfnumber(res, "cmkowner"); + i_cmkrealm = PQfnumber(res, "cmkrealm"); + i_cmkacl = PQfnumber(res, "cmkacl"); + i_acldefault = PQfnumber(res, "acldefault"); + + cmkinfo = pg_malloc(ntups * sizeof(CmkInfo)); + + for (int i = 0; i < ntups; i++) + { + cmkinfo[i].dobj.objType = DO_CMK; + cmkinfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid)); + cmkinfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid)); + AssignDumpId(&cmkinfo[i].dobj); + cmkinfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_cmkname)); + cmkinfo[i].dobj.namespace = findNamespace(atooid(PQgetvalue(res, i, i_cmknamespace))); + cmkinfo[i].dacl.acl = pg_strdup(PQgetvalue(res, i, i_cmkacl)); + cmkinfo[i].dacl.acldefault = pg_strdup(PQgetvalue(res, i, i_acldefault)); + cmkinfo[i].dacl.privtype = 0; + cmkinfo[i].dacl.initprivs = NULL; + cmkinfo[i].rolname = getRoleName(PQgetvalue(res, i, i_cmkowner)); + cmkinfo[i].cmkrealm = pg_strdup(PQgetvalue(res, i, i_cmkrealm)); + + selectDumpableObject(&(cmkinfo[i].dobj), fout); + if (!PQgetisnull(res, i, i_cmkacl)) + cmkinfo[i].dobj.components |= DUMP_COMPONENT_ACL; + } + PQclear(res); + + destroyPQExpBuffer(query); +} + /* * getConversions: * read all conversions in the system catalogs and return them in the @@ -8701,6 +8867,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) int i_typstorage; int i_attidentity; int i_attgenerated; + int i_attcek; + int i_attencalg; + int i_attencdet; int i_attisdropped; int i_attlen; int i_attalign; @@ -8763,29 +8932,31 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) * collation is different from their type's default, we use a CASE here to * suppress uninteresting attcollations cheaply. */ - appendPQExpBufferStr(q, - "SELECT\n" - "a.attrelid,\n" - "a.attnum,\n" - "a.attname,\n" - "a.attstattarget,\n" - "a.attstorage,\n" - "t.typstorage,\n" - "a.atthasdef,\n" - "a.attisdropped,\n" - "a.attlen,\n" - "a.attalign,\n" - "a.attislocal,\n" - "pg_catalog.format_type(t.oid, a.atttypmod) AS atttypname,\n" - "array_to_string(a.attoptions, ', ') AS attoptions,\n" - "CASE WHEN a.attcollation <> t.typcollation " - "THEN a.attcollation ELSE 0 END AS attcollation,\n" - "pg_catalog.array_to_string(ARRAY(" - "SELECT pg_catalog.quote_ident(option_name) || " - "' ' || pg_catalog.quote_literal(option_value) " - "FROM pg_catalog.pg_options_to_table(attfdwoptions) " - "ORDER BY option_name" - "), E',\n ') AS attfdwoptions,\n"); + appendPQExpBuffer(q, "SELECT\n" + "a.attrelid,\n" + "a.attnum,\n" + "a.attname,\n" + "a.attstattarget,\n" + "a.attstorage,\n" + "t.typstorage,\n" + "a.atthasdef,\n" + "a.attisdropped,\n" + "a.attlen,\n" + "a.attalign,\n" + "a.attislocal,\n" + "pg_catalog.format_type(%s) AS atttypname,\n" + "array_to_string(a.attoptions, ', ') AS attoptions,\n" + "CASE WHEN a.attcollation <> t.typcollation " + "THEN a.attcollation ELSE 0 END AS attcollation,\n" + "pg_catalog.array_to_string(ARRAY(" + "SELECT pg_catalog.quote_ident(option_name) || " + "' ' || pg_catalog.quote_literal(option_value) " + "FROM pg_catalog.pg_options_to_table(attfdwoptions) " + "ORDER BY option_name" + "), E',\n ') AS attfdwoptions,\n", + fout->remoteVersion >= 170000 ? + "CASE WHEN a.attusertypid IS NOT NULL THEN a.attusertypid ELSE a.atttypid END, CASE WHEN a.attusertypid IS NOT NULL THEN a.attusertypmod ELSE a.atttypmod END" : + "a.atttypid, a.atttypmod"); /* * Find out any NOT NULL markings for each column. In 17 and up we have @@ -8839,10 +9010,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) if (fout->remoteVersion >= 120000) appendPQExpBufferStr(q, - "a.attgenerated\n"); + "a.attgenerated,\n"); + else + appendPQExpBufferStr(q, + "'' AS attgenerated,\n"); + + if (fout->remoteVersion >= 170000) + appendPQExpBuffer(q, + "a.attcek,\n" + "CASE a.atttypid WHEN %u THEN true WHEN %u THEN false END AS attencdet,\n" + "CASE WHEN a.atttypid IN (%u, %u) THEN a.atttypmod END AS attencalg\n", + PG_ENCRYPTED_DETOID, PG_ENCRYPTED_RNDOID, + PG_ENCRYPTED_DETOID, PG_ENCRYPTED_RNDOID); else appendPQExpBufferStr(q, - "'' AS attgenerated\n"); + "NULL AS attcek,\n" + "NULL AS attencdet,\n" + "NULL AS attencalg\n"); /* need left join to pg_type to not fail on dropped columns ... */ appendPQExpBuffer(q, @@ -8885,6 +9069,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) i_typstorage = PQfnumber(res, "typstorage"); i_attidentity = PQfnumber(res, "attidentity"); i_attgenerated = PQfnumber(res, "attgenerated"); + i_attcek = PQfnumber(res, "attcek"); + i_attencdet = PQfnumber(res, "attencdet"); + i_attencalg = PQfnumber(res, "attencalg"); i_attisdropped = PQfnumber(res, "attisdropped"); i_attlen = PQfnumber(res, "attlen"); i_attalign = PQfnumber(res, "attalign"); @@ -8951,6 +9138,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->typstorage = (char *) pg_malloc(numatts * sizeof(char)); tbinfo->attidentity = (char *) pg_malloc(numatts * sizeof(char)); tbinfo->attgenerated = (char *) pg_malloc(numatts * sizeof(char)); + tbinfo->attcek = (CekInfo **) pg_malloc(numatts * sizeof(CekInfo *)); + tbinfo->attencdet = (bool *) pg_malloc(numatts * sizeof(bool)); + tbinfo->attencalg = (int *) pg_malloc(numatts * sizeof(int)); tbinfo->attisdropped = (bool *) pg_malloc(numatts * sizeof(bool)); tbinfo->attlen = (int *) pg_malloc(numatts * sizeof(int)); tbinfo->attalign = (char *) pg_malloc(numatts * sizeof(char)); @@ -8987,6 +9177,22 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables) tbinfo->attidentity[j] = *(PQgetvalue(res, r, i_attidentity)); tbinfo->attgenerated[j] = *(PQgetvalue(res, r, i_attgenerated)); tbinfo->needs_override = tbinfo->needs_override || (tbinfo->attidentity[j] == ATTRIBUTE_IDENTITY_ALWAYS); + if (!PQgetisnull(res, r, i_attcek)) + { + Oid attcekid = atooid(PQgetvalue(res, r, i_attcek)); + + tbinfo->attcek[j] = findCekByOid(attcekid); + } + else + tbinfo->attcek[j] = NULL; + if (!PQgetisnull(res, r, i_attencdet)) + tbinfo->attencdet[j] = (PQgetvalue(res, r, i_attencdet)[0] == 't'); + else + tbinfo->attencdet[j] = 0; + if (!PQgetisnull(res, r, i_attencalg)) + tbinfo->attencalg[j] = atoi(PQgetvalue(res, r, i_attencalg)); + else + tbinfo->attencalg[j] = 0; tbinfo->attisdropped[j] = (PQgetvalue(res, r, i_attisdropped)[0] == 't'); tbinfo->attlen[j] = atoi(PQgetvalue(res, r, i_attlen)); tbinfo->attalign[j] = *(PQgetvalue(res, r, i_attalign)); @@ -10625,6 +10831,12 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj) case DO_OPFAMILY: dumpOpfamily(fout, (const OpfamilyInfo *) dobj); break; + case DO_CEK: + dumpColumnEncryptionKey(fout, (const CekInfo *) dobj); + break; + case DO_CMK: + dumpColumnMasterKey(fout, (const CmkInfo *) dobj); + break; case DO_COLLATION: dumpCollation(fout, (const CollInfo *) dobj); break; @@ -14100,6 +14312,141 @@ dumpCollation(Archive *fout, const CollInfo *collinfo) free(qcollname); } +/* + * dumpColumnEncryptionKey + * dump the definition of the given column encryption key + */ +static void +dumpColumnEncryptionKey(Archive *fout, const CekInfo *cekinfo) +{ + DumpOptions *dopt = fout->dopt; + PQExpBuffer delq; + PQExpBuffer query; + char *qcekname; + + /* Do nothing in data-only dump */ + if (dopt->dataOnly) + return; + + delq = createPQExpBuffer(); + query = createPQExpBuffer(); + + qcekname = pg_strdup(fmtId(cekinfo->dobj.name)); + + appendPQExpBuffer(delq, "DROP COLUMN ENCRYPTION KEY %s;\n", + fmtQualifiedDumpable(cekinfo)); + + appendPQExpBuffer(query, "CREATE COLUMN ENCRYPTION KEY %s WITH VALUES ", + fmtQualifiedDumpable(cekinfo)); + + for (int i = 0; i < cekinfo->numdata; i++) + { + appendPQExpBuffer(query, "("); + + appendPQExpBuffer(query, "column_master_key = %s, ", fmtQualifiedDumpable(cekinfo->cekcmks[i])); + appendPQExpBuffer(query, "algorithm = '%s', ", get_cmkalg_name(cekinfo->cekcmkalgs[i])); + appendPQExpBuffer(query, "encrypted_value = "); + appendStringLiteralAH(query, cekinfo->cekencvals[i], fout); + + appendPQExpBuffer(query, ")"); + if (i < cekinfo->numdata - 1) + appendPQExpBuffer(query, ", "); + } + + appendPQExpBufferStr(query, ";\n"); + + if (cekinfo->dobj.dump & DUMP_COMPONENT_DEFINITION) + ArchiveEntry(fout, cekinfo->dobj.catId, cekinfo->dobj.dumpId, + ARCHIVE_OPTS(.tag = cekinfo->dobj.name, + .namespace = cekinfo->dobj.namespace->dobj.name, + .owner = cekinfo->rolname, + .description = "COLUMN ENCRYPTION KEY", + .section = SECTION_PRE_DATA, + .createStmt = query->data, + .dropStmt = delq->data)); + + if (cekinfo->dobj.dump & DUMP_COMPONENT_COMMENT) + dumpComment(fout, "COLUMN ENCRYPTION KEY", qcekname, + cekinfo->dobj.namespace->dobj.name, cekinfo->rolname, + cekinfo->dobj.catId, 0, cekinfo->dobj.dumpId); + + if (cekinfo->dobj.dump & DUMP_COMPONENT_SECLABEL) + dumpSecLabel(fout, "COLUMN ENCRYPTION KEY", qcekname, + cekinfo->dobj.namespace->dobj.name, cekinfo->rolname, + cekinfo->dobj.catId, 0, cekinfo->dobj.dumpId); + + if (cekinfo->dobj.dump & DUMP_COMPONENT_ACL) + dumpACL(fout, cekinfo->dobj.dumpId, InvalidDumpId, "COLUMN ENCRYPTION KEY", + qcekname, NULL, cekinfo->dobj.namespace->dobj.name, + NULL, cekinfo->rolname, &cekinfo->dacl); + + destroyPQExpBuffer(delq); + destroyPQExpBuffer(query); + free(qcekname); +} + +/* + * dumpColumnMasterKey + * dump the definition of the given column master key + */ +static void +dumpColumnMasterKey(Archive *fout, const CmkInfo *cmkinfo) +{ + DumpOptions *dopt = fout->dopt; + PQExpBuffer delq; + PQExpBuffer query; + char *qcmkname; + + /* Do nothing in data-only dump */ + if (dopt->dataOnly) + return; + + delq = createPQExpBuffer(); + query = createPQExpBuffer(); + + qcmkname = pg_strdup(fmtId(cmkinfo->dobj.name)); + + appendPQExpBuffer(delq, "DROP COLUMN MASTER KEY %s;\n", + fmtQualifiedDumpable(cmkinfo)); + + appendPQExpBuffer(query, "CREATE COLUMN MASTER KEY %s WITH (", + fmtQualifiedDumpable(cmkinfo)); + + appendPQExpBuffer(query, "realm = "); + appendStringLiteralAH(query, cmkinfo->cmkrealm, fout); + + appendPQExpBufferStr(query, ");\n"); + + if (cmkinfo->dobj.dump & DUMP_COMPONENT_DEFINITION) + ArchiveEntry(fout, cmkinfo->dobj.catId, cmkinfo->dobj.dumpId, + ARCHIVE_OPTS(.tag = cmkinfo->dobj.name, + .namespace = cmkinfo->dobj.namespace->dobj.name, + .owner = cmkinfo->rolname, + .description = "COLUMN MASTER KEY", + .section = SECTION_PRE_DATA, + .createStmt = query->data, + .dropStmt = delq->data)); + + if (cmkinfo->dobj.dump & DUMP_COMPONENT_COMMENT) + dumpComment(fout, "COLUMN MASTER KEY", qcmkname, + cmkinfo->dobj.namespace->dobj.name, cmkinfo->rolname, + cmkinfo->dobj.catId, 0, cmkinfo->dobj.dumpId); + + if (cmkinfo->dobj.dump & DUMP_COMPONENT_SECLABEL) + dumpSecLabel(fout, "COLUMN MASTER KEY", qcmkname, + cmkinfo->dobj.namespace->dobj.name, cmkinfo->rolname, + cmkinfo->dobj.catId, 0, cmkinfo->dobj.dumpId); + + if (cmkinfo->dobj.dump & DUMP_COMPONENT_ACL) + dumpACL(fout, cmkinfo->dobj.dumpId, InvalidDumpId, "COLUMN MASTER KEY", + qcmkname, NULL, cmkinfo->dobj.namespace->dobj.name, + NULL, cmkinfo->rolname, &cmkinfo->dacl); + + destroyPQExpBuffer(delq); + destroyPQExpBuffer(query); + free(qcmkname); +} + /* * dumpConversion * write out a single conversion definition @@ -16193,6 +16540,23 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo) tbinfo->atttypnames[j]); } + if (tbinfo->attcek[j]) + { + appendPQExpBuffer(q, " ENCRYPTED WITH (column_encryption_key = %s, ", + fmtQualifiedDumpable(tbinfo->attcek[j])); + + /* + * To reduce output size, we don't print the default + * of encryption_type, but we do print the default of + * algorithm, since we might want to change to a new + * default algorithm sometime in the future. + */ + if (tbinfo->attencdet[j]) + appendPQExpBuffer(q, "encryption_type = deterministic, "); + appendPQExpBuffer(q, "algorithm = '%s')", + get_cekalg_name(tbinfo->attencalg[j])); + } + if (print_default) { if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED) @@ -18702,6 +19066,8 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs, case DO_ACCESS_METHOD: case DO_OPCLASS: case DO_OPFAMILY: + case DO_CEK: + case DO_CMK: case DO_COLLATION: case DO_CONVERSION: case DO_TABLE: diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h index 2a7c5873a0a..f451281dc74 100644 --- a/src/bin/pg_dump/pg_dump.h +++ b/src/bin/pg_dump/pg_dump.h @@ -47,6 +47,8 @@ typedef enum DO_ACCESS_METHOD, DO_OPCLASS, DO_OPFAMILY, + DO_CEK, + DO_CMK, DO_COLLATION, DO_CONVERSION, DO_TABLE, @@ -338,6 +340,9 @@ typedef struct _tableInfo bool *attisdropped; /* true if attr is dropped; don't dump it */ char *attidentity; char *attgenerated; + struct _CekInfo **attcek; + int *attencalg; + bool *attencdet; int *attlen; /* attribute length, used by binary_upgrade */ char *attalign; /* attribute align, used by binary_upgrade */ bool *attislocal; /* true if attr has local definition */ @@ -698,6 +703,32 @@ typedef struct _SubRelInfo char *srsublsn; } SubRelInfo; +/* + * The CekInfo struct is used to represent column encryption key. + */ +typedef struct _CekInfo +{ + DumpableObject dobj; + DumpableAcl dacl; + const char *rolname; + int numdata; + /* The following are arrays of numdata entries each: */ + struct _CmkInfo **cekcmks; + int *cekcmkalgs; + char **cekencvals; +} CekInfo; + +/* + * The CmkInfo struct is used to represent column master key. + */ +typedef struct _CmkInfo +{ + DumpableObject dobj; + DumpableAcl dacl; + const char *rolname; + char *cmkrealm; +} CmkInfo; + /* * common utility functions */ @@ -719,6 +750,8 @@ extern TableInfo *findTableByOid(Oid oid); extern TypeInfo *findTypeByOid(Oid oid); extern FuncInfo *findFuncByOid(Oid oid); extern OprInfo *findOprByOid(Oid oid); +extern CekInfo *findCekByOid(Oid oid); +extern CmkInfo *findCmkByOid(Oid oid); extern CollInfo *findCollationByOid(Oid oid); extern NamespaceInfo *findNamespaceByOid(Oid oid); extern ExtensionInfo *findExtensionByOid(Oid oid); @@ -747,6 +780,8 @@ extern AccessMethodInfo *getAccessMethods(Archive *fout, int *numAccessMethods); extern OpclassInfo *getOpclasses(Archive *fout, int *numOpclasses); extern OpfamilyInfo *getOpfamilies(Archive *fout, int *numOpfamilies); extern CollInfo *getCollations(Archive *fout, int *numCollations); +extern void getColumnEncryptionKeys(Archive *fout); +extern void getColumnMasterKeys(Archive *fout); extern ConvInfo *getConversions(Archive *fout, int *numConversions); extern TableInfo *getTables(Archive *fout, int *numTables); extern void getOwnedSeqs(Archive *fout, TableInfo tblinfo[], int numTables); diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c index 7362f7c961a..17d7fdd74d5 100644 --- a/src/bin/pg_dump/pg_dump_sort.c +++ b/src/bin/pg_dump/pg_dump_sort.c @@ -71,6 +71,8 @@ enum dbObjectTypePriorities PRIO_TSTEMPLATE, PRIO_TSDICT, PRIO_TSCONFIG, + PRIO_CMK, + PRIO_CEK, PRIO_FDW, PRIO_FOREIGN_SERVER, PRIO_TABLE, @@ -114,6 +116,8 @@ static const int dbObjectTypePriority[] = [DO_ACCESS_METHOD] = PRIO_ACCESS_METHOD, [DO_OPCLASS] = PRIO_OPFAMILY, [DO_OPFAMILY] = PRIO_OPFAMILY, + [DO_CEK] = PRIO_CEK, + [DO_CMK] = PRIO_CMK, [DO_COLLATION] = PRIO_COLLATION, [DO_CONVERSION] = PRIO_CONVERSION, [DO_TABLE] = PRIO_TABLE, @@ -1299,6 +1303,16 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize) "OPERATOR FAMILY %s (ID %d OID %u)", obj->name, obj->dumpId, obj->catId.oid); return; + case DO_CEK: + snprintf(buf, bufsize, + "COLUMN ENCRYPTION KEY (ID %d OID %u)", + obj->dumpId, obj->catId.oid); + return; + case DO_CMK: + snprintf(buf, bufsize, + "COLUMN MASTER KEY (ID %d OID %u)", + obj->dumpId, obj->catId.oid); + return; case DO_COLLATION: snprintf(buf, bufsize, "COLLATION %s (ID %d OID %u)", diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c index 73337f33923..84f623b2c9c 100644 --- a/src/bin/pg_dump/pg_dumpall.c +++ b/src/bin/pg_dump/pg_dumpall.c @@ -93,6 +93,7 @@ static bool dosync = true; static int binary_upgrade = 0; static int column_inserts = 0; +static int decrypt_encrypted_columns = 0; static int disable_dollar_quoting = 0; static int disable_triggers = 0; static int if_exists = 0; @@ -154,6 +155,7 @@ main(int argc, char *argv[]) {"attribute-inserts", no_argument, &column_inserts, 1}, {"binary-upgrade", no_argument, &binary_upgrade, 1}, {"column-inserts", no_argument, &column_inserts, 1}, + {"decrypt-encrypted-columns", no_argument, &decrypt_encrypted_columns, 1}, {"disable-dollar-quoting", no_argument, &disable_dollar_quoting, 1}, {"disable-triggers", no_argument, &disable_triggers, 1}, {"exclude-database", required_argument, NULL, 6}, @@ -429,6 +431,8 @@ main(int argc, char *argv[]) appendPQExpBufferStr(pgdumpopts, " --binary-upgrade"); if (column_inserts) appendPQExpBufferStr(pgdumpopts, " --column-inserts"); + if (decrypt_encrypted_columns) + appendPQExpBufferStr(pgdumpopts, " --decrypt-encrypted-columns"); if (disable_dollar_quoting) appendPQExpBufferStr(pgdumpopts, " --disable-dollar-quoting"); if (disable_triggers) @@ -654,6 +658,7 @@ help(void) printf(_(" -x, --no-privileges do not dump privileges (grant/revoke)\n")); printf(_(" --binary-upgrade for use by upgrade utilities only\n")); printf(_(" --column-inserts dump data as INSERT commands with column names\n")); + printf(_(" --decrypt-encrypted-columns decrypt encrypted columns in the output\n")); printf(_(" --disable-dollar-quoting disable dollar quoting, use SQL standard quoting\n")); printf(_(" --disable-triggers disable triggers during data-only restore\n")); printf(_(" --exclude-database=PATTERN exclude databases whose name matches PATTERN\n")); diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index 0c057fef947..04f94f73489 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -810,6 +810,29 @@ unlike => { no_owner => 1, }, }, + 'ALTER COLUMN ENCRYPTION KEY cek1 OWNER TO' => { + regexp => + qr/^ALTER COLUMN ENCRYPTION KEY dump_test.cek1 OWNER TO .+;/m, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => { + exclude_dump_test_schema => 1, + no_owner => 1, + only_dump_measurement => 1, + }, + }, + + 'ALTER COLUMN MASTER KEY cmk1 OWNER TO' => { + regexp => qr/^ALTER COLUMN MASTER KEY dump_test.cmk1 OWNER TO .+;/m, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => { + exclude_dump_test_schema => 1, + no_owner => 1, + only_dump_measurement => 1, + }, + }, + 'ALTER FOREIGN DATA WRAPPER dummy OWNER TO' => { regexp => qr/^ALTER FOREIGN DATA WRAPPER dummy OWNER TO .+;/m, like => { %full_runs, section_pre_data => 1, }, @@ -1516,6 +1539,34 @@ like => { %full_runs, section_pre_data => 1, }, }, + 'COMMENT ON COLUMN ENCRYPTION KEY cek1' => { + create_order => 55, + create_sql => 'COMMENT ON COLUMN ENCRYPTION KEY dump_test.cek1 + IS \'comment on column encryption key\';', + regexp => + qr/^COMMENT ON COLUMN ENCRYPTION KEY dump_test.cek1 IS 'comment on column encryption key';/m, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => { + exclude_dump_test_schema => 1, + only_dump_measurement => 1, + }, + }, + + 'COMMENT ON COLUMN MASTER KEY cmk1' => { + create_order => 55, + create_sql => 'COMMENT ON COLUMN MASTER KEY dump_test.cmk1 + IS \'comment on column master key\';', + regexp => + qr/^COMMENT ON COLUMN MASTER KEY dump_test.cmk1 IS 'comment on column master key';/m, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => { + exclude_dump_test_schema => 1, + only_dump_measurement => 1, + }, + }, + 'COMMENT ON LARGE OBJECT ...' => { create_order => 65, create_sql => 'DO $$ @@ -1993,6 +2044,32 @@ like => { %full_runs, section_pre_data => 1, }, }, + 'CREATE COLUMN ENCRYPTION KEY cek1' => { + create_order => 51, + create_sql => + "CREATE COLUMN ENCRYPTION KEY dump_test.cek1 WITH VALUES (column_master_key = dump_test.cmk1, algorithm = 'RSAES_OAEP_SHA_1', encrypted_value = '\\xDEADBEEF');", + regexp => qr/^ + \QCREATE COLUMN ENCRYPTION KEY dump_test.cek1 WITH VALUES (column_master_key = dump_test.cmk1, algorithm = 'RSAES_OAEP_SHA_1', encrypted_value = \E + /xm, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => + { exclude_dump_test_schema => 1, only_dump_measurement => 1, }, + }, + + 'CREATE COLUMN MASTER KEY cmk1' => { + create_order => 50, + create_sql => + "CREATE COLUMN MASTER KEY dump_test.cmk1 WITH (realm = 'myrealm');", + regexp => qr/^ + \QCREATE COLUMN MASTER KEY dump_test.cmk1 WITH (realm = 'myrealm');\E + /xm, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => + { exclude_dump_test_schema => 1, only_dump_measurement => 1, }, + }, + 'CREATE DATABASE postgres' => { regexp => qr/^ \QCREATE DATABASE postgres WITH TEMPLATE = template0 \E @@ -4101,6 +4178,38 @@ unlike => { no_privs => 1, }, }, + 'GRANT USAGE ON COLUMN ENCRYPTION KEY cek1' => { + create_order => 85, + create_sql => + 'GRANT USAGE ON COLUMN ENCRYPTION KEY dump_test.cek1 TO regress_dump_test_role;', + regexp => qr/^ + \QGRANT ALL ON COLUMN ENCRYPTION KEY dump_test.cek1 TO regress_dump_test_role;\E + /xm, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => { + exclude_dump_test_schema => 1, + no_privs => 1, + only_dump_measurement => 1, + }, + }, + + 'GRANT USAGE ON COLUMN MASTER KEY cmk1' => { + create_order => 85, + create_sql => + 'GRANT USAGE ON COLUMN MASTER KEY dump_test.cmk1 TO regress_dump_test_role;', + regexp => qr/^ + \QGRANT ALL ON COLUMN MASTER KEY dump_test.cmk1 TO regress_dump_test_role;\E + /xm, + like => + { %full_runs, %dump_test_schema_runs, section_pre_data => 1, }, + unlike => { + exclude_dump_test_schema => 1, + no_privs => 1, + only_dump_measurement => 1, + }, + }, + 'GRANT USAGE ON FOREIGN DATA WRAPPER dummy' => { create_order => 85, create_sql => 'GRANT USAGE ON FOREIGN DATA WRAPPER dummy diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c index 288c1a8c935..125d7cea8c4 100644 --- a/src/bin/psql/command.c +++ b/src/bin/psql/command.c @@ -818,7 +818,11 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd) success = describeTablespaces(pattern, show_verbose); break; case 'c': - if (strncmp(cmd, "dconfig", 7) == 0) + if (strncmp(cmd, "dcek", 4) == 0) + success = listCEKs(pattern, show_verbose); + else if (strncmp(cmd, "dcmk", 4) == 0) + success = listCMKs(pattern, show_verbose); + else if (strncmp(cmd, "dconfig", 7) == 0) success = describeConfigurationParameters(pattern, show_verbose, show_system); diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c index 6433497bcd2..bd7ab861553 100644 --- a/src/bin/psql/describe.c +++ b/src/bin/psql/describe.c @@ -1538,7 +1538,7 @@ describeOneTableDetails(const char *schemaname, bool printTableInitialized = false; int i; char *view_def = NULL; - char *headers[12]; + char *headers[13]; PQExpBufferData title; PQExpBufferData tmpbuf; int cols; @@ -1554,6 +1554,7 @@ describeOneTableDetails(const char *schemaname, fdwopts_col = -1, attstorage_col = -1, attcompression_col = -1, + attcekname_col = -1, attstattarget_col = -1, attdescr_col = -1; int numrows; @@ -1576,6 +1577,8 @@ describeOneTableDetails(const char *schemaname, char *relam; } tableinfo; bool show_column_details = false; + const char *attusertypid; + const char *attusertypmod; myopt.default_footer = false; /* This output looks confusing in expanded mode. */ @@ -1852,7 +1855,17 @@ describeOneTableDetails(const char *schemaname, cols = 0; printfPQExpBuffer(&buf, "SELECT a.attname"); attname_col = cols++; - appendPQExpBufferStr(&buf, ",\n pg_catalog.format_type(a.atttypid, a.atttypmod)"); + if (pset.sversion >= 170000) + { + attusertypid = "CASE WHEN a.attusertypid IS NOT NULL THEN a.attusertypid ELSE a.atttypid END"; + attusertypmod = "CASE WHEN a.attusertypid IS NOT NULL THEN a.attusertypmod ELSE a.atttypmod END"; + } + else + { + attusertypid = "a.atttypid"; + attusertypmod = "a.atttypmod"; + } + appendPQExpBuffer(&buf, ",\n pg_catalog.format_type(%s, %s)", attusertypid, attusertypmod); atttype_col = cols++; if (show_column_details) @@ -1865,7 +1878,8 @@ describeOneTableDetails(const char *schemaname, ",\n a.attnotnull"); attrdef_col = cols++; attnotnull_col = cols++; - appendPQExpBufferStr(&buf, ",\n (SELECT c.collname FROM pg_catalog.pg_collation c, pg_catalog.pg_type t\n" + appendPQExpBufferStr(&buf, ",\n" + " (SELECT c.collname FROM pg_catalog.pg_collation c, pg_catalog.pg_type t\n" " WHERE c.oid = a.attcollation AND t.oid = a.atttypid AND a.attcollation <> t.typcollation) AS attcollation"); attcoll_col = cols++; if (pset.sversion >= 100000) @@ -1917,6 +1931,18 @@ describeOneTableDetails(const char *schemaname, attcompression_col = cols++; } + /* encryption info */ + if (pset.sversion >= 170000 && + !pset.hide_column_encryption && + (tableinfo.relkind == RELKIND_RELATION || + tableinfo.relkind == RELKIND_VIEW || + tableinfo.relkind == RELKIND_MATVIEW || + tableinfo.relkind == RELKIND_PARTITIONED_TABLE)) + { + appendPQExpBufferStr(&buf, ",\n (SELECT cekname FROM pg_colenckey cek WHERE cek.oid = a.attcek) AS attcekname"); + attcekname_col = cols++; + } + /* stats target, if relevant to relkind */ if (tableinfo.relkind == RELKIND_RELATION || tableinfo.relkind == RELKIND_INDEX || @@ -2040,6 +2066,8 @@ describeOneTableDetails(const char *schemaname, headers[cols++] = gettext_noop("Storage"); if (attcompression_col >= 0) headers[cols++] = gettext_noop("Compression"); + if (attcekname_col >= 0) + headers[cols++] = gettext_noop("Encryption"); if (attstattarget_col >= 0) headers[cols++] = gettext_noop("Stats target"); if (attdescr_col >= 0) @@ -2132,6 +2160,17 @@ describeOneTableDetails(const char *schemaname, false, false); } + /* Column encryption */ + if (attcekname_col >= 0) + { + if (!PQgetisnull(res, i, attcekname_col)) + printTableAddCell(&cont, PQgetvalue(res, i, attcekname_col), + false, false); + else + printTableAddCell(&cont, "", + false, false); + } + /* Statistics target, if the relkind supports this feature */ if (attstattarget_col >= 0) printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col), @@ -4589,6 +4628,152 @@ listConversions(const char *pattern, bool verbose, bool showSystem) return true; } +/* + * \dcek + * + * Lists column encryption keys. + */ +bool +listCEKs(const char *pattern, bool verbose) +{ + PQExpBufferData buf; + PGresult *res; + printQueryOpt myopt = pset.popt; + + if (pset.sversion < 170000) + { + char sverbuf[32]; + + pg_log_error("The server (version %s) does not support column encryption.", + formatPGVersionNumber(pset.sversion, false, + sverbuf, sizeof(sverbuf))); + return true; + } + + initPQExpBuffer(&buf); + + printfPQExpBuffer(&buf, + "SELECT " + "n.nspname AS \"%s\", " + "cekname AS \"%s\", " + "pg_catalog.pg_get_userbyid(cekowner) AS \"%s\", " + "cmkname AS \"%s\"", + gettext_noop("Schema"), + gettext_noop("Name"), + gettext_noop("Owner"), + gettext_noop("Master key")); + if (verbose) + { + appendPQExpBuffer(&buf, ", "); + printACLColumn(&buf, "cekacl"); + appendPQExpBuffer(&buf, + ", pg_catalog.obj_description(cek.oid, 'pg_colenckey') AS \"%s\"", + gettext_noop("Description")); + } + appendPQExpBufferStr(&buf, + "\nFROM pg_catalog.pg_colenckey cek " + "LEFT JOIN pg_catalog.pg_namespace n ON n.oid = cek.ceknamespace " + "JOIN pg_catalog.pg_colenckeydata ckd ON (cek.oid = ckd.ckdcekid) " + "JOIN pg_catalog.pg_colmasterkey cmk ON (ckd.ckdcmkid = cmk.oid) "); + + if (!validateSQLNamePattern(&buf, pattern, false, false, + "n.nspname", "cekname", NULL, + "pg_catalog.pg_cek_is_visible(cek.oid)", + NULL, 3)) + { + termPQExpBuffer(&buf); + return false; + } + + appendPQExpBufferStr(&buf, "ORDER BY 1, 2, 4"); + + res = PSQLexec(buf.data); + termPQExpBuffer(&buf); + if (!res) + return false; + + myopt.nullPrint = NULL; + myopt.title = _("List of column encryption keys"); + myopt.translate_header = true; + + printQuery(res, &myopt, pset.queryFout, false, pset.logfile); + + PQclear(res); + return true; +} + +/* + * \dcmk + * + * Lists column master keys. + */ +bool +listCMKs(const char *pattern, bool verbose) +{ + PQExpBufferData buf; + PGresult *res; + printQueryOpt myopt = pset.popt; + + if (pset.sversion < 170000) + { + char sverbuf[32]; + + pg_log_error("The server (version %s) does not support column encryption.", + formatPGVersionNumber(pset.sversion, false, + sverbuf, sizeof(sverbuf))); + return true; + } + + initPQExpBuffer(&buf); + + printfPQExpBuffer(&buf, + "SELECT " + "n.nspname AS \"%s\", " + "cmkname AS \"%s\", " + "pg_catalog.pg_get_userbyid(cmkowner) AS \"%s\", " + "cmkrealm AS \"%s\"", + gettext_noop("Schema"), + gettext_noop("Name"), + gettext_noop("Owner"), + gettext_noop("Realm")); + if (verbose) + { + appendPQExpBuffer(&buf, ", "); + printACLColumn(&buf, "cmkacl"); + appendPQExpBuffer(&buf, + ", pg_catalog.obj_description(cmk.oid, 'pg_colmasterkey') AS \"%s\"", + gettext_noop("Description")); + } + appendPQExpBufferStr(&buf, + "\nFROM pg_catalog.pg_colmasterkey cmk " + "LEFT JOIN pg_catalog.pg_namespace n ON n.oid = cmk.cmknamespace "); + + if (!validateSQLNamePattern(&buf, pattern, false, false, + "n.nspname", "cmkname", NULL, + "pg_catalog.pg_cmk_is_visible(cmk.oid)", + NULL, 3)) + { + termPQExpBuffer(&buf); + return false; + } + + appendPQExpBufferStr(&buf, "ORDER BY 1, 2"); + + res = PSQLexec(buf.data); + termPQExpBuffer(&buf); + if (!res) + return false; + + myopt.nullPrint = NULL; + myopt.title = _("List of column master keys"); + myopt.translate_header = true; + + printQuery(res, &myopt, pset.queryFout, false, pset.logfile); + + PQclear(res); + return true; +} + /* * \dconfig * diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h index 273f974538e..8fbab5b361f 100644 --- a/src/bin/psql/describe.h +++ b/src/bin/psql/describe.h @@ -79,6 +79,12 @@ extern bool listDomains(const char *pattern, bool verbose, bool showSystem); /* \dc */ extern bool listConversions(const char *pattern, bool verbose, bool showSystem); +/* \dcek */ +extern bool listCEKs(const char *pattern, bool verbose); + +/* \dcmk */ +extern bool listCMKs(const char *pattern, bool verbose); + /* \dconfig */ extern bool describeConfigurationParameters(const char *pattern, bool verbose, bool showSystem); diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c index 4e79a819d87..f76b1bb5d0b 100644 --- a/src/bin/psql/help.c +++ b/src/bin/psql/help.c @@ -229,6 +229,8 @@ slashUsage(unsigned short int pager) HELP0(" \\dAp[+] [AMPTRN [OPFPTRN]] list support functions of operator families\n"); HELP0(" \\db[+] [PATTERN] list tablespaces\n"); HELP0(" \\dc[S+] [PATTERN] list conversions\n"); + HELP0(" \\dcek[+] [PATTERN] list column encryption keys\n"); + HELP0(" \\dcmk[+] [PATTERN] list column master keys\n"); HELP0(" \\dconfig[+] [PATTERN] list configuration parameters\n"); HELP0(" \\dC[+] [PATTERN] list casts\n"); HELP0(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"); @@ -391,6 +393,8 @@ helpVariables(unsigned short int pager) " \"true\" if last query failed, else \"false\"\n"); HELP0(" FETCH_COUNT\n" " the number of result rows to fetch and display at a time (0 = unlimited)\n"); + HELP0(" HIDE_COLUMN_ENCRYPTION\n" + " if set, column encryption details are not displayed\n"); HELP0(" HIDE_TABLEAM\n" " if set, table access methods are not displayed\n"); HELP0(" HIDE_TOAST_COMPRESSION\n" diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h index 505f99d8e47..f2207220635 100644 --- a/src/bin/psql/settings.h +++ b/src/bin/psql/settings.h @@ -138,6 +138,7 @@ typedef struct _psqlSettings bool quiet; bool singleline; bool singlestep; + bool hide_column_encryption; bool hide_compression; bool hide_tableam; int fetch_count; diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c index 036caaec2ff..92ed10de189 100644 --- a/src/bin/psql/startup.c +++ b/src/bin/psql/startup.c @@ -1188,6 +1188,13 @@ hide_compression_hook(const char *newval) &pset.hide_compression); } +static bool +hide_column_encryption_hook(const char *newval) +{ + return ParseVariableBool(newval, "HIDE_COLUMN_ENCRYPTION", + &pset.hide_column_encryption); +} + static bool hide_tableam_hook(const char *newval) { @@ -1259,6 +1266,9 @@ EstablishVariableSpace(void) SetVariableHooks(pset.vars, "SHOW_CONTEXT", show_context_substitute_hook, show_context_hook); + SetVariableHooks(pset.vars, "HIDE_COLUMN_ENCRYPTION", + bool_substitute_hook, + hide_column_encryption_hook); SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION", bool_substitute_hook, hide_compression_hook); diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index 6fee3160f02..d5b5a9b4366 100644 --- a/src/bin/psql/tab-complete.c +++ b/src/bin/psql/tab-complete.c @@ -934,6 +934,20 @@ static const SchemaQuery Query_for_list_of_collations = { .result = "c.collname", }; +static const SchemaQuery Query_for_list_of_ceks = { + .catname = "pg_catalog.pg_colenckey c", + .viscondition = "pg_catalog.pg_cek_is_visible(c.oid)", + .namespace = "c.ceknamespace", + .result = "c.cekname", +}; + +static const SchemaQuery Query_for_list_of_cmks = { + .catname = "pg_catalog.pg_colmasterkey c", + .viscondition = "pg_catalog.pg_cmk_is_visible(c.oid)", + .namespace = "c.cmknamespace", + .result = "c.cmkname", +}; + static const SchemaQuery Query_for_partition_of_table = { .catname = "pg_catalog.pg_class c1, pg_catalog.pg_class c2, pg_catalog.pg_inherits i", .selcondition = "c1.oid=i.inhparent and i.inhrelid=c2.oid and c2.relispartition", @@ -1228,6 +1242,8 @@ static const pgsql_thing_t words_after_create[] = { {"CAST", NULL, NULL, NULL}, /* Casts have complex structures for names, so * skip it */ {"COLLATION", NULL, NULL, &Query_for_list_of_collations}, + {"COLUMN ENCRYPTION KEY", NULL, NULL, NULL}, + {"COLUMN MASTER KEY KEY", NULL, NULL, NULL}, /* * CREATE CONSTRAINT TRIGGER is not supported here because it is designed @@ -1717,7 +1733,7 @@ psql_completion(const char *text, int start, int end) "\\connect", "\\conninfo", "\\C", "\\cd", "\\copy", "\\copyright", "\\crosstabview", "\\d", "\\da", "\\dA", "\\dAc", "\\dAf", "\\dAo", "\\dAp", - "\\db", "\\dc", "\\dconfig", "\\dC", "\\dd", "\\ddp", "\\dD", + "\\db", "\\dc", "\\dcek", "\\dcmk", "\\dconfig", "\\dC", "\\dd", "\\ddp", "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df", "\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL", "\\dm", "\\dn", "\\do", "\\dO", "\\dp", "\\dP", "\\dPi", "\\dPt", @@ -1974,6 +1990,22 @@ psql_completion(const char *text, int start, int end) else if (Matches("ALTER", "COLLATION", MatchAny)) COMPLETE_WITH("OWNER TO", "REFRESH VERSION", "RENAME TO", "SET SCHEMA"); + /* ALTER/DROP COLUMN ENCRYPTION KEY */ + else if (Matches("ALTER|DROP", "COLUMN", "ENCRYPTION", "KEY")) + COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_ceks); + + /* ALTER COLUMN ENCRYPTION KEY */ + else if (Matches("ALTER", "COLUMN", "ENCRYPTION", "KEY", MatchAny)) + COMPLETE_WITH("ADD VALUE (", "DROP VALUE (", "OWNER TO", "RENAME TO", "SET SCHEMA"); + + /* ALTER/DROP COLUMN MASTER KEY */ + else if (Matches("ALTER|DROP", "COLUMN", "MASTER", "KEY")) + COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_cmks); + + /* ALTER COLUMN MASTER KEY */ + else if (Matches("ALTER", "COLUMN", "MASTER", "KEY", MatchAny)) + COMPLETE_WITH("(", "OWNER TO", "RENAME TO", "SET SCHEMA"); + /* ALTER CONVERSION */ else if (Matches("ALTER", "CONVERSION", MatchAny)) COMPLETE_WITH("OWNER TO", "RENAME TO", "SET SCHEMA"); @@ -2949,6 +2981,26 @@ psql_completion(const char *text, int start, int end) COMPLETE_WITH("true", "false"); } + /* CREATE/ALTER/DROP COLUMN ... KEY */ + else if (Matches("CREATE|ALTER|DROP", "COLUMN")) + COMPLETE_WITH("ENCRYPTION", "MASTER"); + else if (Matches("CREATE|ALTER|DROP", "COLUMN", "ENCRYPTION|MASTER")) + COMPLETE_WITH("KEY"); + + /* CREATE COLUMN ENCRYPTION KEY */ + else if (Matches("CREATE", "COLUMN", "ENCRYPTION", "KEY", MatchAny)) + COMPLETE_WITH("WITH"); + else if (Matches("CREATE", "COLUMN", "ENCRYPTION", "KEY", MatchAny, "WITH")) + COMPLETE_WITH("VALUES"); + else if (Matches("CREATE", "COLUMN", "ENCRYPTION", "KEY", MatchAny, "WITH", "VALUES")) + COMPLETE_WITH("("); + + /* CREATE COLUMN MASTER KEY */ + else if (Matches("CREATE", "COLUMN", "MASTER", "KEY", MatchAny)) + COMPLETE_WITH("WITH"); + else if (Matches("CREATE", "COLUMN", "MASTER", "KEY", MatchAny, "WITH")) + COMPLETE_WITH("("); + /* CREATE DATABASE */ else if (Matches("CREATE", "DATABASE", MatchAny)) COMPLETE_WITH("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE", @@ -3708,7 +3760,7 @@ psql_completion(const char *text, int start, int end) /* DISCARD */ else if (Matches("DISCARD")) - COMPLETE_WITH("ALL", "PLANS", "SEQUENCES", "TEMP"); + COMPLETE_WITH("ALL", "COLUMN ENCRYPTION KEYS", "PLANS", "SEQUENCES", "TEMP"); /* DO */ else if (Matches("DO")) @@ -3722,6 +3774,7 @@ psql_completion(const char *text, int start, int end) Matches("DROP", "ACCESS", "METHOD", MatchAny) || (Matches("DROP", "AGGREGATE|FUNCTION|PROCEDURE|ROUTINE", MatchAny, MatchAny) && ends_with(prev_wd, ')')) || + Matches("DROP", "COLUMN", "ENCRYPTION|MASTER", "KEY", MatchAny) || Matches("DROP", "EVENT", "TRIGGER", MatchAny) || Matches("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) || Matches("DROP", "FOREIGN", "TABLE", MatchAny) || @@ -4043,6 +4096,8 @@ psql_completion(const char *text, int start, int end) "ALL ROUTINES IN SCHEMA", "ALL SEQUENCES IN SCHEMA", "ALL TABLES IN SCHEMA", + "COLUMN ENCRYPTION KEY", + "COLUMN MASTER KEY", "DATABASE", "DOMAIN", "FOREIGN DATA WRAPPER", @@ -4161,6 +4216,19 @@ psql_completion(const char *text, int start, int end) COMPLETE_WITH("FROM"); } + /* + * Complete "GRANT/REVOKE * ON COLUMN ENCRYPTION|MASTER KEY *" with + * TO/FROM + */ + else if (TailMatches("GRANT|REVOKE", MatchAny, "ON", "COLUMN", "ENCRYPTION|MASTER", "KEY", MatchAny) || + TailMatches("REVOKE", "GRANT", "OPTION", "FOR", MatchAny, "ON", "COLUMN", "ENCRYPTION|MASTER", "KEY", MatchAny)) + { + if (TailMatches("GRANT", MatchAny, MatchAny, MatchAny, MatchAny, MatchAny, MatchAny)) + COMPLETE_WITH("TO"); + else + COMPLETE_WITH("FROM"); + } + /* Complete "GRANT/REVOKE * ON FOREIGN DATA WRAPPER *" with TO/FROM */ else if (TailMatches("GRANT|REVOKE", MatchAny, "ON", "FOREIGN", "DATA", "WRAPPER", MatchAny) || TailMatches("REVOKE", "GRANT", "OPTION", "FOR", MatchAny, "ON", "FOREIGN", "DATA", "WRAPPER", MatchAny)) diff --git a/src/common/Makefile b/src/common/Makefile index 3d83299432b..448a19d6845 100644 --- a/src/common/Makefile +++ b/src/common/Makefile @@ -49,6 +49,7 @@ OBJS_COMMON = \ binaryheap.o \ blkreftable.o \ checksum_helper.o \ + colenc.o \ compression.o \ config_info.o \ controldata_utils.o \ diff --git a/src/common/colenc.c b/src/common/colenc.c new file mode 100644 index 00000000000..b3ba84087ef --- /dev/null +++ b/src/common/colenc.c @@ -0,0 +1,107 @@ +/*------------------------------------------------------------------------- + * + * colenc.c + * + * Shared code for column encryption algorithms. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/common/colenc.c + *------------------------------------------------------------------------- + */ + +#ifndef FRONTEND +#include "postgres.h" +#else +#include "postgres_fe.h" +#endif + +#include "common/colenc.h" + +int +get_cmkalg_num(const char *name) +{ + if (strcmp(name, "unspecified") == 0) + return PG_CMK_UNSPECIFIED; + else if (strcmp(name, "RSAES_OAEP_SHA_1") == 0) + return PG_CMK_RSAES_OAEP_SHA_1; + else if (strcmp(name, "RSAES_OAEP_SHA_256") == 0) + return PG_CMK_RSAES_OAEP_SHA_256; + else + return 0; +} + +const char * +get_cmkalg_name(int num) +{ + switch (num) + { + case PG_CMK_UNSPECIFIED: + return "unspecified"; + case PG_CMK_RSAES_OAEP_SHA_1: + return "RSAES_OAEP_SHA_1"; + case PG_CMK_RSAES_OAEP_SHA_256: + return "RSAES_OAEP_SHA_256"; + } + + return NULL; +} + +/* + * JSON Web Algorithms (JWA) names (RFC 7518) + * + * This is useful for some key management systems that use these names + * natively. + * + * This only supports algorithms that have a mapping in JWA. For any other + * ones, it returns NULL. + */ +const char * +get_cmkalg_jwa_name(int num) +{ + switch (num) + { + case PG_CMK_UNSPECIFIED: + return NULL; + case PG_CMK_RSAES_OAEP_SHA_1: + return "RSA-OAEP"; + case PG_CMK_RSAES_OAEP_SHA_256: + return "RSA-OAEP-256"; + } + + return NULL; +} + +int +get_cekalg_num(const char *name) +{ + if (strcmp(name, "AEAD_AES_128_CBC_HMAC_SHA_256") == 0) + return PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256; + else if (strcmp(name, "AEAD_AES_192_CBC_HMAC_SHA_384") == 0) + return PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384; + else if (strcmp(name, "AEAD_AES_256_CBC_HMAC_SHA_384") == 0) + return PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384; + else if (strcmp(name, "AEAD_AES_256_CBC_HMAC_SHA_512") == 0) + return PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512; + else + return 0; +} + +const char * +get_cekalg_name(int num) +{ + switch (num) + { + case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256: + return "AEAD_AES_128_CBC_HMAC_SHA_256"; + case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384: + return "AEAD_AES_192_CBC_HMAC_SHA_384"; + case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384: + return "AEAD_AES_256_CBC_HMAC_SHA_384"; + case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512: + return "AEAD_AES_256_CBC_HMAC_SHA_512"; + } + + return NULL; +} diff --git a/src/common/meson.build b/src/common/meson.build index de68e408fa3..9e88f31c061 100644 --- a/src/common/meson.build +++ b/src/common/meson.build @@ -6,6 +6,7 @@ common_sources = files( 'binaryheap.c', 'blkreftable.c', 'checksum_helper.c', + 'colenc.c', 'compression.c', 'controldata_utils.c', 'encnames.c', diff --git a/src/include/access/printtup.h b/src/include/access/printtup.h index b1fecf873b4..13e38ba5f3a 100644 --- a/src/include/access/printtup.h +++ b/src/include/access/printtup.h @@ -20,9 +20,13 @@ extern DestReceiver *printtup_create_DR(CommandDest dest); extern void SetRemoteDestReceiverParams(DestReceiver *self, Portal portal); +extern void MaybeSendColumnEncryptionKeyMessage(Oid attcek); + extern void SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo, List *targetlist, int16 *formats); +extern void DiscardColumnEncryptionKeys(void); + extern void debugStartup(DestReceiver *self, int operation, TupleDesc typeinfo); extern bool debugtup(TupleTableSlot *slot, DestReceiver *self); diff --git a/src/include/catalog/Makefile b/src/include/catalog/Makefile index 167f91a6e3f..9cf93c8735d 100644 --- a/src/include/catalog/Makefile +++ b/src/include/catalog/Makefile @@ -81,7 +81,10 @@ CATALOG_HEADERS := \ pg_publication_namespace.h \ pg_publication_rel.h \ pg_subscription.h \ - pg_subscription_rel.h + pg_subscription_rel.h \ + pg_colmasterkey.h \ + pg_colenckey.h \ + pg_colenckeydata.h GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h) diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h index 21e31f9c974..cfe383799c1 100644 --- a/src/include/catalog/heap.h +++ b/src/include/catalog/heap.h @@ -23,6 +23,7 @@ #define CHKATYPE_ANYARRAY 0x01 /* allow ANYARRAY */ #define CHKATYPE_ANYRECORD 0x02 /* allow RECORD and RECORD[] */ #define CHKATYPE_IS_PARTKEY 0x04 /* attname is part key # not column */ +#define CHKATYPE_ENCRYPTED 0x08 /* allow internal encrypted types */ typedef struct RawColumnDefault { @@ -72,6 +73,7 @@ extern Oid heap_create_with_catalog(const char *relname, Oid ownerid, Oid accessmtd, TupleDesc tupdesc, + const FormExtraData_pg_attribute tupdesc_extra[], List *cooked_constraints, char relkind, char relpersistence, @@ -140,7 +142,7 @@ extern const FormData_pg_attribute *SystemAttributeDefinition(AttrNumber attno); extern const FormData_pg_attribute *SystemAttributeByName(const char *attname); -extern void CheckAttributeNamesTypes(TupleDesc tupdesc, char relkind, +extern void CheckAttributeNamesTypes(TupleDesc tupdesc, const FormExtraData_pg_attribute tupdesc_extra[], char relkind, int flags); extern void CheckAttributeType(const char *attname, diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build index f70d1daba52..a5af2a86bf9 100644 --- a/src/include/catalog/meson.build +++ b/src/include/catalog/meson.build @@ -69,6 +69,9 @@ catalog_headers = [ 'pg_publication_rel.h', 'pg_subscription.h', 'pg_subscription_rel.h', + 'pg_colmasterkey.h', + 'pg_colenckey.h', + 'pg_colenckeydata.h', ] # The .dat files we need can just be listed alphabetically. diff --git a/src/include/catalog/namespace.h b/src/include/catalog/namespace.h index 8d434d48d57..553780b6ff3 100644 --- a/src/include/catalog/namespace.h +++ b/src/include/catalog/namespace.h @@ -116,6 +116,12 @@ extern bool OpclassIsVisible(Oid opcid); extern Oid OpfamilynameGetOpfid(Oid amid, const char *opfname); extern bool OpfamilyIsVisible(Oid opfid); +extern Oid get_cek_oid(List *names, bool missing_ok); +extern bool CEKIsVisible(Oid cekid); + +extern Oid get_cmk_oid(List *names, bool missing_ok); +extern bool CMKIsVisible(Oid cmkid); + extern Oid CollationGetCollid(const char *collname); extern bool CollationIsVisible(Oid collid); diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index d8a05214b11..a10077d1f13 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1028,6 +1028,11 @@ amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)', amopmethod => 'hash' }, +# pg_encrypted_det_ops +{ amopfamily => 'hash/pg_encrypted_det_ops', amoplefttype => 'pg_encrypted_det', + amoprighttype => 'pg_encrypted_det', amopstrategy => '1', + amopopr => '=(pg_encrypted_det,pg_encrypted_det)', amopmethod => 'hash' }, + # xid_ops { amopfamily => 'hash/xid_ops', amoplefttype => 'xid', amoprighttype => 'xid', amopstrategy => '1', amopopr => '=(xid,xid)', amopmethod => 'hash' }, diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat index 352558c1f06..b004bba7620 100644 --- a/src/include/catalog/pg_amproc.dat +++ b/src/include/catalog/pg_amproc.dat @@ -400,6 +400,12 @@ { amprocfamily => 'hash/bytea_ops', amproclefttype => 'bytea', amprocrighttype => 'bytea', amprocnum => '2', amproc => 'hashvarlenaextended' }, +{ amprocfamily => 'hash/pg_encrypted_det_ops', + amproclefttype => 'pg_encrypted_det', amprocrighttype => 'pg_encrypted_det', + amprocnum => '1', amproc => 'hashvarlena' }, +{ amprocfamily => 'hash/pg_encrypted_det_ops', + amproclefttype => 'pg_encrypted_det', amprocrighttype => 'pg_encrypted_det', + amprocnum => '2', amproc => 'hashvarlenaextended' }, { amprocfamily => 'hash/xid_ops', amproclefttype => 'xid', amprocrighttype => 'xid', amprocnum => '1', amproc => 'hashint4' }, { amprocfamily => 'hash/xid_ops', amproclefttype => 'xid', diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h index 1c62b8bfcb5..3cc32452820 100644 --- a/src/include/catalog/pg_attribute.h +++ b/src/include/catalog/pg_attribute.h @@ -184,6 +184,15 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75, /* Column-level FDW options */ text attfdwoptions[1] BKI_DEFAULT(_null_); + /* column encryption key */ + Oid attcek BKI_DEFAULT(_null_) BKI_FORCE_NULL BKI_LOOKUP_OPT(pg_colenckey); + + /* + * User-visible type and typmod, currently used for encrypted columns. + */ + Oid attusertypid BKI_DEFAULT(_null_) BKI_FORCE_NULL BKI_LOOKUP_OPT(pg_type); + int32 attusertypmod BKI_DEFAULT(_null_) BKI_FORCE_NULL; + /* * Missing value for added columns. This is a one element array which lets * us store a value of the attribute type here. @@ -199,7 +208,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75, * can access the variable-length fields except in a real tuple! */ #define ATTRIBUTE_FIXED_PART_SIZE \ - (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid)) + (offsetof(FormData_pg_attribute,attcollation) + sizeof(int32)) /* ---------------- * Form_pg_attribute corresponds to a pointer to a tuple with @@ -220,6 +229,9 @@ typedef struct FormExtraData_pg_attribute { NullableDatum attstattarget; NullableDatum attoptions; + NullableDatum attcek; + NullableDatum attusertypid; + NullableDatum attusertypmod; } FormExtraData_pg_attribute; DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, AttributeRelidNameIndexId, pg_attribute, btree(attrelid oid_ops, attname name_ops)); diff --git a/src/include/catalog/pg_colenckey.h b/src/include/catalog/pg_colenckey.h new file mode 100644 index 00000000000..c69d245109e --- /dev/null +++ b/src/include/catalog/pg_colenckey.h @@ -0,0 +1,49 @@ +/*------------------------------------------------------------------------- + * + * pg_colenckey.h + * definition of the "column encryption key" system catalog + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/catalog/pg_colenkey.h + * + * NOTES + * The Catalog.pm module reads this file and derives schema + * information. + * + *------------------------------------------------------------------------- + */ +#ifndef PG_COLENCKEY_H +#define PG_COLENCKEY_H + +#include "catalog/genbki.h" +#include "catalog/pg_colenckey_d.h" + +/* ---------------- + * pg_colenckey definition. cpp turns this into + * typedef struct FormData_pg_colenckey + * ---------------- + */ +CATALOG(pg_colenckey,8234,ColumnEncKeyRelationId) +{ + Oid oid; + NameData cekname; + Oid ceknamespace BKI_DEFAULT(pg_catalog) BKI_LOOKUP(pg_namespace); + Oid cekowner BKI_LOOKUP(pg_authid); +#ifdef CATALOG_VARLEN /* variable-length fields start here */ + aclitem cekacl[1] BKI_DEFAULT(_null_); +#endif +} FormData_pg_colenckey; + +typedef FormData_pg_colenckey *Form_pg_colenckey; + +DECLARE_TOAST(pg_colenckey, 8263, 8264); + +DECLARE_UNIQUE_INDEX_PKEY(pg_colenckey_oid_index, 8240, ColumnEncKeyOidIndexId, pg_colenckey, btree(oid oid_ops)); +DECLARE_UNIQUE_INDEX(pg_colenckey_cekname_nsp_index, 8242, ColumnEncKeyNameNspIndexId, pg_colenckey, btree(cekname name_ops, ceknamespace oid_ops)); + +MAKE_SYSCACHE(CEKOID, pg_colenckey_oid_index, 8); +MAKE_SYSCACHE(CEKNAMENSP, pg_colenckey_cekname_nsp_index, 8); + +#endif diff --git a/src/include/catalog/pg_colenckeydata.h b/src/include/catalog/pg_colenckeydata.h new file mode 100644 index 00000000000..e6ce40ad481 --- /dev/null +++ b/src/include/catalog/pg_colenckeydata.h @@ -0,0 +1,49 @@ +/*------------------------------------------------------------------------- + * + * pg_colenckeydata.h + * definition of the "column encryption key data" system catalog + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/catalog/pg_colenkeydata.h + * + * NOTES + * The Catalog.pm module reads this file and derives schema + * information. + * + *------------------------------------------------------------------------- + */ +#ifndef PG_COLENCKEYDATA_H +#define PG_COLENCKEYDATA_H + +#include "catalog/genbki.h" +#include "catalog/pg_colenckeydata_d.h" + +/* ---------------- + * pg_colenckeydata definition. cpp turns this into + * typedef struct FormData_pg_colenckeydata + * ---------------- + */ +CATALOG(pg_colenckeydata,8250,ColumnEncKeyDataRelationId) +{ + Oid oid; + Oid ckdcekid BKI_LOOKUP(pg_colenckey); + Oid ckdcmkid BKI_LOOKUP(pg_colmasterkey); + int32 ckdcmkalg; /* PG_CMK_* values */ +#ifdef CATALOG_VARLEN /* variable-length fields start here */ + bytea ckdencval BKI_FORCE_NOT_NULL; +#endif +} FormData_pg_colenckeydata; + +typedef FormData_pg_colenckeydata *Form_pg_colenckeydata; + +DECLARE_TOAST(pg_colenckeydata, 8237, 8238); + +DECLARE_UNIQUE_INDEX_PKEY(pg_colenckeydata_oid_index, 8251, ColumnEncKeyDataOidIndexId, pg_colenckeydata, btree(oid oid_ops)); +DECLARE_UNIQUE_INDEX(pg_colenckeydata_ckdcekid_ckdcmkid_index, 8252, ColumnEncKeyCekidCmkidIndexId, pg_colenckeydata, btree(ckdcekid oid_ops, ckdcmkid oid_ops)); + +MAKE_SYSCACHE(CEKDATAOID, pg_colenckeydata_oid_index, 8); +MAKE_SYSCACHE(CEKDATACEKCMK, pg_colenckeydata_ckdcekid_ckdcmkid_index, 8); + +#endif diff --git a/src/include/catalog/pg_colmasterkey.h b/src/include/catalog/pg_colmasterkey.h new file mode 100644 index 00000000000..1629760a9b9 --- /dev/null +++ b/src/include/catalog/pg_colmasterkey.h @@ -0,0 +1,50 @@ +/*------------------------------------------------------------------------- + * + * pg_colmasterkey.h + * definition of the "column master key" system catalog + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/catalog/pg_colmasterkey.h + * + * NOTES + * The Catalog.pm module reads this file and derives schema + * information. + * + *------------------------------------------------------------------------- + */ +#ifndef PG_COLMASTERKEY_H +#define PG_COLMASTERKEY_H + +#include "catalog/genbki.h" +#include "catalog/pg_colmasterkey_d.h" + +/* ---------------- + * pg_colmasterkey definition. cpp turns this into + * typedef struct FormData_pg_colmasterkey + * ---------------- + */ +CATALOG(pg_colmasterkey,8233,ColumnMasterKeyRelationId) +{ + Oid oid; + NameData cmkname; + Oid cmknamespace BKI_DEFAULT(pg_catalog) BKI_LOOKUP(pg_namespace); + Oid cmkowner BKI_LOOKUP(pg_authid); +#ifdef CATALOG_VARLEN /* variable-length fields start here */ + text cmkrealm BKI_FORCE_NOT_NULL; + aclitem cmkacl[1] BKI_DEFAULT(_null_); +#endif +} FormData_pg_colmasterkey; + +typedef FormData_pg_colmasterkey *Form_pg_colmasterkey; + +DECLARE_TOAST(pg_colmasterkey, 8235, 8236); + +DECLARE_UNIQUE_INDEX_PKEY(pg_colmasterkey_oid_index, 8239, ColumnMasterKeyOidIndexId, pg_colmasterkey, btree(oid oid_ops)); +DECLARE_UNIQUE_INDEX(pg_colmasterkey_cmkname_nsp_index, 8241, ColumnMasterKeyNameNspIndexId, pg_colmasterkey, btree(cmkname name_ops, cmknamespace oid_ops)); + +MAKE_SYSCACHE(CMKOID, pg_colmasterkey_oid_index, 8); +MAKE_SYSCACHE(CMKNAMENSP, pg_colmasterkey_cmkname_nsp_index, 8); + +#endif diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat index 6c30770fe7c..c3cc8fd919a 100644 --- a/src/include/catalog/pg_opclass.dat +++ b/src/include/catalog/pg_opclass.dat @@ -166,6 +166,8 @@ opcintype => 'bool' }, { opcmethod => 'hash', opcname => 'bytea_ops', opcfamily => 'hash/bytea_ops', opcintype => 'bytea' }, +{ opcmethod => 'hash', opcname => 'pg_encrypted_det_ops', + opcfamily => 'hash/pg_encrypted_det_ops', opcintype => 'pg_encrypted_det' }, { opcmethod => 'btree', opcname => 'tid_ops', opcfamily => 'btree/tid_ops', opcintype => 'tid' }, { opcmethod => 'hash', opcname => 'xid_ops', opcfamily => 'hash/xid_ops', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0e7511dde1c..a0b135cf19e 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -3458,4 +3458,19 @@ oprcode => 'multirange_after_multirange', oprrest => 'multirangesel', oprjoin => 'scalargtjoinsel' }, +{ oid => '8247', descr => 'equal', + oprname => '=', oprcanhash => 't', oprleft => 'pg_encrypted_det', + oprright => 'pg_encrypted_det', oprresult => 'bool', + oprcom => '=(pg_encrypted_det,pg_encrypted_det)', + oprnegate => '<>(pg_encrypted_det,pg_encrypted_det)', + oprcode => 'pg_encrypted_det_eq', oprrest => 'eqsel', + oprjoin => 'eqjoinsel' }, +{ oid => '8248', descr => 'not equal', + oprname => '<>', oprleft => 'pg_encrypted_det', + oprright => 'pg_encrypted_det', oprresult => 'bool', + oprcom => '<>(pg_encrypted_det,pg_encrypted_det)', + oprnegate => '=(pg_encrypted_det,pg_encrypted_det)', + oprcode => 'pg_encrypted_det_ne', oprrest => 'neqsel', + oprjoin => 'neqjoinsel' }, + ] diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat index c8ac8c73def..9b7308592fe 100644 --- a/src/include/catalog/pg_opfamily.dat +++ b/src/include/catalog/pg_opfamily.dat @@ -108,6 +108,8 @@ opfmethod => 'hash', opfname => 'bool_ops' }, { oid => '2223', opfmethod => 'hash', opfname => 'bytea_ops' }, +{ oid => '8249', + opfmethod => 'hash', opfname => 'pg_encrypted_det_ops' }, { oid => '2789', opfmethod => 'btree', opfname => 'tid_ops' }, { oid => '2225', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 153d816a053..19d87c56194 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6439,6 +6439,12 @@ proname => 'pg_collation_is_visible', procost => '10', provolatile => 's', prorettype => 'bool', proargtypes => 'oid', prosrc => 'pg_collation_is_visible' }, +{ oid => '8261', descr => 'is column encryption key visible in search path?', + proname => 'pg_cek_is_visible', procost => '10', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid', prosrc => 'pg_cek_is_visible' }, +{ oid => '8262', descr => 'is column master key visible in search path?', + proname => 'pg_cmk_is_visible', procost => '10', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid', prosrc => 'pg_cmk_is_visible' }, { oid => '2854', descr => 'get OID of current session\'s temp schema, if any', proname => 'pg_my_temp_schema', provolatile => 's', proparallel => 'r', @@ -7226,6 +7232,68 @@ proname => 'fmgr_sql_validator', provolatile => 's', prorettype => 'void', proargtypes => 'oid', prosrc => 'fmgr_sql_validator' }, +{ oid => '8265', + descr => 'user privilege on column encryption key by username, column encryption key name', + proname => 'has_column_encryption_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'name text text', + prosrc => 'has_column_encryption_key_privilege_name_name' }, +{ oid => '8266', + descr => 'user privilege on column encryption key by username, column encryption key oid', + proname => 'has_column_encryption_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'name oid text', + prosrc => 'has_column_encryption_key_privilege_name_id' }, +{ oid => '8267', + descr => 'user privilege on column encryption key by user oid, column encryption key name', + proname => 'has_column_encryption_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid text text', + prosrc => 'has_column_encryption_key_privilege_id_name' }, +{ oid => '8268', + descr => 'user privilege on column encryption key by user oid, column encryption key oid', + proname => 'has_column_encryption_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid oid text', + prosrc => 'has_column_encryption_key_privilege_id_id' }, +{ oid => '8269', + descr => 'current user privilege on column encryption key by column encryption key name', + proname => 'has_column_encryption_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'text text', + prosrc => 'has_column_encryption_key_privilege_name' }, +{ oid => '8270', + descr => 'current user privilege on column encryption key by column encryption key oid', + proname => 'has_column_encryption_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid text', + prosrc => 'has_column_encryption_key_privilege_id' }, + +{ oid => '8271', + descr => 'user privilege on column master key by username, column master key name', + proname => 'has_column_master_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'name text text', + prosrc => 'has_column_master_key_privilege_name_name' }, +{ oid => '8272', + descr => 'user privilege on column master key by username, column master key oid', + proname => 'has_column_master_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'name oid text', + prosrc => 'has_column_master_key_privilege_name_id' }, +{ oid => '8273', + descr => 'user privilege on column master key by user oid, column master key name', + proname => 'has_column_master_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid text text', + prosrc => 'has_column_master_key_privilege_id_name' }, +{ oid => '8274', + descr => 'user privilege on column master key by user oid, column master key oid', + proname => 'has_column_master_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid oid text', + prosrc => 'has_column_master_key_privilege_id_id' }, +{ oid => '8275', + descr => 'current user privilege on column master key by column master key name', + proname => 'has_column_master_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'text text', + prosrc => 'has_column_master_key_privilege_name' }, +{ oid => '8276', + descr => 'current user privilege on column master key by column master key oid', + proname => 'has_column_master_key_privilege', provolatile => 's', + prorettype => 'bool', proargtypes => 'oid text', + prosrc => 'has_column_master_key_privilege_id' }, + { oid => '2250', descr => 'user privilege on database by username, database name', proname => 'has_database_privilege', provolatile => 's', prorettype => 'bool', @@ -12200,4 +12268,36 @@ proargtypes => 'int2', prosrc => 'gist_stratnum_identity' }, +{ oid => '8253', descr => 'I/O', + proname => 'pg_encrypted_det_in', prorettype => 'pg_encrypted_det', + proargtypes => 'cstring', prosrc => 'pg_encrypted_in' }, +{ oid => '8254', descr => 'I/O', + proname => 'pg_encrypted_det_out', prorettype => 'cstring', + proargtypes => 'pg_encrypted_det', prosrc => 'pg_encrypted_out' }, +{ oid => '8255', descr => 'I/O', + proname => 'pg_encrypted_det_recv', prorettype => 'pg_encrypted_det', + proargtypes => 'internal', prosrc => 'pg_encrypted_recv' }, +{ oid => '8256', descr => 'I/O', + proname => 'pg_encrypted_det_send', prorettype => 'bytea', + proargtypes => 'pg_encrypted_det', prosrc => 'pg_encrypted_send' }, + +{ oid => '8257', descr => 'I/O', + proname => 'pg_encrypted_rnd_in', prorettype => 'pg_encrypted_rnd', + proargtypes => 'cstring', prosrc => 'pg_encrypted_in' }, +{ oid => '8258', descr => 'I/O', + proname => 'pg_encrypted_rnd_out', prorettype => 'cstring', + proargtypes => 'pg_encrypted_rnd', prosrc => 'pg_encrypted_out' }, +{ oid => '8259', descr => 'I/O', + proname => 'pg_encrypted_rnd_recv', prorettype => 'pg_encrypted_rnd', + proargtypes => 'internal', prosrc => 'pg_encrypted_recv' }, +{ oid => '8260', descr => 'I/O', + proname => 'pg_encrypted_rnd_send', prorettype => 'bytea', + proargtypes => 'pg_encrypted_rnd', prosrc => 'pg_encrypted_send' }, + +{ oid => '8245', + proname => 'pg_encrypted_det_eq', proleakproof => 't', prorettype => 'bool', + proargtypes => 'pg_encrypted_det pg_encrypted_det', prosrc => 'byteaeq' }, +{ oid => '8246', + proname => 'pg_encrypted_det_ne', proleakproof => 't', prorettype => 'bool', + proargtypes => 'pg_encrypted_det pg_encrypted_det', prosrc => 'byteane' }, ] diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat index d29194da31f..5f7d5a37e62 100644 --- a/src/include/catalog/pg_type.dat +++ b/src/include/catalog/pg_type.dat @@ -694,4 +694,17 @@ typreceive => 'brin_minmax_multi_summary_recv', typsend => 'brin_minmax_multi_summary_send', typalign => 'i', typstorage => 'x', typcollation => 'default' }, + +# Note: typstorage 'e' since compression is not useful for encrypted data +{ oid => '8243', descr => 'encrypted column (deterministic)', + typname => 'pg_encrypted_det', typlen => '-1', typbyval => 'f', + typcategory => 'Y', typinput => 'pg_encrypted_det_in', + typoutput => 'pg_encrypted_det_out', typreceive => 'pg_encrypted_det_recv', + typsend => 'pg_encrypted_det_send', typalign => 'i', typstorage => 'e' }, +{ oid => '8244', descr => 'encrypted column (randomized)', + typname => 'pg_encrypted_rnd', typlen => '-1', typbyval => 'f', + typcategory => 'Y', typinput => 'pg_encrypted_rnd_in', + typoutput => 'pg_encrypted_rnd_out', typreceive => 'pg_encrypted_rnd_recv', + typsend => 'pg_encrypted_rnd_send', typalign => 'i', typstorage => 'e' }, + ] diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h index e9259697321..533b81782c4 100644 --- a/src/include/catalog/pg_type.h +++ b/src/include/catalog/pg_type.h @@ -297,6 +297,7 @@ MAKE_SYSCACHE(TYPENAMENSP, pg_type_typname_nsp_index, 64); #define TYPCATEGORY_USER 'U' #define TYPCATEGORY_BITSTRING 'V' /* er ... "varbit"? */ #define TYPCATEGORY_UNKNOWN 'X' +#define TYPCATEGORY_ENCRYPTED 'Y' #define TYPCATEGORY_INTERNAL 'Z' #define TYPALIGN_CHAR 'c' /* char alignment (i.e. unaligned) */ diff --git a/src/include/commands/colenccmds.h b/src/include/commands/colenccmds.h new file mode 100644 index 00000000000..6b881d969d2 --- /dev/null +++ b/src/include/commands/colenccmds.h @@ -0,0 +1,26 @@ +/*------------------------------------------------------------------------- + * + * colenccmds.h + * prototypes for colenccmds.c. + * + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * src/include/commands/colenccmds.h + * + *------------------------------------------------------------------------- + */ + +#ifndef COLENCCMDS_H +#define COLENCCMDS_H + +#include "catalog/objectaddress.h" +#include "parser/parse_node.h" + +extern ObjectAddress CreateCEK(ParseState *pstate, DefineStmt *stmt); +extern ObjectAddress AlterColumnEncryptionKey(ParseState *pstate, AlterColumnEncryptionKeyStmt *stmt); +extern ObjectAddress CreateCMK(ParseState *pstate, DefineStmt *stmt); +extern ObjectAddress AlterColumnMasterKey(ParseState *pstate, AlterColumnMasterKeyStmt *stmt); + +#endif /* COLENCCMDS_H */ diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h index 85cbad3d0c2..5169e03cedf 100644 --- a/src/include/commands/tablecmds.h +++ b/src/include/commands/tablecmds.h @@ -27,7 +27,7 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */ extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId, ObjectAddress *typaddress, const char *queryString); -extern TupleDesc BuildDescForRelation(const List *columns); +extern TupleDesc BuildDescForRelation(const List *columns, FormExtraData_pg_attribute **tupdesc_extra_p); extern void RemoveRelations(DropStmt *drop); @@ -107,4 +107,6 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation, extern bool PartConstraintImpliedByRelConstraint(Relation scanrel, List *partConstraint); +extern List *makeColumnEncryption(HeapTuple attrtup); + #endif /* TABLECMDS_H */ diff --git a/src/include/common/colenc.h b/src/include/common/colenc.h new file mode 100644 index 00000000000..86a5134164c --- /dev/null +++ b/src/include/common/colenc.h @@ -0,0 +1,51 @@ +/*------------------------------------------------------------------------- + * + * colenc.h + * + * Shared definitions for column encryption algorithms. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/include/common/colenc.h + *------------------------------------------------------------------------- + */ + +#ifndef COMMON_COLENC_H +#define COMMON_COLENC_H + +/* + * Constants for CMK and CEK algorithms. Note that these are part of the + * protocol. In either case, don't assign zero, so that that can be used as + * an invalid value. + * + * Names should use IANA-style capitalization and punctuation ("LIKE_THIS"). + * + * When making changes, also update protocol.sgml. + */ + +#define PG_CMK_UNSPECIFIED 1 +#define PG_CMK_RSAES_OAEP_SHA_1 2 +#define PG_CMK_RSAES_OAEP_SHA_256 3 + +/* + * These algorithms are part of the RFC 5116 realm of AEAD algorithms (even + * though they never became an official IETF standard). So for propriety, we + * use "private use" numbers from + * . + */ +#define PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256 32768 +#define PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384 32769 +#define PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384 32770 +#define PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512 32771 + +/* + * Functions to convert between names and numbers + */ +extern int get_cmkalg_num(const char *name); +extern const char *get_cmkalg_name(int num); +extern const char *get_cmkalg_jwa_name(int num); +extern int get_cekalg_num(const char *name); +extern const char *get_cekalg_name(int num); + +#endif diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h index 05cb1874c58..0f0be990797 100644 --- a/src/include/libpq/libpq-be.h +++ b/src/include/libpq/libpq-be.h @@ -150,6 +150,7 @@ typedef struct Port */ char *database_name; char *user_name; + bool column_encryption_enabled; char *cmdline_options; List *guc_options; diff --git a/src/include/libpq/protocol.h b/src/include/libpq/protocol.h index 4b8d4403656..a2db4547767 100644 --- a/src/include/libpq/protocol.h +++ b/src/include/libpq/protocol.h @@ -57,6 +57,8 @@ #define PqMsg_PortalSuspended 's' #define PqMsg_ParameterDescription 't' #define PqMsg_NegotiateProtocolVersion 'v' +#define PqMsg_ColumnMasterKey 'y' +#define PqMsg_ColumnEncryptionKey 'Y' /* These are the codes sent by both the frontend and backend. */ diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index f763f790b18..01913659bc6 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -726,6 +726,7 @@ typedef struct ColumnDef char *colname; /* name of column */ TypeName *typeName; /* type of column */ char *compression; /* compression method for column */ + List *encryption; /* encryption info for column */ int inhcount; /* number of times column is inherited */ bool is_local; /* column has local (non-inherited) def'n */ bool is_not_null; /* NOT NULL constraint specified? */ @@ -762,11 +763,12 @@ typedef enum TableLikeOption CREATE_TABLE_LIKE_COMPRESSION = 1 << 1, CREATE_TABLE_LIKE_CONSTRAINTS = 1 << 2, CREATE_TABLE_LIKE_DEFAULTS = 1 << 3, - CREATE_TABLE_LIKE_GENERATED = 1 << 4, - CREATE_TABLE_LIKE_IDENTITY = 1 << 5, - CREATE_TABLE_LIKE_INDEXES = 1 << 6, - CREATE_TABLE_LIKE_STATISTICS = 1 << 7, - CREATE_TABLE_LIKE_STORAGE = 1 << 8, + CREATE_TABLE_LIKE_ENCRYPTED = 1 << 4, + CREATE_TABLE_LIKE_GENERATED = 1 << 5, + CREATE_TABLE_LIKE_IDENTITY = 1 << 6, + CREATE_TABLE_LIKE_INDEXES = 1 << 7, + CREATE_TABLE_LIKE_STATISTICS = 1 << 8, + CREATE_TABLE_LIKE_STORAGE = 1 << 9, CREATE_TABLE_LIKE_ALL = PG_INT32_MAX } TableLikeOption; @@ -2266,6 +2268,9 @@ typedef enum ObjectType OBJECT_CAST, OBJECT_COLUMN, OBJECT_COLLATION, + OBJECT_CEK, + OBJECT_CEKDATA, + OBJECT_CMK, OBJECT_CONVERSION, OBJECT_DATABASE, OBJECT_DEFAULT, @@ -2456,6 +2461,31 @@ typedef struct AlterCollationStmt } AlterCollationStmt; +/* ---------------------- + * Alter Column Encryption Key + * ---------------------- + */ +typedef struct AlterColumnEncryptionKeyStmt +{ + NodeTag type; + List *cekname; + bool isDrop; /* ADD or DROP the items? */ + List *definition; +} AlterColumnEncryptionKeyStmt; + + +/* ---------------------- + * Alter Column Master Key + * ---------------------- + */ +typedef struct AlterColumnMasterKeyStmt +{ + NodeTag type; + List *cmkname; + List *definition; +} AlterColumnMasterKeyStmt; + + /* ---------------------- * Alter Domain * @@ -3934,6 +3964,7 @@ typedef struct CheckPointStmt typedef enum DiscardMode { DISCARD_ALL, + DISCARD_COLUMN_ENCRYPTION_KEYS, DISCARD_PLANS, DISCARD_SEQUENCES, DISCARD_TEMP, diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h index f9a4afd4723..02dc1fc8653 100644 --- a/src/include/parser/kwlist.h +++ b/src/include/parser/kwlist.h @@ -152,6 +152,7 @@ PG_KEYWORD("empty", EMPTY_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("enable", ENABLE_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("encoding", ENCODING, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("encrypted", ENCRYPTED, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("encryption", ENCRYPTION, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("end", END_P, RESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("enum", ENUM_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("error", ERROR_P, UNRESERVED_KEYWORD, BARE_LABEL) @@ -269,6 +270,7 @@ PG_KEYWORD("lock", LOCK_P, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("locked", LOCKED, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("logged", LOGGED, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("mapping", MAPPING, UNRESERVED_KEYWORD, BARE_LABEL) +PG_KEYWORD("master", MASTER, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("match", MATCH, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("matched", MATCHED, UNRESERVED_KEYWORD, BARE_LABEL) PG_KEYWORD("materialized", MATERIALIZED, UNRESERVED_KEYWORD, BARE_LABEL) diff --git a/src/include/parser/parse_param.h b/src/include/parser/parse_param.h index 6459d4ab6f3..08008b1973f 100644 --- a/src/include/parser/parse_param.h +++ b/src/include/parser/parse_param.h @@ -21,5 +21,6 @@ extern void setup_parse_variable_parameters(ParseState *pstate, Oid **paramTypes, int *numParams); extern void check_variable_parameters(ParseState *pstate, Query *query); extern bool query_contains_extern_params(Query *query); +extern void find_param_origs(List *query_list, Oid **param_orig_tbls, AttrNumber **param_orig_cols); #endif /* PARSE_PARAM_H */ diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h index 7fdcec6dd93..a4715baf0a0 100644 --- a/src/include/tcop/cmdtaglist.h +++ b/src/include/tcop/cmdtaglist.h @@ -29,6 +29,8 @@ PG_CMDTAG(CMDTAG_ALTER_ACCESS_METHOD, "ALTER ACCESS METHOD", true, false, false) PG_CMDTAG(CMDTAG_ALTER_AGGREGATE, "ALTER AGGREGATE", true, false, false) PG_CMDTAG(CMDTAG_ALTER_CAST, "ALTER CAST", true, false, false) PG_CMDTAG(CMDTAG_ALTER_COLLATION, "ALTER COLLATION", true, false, false) +PG_CMDTAG(CMDTAG_ALTER_COLUMN_ENCRYPTION_KEY, "ALTER COLUMN ENCRYPTION KEY", true, false, false) +PG_CMDTAG(CMDTAG_ALTER_COLUMN_MASTER_KEY, "ALTER COLUMN MASTER KEY", true, false, false) PG_CMDTAG(CMDTAG_ALTER_CONSTRAINT, "ALTER CONSTRAINT", true, false, false) PG_CMDTAG(CMDTAG_ALTER_CONVERSION, "ALTER CONVERSION", true, false, false) PG_CMDTAG(CMDTAG_ALTER_DATABASE, "ALTER DATABASE", false, false, false) @@ -86,6 +88,8 @@ PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, fals PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false) PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false) PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false) +PG_CMDTAG(CMDTAG_CREATE_COLUMN_ENCRYPTION_KEY, "CREATE COLUMN ENCRYPTION KEY", true, false, false) +PG_CMDTAG(CMDTAG_CREATE_COLUMN_MASTER_KEY, "CREATE COLUMN MASTER KEY", true, false, false) PG_CMDTAG(CMDTAG_CREATE_CONSTRAINT, "CREATE CONSTRAINT", true, false, false) PG_CMDTAG(CMDTAG_CREATE_CONVERSION, "CREATE CONVERSION", true, false, false) PG_CMDTAG(CMDTAG_CREATE_DATABASE, "CREATE DATABASE", false, false, false) @@ -130,6 +134,7 @@ PG_CMDTAG(CMDTAG_DECLARE_CURSOR, "DECLARE CURSOR", false, false, false) PG_CMDTAG(CMDTAG_DELETE, "DELETE", false, false, true) PG_CMDTAG(CMDTAG_DISCARD, "DISCARD", false, false, false) PG_CMDTAG(CMDTAG_DISCARD_ALL, "DISCARD ALL", false, false, false) +PG_CMDTAG(CMDTAG_DISCARD_COLUMN_ENCRYPTION_KEYS, "DISCARD COLUMN ENCRYPTION KEYS", false, false, false) PG_CMDTAG(CMDTAG_DISCARD_PLANS, "DISCARD PLANS", false, false, false) PG_CMDTAG(CMDTAG_DISCARD_SEQUENCES, "DISCARD SEQUENCES", false, false, false) PG_CMDTAG(CMDTAG_DISCARD_TEMP, "DISCARD TEMP", false, false, false) @@ -138,6 +143,8 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false) PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false) PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false) PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false) +PG_CMDTAG(CMDTAG_DROP_COLUMN_ENCRYPTION_KEY, "DROP COLUMN ENCRYPTION KEY", true, false, false) +PG_CMDTAG(CMDTAG_DROP_COLUMN_MASTER_KEY, "DROP COLUMN MASTER KEY", true, false, false) PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false) PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false) PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false) diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h index 3a0baf30395..152d2b0da90 100644 --- a/src/include/utils/acl.h +++ b/src/include/utils/acl.h @@ -159,6 +159,8 @@ typedef struct ArrayType Acl; #define ACL_ALL_RIGHTS_COLUMN (ACL_INSERT|ACL_SELECT|ACL_UPDATE|ACL_REFERENCES) #define ACL_ALL_RIGHTS_RELATION (ACL_INSERT|ACL_SELECT|ACL_UPDATE|ACL_DELETE|ACL_TRUNCATE|ACL_REFERENCES|ACL_TRIGGER|ACL_MAINTAIN) #define ACL_ALL_RIGHTS_SEQUENCE (ACL_USAGE|ACL_SELECT|ACL_UPDATE) +#define ACL_ALL_RIGHTS_CEK (ACL_USAGE) +#define ACL_ALL_RIGHTS_CMK (ACL_USAGE) #define ACL_ALL_RIGHTS_DATABASE (ACL_CREATE|ACL_CREATE_TEMP|ACL_CONNECT) #define ACL_ALL_RIGHTS_FDW (ACL_USAGE) #define ACL_ALL_RIGHTS_FOREIGN_SERVER (ACL_USAGE) diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h index 35a8dec2b9f..6e512455c35 100644 --- a/src/include/utils/lsyscache.h +++ b/src/include/utils/lsyscache.h @@ -164,6 +164,7 @@ extern bool type_is_rowtype(Oid typid); extern bool type_is_enum(Oid typid); extern bool type_is_range(Oid typid); extern bool type_is_multirange(Oid typid); +extern bool type_is_encrypted(Oid typid); extern void get_type_category_preferred(Oid typid, char *typcategory, bool *typispreferred); @@ -203,6 +204,9 @@ extern Oid get_publication_oid(const char *pubname, bool missing_ok); extern char *get_publication_name(Oid pubid, bool missing_ok); extern Oid get_subscription_oid(const char *subname, bool missing_ok); extern char *get_subscription_name(Oid subid, bool missing_ok); +extern char *get_cek_name(Oid cekid, bool missing_ok); +extern Oid get_cekdata_oid(Oid cekid, Oid cmkid, bool missing_ok); +extern char *get_cmk_name(Oid cmkid, bool missing_ok); #define type_is_array(typid) (get_element_type(typid) != InvalidOid) /* type_is_array_domain accepts both plain arrays and domains over arrays */ diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h index a90dfdf9067..d3c37614389 100644 --- a/src/include/utils/plancache.h +++ b/src/include/utils/plancache.h @@ -209,6 +209,9 @@ extern void CompleteCachedPlan(CachedPlanSource *plansource, extern void SaveCachedPlan(CachedPlanSource *plansource); extern void DropCachedPlan(CachedPlanSource *plansource); +extern List *RevalidateCachedQuery(CachedPlanSource *plansource, + QueryEnvironment *queryEnv); + extern void CachedPlanSetParentContext(CachedPlanSource *plansource, MemoryContext newcontext); diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile index fe2af575c5d..5539222b38a 100644 --- a/src/interfaces/libpq/Makefile +++ b/src/interfaces/libpq/Makefile @@ -52,6 +52,7 @@ endif ifeq ($(with_ssl),openssl) OBJS += \ + fe-encrypt-openssl.o \ fe-secure-openssl.o endif diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt index 8ee08115100..04a86c09a12 100644 --- a/src/interfaces/libpq/exports.txt +++ b/src/interfaces/libpq/exports.txt @@ -204,3 +204,7 @@ PQcancelReset 201 PQcancelFinish 202 PQsocketPoll 203 PQsetChunkedRowsMode 204 +PQexecPreparedDescribed 205 +PQsendQueryPreparedDescribed 206 +PQfisencrypted 207 +PQparamisencrypted 208 diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index e35bdc40361..6425c80032c 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -359,6 +359,14 @@ static const internalPQconninfoOption PQconninfoOptions[] = { "Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */ offsetof(struct pg_conn, target_session_attrs)}, + {"cmklookup", "PGCMKLOOKUP", "", NULL, + "CMK-Lookup", "", 64, + offsetof(struct pg_conn, cmklookup)}, + + {"column_encryption", "PGCOLUMNENCRYPTION", "0", NULL, + "Column-Encryption", "", 1, + offsetof(struct pg_conn, column_encryption_setting)}, + {"load_balance_hosts", "PGLOADBALANCEHOSTS", DefaultLoadBalanceHosts, NULL, "Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */ @@ -1822,6 +1830,28 @@ pqConnectOptions2(PGconn *conn) goto oom_error; } + /* + * validate column_encryption option + */ + if (conn->column_encryption_setting) + { + if (strcmp(conn->column_encryption_setting, "on") == 0 || + strcmp(conn->column_encryption_setting, "true") == 0 || + strcmp(conn->column_encryption_setting, "1") == 0) + conn->column_encryption_enabled = true; + else if (strcmp(conn->column_encryption_setting, "off") == 0 || + strcmp(conn->column_encryption_setting, "false") == 0 || + strcmp(conn->column_encryption_setting, "0") == 0) + conn->column_encryption_enabled = false; + else + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", + "column_encryption", conn->column_encryption_setting); + return false; + } + } + /* * Only if we get this far is it appropriate to try to connect. (We need a * state flag, rather than just the boolean result of this function, in @@ -4679,6 +4709,22 @@ freePGconn(PGconn *conn) free(conn->gsslib); free(conn->gssdelegation); free(conn->connip); + free(conn->cmklookup); + for (int i = 0; i < conn->ncmks; i++) + { + free(conn->cmks[i].cmkname); + free(conn->cmks[i].cmkrealm); + } + free(conn->cmks); + for (int i = 0; i < conn->nceks; i++) + { + if (conn->ceks[i].cekdata) + { + explicit_bzero(conn->ceks[i].cekdata, conn->ceks[i].cekdatalen); + free(conn->ceks[i].cekdata); + } + } + free(conn->ceks); /* Note that conn->Pfdebug is not ours to close or free */ free(conn->write_err_msg); free(conn->inBuffer); diff --git a/src/interfaces/libpq/fe-encrypt-openssl.c b/src/interfaces/libpq/fe-encrypt-openssl.c new file mode 100644 index 00000000000..462535c3832 --- /dev/null +++ b/src/interfaces/libpq/fe-encrypt-openssl.c @@ -0,0 +1,840 @@ +/*------------------------------------------------------------------------- + * + * fe-encrypt-openssl.c + * + * client-side column encryption support using OpenSSL + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/interfaces/libpq/fe-encrypt-openssl.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres_fe.h" + +#include "fe-encrypt.h" +#include "libpq-int.h" + +#include "common/colenc.h" +#include "port/pg_bswap.h" + +#include + + +/* + * When TEST_ENCRYPT is defined, this file builds a standalone program that + * checks encryption test cases against the specification document. + * + * We have to replace some functions that are not available in that + * environment. + */ +#ifdef TEST_ENCRYPT + +#define libpq_gettext(x) (x) +#define libpq_append_conn_error(conn, ...) do { fprintf(stderr, __VA_ARGS__); fprintf(stderr, "\n"); exit(1); } while(0) +#define pqResultAlloc(res, nBytes, isBinary) malloc(nBytes) + +#endif /* TEST_ENCRYPT */ + + +/* + * Decrypt the CEK given by "from" and "fromlen" (data typically sent from the + * server) using the CMK in cmkfilename. + */ +unsigned char * +decrypt_cek_from_file(PGconn *conn, const char *cmkfilename, int cmkalg, + int fromlen, const unsigned char *from, + int *tolen) +{ + const EVP_MD *md = NULL; + EVP_PKEY *key = NULL; + RSA *rsa = NULL; + BIO *bio = NULL; + EVP_PKEY_CTX *ctx = NULL; + unsigned char *out = NULL; + size_t outlen; + + switch (cmkalg) + { + case PG_CMK_RSAES_OAEP_SHA_1: + md = EVP_sha1(); + break; + case PG_CMK_RSAES_OAEP_SHA_256: + md = EVP_sha256(); + break; + case PG_CMK_UNSPECIFIED: + libpq_append_conn_error(conn, "unspecified CMK algorithm not supported with file lookup scheme"); + goto fail; + default: + libpq_append_conn_error(conn, "unsupported CMK algorithm ID: %d", cmkalg); + goto fail; + } + + bio = BIO_new_file(cmkfilename, "r"); + if (!bio) + { + libpq_append_conn_error(conn, "could not open file \"%s\": %m", cmkfilename); + goto fail; + } + + rsa = RSA_new(); + if (!rsa) + { + libpq_append_conn_error(conn, "could not allocate RSA structure: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + /* + * Note: We must go through BIO and not say use PEM_read_RSAPrivateKey() + * directly on a FILE. Otherwise, we get into "no OPENSSL_Applink" hell + * on Windows (which happens whenever you pass a stdio handle from the + * application into OpenSSL). + */ + rsa = PEM_read_bio_RSAPrivateKey(bio, &rsa, NULL, NULL); + if (!rsa) + { + libpq_append_conn_error(conn, "could not read RSA private key: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + key = EVP_PKEY_new(); + if (!key) + { + libpq_append_conn_error(conn, "could not allocate private key structure: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + if (!EVP_PKEY_assign_RSA(key, rsa)) + { + libpq_append_conn_error(conn, "could not assign private key: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + ctx = EVP_PKEY_CTX_new(key, NULL); + if (!ctx) + { + libpq_append_conn_error(conn, "could not allocate public key algorithm context: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + if (EVP_PKEY_decrypt_init(ctx) <= 0) + { + libpq_append_conn_error(conn, "decryption initialization failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + if (EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PKCS1_OAEP_PADDING) <= 0 || + EVP_PKEY_CTX_set_rsa_mgf1_md(ctx, md) <= 0 || + EVP_PKEY_CTX_set_rsa_oaep_md(ctx, md) <= 0) + { + libpq_append_conn_error(conn, "could not set RSA parameter: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + /* get output length */ + if (EVP_PKEY_decrypt(ctx, NULL, &outlen, from, fromlen) <= 0) + { + libpq_append_conn_error(conn, "RSA decryption setup failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + out = malloc(outlen); + if (!out) + { + libpq_append_conn_error(conn, "out of memory"); + goto fail; + } + + if (EVP_PKEY_decrypt(ctx, out, &outlen, from, fromlen) <= 0) + { + libpq_append_conn_error(conn, "RSA decryption failed: %s", + ERR_reason_error_string(ERR_get_error())); + free(out); + out = NULL; + goto fail; + } + + *tolen = outlen; + +fail: + EVP_PKEY_CTX_free(ctx); + EVP_PKEY_free(key); + BIO_free(bio); + + return out; +} + + +/* + * The routines below implement the AEAD algorithms specified in + * + * for encrypting and decrypting column values. + */ + +#ifdef TEST_ENCRYPT + +/* + * Test data from + * + */ + +/* + * The different test cases just use different prefixes of K, so one constant + * is enough here. + */ +static const unsigned char K[] = { + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, + 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, + 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f, + 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f, +}; + +static const unsigned char P[] = { + 0x41, 0x20, 0x63, 0x69, 0x70, 0x68, 0x65, 0x72, 0x20, 0x73, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x20, + 0x6d, 0x75, 0x73, 0x74, 0x20, 0x6e, 0x6f, 0x74, 0x20, 0x62, 0x65, 0x20, 0x72, 0x65, 0x71, 0x75, + 0x69, 0x72, 0x65, 0x64, 0x20, 0x74, 0x6f, 0x20, 0x62, 0x65, 0x20, 0x73, 0x65, 0x63, 0x72, 0x65, + 0x74, 0x2c, 0x20, 0x61, 0x6e, 0x64, 0x20, 0x69, 0x74, 0x20, 0x6d, 0x75, 0x73, 0x74, 0x20, 0x62, + 0x65, 0x20, 0x61, 0x62, 0x6c, 0x65, 0x20, 0x74, 0x6f, 0x20, 0x66, 0x61, 0x6c, 0x6c, 0x20, 0x69, + 0x6e, 0x74, 0x6f, 0x20, 0x74, 0x68, 0x65, 0x20, 0x68, 0x61, 0x6e, 0x64, 0x73, 0x20, 0x6f, 0x66, + 0x20, 0x74, 0x68, 0x65, 0x20, 0x65, 0x6e, 0x65, 0x6d, 0x79, 0x20, 0x77, 0x69, 0x74, 0x68, 0x6f, + 0x75, 0x74, 0x20, 0x69, 0x6e, 0x63, 0x6f, 0x6e, 0x76, 0x65, 0x6e, 0x69, 0x65, 0x6e, 0x63, 0x65, +}; + +static const unsigned char test_IV[] = { + 0x1a, 0xf3, 0x8c, 0x2d, 0xc2, 0xb9, 0x6f, 0xfd, 0xd8, 0x66, 0x94, 0x09, 0x23, 0x41, 0xbc, 0x04, +}; + +static const unsigned char test_A[] = { + 0x54, 0x68, 0x65, 0x20, 0x73, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x20, 0x70, 0x72, 0x69, 0x6e, 0x63, + 0x69, 0x70, 0x6c, 0x65, 0x20, 0x6f, 0x66, 0x20, 0x41, 0x75, 0x67, 0x75, 0x73, 0x74, 0x65, 0x20, + 0x4b, 0x65, 0x72, 0x63, 0x6b, 0x68, 0x6f, 0x66, 0x66, 0x73, +}; + +#endif /* TEST_ENCRYPT */ + + +/* + * Get OpenSSL cipher that corresponds to the CEK algorithm number. + */ +static const EVP_CIPHER * +pg_cekalg_to_openssl_cipher(int cekalg) +{ + const EVP_CIPHER *cipher; + + switch (cekalg) + { + case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256: + cipher = EVP_aes_128_cbc(); + break; + case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384: + cipher = EVP_aes_192_cbc(); + break; + case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384: + case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512: + cipher = EVP_aes_256_cbc(); + break; + default: + cipher = NULL; + } + + return cipher; +} + +/* + * Get OpenSSL digest (for MAC) that corresponds to the CEK algorithm number. + */ +static const EVP_MD * +pg_cekalg_to_openssl_md(int cekalg) +{ + const EVP_MD *md; + + switch (cekalg) + { + case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256: + md = EVP_sha256(); + break; + case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384: + case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384: + md = EVP_sha384(); + break; + case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512: + md = EVP_sha512(); + break; + default: + md = NULL; + } + + return md; +} + +/* + * Get the MAC key length (in octets) that corresponds to the CEK algorithm number. + * + * This is MAC_KEY_LEN in the mcgrew paper. + */ +static int +md_key_length(const EVP_MD *md) +{ + if (md == EVP_sha256()) + return 16; + else if (md == EVP_sha384()) + return 24; + else if (md == EVP_sha512()) + return 32; + else + return -1; +} + +/* + * Get the HMAC output length (in octets) that corresponds to the CEK + * algorithm number. + */ +static int +md_hash_length(const EVP_MD *md) +{ + if (md == EVP_sha256()) + return 32; + else if (md == EVP_sha384()) + return 48; + else if (md == EVP_sha512()) + return 64; + else + return -1; +} + +/* + * Length of associated data (A in mcgrew paper) + */ +#ifndef TEST_ENCRYPT +#define PG_AD_LEN 4 +#else +#define PG_AD_LEN sizeof(test_A) +#endif + +/* + * Compute message authentication tag (T in the mcgrew paper), from MAC key + * and ciphertext. + * + * Returns false on error, with error message in errmsgp. + */ +static bool +get_message_auth_tag(const EVP_MD *md, + const unsigned char *mac_key, int mac_key_len, + const unsigned char *encr, int encrlen, + unsigned char *md_value, size_t *md_len_p, + const char **errmsgp) +{ + static char msgbuf[1024]; + EVP_MD_CTX *evp_md_ctx = NULL; + EVP_PKEY *pkey = NULL; + size_t bufsize; + unsigned char *buf = NULL; + int64 al; + bool result = false; + + if (encrlen < 0) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("encrypted value has invalid length")); + *errmsgp = msgbuf; + goto fail; + } + + evp_md_ctx = EVP_MD_CTX_new(); + + pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, NULL, mac_key, mac_key_len); + if (!pkey) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("could not allocate key for HMAC: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + + /* + * Build input to MAC call (A || S || AL in mcgrew paper) + */ + bufsize = PG_AD_LEN + encrlen + sizeof(int64); + buf = malloc(bufsize); + if (!buf) + { + *errmsgp = libpq_gettext("out of memory"); + goto fail; + } + /* A (associated data) */ +#ifndef TEST_ENCRYPT + buf[0] = 'P'; + buf[1] = 'G'; + *(int16 *) (buf + 2) = pg_hton16(1); +#else + memcpy(buf, test_A, sizeof(test_A)); +#endif + /* S (ciphertext) */ + memcpy(buf + PG_AD_LEN, encr, encrlen); + /* AL (number of *bits* in A) */ + al = pg_hton64(PG_AD_LEN * 8); + memcpy(buf + PG_AD_LEN + encrlen, &al, sizeof(al)); + + /* + * Call MAC + */ + if (!EVP_DigestSignInit(evp_md_ctx, NULL, md, NULL, pkey)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("digest initialization failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + + if (!EVP_DigestSignUpdate(evp_md_ctx, buf, bufsize)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("digest signing failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + + if (!EVP_DigestSignFinal(evp_md_ctx, md_value, md_len_p)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("digest signing failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + Assert(*md_len_p == md_hash_length(md)); + + /* truncate output to half the length, per spec */ + *md_len_p /= 2; + + result = true; +fail: + free(buf); + EVP_PKEY_free(pkey); + EVP_MD_CTX_free(evp_md_ctx); + return result; +} + +/* + * Decrypt a column value + */ +unsigned char * +decrypt_value(PGresult *res, const PGCEK *cek, int cekalg, const unsigned char *input, int inputlen, const char **errmsgp) +{ + static char msgbuf[1024]; + + const unsigned char *iv = NULL; + size_t ivlen; + + const EVP_CIPHER *cipher; + const EVP_MD *md; + EVP_CIPHER_CTX *evp_cipher_ctx = NULL; + int enc_key_len; + int mac_key_len; + int iv_key_len; + int key_len; + const unsigned char *enc_key; + const unsigned char *mac_key; + unsigned char md_value[EVP_MAX_MD_SIZE]; + size_t md_len = sizeof(md_value); + size_t bufsize; + unsigned char *buf = NULL; + unsigned char *decr; + int decrlen, + decrlen2; + + unsigned char *result = NULL; + + cipher = pg_cekalg_to_openssl_cipher(cekalg); + if (!cipher) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("unrecognized encryption algorithm identifier: %d"), cekalg); + *errmsgp = msgbuf; + goto fail; + } + + md = pg_cekalg_to_openssl_md(cekalg); + if (!md) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("unrecognized digest algorithm identifier: %d"), cekalg); + *errmsgp = msgbuf; + goto fail; + } + + evp_cipher_ctx = EVP_CIPHER_CTX_new(); + + if (!EVP_DecryptInit_ex(evp_cipher_ctx, cipher, NULL, NULL, NULL)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("decryption initialization failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + + enc_key_len = EVP_CIPHER_CTX_key_length(evp_cipher_ctx); + mac_key_len = iv_key_len = md_key_length(md); + key_len = mac_key_len + enc_key_len + iv_key_len; + + if (cek->cekdatalen != key_len) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("column encryption key has wrong length for algorithm (has: %zu, required: %d)"), + cek->cekdatalen, key_len); + *errmsgp = msgbuf; + goto fail; + } + + mac_key = cek->cekdata; + enc_key = cek->cekdata + mac_key_len; + + if (!get_message_auth_tag(md, mac_key, mac_key_len, + input, inputlen - (md_hash_length(md) / 2), + md_value, &md_len, + errmsgp)) + { + goto fail; + } + + /* use constant-time comparison, per mcgrew paper */ + if (CRYPTO_memcmp(input + (inputlen - md_len), md_value, md_len) != 0) + { + *errmsgp = libpq_gettext("MAC mismatch"); + goto fail; + } + + ivlen = EVP_CIPHER_CTX_iv_length(evp_cipher_ctx); + iv = input; + input += ivlen; + inputlen -= ivlen; + if (!EVP_DecryptInit_ex(evp_cipher_ctx, NULL, NULL, enc_key, iv)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("decryption initialization failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + + bufsize = inputlen + EVP_CIPHER_CTX_block_size(evp_cipher_ctx) + 1; + buf = pqResultAlloc(res, bufsize, false); + if (!buf) + { + *errmsgp = libpq_gettext("out of memory"); + goto fail; + } + decr = buf; + if (!EVP_DecryptUpdate(evp_cipher_ctx, decr, &decrlen, input, inputlen - md_len)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("decryption failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + if (!EVP_DecryptFinal_ex(evp_cipher_ctx, decr + decrlen, &decrlen2)) + { + snprintf(msgbuf, sizeof(msgbuf), + libpq_gettext("decryption failed: %s"), + ERR_reason_error_string(ERR_get_error())); + *errmsgp = msgbuf; + goto fail; + } + decrlen += decrlen2; + Assert(decrlen < bufsize); + decr[decrlen] = '\0'; + result = decr; + +fail: + EVP_CIPHER_CTX_free(evp_cipher_ctx); + + return result; +} + +/* + * Compute a synthetic initialization vector (SIV), for deterministic + * encryption. + * + * Per protocol specification, the SIV is computed as: + * + * SUBSTRING(HMAC(K, P) FOR IVLEN) + */ +#ifndef TEST_ENCRYPT +static bool +make_siv(PGconn *conn, + unsigned char *iv, size_t ivlen, + const EVP_MD *md, + const unsigned char *iv_key, int iv_key_len, + const unsigned char *plaintext, int plaintext_len) +{ + EVP_MD_CTX *evp_md_ctx = NULL; + EVP_PKEY *pkey = NULL; + unsigned char md_value[EVP_MAX_MD_SIZE]; + size_t md_len = sizeof(md_value); + bool result = false; + + evp_md_ctx = EVP_MD_CTX_new(); + + pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, NULL, iv_key, iv_key_len); + if (!pkey) + { + libpq_append_conn_error(conn, "could not allocate key for HMAC: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + if (!EVP_DigestSignInit(evp_md_ctx, NULL, md, NULL, pkey)) + { + libpq_append_conn_error(conn, "digest initialization failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + if (!EVP_DigestSignUpdate(evp_md_ctx, plaintext, plaintext_len)) + { + libpq_append_conn_error(conn, "digest signing failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + if (!EVP_DigestSignFinal(evp_md_ctx, md_value, &md_len)) + { + libpq_append_conn_error(conn, "digest signing failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + Assert(md_len == md_hash_length(md)); + memcpy(iv, md_value, ivlen); + + result = true; +fail: + EVP_PKEY_free(pkey); + EVP_MD_CTX_free(evp_md_ctx); + return result; +} +#endif /* TEST_ENCRYPT */ + +/* + * Encrypt a column value + */ +unsigned char * +encrypt_value(PGconn *conn, const PGCEK *cek, int cekalg, const unsigned char *value, int *nbytesp, bool enc_det) +{ + int nbytes = *nbytesp; + unsigned char iv[EVP_MAX_IV_LENGTH]; + size_t ivlen; + const EVP_CIPHER *cipher; + const EVP_MD *md; + EVP_CIPHER_CTX *evp_cipher_ctx = NULL; + int enc_key_len; + int mac_key_len; + int iv_key_len; + int key_len; + const unsigned char *enc_key; + const unsigned char *mac_key; + const unsigned char *iv_key; + size_t bufsize; + unsigned char *buf = NULL; + unsigned char *encr; + int encrlen, + encrlen2; + + const char *errmsg; + unsigned char md_value[EVP_MAX_MD_SIZE]; + size_t md_len = sizeof(md_value); + size_t buf2size; + unsigned char *buf2 = NULL; + + unsigned char *result = NULL; + + cipher = pg_cekalg_to_openssl_cipher(cekalg); + if (!cipher) + { + libpq_append_conn_error(conn, "unrecognized encryption algorithm identifier: %d", cekalg); + goto fail; + } + + md = pg_cekalg_to_openssl_md(cekalg); + if (!md) + { + libpq_append_conn_error(conn, "unrecognized digest algorithm identifier: %d", cekalg); + goto fail; + } + + evp_cipher_ctx = EVP_CIPHER_CTX_new(); + + if (!EVP_EncryptInit_ex(evp_cipher_ctx, cipher, NULL, NULL, NULL)) + { + libpq_append_conn_error(conn, "encryption initialization failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + enc_key_len = EVP_CIPHER_CTX_key_length(evp_cipher_ctx); + mac_key_len = iv_key_len = md_key_length(md); + key_len = mac_key_len + enc_key_len + iv_key_len; + + if (cek->cekdatalen != key_len) + { + libpq_append_conn_error(conn, "column encryption key has wrong length for algorithm (has: %zu, required: %d)", + cek->cekdatalen, key_len); + goto fail; + } + + mac_key = cek->cekdata; + enc_key = cek->cekdata + mac_key_len; + iv_key = cek->cekdata + mac_key_len + enc_key_len; + + ivlen = EVP_CIPHER_CTX_iv_length(evp_cipher_ctx); + Assert(ivlen <= sizeof(iv)); + if (enc_det) + { +#ifndef TEST_ENCRYPT + make_siv(conn, iv, ivlen, md, iv_key, iv_key_len, value, nbytes); +#else + (void) iv_key; /* unused */ + memcpy(iv, test_IV, ivlen); +#endif + } + else + pg_strong_random(iv, ivlen); + if (!EVP_EncryptInit_ex(evp_cipher_ctx, NULL, NULL, enc_key, iv)) + { + libpq_append_conn_error(conn, "encryption initialization failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + + bufsize = ivlen + (nbytes + 2 * EVP_CIPHER_CTX_block_size(evp_cipher_ctx) - 1); + buf = malloc(bufsize); + if (!buf) + { + libpq_append_conn_error(conn, "out of memory"); + goto fail; + } + memcpy(buf, iv, ivlen); + encr = buf + ivlen; + if (!EVP_EncryptUpdate(evp_cipher_ctx, encr, &encrlen, value, nbytes)) + { + libpq_append_conn_error(conn, "encryption failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + if (!EVP_EncryptFinal_ex(evp_cipher_ctx, encr + encrlen, &encrlen2)) + { + libpq_append_conn_error(conn, "encryption failed: %s", + ERR_reason_error_string(ERR_get_error())); + goto fail; + } + encrlen += encrlen2; + + encr -= ivlen; + encrlen += ivlen; + + Assert(encrlen <= bufsize); + + if (!get_message_auth_tag(md, mac_key, mac_key_len, + encr, encrlen, + md_value, &md_len, + &errmsg)) + { + appendPQExpBuffer(&conn->errorMessage, "%s\n", errmsg); + goto fail; + } + + buf2size = encrlen + md_len; + buf2 = malloc(buf2size); + if (!buf2) + { + libpq_append_conn_error(conn, "out of memory"); + goto fail; + } + memcpy(buf2, encr, encrlen); + memcpy(buf2 + encrlen, md_value, md_len); + + result = buf2; + nbytes = buf2size; + +fail: + free(buf); + EVP_CIPHER_CTX_free(evp_cipher_ctx); + + *nbytesp = nbytes; + return result; +} + + +/* + * Run test cases + */ +#ifdef TEST_ENCRYPT + +static void +debug_print_hex(const char *name, const unsigned char *val, int len) +{ + printf("%s =", name); + for (int i = 0; i < len; i++) + { + if (i % 16 == 0) + printf("\n"); + else + printf(" "); + printf("%02x", val[i]); + } + printf("\n"); +} + +/* + * K and P are from the mcgrew paper, K_len and P_len are their respective + * lengths. encrypt_value() requires the key length to contain the IV key, so + * we pass it here, too, but it will not be used. + */ +static void +test_case(int alg, const unsigned char *K, size_t K_len, size_t IV_key_len, const unsigned char *P, size_t P_len) +{ + unsigned char *C; + int nbytes; + PGCEK cek; + + nbytes = P_len; + cek.cekdatalen = K_len + IV_key_len; + cek.cekdata = malloc(cek.cekdatalen); + memcpy(cek.cekdata, K, K_len); + + C = encrypt_value(NULL, &cek, alg, P, &nbytes, true); + debug_print_hex("C", C, nbytes); +} + +int +main(int argc, char **argv) +{ + printf("5.1\n"); + test_case(PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256, K, 32, 16, P, sizeof(P)); + printf("5.2\n"); + test_case(PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384, K, 48, 24, P, sizeof(P)); + printf("5.3\n"); + test_case(PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384, K, 56, 24, P, sizeof(P)); + printf("5.4\n"); + test_case(PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512, K, 64, 32, P, sizeof(P)); + + return 0; +} + +#endif /* TEST_ENCRYPT */ diff --git a/src/interfaces/libpq/fe-encrypt.h b/src/interfaces/libpq/fe-encrypt.h new file mode 100644 index 00000000000..252d271436c --- /dev/null +++ b/src/interfaces/libpq/fe-encrypt.h @@ -0,0 +1,33 @@ +/*------------------------------------------------------------------------- + * + * fe-encrypt.h + * + * client-side column encryption support + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * IDENTIFICATION + * src/interfaces/libpq/fe-encrypt.h + * + *------------------------------------------------------------------------- + */ + +#ifndef FE_ENCRYPT_H +#define FE_ENCRYPT_H + +#include "libpq-fe.h" +#include "libpq-int.h" + +extern unsigned char *decrypt_cek_from_file(PGconn *conn, const char *cmkfilename, int cmkalg, + int fromlen, const unsigned char *from, + int *tolen); + +extern unsigned char *decrypt_value(PGresult *res, const PGCEK *cek, int cekalg, + const unsigned char *input, int inputlen, + const char **errmsgp); + +extern unsigned char *encrypt_value(PGconn *conn, const PGCEK *cek, int cekalg, + const unsigned char *value, int *nbytesp, bool enc_det); + +#endif /* FE_ENCRYPT_H */ diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c index 7bdfc4c21aa..fc1d5fe39d8 100644 --- a/src/interfaces/libpq/fe-exec.c +++ b/src/interfaces/libpq/fe-exec.c @@ -24,6 +24,8 @@ #include #endif +#include "common/colenc.h" +#include "fe-encrypt.h" #include "libpq-fe.h" #include "libpq-int.h" #include "mb/pg_wchar.h" @@ -73,7 +75,8 @@ static int PQsendQueryGuts(PGconn *conn, const char *const *paramValues, const int *paramLengths, const int *paramFormats, - int resultFormat); + int resultFormat, + PGresult *paramDesc); static void parseInput(PGconn *conn); static PGresult *getCopyResult(PGconn *conn, ExecStatusType copytype); static bool PQexecStart(PGconn *conn); @@ -1192,6 +1195,421 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value) } } +/* + * pqSaveColumnMasterKey - save column master key sent by backend + */ +int +pqSaveColumnMasterKey(PGconn *conn, int keyid, const char *keyname, + const char *keyrealm) +{ + char *keyname_copy; + char *keyrealm_copy; + bool found; + + keyname_copy = strdup(keyname); + if (!keyname_copy) + return EOF; + keyrealm_copy = strdup(keyrealm); + if (!keyrealm_copy) + { + free(keyname_copy); + return EOF; + } + + found = false; + for (int i = 0; i < conn->ncmks; i++) + { + struct pg_cmk *checkcmk = &conn->cmks[i]; + + /* replace existing? */ + if (checkcmk->cmkid == keyid) + { + free(checkcmk->cmkname); + free(checkcmk->cmkrealm); + checkcmk->cmkname = keyname_copy; + checkcmk->cmkrealm = keyrealm_copy; + found = true; + break; + } + } + + /* append new? */ + if (!found) + { + int newncmks; + struct pg_cmk *newcmks; + struct pg_cmk *newcmk; + + newncmks = conn->ncmks + 1; + if (newncmks <= 0) + return EOF; + newcmks = realloc(conn->cmks, newncmks * sizeof(struct pg_cmk)); + if (!newcmks) + { + free(keyname_copy); + free(keyrealm_copy); + return EOF; + } + + newcmk = &newcmks[newncmks - 1]; + newcmk->cmkid = keyid; + newcmk->cmkname = keyname_copy; + newcmk->cmkrealm = keyrealm_copy; + + conn->ncmks = newncmks; + conn->cmks = newcmks; + } + + return 0; +} + +/* + * Replace placeholders in input string. Return value malloc'ed. + */ +static char * +replace_cmk_placeholders(const char *in, const char *cmkname, const char *cmkrealm, int cmkalg, const char *tmpfile) +{ + PQExpBufferData buf; + + initPQExpBuffer(&buf); + + for (const char *p = in; *p; p++) + { + if (p[0] == '%') + { + switch (p[1]) + { + case 'a': + { + const char *s = get_cmkalg_name(cmkalg); + + appendPQExpBufferStr(&buf, s ? s : "INVALID"); + } + p++; + break; + case 'j': + { + const char *s = get_cmkalg_jwa_name(cmkalg); + + appendPQExpBufferStr(&buf, s ? s : "INVALID"); + } + p++; + break; + case 'k': + appendPQExpBufferStr(&buf, cmkname); + p++; + break; + case 'p': + appendPQExpBufferStr(&buf, tmpfile); + p++; + break; + case 'r': + appendPQExpBufferStr(&buf, cmkrealm); + p++; + break; + default: + appendPQExpBufferChar(&buf, p[0]); + } + } + else + appendPQExpBufferChar(&buf, p[0]); + } + + return buf.data; +} + +#ifndef USE_SSL +/* + * Dummy implementation for non-SSL builds + */ +unsigned char * +decrypt_cek_from_file(PGconn *conn, const char *cmkfilename, int cmkalg, + int fromlen, const unsigned char *from, + int *tolen) +{ + libpq_append_conn_error(conn, "column encryption not supported by this build"); + return NULL; +} +#endif + +/* + * Decrypt a CEK using the given CMK. The ciphertext is passed in + * "from" and "fromlen". Return the decrypted value in a malloc'ed area, its + * length via "tolen". Return NULL on error; add error messages directly to + * "conn". + */ +static unsigned char * +decrypt_cek(PGconn *conn, const PGCMK *cmk, int cmkalg, + int fromlen, const unsigned char *from, + int *tolen) +{ + char *cmklookup; + bool found = false; + unsigned char *result = NULL; + + if (!conn->cmklookup || !conn->cmklookup[0]) + { + libpq_append_conn_error(conn, "column master key lookup is not configured"); + return NULL; + } + + cmklookup = strdup(conn->cmklookup ? conn->cmklookup : ""); + + if (!cmklookup) + { + libpq_append_conn_error(conn, "out of memory"); + return NULL; + } + + /* + * Analyze semicolon-separated list + */ + for (char *s = strtok(cmklookup, ";"); s; s = strtok(NULL, ";")) + { + char *sep; + + /* split found token at '=' */ + sep = strchr(s, '='); + if (!sep) + { + libpq_append_conn_error(conn, "syntax error in CMK lookup specification, missing \"%c\": %s", '=', s); + break; + } + + /* matching realm? */ + if (strncmp(s, "*", sep - s) == 0 || strncmp(s, cmk->cmkrealm, sep - s) == 0) + { + char *sep2; + + found = true; + + sep2 = strchr(sep, ':'); + if (!sep2) + { + libpq_append_conn_error(conn, "syntax error in CMK lookup specification, missing \"%c\": %s", ':', s); + goto fail; + } + + if (strncmp(sep + 1, "file", sep2 - (sep + 1)) == 0) + { + char *cmkfilename; + + cmkfilename = replace_cmk_placeholders(sep2 + 1, cmk->cmkname, cmk->cmkrealm, cmkalg, "INVALID"); + result = decrypt_cek_from_file(conn, cmkfilename, cmkalg, fromlen, from, tolen); + free(cmkfilename); + } + else if (strncmp(sep + 1, "run", sep2 - (sep + 1)) == 0) + { + char tmpfile[MAXPGPATH] = {0}; + int fd; + char *command; + FILE *fp; + + /* only needs enough room for CEK key material */ + char buf[1024]; + size_t nread; + int rc; + +#ifndef WIN32 + { + const char *tmpdir; + + tmpdir = getenv("TMPDIR"); + if (!tmpdir) + tmpdir = "/tmp"; + strlcpy(tmpfile, tmpdir, sizeof(tmpfile)); + strlcat(tmpfile, "/libpq-XXXXXX", sizeof(tmpfile)); + fd = mkstemp(tmpfile); + if (fd < 0) + { + libpq_append_conn_error(conn, "could not run create temporary file: %m"); + goto fail; + } + } +#else + { + char tmpdir[MAXPGPATH]; + int ret; + + ret = GetTempPath(MAXPGPATH, tmpdir); + if (ret == 0 || ret > MAXPGPATH) + { + libpq_append_conn_error(conn, "could not locate temporary directory: %s", + !ret ? strerror(errno) : ""); + return false; + } + + if (GetTempFileName(tmpdir, "libpq", 0, tmpfile) == 0) + { + libpq_append_conn_error(conn, "could not run create temporary file: error code %lu", + GetLastError()); + goto fail; + } + + fd = open(tmpfile, O_WRONLY | O_TRUNC | PG_BINARY, 0); + if (fd < 0) + { + libpq_append_conn_error(conn, "could not run open temporary file: %m"); + goto fail; + } + } +#endif + if (write(fd, from, fromlen) < fromlen) + { + libpq_append_conn_error(conn, "could not write to temporary file: %m"); + close(fd); + unlink(tmpfile); + goto fail; + } + if (close(fd) < 0) + { + libpq_append_conn_error(conn, "could not close temporary file: %m"); + unlink(tmpfile); + goto fail; + } + + command = replace_cmk_placeholders(sep2 + 1, cmk->cmkname, cmk->cmkrealm, cmkalg, tmpfile); + fp = popen(command, PG_BINARY_R); + if (!fp) + { + libpq_append_conn_error(conn, "could not run command \"%s\": %m", command); + free(command); + unlink(tmpfile); + goto fail; + } + nread = fread(buf, 1, sizeof(buf), fp); + if (ferror(fp)) + { + libpq_append_conn_error(conn, "could not read from command: %m"); + pclose(fp); + free(command); + unlink(tmpfile); + goto fail; + } + else if (!feof(fp)) + { + libpq_append_conn_error(conn, "output from command too long"); + pclose(fp); + free(command); + unlink(tmpfile); + goto fail; + } + rc = pclose(fp); + if (rc != 0) + { + /* + * XXX would like to use wait_result_to_str(rc) but that + * cannot be called from libpq because it calls exit() + */ + libpq_append_conn_error(conn, "could not run command \"%s\"", command); + free(command); + unlink(tmpfile); + goto fail; + } + free(command); + unlink(tmpfile); + + result = malloc(nread); + if (!result) + { + libpq_append_conn_error(conn, "out of memory"); + goto fail; + } + memcpy(result, buf, nread); + *tolen = nread; + } + else + { + libpq_append_conn_error(conn, "CMK lookup scheme \"%s\" not recognized", sep + 1); + goto fail; + } + } + } + + if (!found) + { + libpq_append_conn_error(conn, "no CMK lookup found for realm \"%s\"", cmk->cmkrealm); + } + +fail: + free(cmklookup); + return result; +} + +/* + * pqSaveColumnEncryptionKey - save column encryption key sent by backend + */ +int +pqSaveColumnEncryptionKey(PGconn *conn, int keyid, int cmkid, int cmkalg, const unsigned char *value, int len) +{ + PGCMK *cmk = NULL; + unsigned char *plainval = NULL; + int plainvallen = 0; + bool found; + + for (int i = 0; i < conn->ncmks; i++) + { + if (conn->cmks[i].cmkid == cmkid) + { + cmk = &conn->cmks[i]; + break; + } + } + + if (!cmk) + return EOF; + + plainval = decrypt_cek(conn, cmk, cmkalg, len, value, &plainvallen); + if (!plainval) + return EOF; + + found = false; + for (int i = 0; i < conn->nceks; i++) + { + struct pg_cek *checkcek = &conn->ceks[i]; + + /* replace existing? */ + if (checkcek->cekid == keyid) + { + free(checkcek->cekdata); + checkcek->cekdata = plainval; + checkcek->cekdatalen = plainvallen; + found = true; + break; + } + } + + /* append new? */ + if (!found) + { + int newnceks; + struct pg_cek *newceks; + struct pg_cek *newcek; + + newnceks = conn->nceks + 1; + if (newnceks <= 0) + { + free(plainval); + return EOF; + } + newceks = realloc(conn->ceks, newnceks * sizeof(struct pg_cek)); + if (!newceks) + { + free(plainval); + return EOF; + } + + newcek = &newceks[newnceks - 1]; + newcek->cekid = keyid; + newcek->cekdata = plainval; + newcek->cekdatalen = plainvallen; + + conn->nceks = newnceks; + conn->ceks = newceks; + } + + return 0; +} /* * pqRowProcessor @@ -1263,13 +1681,51 @@ pqRowProcessor(PGconn *conn, const char **errmsgp) bool isbinary = (res->attDescs[i].format != 0); char *val; - val = (char *) pqResultAlloc(res, clen + 1, isbinary); - if (val == NULL) + if (res->attDescs[i].cekid) + { + /* encrypted column */ +#ifdef USE_SSL + PGCEK *cek = NULL; + + if (!isbinary) + { + *errmsgp = libpq_gettext("encrypted column was not sent in binary format"); + return 0; + } + + for (int j = 0; j < conn->nceks; j++) + { + if (conn->ceks[j].cekid == res->attDescs[i].cekid) + { + cek = &conn->ceks[j]; + break; + } + } + if (!cek) + { + *errmsgp = libpq_gettext("protocol error: column encryption key associated with encrypted column was not sent by the server"); + return 0; + } + + val = (char *) decrypt_value(res, cek, res->attDescs[i].cekalg, + (const unsigned char *) columns[i].value, clen, errmsgp); + if (val == NULL) + return 0; +#else + *errmsgp = libpq_gettext("column encryption not supported by this build"); return 0; +#endif + } + else + { + val = (char *) pqResultAlloc(res, clen + 1, isbinary); + if (val == NULL) + return 0; - /* copy and zero-terminate the data (even if it's binary) */ - memcpy(val, columns[i].value, clen); - val[clen] = '\0'; + /* copy and zero-terminate the data (even if it's binary) */ + memcpy(val, columns[i].value, clen); + val[clen] = '\0'; + } tup[i].len = clen; tup[i].value = val; @@ -1498,6 +1954,8 @@ PQsendQueryParams(PGconn *conn, const int *paramFormats, int resultFormat) { + PGresult *paramDesc = NULL; + if (!PQsendQueryStart(conn, true)) return 0; @@ -1514,6 +1972,37 @@ PQsendQueryParams(PGconn *conn, return 0; } + if (conn->column_encryption_enabled) + { + PGresult *res; + bool error; + + if (conn->pipelineStatus != PQ_PIPELINE_OFF) + { + libpq_append_conn_error(conn, "synchronous command execution functions are not allowed in pipeline mode"); + return 0; + } + + if (!PQsendPrepare(conn, "", command, nParams, paramTypes)) + return 0; + error = false; + while ((res = PQgetResult(conn)) != NULL) + { + if (PQresultStatus(res) != PGRES_COMMAND_OK) + error = true; + PQclear(res); + } + if (error) + return 0; + + paramDesc = PQdescribePrepared(conn, ""); + if (PQresultStatus(paramDesc) != PGRES_COMMAND_OK) + return 0; + + command = NULL; + paramTypes = NULL; + } + return PQsendQueryGuts(conn, command, "", /* use unnamed statement */ @@ -1522,7 +2011,8 @@ PQsendQueryParams(PGconn *conn, paramValues, paramLengths, paramFormats, - resultFormat); + resultFormat, + paramDesc); } /* @@ -1637,6 +2127,24 @@ PQsendQueryPrepared(PGconn *conn, const int *paramLengths, const int *paramFormats, int resultFormat) +{ + return PQsendQueryPreparedDescribed(conn, stmtName, nParams, paramValues, paramLengths, paramFormats, resultFormat, NULL); +} + +/* + * PQsendQueryPreparedDescribed + * Like PQsendQueryPrepared, but with additional argument to pass + * parameter descriptions, for column encryption. + */ +int +PQsendQueryPreparedDescribed(PGconn *conn, + const char *stmtName, + int nParams, + const char *const *paramValues, + const int *paramLengths, + const int *paramFormats, + int resultFormat, + PGresult *paramDesc) { if (!PQsendQueryStart(conn, true)) return 0; @@ -1662,7 +2170,8 @@ PQsendQueryPrepared(PGconn *conn, paramValues, paramLengths, paramFormats, - resultFormat); + resultFormat, + paramDesc); } /* @@ -1762,7 +2271,8 @@ PQsendQueryGuts(PGconn *conn, const char *const *paramValues, const int *paramLengths, const int *paramFormats, - int resultFormat) + int resultFormat, + PGresult *paramDesc) { int i; PGcmdQueueEntry *entry; @@ -1810,13 +2320,47 @@ PQsendQueryGuts(PGconn *conn, goto sendFailed; /* Send parameter formats */ - if (nParams > 0 && paramFormats) + if (nParams > 0 && (paramFormats || (paramDesc && paramDesc->paramDescs))) { if (pqPutInt(nParams, 2, conn) < 0) goto sendFailed; + for (i = 0; i < nParams; i++) { - if (pqPutInt(paramFormats[i], 2, conn) < 0) + int format = paramFormats ? paramFormats[i] : 0; + + /* Check force column encryption */ + if (format & 0x10) + { + if (!(paramDesc && + paramDesc->paramDescs && + paramDesc->paramDescs[i].cekid)) + { + libpq_append_conn_error(conn, "parameter with forced encryption is not to be encrypted"); + goto sendFailed; + } + } + format &= ~0x10; + + if (paramDesc && paramDesc->paramDescs) + { + PGresParamDesc *pd = ¶mDesc->paramDescs[i]; + + if (pd->cekid) + { + if (format != 0) + { + libpq_append_conn_error(conn, "format must be text for encrypted parameter"); + goto sendFailed; + } + /* Send encrypted value in binary */ + format = 1; + /* And mark it as encrypted */ + format |= 0x10; + } + } + + if (pqPutInt(format, 2, conn) < 0) goto sendFailed; } } @@ -1835,8 +2379,9 @@ PQsendQueryGuts(PGconn *conn, if (paramValues && paramValues[i]) { int nbytes; + const char *paramValue; - if (paramFormats && paramFormats[i] != 0) + if (paramFormats && (paramFormats[i] & 0x01) != 0) { /* binary parameter */ if (paramLengths) @@ -1852,9 +2397,53 @@ PQsendQueryGuts(PGconn *conn, /* text parameter, do not use paramLengths */ nbytes = strlen(paramValues[i]); } - if (pqPutInt(nbytes, 4, conn) < 0 || - pqPutnchar(paramValues[i], nbytes, conn) < 0) + + paramValue = paramValues[i]; + + if (paramDesc && paramDesc->paramDescs && paramDesc->paramDescs[i].cekid) + { + /* encrypted column */ +#ifdef USE_SSL + bool enc_det = (paramDesc->paramDescs[i].flags & 0x01) != 0; + PGCEK *cek = NULL; + char *enc_paramValue; + int enc_nbytes = nbytes; + + for (int j = 0; j < conn->nceks; j++) + { + if (conn->ceks[j].cekid == paramDesc->paramDescs[i].cekid) + { + cek = &conn->ceks[j]; + break; + } + } + if (!cek) + { + libpq_append_conn_error(conn, "protocol error: column encryption key associated with encrypted parameter was not sent by the server"); + goto sendFailed; + } + + enc_paramValue = (char *) encrypt_value(conn, cek, paramDesc->paramDescs[i].cekalg, + (const unsigned char *) paramValue, &enc_nbytes, enc_det); + if (!enc_paramValue) + goto sendFailed; + + if (pqPutInt(enc_nbytes, 4, conn) < 0 || + pqPutnchar(enc_paramValue, enc_nbytes, conn) < 0) + goto sendFailed; + + free(enc_paramValue); +#else + libpq_append_conn_error(conn, "column encryption not supported by this build"); goto sendFailed; +#endif + } + else + { + if (pqPutInt(nbytes, 4, conn) < 0 || + pqPutnchar(paramValue, nbytes, conn) < 0) + goto sendFailed; + } } else { @@ -2327,12 +2916,31 @@ PQexecPrepared(PGconn *conn, const int *paramLengths, const int *paramFormats, int resultFormat) +{ + return PQexecPreparedDescribed(conn, stmtName, nParams, paramValues, paramLengths, paramFormats, resultFormat, NULL); +} + +/* + * PQexecPreparedDescribed + * Like PQexecPrepared, but with additional argument to pass parameter + * descriptions, for column encryption. + */ +PGresult * +PQexecPreparedDescribed(PGconn *conn, + const char *stmtName, + int nParams, + const char *const *paramValues, + const int *paramLengths, + const int *paramFormats, + int resultFormat, + PGresult *paramDesc) { if (!PQexecStart(conn)) return NULL; - if (!PQsendQueryPrepared(conn, stmtName, - nParams, paramValues, paramLengths, - paramFormats, resultFormat)) + if (!PQsendQueryPreparedDescribed(conn, stmtName, + nParams, paramValues, paramLengths, + paramFormats, + resultFormat, paramDesc)) return NULL; return PQexecFinish(conn); } @@ -3710,7 +4318,17 @@ PQfformat(const PGresult *res, int field_num) if (!check_field_number(res, field_num)) return 0; if (res->attDescs) + { + /* + * An encrypted column is always presented to the application in text + * format. The .format field applies to the ciphertext, which might + * be in either format, but the plaintext inside is always in text + * format. + */ + if (res->attDescs[field_num].cekid != 0) + return 0; return res->attDescs[field_num].format; + } else return 0; } @@ -3748,6 +4366,17 @@ PQfmod(const PGresult *res, int field_num) return 0; } +int +PQfisencrypted(const PGresult *res, int field_num) +{ + if (!check_field_number(res, field_num)) + return false; + if (res->attDescs) + return (res->attDescs[field_num].cekid != 0); + else + return false; +} + char * PQcmdStatus(PGresult *res) { @@ -3933,6 +4562,17 @@ PQparamtype(const PGresult *res, int param_num) return InvalidOid; } +int +PQparamisencrypted(const PGresult *res, int param_num) +{ + if (!check_param_number(res, param_num)) + return false; + if (res->paramDescs) + return (res->paramDescs[param_num].cekid != 0); + else + return false; +} + /* PQsetnonblocking: * sets the PGconn's database connection non-blocking if the arg is true diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index 3170d484f02..c453546b46d 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -48,6 +48,8 @@ static int getRowDescriptions(PGconn *conn, int msgLength); static int getParamDescriptions(PGconn *conn, int msgLength); static int getAnotherTuple(PGconn *conn, int msgLength); static int getParameterStatus(PGconn *conn); +static int getColumnMasterKey(PGconn *conn); +static int getColumnEncryptionKey(PGconn *conn); static int getNotify(PGconn *conn); static int getCopyStart(PGconn *conn, ExecStatusType copytype); static int getReadyForQuery(PGconn *conn); @@ -313,6 +315,12 @@ pqParseInput3(PGconn *conn) if (pqGetInt(&(conn->be_key), 4, conn)) return; break; + case PqMsg_ColumnMasterKey: + getColumnMasterKey(conn); + break; + case PqMsg_ColumnEncryptionKey: + getColumnEncryptionKey(conn); + break; case PqMsg_RowDescription: if (conn->error_result || (conn->result != NULL && @@ -374,8 +382,21 @@ pqParseInput3(PGconn *conn) } break; case PqMsg_ParameterDescription: - if (getParamDescriptions(conn, msgLength)) - return; + if (conn->error_result || + (conn->result != NULL && + conn->result->resultStatus == PGRES_FATAL_ERROR)) + { + /* + * We've already choked for some reason. Just discard + * the data till we get to the end of the query. + */ + conn->inCursor += msgLength; + } + else + { + if (getParamDescriptions(conn, msgLength)) + return; + } break; case PqMsg_DataRow: if (conn->result != NULL && @@ -564,6 +585,9 @@ getRowDescriptions(PGconn *conn, int msgLength) int typlen; int atttypmod; int format; + int cekid; + int cekalg; + int flags; if (pqGets(&conn->workBuffer, conn) || pqGetInt(&tableid, 4, conn) || @@ -578,6 +602,23 @@ getRowDescriptions(PGconn *conn, int msgLength) goto advance_and_error; } + if (conn->column_encryption_enabled) + { + if (pqGetInt(&cekid, 4, conn) || + pqGetInt(&cekalg, 4, conn) || + pqGetInt(&flags, 2, conn)) + { + errmsg = libpq_gettext("insufficient data in \"T\" message"); + goto advance_and_error; + } + } + else + { + cekid = 0; + cekalg = 0; + flags = 0; + } + /* * Since pqGetInt treats 2-byte integers as unsigned, we need to * coerce these results to signed form. @@ -599,6 +640,8 @@ getRowDescriptions(PGconn *conn, int msgLength) result->attDescs[i].typid = typid; result->attDescs[i].typlen = typlen; result->attDescs[i].atttypmod = atttypmod; + result->attDescs[i].cekid = cekid; + result->attDescs[i].cekalg = cekalg; if (format != 1) result->binary = 0; @@ -702,10 +745,31 @@ getParamDescriptions(PGconn *conn, int msgLength) for (i = 0; i < nparams; i++) { int typid; + int cekid; + int cekalg; + int flags; if (pqGetInt(&typid, 4, conn)) goto not_enough_data; + if (conn->column_encryption_enabled) + { + if (pqGetInt(&cekid, 4, conn)) + goto not_enough_data; + if (pqGetInt(&cekalg, 4, conn)) + goto not_enough_data; + if (pqGetInt(&flags, 2, conn)) + goto not_enough_data; + } + else + { + cekid = 0; + cekalg = 0; + flags = 0; + } result->paramDescs[i].typid = typid; + result->paramDescs[i].cekid = cekid; + result->paramDescs[i].cekalg = cekalg; + result->paramDescs[i].flags = flags; } /* Success! */ @@ -1486,6 +1550,92 @@ getParameterStatus(PGconn *conn) return 0; } +/* + * Attempt to read a ColumnMasterKey message. + * Entry: 'y' message type and length have already been consumed. + * Exit: returns 0 if successfully consumed message. + * returns EOF if not enough data. + */ +static int +getColumnMasterKey(PGconn *conn) +{ + int keyid; + char *keyname; + char *keyrealm; + int ret; + + /* Get the key ID */ + if (pqGetInt(&keyid, 4, conn) != 0) + return EOF; + /* Get the key name */ + if (pqGets(&conn->workBuffer, conn) != 0) + return EOF; + keyname = strdup(conn->workBuffer.data); + if (!keyname) + return EOF; + /* Get the key realm */ + if (pqGets(&conn->workBuffer, conn) != 0) + return EOF; + keyrealm = strdup(conn->workBuffer.data); + if (!keyrealm) + return EOF; + /* And save it */ + ret = pqSaveColumnMasterKey(conn, keyid, keyname, keyrealm); + if (ret != 0) + pqSaveErrorResult(conn); + + free(keyname); + free(keyrealm); + + return ret; +} + +/* + * Attempt to read a ColumnEncryptionKey message. + * Entry: 'Y' message type and length have already been consumed. + * Exit: returns 0 if successfully consumed message. + * returns EOF if not enough data. + */ +static int +getColumnEncryptionKey(PGconn *conn) +{ + int keyid; + int cmkid; + int cmkalg; + char *buf; + int vallen; + int ret; + + /* Get the key ID */ + if (pqGetInt(&keyid, 4, conn) != 0) + return EOF; + /* Get the CMK ID */ + if (pqGetInt(&cmkid, 4, conn) != 0) + return EOF; + /* Get the CMK algorithm */ + if (pqGetInt(&cmkalg, 4, conn) != 0) + return EOF; + /* Get the key data len */ + if (pqGetInt(&vallen, 4, conn) != 0) + return EOF; + /* Get the key data */ + buf = malloc(vallen); + if (!buf) + return EOF; + if (pqGetnchar(buf, vallen, conn) != 0) + { + free(buf); + return EOF; + } + /* And save it */ + ret = pqSaveColumnEncryptionKey(conn, keyid, cmkid, cmkalg, (unsigned char *) buf, vallen); + if (ret != 0) + pqSaveErrorResult(conn); + + free(buf); + + return ret; +} /* * Attempt to read a Notify response message. @@ -2304,6 +2454,9 @@ build_startup_packet(const PGconn *conn, char *packet, if (conn->client_encoding_initial && conn->client_encoding_initial[0]) ADD_STARTUP_OPTION("client_encoding", conn->client_encoding_initial); + if (conn->column_encryption_enabled) + ADD_STARTUP_OPTION("_pq_.column_encryption", "1"); + /* Add any environment-driven GUC settings needed */ for (next_eo = options; next_eo->envName; next_eo++) { diff --git a/src/interfaces/libpq/fe-trace.c b/src/interfaces/libpq/fe-trace.c index c9932fc8a6b..740dd15e5b2 100644 --- a/src/interfaces/libpq/fe-trace.c +++ b/src/interfaces/libpq/fe-trace.c @@ -450,7 +450,8 @@ pqTraceOutputS(FILE *f, const char *message, int *cursor) /* ParameterDescription */ static void -pqTraceOutputt(FILE *f, const char *message, int *cursor, bool regress) +pqTraceOutputt(FILE *f, const char *message, int *cursor, bool regress, + bool column_encryption_enabled) { int nfields; @@ -458,12 +459,21 @@ pqTraceOutputt(FILE *f, const char *message, int *cursor, bool regress) nfields = pqTraceOutputInt16(f, message, cursor); for (int i = 0; i < nfields; i++) + { pqTraceOutputInt32(f, message, cursor, regress); + if (column_encryption_enabled) + { + pqTraceOutputInt32(f, message, cursor, regress); + pqTraceOutputInt32(f, message, cursor, false); + pqTraceOutputInt16(f, message, cursor); + } + } } /* RowDescription */ static void -pqTraceOutputT(FILE *f, const char *message, int *cursor, bool regress) +pqTraceOutputT(FILE *f, const char *message, int *cursor, bool regress, + bool column_encryption_enabled) { int nfields; @@ -479,6 +489,12 @@ pqTraceOutputT(FILE *f, const char *message, int *cursor, bool regress) pqTraceOutputInt16(f, message, cursor); pqTraceOutputInt32(f, message, cursor, false); pqTraceOutputInt16(f, message, cursor); + if (column_encryption_enabled) + { + pqTraceOutputInt32(f, message, cursor, regress); + pqTraceOutputInt32(f, message, cursor, false); + pqTraceOutputInt16(f, message, cursor); + } } } @@ -514,6 +530,30 @@ pqTraceOutputW(FILE *f, const char *message, int *cursor, int length) pqTraceOutputInt16(f, message, cursor); } +/* ColumnMasterKey */ +static void +pqTraceOutputy(FILE *f, const char *message, int *cursor, bool regress) +{ + fprintf(f, "ColumnMasterKey\t"); + pqTraceOutputInt32(f, message, cursor, regress); + pqTraceOutputString(f, message, cursor, false); + pqTraceOutputString(f, message, cursor, false); +} + +/* ColumnEncryptionKey */ +static void +pqTraceOutputY(FILE *f, const char *message, int *cursor, bool regress) +{ + int len; + + fprintf(f, "ColumnEncryptionKey\t"); + pqTraceOutputInt32(f, message, cursor, regress); + pqTraceOutputInt32(f, message, cursor, regress); + pqTraceOutputInt32(f, message, cursor, false); + len = pqTraceOutputInt32(f, message, cursor, false); + pqTraceOutputNchar(f, len, message, cursor); +} + /* ReadyForQuery */ static void pqTraceOutputZ(FILE *f, const char *message, int *cursor) @@ -657,10 +697,12 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer) fprintf(conn->Pfdebug, "Sync"); /* no message content */ break; case PqMsg_ParameterDescription: - pqTraceOutputt(conn->Pfdebug, message, &logCursor, regress); + pqTraceOutputt(conn->Pfdebug, message, &logCursor, regress, + conn->column_encryption_enabled); break; case PqMsg_RowDescription: - pqTraceOutputT(conn->Pfdebug, message, &logCursor, regress); + pqTraceOutputT(conn->Pfdebug, message, &logCursor, regress, + conn->column_encryption_enabled); break; case PqMsg_NegotiateProtocolVersion: pqTraceOutputv(conn->Pfdebug, message, &logCursor); @@ -675,6 +717,12 @@ pqTraceOutputMessage(PGconn *conn, const char *message, bool toServer) fprintf(conn->Pfdebug, "Terminate"); /* No message content */ break; + case PqMsg_ColumnMasterKey: + pqTraceOutputy(conn->Pfdebug, message, &logCursor, regress); + break; + case PqMsg_ColumnEncryptionKey: + pqTraceOutputY(conn->Pfdebug, message, &logCursor, regress); + break; case PqMsg_ReadyForQuery: pqTraceOutputZ(conn->Pfdebug, message, &logCursor); break; diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h index 73f6e65ae55..91717ab23b9 100644 --- a/src/interfaces/libpq/libpq-fe.h +++ b/src/interfaces/libpq/libpq-fe.h @@ -276,6 +276,8 @@ typedef struct pgresAttDesc Oid typid; /* type id */ int typlen; /* type size */ int atttypmod; /* type-specific modifier info */ + Oid cekid; + int cekalg; } PGresAttDesc; /* ---------------- @@ -466,6 +468,14 @@ extern PGresult *PQexecPrepared(PGconn *conn, const int *paramLengths, const int *paramFormats, int resultFormat); +extern PGresult *PQexecPreparedDescribed(PGconn *conn, + const char *stmtName, + int nParams, + const char *const *paramValues, + const int *paramLengths, + const int *paramFormats, + int resultFormat, + PGresult *paramDesc); /* Interface for multiple-result or asynchronous queries */ #define PQ_QUERY_PARAM_MAX_LIMIT 65535 @@ -489,6 +499,14 @@ extern int PQsendQueryPrepared(PGconn *conn, const int *paramLengths, const int *paramFormats, int resultFormat); +extern int PQsendQueryPreparedDescribed(PGconn *conn, + const char *stmtName, + int nParams, + const char *const *paramValues, + const int *paramLengths, + const int *paramFormats, + int resultFormat, + PGresult *paramDesc); extern int PQsetSingleRowMode(PGconn *conn); extern int PQsetChunkedRowsMode(PGconn *conn, int chunkSize); extern PGresult *PQgetResult(PGconn *conn); @@ -561,6 +579,7 @@ extern int PQfformat(const PGresult *res, int field_num); extern Oid PQftype(const PGresult *res, int field_num); extern int PQfsize(const PGresult *res, int field_num); extern int PQfmod(const PGresult *res, int field_num); +extern int PQfisencrypted(const PGresult *res, int field_num); extern char *PQcmdStatus(PGresult *res); extern char *PQoidStatus(const PGresult *res); /* old and ugly */ extern Oid PQoidValue(const PGresult *res); /* new and improved */ @@ -570,6 +589,7 @@ extern int PQgetlength(const PGresult *res, int tup_num, int field_num); extern int PQgetisnull(const PGresult *res, int tup_num, int field_num); extern int PQnparams(const PGresult *res); extern Oid PQparamtype(const PGresult *res, int param_num); +extern int PQparamisencrypted(const PGresult *res, int param_num); /* Describe prepared statements and portals */ extern PGresult *PQdescribePrepared(PGconn *conn, const char *stmt); diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h index 3691e5ee969..7b1c7133379 100644 --- a/src/interfaces/libpq/libpq-int.h +++ b/src/interfaces/libpq/libpq-int.h @@ -113,6 +113,9 @@ union pgresult_data typedef struct pgresParamDesc { Oid typid; /* type id */ + Oid cekid; + int cekalg; + int flags; } PGresParamDesc; /* @@ -358,6 +361,26 @@ typedef struct pg_conn_host * found in password file. */ } pg_conn_host; +/* + * Column encryption support data + */ + +/* column master key */ +typedef struct pg_cmk +{ + Oid cmkid; + char *cmkname; + char *cmkrealm; +} PGCMK; + +/* column encryption key */ +typedef struct pg_cek +{ + Oid cekid; + unsigned char *cekdata; /* (decrypted) */ + size_t cekdatalen; +} PGCEK; + /* * PGconn stores all the state data associated with a single connection * to a backend. @@ -416,6 +439,9 @@ struct pg_conn char *ssl_max_protocol_version; /* maximum TLS protocol version */ char *target_session_attrs; /* desired session properties */ char *require_auth; /* name of the expected auth method */ + char *cmklookup; /* CMK lookup specification */ + char *column_encryption_setting; /* column_encryption connection + * parameter */ char *load_balance_hosts; /* load balance over hosts */ bool cancelRequest; /* true if this connection is used to send a @@ -517,8 +543,15 @@ struct pg_conn PGVerbosity verbosity; /* error/notice message verbosity */ PGContextVisibility show_context; /* whether to show CONTEXT field */ PGlobjfuncs *lobjfuncs; /* private state for large-object access fns */ + bool column_encryption_enabled; /* parsed version of + * column_encryption_setting */ pg_prng_state prng_state; /* prng state for load balancing connections */ + /* Column encryption support data */ + int ncmks; + PGCMK *cmks; + int nceks; + PGCEK *ceks; /* Buffer for data received from backend and not yet processed */ char *inBuffer; /* currently allocated buffer */ @@ -709,6 +742,10 @@ extern void pqSaveMessageField(PGresult *res, char code, const char *value); extern void pqSaveParameterStatus(PGconn *conn, const char *name, const char *value); +extern int pqSaveColumnMasterKey(PGconn *conn, int keyid, const char *keyname, + const char *keyrealm); +extern int pqSaveColumnEncryptionKey(PGconn *conn, int keyid, int cmkid, int cmkalg, + const unsigned char *value, int len); extern int pqRowProcessor(PGconn *conn, const char **errmsgp); extern void pqCommandQueueAdvance(PGconn *conn, bool isReadyForQuery, bool gotSync); diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build index be6fadaea23..60fbdce602f 100644 --- a/src/interfaces/libpq/meson.build +++ b/src/interfaces/libpq/meson.build @@ -30,6 +30,7 @@ endif if ssl.found() libpq_sources += files('fe-secure-common.c') + libpq_sources += files('fe-encrypt-openssl.c') libpq_sources += files('fe-secure-openssl.c') endif @@ -118,6 +119,7 @@ tests += { 't/002_api.pl', 't/003_load_balance_host_list.pl', 't/004_load_balance_dns.pl', + 't/010_encrypt.pl', ], 'env': {'with_ssl': ssl_library}, }, diff --git a/src/interfaces/libpq/nls.mk b/src/interfaces/libpq/nls.mk index 40b662dc08b..b6e9743a98a 100644 --- a/src/interfaces/libpq/nls.mk +++ b/src/interfaces/libpq/nls.mk @@ -3,6 +3,7 @@ CATALOG_NAME = libpq GETTEXT_FILES = fe-auth.c \ fe-auth-scram.c \ fe-connect.c \ + fe-encrypt-openssl.c \ fe-exec.c \ fe-gssapi-common.c \ fe-lobj.c \ diff --git a/src/interfaces/libpq/t/010_encrypt.pl b/src/interfaces/libpq/t/010_encrypt.pl new file mode 100644 index 00000000000..db588a52563 --- /dev/null +++ b/src/interfaces/libpq/t/010_encrypt.pl @@ -0,0 +1,72 @@ +# Copyright (c) 2024, PostgreSQL Global Development Group +use strict; +use warnings; + +use PostgreSQL::Test::Utils; +use Test::More; + +plan skip_all => 'OpenSSL not supported by this build' + if $ENV{with_ssl} ne 'openssl'; + +# test data from https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05#section-5 +command_like( + ['libpq_test_encrypt'], + qr{5.1 +C = +1a f3 8c 2d c2 b9 6f fd d8 66 94 09 23 41 bc 04 +c8 0e df a3 2d df 39 d5 ef 00 c0 b4 68 83 42 79 +a2 e4 6a 1b 80 49 f7 92 f7 6b fe 54 b9 03 a9 c9 +a9 4a c9 b4 7a d2 65 5c 5f 10 f9 ae f7 14 27 e2 +fc 6f 9b 3f 39 9a 22 14 89 f1 63 62 c7 03 23 36 +09 d4 5a c6 98 64 e3 32 1c f8 29 35 ac 40 96 c8 +6e 13 33 14 c5 40 19 e8 ca 79 80 df a4 b9 cf 1b +38 4c 48 6f 3a 54 c5 10 78 15 8e e5 d7 9d e5 9f +bd 34 d8 48 b3 d6 95 50 a6 76 46 34 44 27 ad e5 +4b 88 51 ff b5 98 f7 f8 00 74 b9 47 3c 82 e2 db +65 2c 3f a3 6b 0a 7c 5b 32 19 fa b3 a3 0b c1 c4 +5.2 +C = +1a f3 8c 2d c2 b9 6f fd d8 66 94 09 23 41 bc 04 +ea 65 da 6b 59 e6 1e db 41 9b e6 2d 19 71 2a e5 +d3 03 ee b5 00 52 d0 df d6 69 7f 77 22 4c 8e db +00 0d 27 9b dc 14 c1 07 26 54 bd 30 94 42 30 c6 +57 be d4 ca 0c 9f 4a 84 66 f2 2b 22 6d 17 46 21 +4b f8 cf c2 40 0a dd 9f 51 26 e4 79 66 3f c9 0b +3b ed 78 7a 2f 0f fc bf 39 04 be 2a 64 1d 5c 21 +05 bf e5 91 ba e2 3b 1d 74 49 e5 32 ee f6 0a 9a +c8 bb 6c 6b 01 d3 5d 49 78 7b cd 57 ef 48 49 27 +f2 80 ad c9 1a c0 c4 e7 9c 7b 11 ef c6 00 54 e3 +84 90 ac 0e 58 94 9b fe 51 87 5d 73 3f 93 ac 20 +75 16 80 39 cc c7 33 d7 +5.3 +C = +1a f3 8c 2d c2 b9 6f fd d8 66 94 09 23 41 bc 04 +89 31 29 b0 f4 ee 9e b1 8d 75 ed a6 f2 aa a9 f3 +60 7c 98 c4 ba 04 44 d3 41 62 17 0d 89 61 88 4e +58 f2 7d 4a 35 a5 e3 e3 23 4a a9 94 04 f3 27 f5 +c2 d7 8e 98 6e 57 49 85 8b 88 bc dd c2 ba 05 21 +8f 19 51 12 d6 ad 48 fa 3b 1e 89 aa 7f 20 d5 96 +68 2f 10 b3 64 8d 3b b0 c9 83 c3 18 5f 59 e3 6d +28 f6 47 c1 c1 39 88 de 8e a0 d8 21 19 8c 15 09 +77 e2 8c a7 68 08 0b c7 8c 35 fa ed 69 d8 c0 b7 +d9 f5 06 23 21 98 a4 89 a1 a6 ae 03 a3 19 fb 30 +dd 13 1d 05 ab 34 67 dd 05 6f 8e 88 2b ad 70 63 +7f 1e 9a 54 1d 9c 23 e7 +5.4 +C = +1a f3 8c 2d c2 b9 6f fd d8 66 94 09 23 41 bc 04 +4a ff aa ad b7 8c 31 c5 da 4b 1b 59 0d 10 ff bd +3d d8 d5 d3 02 42 35 26 91 2d a0 37 ec bc c7 bd +82 2c 30 1d d6 7c 37 3b cc b5 84 ad 3e 92 79 c2 +e6 d1 2a 13 74 b7 7f 07 75 53 df 82 94 10 44 6b +36 eb d9 70 66 29 6a e6 42 7e a7 5c 2e 08 46 a1 +1a 09 cc f5 37 0d c8 0b fe cb ad 28 c7 3f 09 b3 +a3 b7 5e 66 2a 25 94 41 0a e4 96 b2 e2 e6 60 9e +31 e6 e0 2c c8 37 f0 53 d2 1f 37 ff 4f 51 95 0b +be 26 38 d0 9d d7 a4 93 09 30 80 6d 07 03 b1 f6 +4d d3 b4 c0 88 a7 f4 5c 21 68 39 64 5b 20 12 bf +2e 62 69 a8 c5 6a 81 6d bc 1b 26 77 61 95 5b c5 +}, + 'AEAD_AES_*_CBC_HMAC_SHA_* test cases'); + +done_testing(); diff --git a/src/interfaces/libpq/test/.gitignore b/src/interfaces/libpq/test/.gitignore index 6ba78adb678..1846594ec51 100644 --- a/src/interfaces/libpq/test/.gitignore +++ b/src/interfaces/libpq/test/.gitignore @@ -1,2 +1,3 @@ +/libpq_test_encrypt /libpq_testclient /libpq_uri_regress diff --git a/src/interfaces/libpq/test/Makefile b/src/interfaces/libpq/test/Makefile index 4e17ec15141..11b66337b83 100644 --- a/src/interfaces/libpq/test/Makefile +++ b/src/interfaces/libpq/test/Makefile @@ -15,10 +15,17 @@ override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS) LDFLAGS_INTERNAL += $(libpq_pgport) PROGS = libpq_testclient libpq_uri_regress +ifeq ($(with_ssl),openssl) +PROGS += libpq_test_encrypt +endif + all: $(PROGS) $(PROGS): $(WIN32RES) +libpq_test_encrypt: ../fe-encrypt-openssl.c + $(CC) $(CPPFLAGS) -DTEST_ENCRYPT $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X) + clean distclean: rm -f $(PROGS) *.o diff --git a/src/interfaces/libpq/test/meson.build b/src/interfaces/libpq/test/meson.build index 21dd37f69bc..666771a99e8 100644 --- a/src/interfaces/libpq/test/meson.build +++ b/src/interfaces/libpq/test/meson.build @@ -36,3 +36,26 @@ testprep_targets += executable('libpq_testclient', 'install': false, } ) + + +libpq_test_encrypt_sources = files( + '../fe-encrypt-openssl.c', +) + +if host_system == 'windows' + libpq_test_encrypt_sources += rc_bin_gen.process(win32ver_rc, extra_args: [ + '--NAME', 'libpq_test_encrypt', + '--FILEDESC', 'libpq test program',]) +endif + +if ssl.found() + executable('libpq_test_encrypt', + libpq_test_encrypt_sources, + include_directories: include_directories('../../../port'), + dependencies: [frontend_code, libpq, ssl], + c_args: ['-DTEST_ENCRYPT'], + kwargs: default_bin_args + { + 'install': false, + } + ) +endif diff --git a/src/test/Makefile b/src/test/Makefile index dbd3192874d..c9a38680531 100644 --- a/src/test/Makefile +++ b/src/test/Makefile @@ -24,7 +24,7 @@ ifeq ($(with_ldap),yes) SUBDIRS += ldap endif ifeq ($(with_ssl),openssl) -SUBDIRS += ssl +SUBDIRS += column_encryption ssl endif # Test suites that are not safe by default but can be run if selected @@ -36,7 +36,7 @@ export PG_TEST_EXTRA # clean" etc to recurse into them. (We must filter out those that we # have conditionally included into SUBDIRS above, else there will be # make confusion.) -ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl) +ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),column_encryption examples kerberos icu ldap ssl) # We want to recurse to all subdirs for all standard targets, except that # installcheck and install should not recurse into the subdirectory "modules". diff --git a/src/test/column_encryption/.gitignore b/src/test/column_encryption/.gitignore new file mode 100644 index 00000000000..456dbf69d2a --- /dev/null +++ b/src/test/column_encryption/.gitignore @@ -0,0 +1,3 @@ +/test_client +# Generated by test suite +/tmp_check/ diff --git a/src/test/column_encryption/Makefile b/src/test/column_encryption/Makefile new file mode 100644 index 00000000000..7b6471e43e0 --- /dev/null +++ b/src/test/column_encryption/Makefile @@ -0,0 +1,31 @@ +#------------------------------------------------------------------------- +# +# Makefile for src/test/column_encryption +# +# Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group +# Portions Copyright (c) 1994, Regents of the University of California +# +# src/test/column_encryption/Makefile +# +#------------------------------------------------------------------------- + +subdir = src/test/column_encryption +top_builddir = ../../.. +include $(top_builddir)/src/Makefile.global + +export OPENSSL + +override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS) +LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) + +all: test_client + +check: all + $(prove_check) + +installcheck: + $(prove_installcheck) + +clean distclean maintainer-clean: + rm -f test_client.o test_client + rm -rf tmp_check diff --git a/src/test/column_encryption/meson.build b/src/test/column_encryption/meson.build new file mode 100644 index 00000000000..84cfa84e12f --- /dev/null +++ b/src/test/column_encryption/meson.build @@ -0,0 +1,23 @@ +column_encryption_test_client = executable('test_client', + files('test_client.c'), + dependencies: [frontend_code, libpq], + kwargs: default_bin_args + { + 'install': false, + }, +) +testprep_targets += column_encryption_test_client + +tests += { + 'name': 'column_encryption', + 'sd': meson.current_source_dir(), + 'bd': meson.current_build_dir(), + 'tap': { + 'tests': [ + 't/001_column_encryption.pl', + 't/002_cmk_rotation.pl', + ], + 'env': { + 'OPENSSL': openssl.path(), + }, + }, +} diff --git a/src/test/column_encryption/t/001_column_encryption.pl b/src/test/column_encryption/t/001_column_encryption.pl new file mode 100644 index 00000000000..7a4c86e51f6 --- /dev/null +++ b/src/test/column_encryption/t/001_column_encryption.pl @@ -0,0 +1,307 @@ +# Copyright (c) 2021-2024, PostgreSQL Global Development Group + +use strict; +use warnings; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +my $openssl = $ENV{OPENSSL}; + +my $perlbin = $^X; +$perlbin =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os; + +# Can be changed manually for testing other algorithms. Note that +# RSAES_OAEP_SHA_256 requires OpenSSL 1.1.0. +my $cmkalg = 'RSAES_OAEP_SHA_1'; + +my $node = PostgreSQL::Test::Cluster->new('node'); +$node->init; +$node->start; + + +sub create_cmk +{ + my ($cmkname) = @_; + my $cmkfilename = "${PostgreSQL::Test::Utils::tmp_check}/${cmkname}.pem"; + system_or_bail $openssl, 'genpkey', '-algorithm', 'rsa', '-out', + $cmkfilename; + $node->safe_psql('postgres', qq{CREATE COLUMN MASTER KEY ${cmkname}}); + return $cmkfilename; +} + +sub create_cek +{ + my ($cekname, $bytes, $cmkname, $cmkfilename) = @_; + + my $digest = $cmkalg; + $digest =~ s/.*(?=SHA)//; + $digest =~ s/_//g; + + # generate random bytes + system_or_bail $openssl, 'rand', '-out', + "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", $bytes; + + # encrypt CEK using CMK + my @cmd = ( + $openssl, + 'pkeyutl', + '-encrypt', + '-inkey', + $cmkfilename, + '-pkeyopt', + 'rsa_padding_mode:oaep', + '-in', + "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", + '-out', + "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc"); + if ($digest ne 'SHA1') + { + # These options require OpenSSL >=1.1.0, so if the digest is + # SHA1, which is the default, omit the options. + push @cmd, + '-pkeyopt', "rsa_mgf1_md:$digest", + '-pkeyopt', "rsa_oaep_md:$digest"; + } + system_or_bail @cmd; + + my $cekenchex = unpack('H*', + slurp_file + "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc"); + + # create CEK in database + $node->safe_psql('postgres', + qq{CREATE COLUMN ENCRYPTION KEY ${cekname} WITH VALUES (column_master_key = ${cmkname}, algorithm = '${cmkalg}', encrypted_value = '\\x${cekenchex}');} + ); + + return; +} + + +my $cmk1filename = create_cmk('cmk1'); +my $cmk2filename = create_cmk('cmk2'); +create_cek('cek1', 48, 'cmk1', $cmk1filename); +create_cek('cek2', 72, 'cmk2', $cmk2filename); + +$ENV{PGCOLUMNENCRYPTION} = 'on'; +$ENV{PGCMKLOOKUP} = + '*=file:' . ${PostgreSQL::Test::Utils::tmp_check} . '/%k.pem'; + + +$node->safe_psql( + 'postgres', qq{ +CREATE TABLE tbl1 ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1), + c smallint ENCRYPTED WITH (column_encryption_key = cek1) +); +}); + +$node->safe_psql( + 'postgres', q{ +INSERT INTO tbl1 (a, b, c) VALUES (1, $1, $2) \bind 'val1' 11 \g +INSERT INTO tbl1 (a, b, c) VALUES (2, $1, $2) \bind 'val2' 22 \g +}); + +# Expected ciphertext length is 2 blocks of AES output (2 * 16) plus +# half SHA-256 output (16) in hex encoding: (2 * 16 + 16) * 2 = 96 +like( + $node->safe_psql('postgres', q{COPY (SELECT * FROM tbl1) TO STDOUT}), + qr/1\tencrypted\$[0-9a-f]{96}\tencrypted\$[0-9a-f]{96}\n2\tencrypted\$[0-9a-f]{96}\tencrypted\$[0-9a-f]{96}/, + 'inserted data is encrypted'); + +my $result; + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl1 \gdesc}); +is( $result, + q(a|integer +b|text +c|smallint), + 'query result description has original type'); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl1}); +is( $result, + q(1|val1|11 +2|val2|22), + 'decrypted query result'); + +{ + local $ENV{PGCMKLOOKUP} = '*=run:broken %k %p'; + $result = $node->psql('postgres', q{SELECT a, b, c FROM tbl1}); + isnt($result, 0, 'query fails with broken cmklookup run setting'); +} + +{ + local $ENV{TESTWORKDIR} = ${PostgreSQL::Test::Utils::tmp_check}; + local $ENV{PGCMKLOOKUP} = + qq{*=run:"$perlbin" ./test_run_decrypt.pl %k %a %p}; + + my $stdout; + $result = $node->psql('postgres', q{SELECT a, b, c FROM tbl1}, + stdout => \$stdout); + is( $stdout, + q(1|val1|11 +2|val2|22), + 'decrypted query result with cmklookup run'); +} + + +$node->command_fails_like( + [ 'test_client', 'test1' ], + qr/not encrypted/, + 'test client fails because parameters not encrypted'); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl1}); +is( $result, + q(1|val1|11 +2|val2|22), + 'decrypted query result after test client insert'); + +$node->command_ok([ 'test_client', 'test2' ], 'test client test 2'); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl1}); +is( $result, + q(1|val1|11 +2|val2|22 +3|val3|33), + 'decrypted query result after test client insert 2'); + +like( + $node->safe_psql( + 'postgres', q{COPY (SELECT * FROM tbl1 WHERE a = 3) TO STDOUT}), + qr/3\tencrypted\$[0-9a-f]{96}/, + 'inserted data is encrypted'); + + +# Test copy and restore + +my $copy_out = $node->safe_psql('postgres', q{COPY tbl1 TO STDOUT;}); +$node->safe_psql('postgres', + q{CREATE TABLE tbl1_copy (LIKE tbl1 INCLUDING ENCRYPTED)}); +$node->safe_psql('postgres', + q{COPY tbl1_copy FROM STDIN;} . "\n" . $copy_out . "\\\.\n"); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl1_copy}); +is( $result, + q(1|val1|11 +2|val2|22 +3|val3|33), + 'decrypted query result after COPY dump and restore'); + + +# Tests with binary format + +# Supplying a parameter in binary format when the parameter is to be +# encrypted results in an error from libpq. +$node->command_fails_like( + [ 'test_client', 'test3' ], + qr/format must be text for encrypted parameter/, + 'test client fails because to-be-encrypted parameter is in binary format' +); + +# Requesting a binary result set still causes any encrypted columns to +# be returned as text from the libpq API. +$node->command_like( + [ 'test_client', 'test4' ], + qr/<0,0>=1:\n<0,1>=0:val1\n<0,2>=0:11/, + 'binary result set with encrypted columns: encrypted columns returned as text' +); + + +# Test UPDATE + +$node->safe_psql( + 'postgres', q{ +UPDATE tbl1 SET b = $2 WHERE a = $1 \bind '3' 'val3upd' \g +}); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl1}); +is( $result, + q(1|val1|11 +2|val2|22 +3|val3upd|33), + 'decrypted query result after update'); + + +# Test views + +$node->safe_psql('postgres', q{CREATE VIEW v1 AS SELECT a, b, c FROM tbl1}); + +$node->safe_psql('postgres', + q{UPDATE v1 SET b = $2 WHERE a = $1 \bind '3' 'val3upd2' \g}); + +$result = + $node->safe_psql('postgres', q{SELECT a, b, c FROM v1 WHERE a IN (1, 3)}); +is( $result, + q(1|val1|11 +3|val3upd2|33), + 'decrypted query result from view'); + + +# Test deterministic encryption + +$node->safe_psql( + 'postgres', qq{ +CREATE TABLE tbl2 ( + a int, + b text ENCRYPTED WITH (encryption_type = deterministic, column_encryption_key = cek1) +); +}); + +$node->safe_psql( + 'postgres', q{ +INSERT INTO tbl2 (a, b) VALUES ($1, $2), ($3, $4), ($5, $6) \bind '1' 'valA' '2' 'valB' '3' 'valA' \g +}); + +$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl2}); +is( $result, + q(1|valA +2|valB +3|valA), + 'decrypted query result in table for deterministic encryption'); + +is( $node->safe_psql( + 'postgres', q{SELECT b, count(*) FROM tbl2 GROUP BY b ORDER BY 2}), + q(valB|1 +valA|2), + 'group by deterministically encrypted column'); + +is( $node->safe_psql( + 'postgres', q{SELECT a FROM tbl2 WHERE b = $1 \bind 'valB' \g}), + q(2), + 'select by deterministically encrypted column'); + + +# Test multiple keys in one table + +$node->safe_psql( + 'postgres', qq{ +CREATE TABLE tbl3 ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1), + c text ENCRYPTED WITH (column_encryption_key = cek2, algorithm = 'AEAD_AES_192_CBC_HMAC_SHA_384') +); +}); + +$node->safe_psql( + 'postgres', q{ +INSERT INTO tbl3 (a, b, c) VALUES (1, $1, $2) \bind 'valB1' 'valC1' \g +}); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl3}); +is($result, q(1|valB1|valC1), 'decrypted query result multiple keys'); + +$node->safe_psql( + 'postgres', q{ +INSERT INTO tbl3 (a, b, c) VALUES ($1, $2, $3), ($4, $5, $6) \bind '2' 'valB2' 'valC2' '3' 'valB3' 'valC3' \g +}); + +$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl3}); +is( $result, + q(1|valB1|valC1 +2|valB2|valC2 +3|valB3|valC3), + 'decrypted query result multiple keys after second insert'); + + +done_testing(); diff --git a/src/test/column_encryption/t/002_cmk_rotation.pl b/src/test/column_encryption/t/002_cmk_rotation.pl new file mode 100644 index 00000000000..2f2dac4e143 --- /dev/null +++ b/src/test/column_encryption/t/002_cmk_rotation.pl @@ -0,0 +1,125 @@ +# Copyright (c) 2021-2024, PostgreSQL Global Development Group + +# Test column master key rotation. First, we generate CMK1 and a CEK +# encrypted with it. Then we add a CMK2 and encrypt the CEK with it +# as well. (Recall that a CEK can be associated with multiple CMKs, +# for this reason. That's why pg_colenckeydata is split out from +# pg_colenckey.) Then we remove CMK1. We test that we can get +# decrypted query results at each step. + +use strict; +use warnings; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +my $openssl = $ENV{OPENSSL}; + +my $cmkalg = 'RSAES_OAEP_SHA_1'; + +my $node = PostgreSQL::Test::Cluster->new('node'); +$node->init; +$node->start; + + +sub create_cmk +{ + my ($cmkname) = @_; + my $cmkfilename = "${PostgreSQL::Test::Utils::tmp_check}/${cmkname}.pem"; + system_or_bail $openssl, 'genpkey', '-algorithm', 'rsa', '-out', + $cmkfilename; + $node->safe_psql('postgres', qq{CREATE COLUMN MASTER KEY ${cmkname}}); + return $cmkfilename; +} + + +my $cmk1filename = create_cmk('cmk1'); + +# create CEK +my ($cekname, $bytes) = ('cek1', 48); + +# generate random bytes +system_or_bail $openssl, 'rand', '-out', + "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", $bytes; + +# encrypt CEK using CMK +system_or_bail $openssl, 'pkeyutl', '-encrypt', + '-inkey', $cmk1filename, + '-pkeyopt', 'rsa_padding_mode:oaep', + '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", + '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc"; + +my $cekenchex = unpack('H*', + slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc"); + +# create CEK in database +$node->safe_psql('postgres', + qq{CREATE COLUMN ENCRYPTION KEY ${cekname} WITH VALUES (column_master_key = cmk1, algorithm = '$cmkalg', encrypted_value = '\\x${cekenchex}');} +); + +$ENV{PGCOLUMNENCRYPTION} = 'on'; +$ENV{PGCMKLOOKUP} = + '*=file:' . ${PostgreSQL::Test::Utils::tmp_check} . '/%k.pem'; + +$node->safe_psql( + 'postgres', qq{ +CREATE TABLE tbl1 ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +); +}); + +$node->safe_psql( + 'postgres', q{ +INSERT INTO tbl1 (a, b) VALUES (1, $1) \bind 'val1' \g +INSERT INTO tbl1 (a, b) VALUES (2, $1) \bind 'val2' \g +}); + +is( $node->safe_psql('postgres', q{SELECT a, b FROM tbl1}), + q(1|val1 +2|val2), + 'decrypted query result with one CMK'); + + +# create new CMK +my $cmk2filename = create_cmk('cmk2'); + +# encrypt CEK using new CMK +# +# (Here, we still have the plaintext of the CEK available from +# earlier. In reality, one would decrypt the CEK with the first CMK +# and then re-encrypt it with the second CMK.) +system_or_bail $openssl, 'pkeyutl', '-encrypt', + '-inkey', $cmk2filename, + '-pkeyopt', 'rsa_padding_mode:oaep', + '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", + '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc"; + +$cekenchex = unpack('H*', + slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc"); + +# add new data record for CEK in database +$node->safe_psql('postgres', + qq{ALTER COLUMN ENCRYPTION KEY ${cekname} ADD VALUE (column_master_key = cmk2, algorithm = '$cmkalg', encrypted_value = '\\x${cekenchex}');} +); + + +is( $node->safe_psql('postgres', q{SELECT a, b FROM tbl1}), + q(1|val1 +2|val2), + 'decrypted query result with two CMKs'); + + +# delete CEK record for first CMK +$node->safe_psql('postgres', + qq{ALTER COLUMN ENCRYPTION KEY ${cekname} DROP VALUE (column_master_key = cmk1);} +); + + +is( $node->safe_psql('postgres', q{SELECT a, b FROM tbl1}), + q(1|val1 +2|val2), + 'decrypted query result with only new CMK'); + + +done_testing(); diff --git a/src/test/column_encryption/test_client.c b/src/test/column_encryption/test_client.c new file mode 100644 index 00000000000..c5df88d47d2 --- /dev/null +++ b/src/test/column_encryption/test_client.c @@ -0,0 +1,161 @@ +/* + * Copyright (c) 2021-2024, PostgreSQL Global Development Group + */ + +#include "postgres_fe.h" + +#include "libpq-fe.h" + + +/* + * Test calls that don't support encryption + */ +static int +test1(PGconn *conn) +{ + PGresult *res; + const char *values[] = {"3", "val3", "33"}; + + res = PQprepare(conn, "", "INSERT INTO tbl1 (a, b, c) VALUES ($1, $2, $3)", + 3, NULL); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + { + fprintf(stderr, "PQprepare() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + res = PQexecPrepared(conn, "", 3, values, NULL, NULL, 0); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + { + fprintf(stderr, "PQexecPrepared() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + return 0; +} + +/* + * Test forced encryption + */ +static int +test2(PGconn *conn) +{ + PGresult *res, + *res2; + const char *values[] = {"3", "val3", "33"}; + int formats[] = {0x00, 0x10, 0x00}; + + res = PQprepare(conn, "", "INSERT INTO tbl1 (a, b, c) VALUES ($1, $2, $3)", + 3, NULL); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + { + fprintf(stderr, "PQprepare() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + res2 = PQdescribePrepared(conn, ""); + if (PQresultStatus(res2) != PGRES_COMMAND_OK) + { + fprintf(stderr, "PQdescribePrepared() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + if (!(!PQparamisencrypted(res2, 0) && + PQparamisencrypted(res2, 1))) + { + fprintf(stderr, "wrong results from PQparamisencrypted()\n"); + return 1; + } + + res = PQexecPreparedDescribed(conn, "", 3, values, NULL, formats, 0, res2); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + { + fprintf(stderr, "PQexecPrepared() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + return 0; +} + +/* + * Test what happens when you supply a binary parameter that is required to be + * encrypted. + */ +static int +test3(PGconn *conn) +{ + PGresult *res; + const char *values[] = {""}; + int lengths[] = {1}; + int formats[] = {1}; + + res = PQexecParams(conn, "INSERT INTO tbl1 (a, b, c) VALUES (100, NULL, $1)", + 3, NULL, values, lengths, formats, 0); + if (PQresultStatus(res) != PGRES_COMMAND_OK) + { + fprintf(stderr, "PQexecParams() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + return 0; +} + +/* + * Test what happens when you request results in binary and the result rows + * contain an encrypted column. + */ +static int +test4(PGconn *conn) +{ + PGresult *res; + + res = PQexecParams(conn, "SELECT a, b, c FROM tbl1 WHERE a = 1", 0, NULL, NULL, NULL, NULL, 1); + if (PQresultStatus(res) != PGRES_TUPLES_OK) + { + fprintf(stderr, "PQexecParams() failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + for (int row = 0; row < PQntuples(res); row++) + for (int col = 0; col < PQnfields(res); col++) + printf("<%d,%d>=%d:%s\n", row, col, PQfformat(res, col), PQgetvalue(res, row, col)); + return 0; +} + +int +main(int argc, char **argv) +{ + PGconn *conn; + int ret = 0; + + conn = PQconnectdb(""); + if (PQstatus(conn) != CONNECTION_OK) + { + fprintf(stderr, "Connection to database failed: %s\n", + PQerrorMessage(conn)); + return 1; + } + + if (argc < 2 || argv[1] == NULL) + return 87; + else if (strcmp(argv[1], "test1") == 0) + ret = test1(conn); + else if (strcmp(argv[1], "test2") == 0) + ret = test2(conn); + else if (strcmp(argv[1], "test3") == 0) + ret = test3(conn); + else if (strcmp(argv[1], "test4") == 0) + ret = test4(conn); + else + ret = 88; + + PQfinish(conn); + return ret; +} diff --git a/src/test/column_encryption/test_run_decrypt.pl b/src/test/column_encryption/test_run_decrypt.pl new file mode 100755 index 00000000000..fef25dc6785 --- /dev/null +++ b/src/test/column_encryption/test_run_decrypt.pl @@ -0,0 +1,61 @@ +#!/usr/bin/perl + +# Test/sample command for libpq cmklookup run scheme +# +# This just places the data into temporary files and runs the openssl +# command on it. (In practice, this could more simply be written as a +# shell script, but this way it's more portable.) + +# Copyright (c) 2021-2024, PostgreSQL Global Development Group + +use strict; +use warnings; + +my ($cmkname, $alg, $filename) = @ARGV; + +die unless $alg =~ 'RSAES_OAEP_SHA'; + +my $digest = $alg; +$digest =~ s/.*(?=SHA)//; +$digest =~ s/_//g; + +my $tmpdir = $ENV{TESTWORKDIR}; + +my $openssl = $ENV{OPENSSL}; + +my @cmd = ( + $openssl, 'pkeyutl', + '-decrypt', '-inkey', + "${tmpdir}/${cmkname}.pem", '-pkeyopt', + 'rsa_padding_mode:oaep', '-in', + $filename, '-out', + "${tmpdir}/output.tmp"); + +if ($digest ne 'SHA1') +{ + # These options require OpenSSL >=1.1.0, so if the digest is + # SHA1, which is the default, omit the options. + push @cmd, + '-pkeyopt', "rsa_mgf1_md:$digest", + '-pkeyopt', "rsa_oaep_md:$digest"; +} + +system(@cmd) == 0 or die "system failed: $?"; + +open my $fh, '<:raw', "${tmpdir}/output.tmp" or die $!; +my $data = ''; + +while (1) +{ + my $success = read $fh, $data, 100, length($data); + die $! if not defined $success; + last if not $success; +} + +close $fh; + +unlink "${tmpdir}/output.tmp"; + +binmode STDOUT; + +print $data; diff --git a/src/test/meson.build b/src/test/meson.build index 702213bc6f6..213f7edf66e 100644 --- a/src/test/meson.build +++ b/src/test/meson.build @@ -10,6 +10,7 @@ subdir('subscription') subdir('modules') if ssl.found() + subdir('column_encryption') subdir('ssl') endif diff --git a/src/test/regress/expected/column_encryption.out b/src/test/regress/expected/column_encryption.out new file mode 100644 index 00000000000..b64b3241b93 --- /dev/null +++ b/src/test/regress/expected/column_encryption.out @@ -0,0 +1,541 @@ +\set HIDE_COLUMN_ENCRYPTION false +CREATE ROLE regress_enc_user1; +CREATE COLUMN MASTER KEY fail WITH ( + foo = bar +); +ERROR: column master key attribute "foo" not recognized +LINE 2: foo = bar + ^ +CREATE COLUMN MASTER KEY fail WITH ( + realm = 'test', + realm = 'test' +); +ERROR: conflicting or redundant options +LINE 3: realm = 'test' + ^ +CREATE COLUMN MASTER KEY cmk1 WITH ( + realm = 'test' +); +COMMENT ON COLUMN MASTER KEY cmk1 IS 'column master key'; +-- duplicate +CREATE COLUMN MASTER KEY cmk1 WITH ( + realm = 'test' +); +ERROR: column master key "cmk1" already exists +CREATE COLUMN MASTER KEY cmk1a WITH ( + realm = 'test' +); +CREATE COLUMN MASTER KEY cmk2; +CREATE COLUMN MASTER KEY cmk2a WITH ( + realm = 'testx' +); +ALTER COLUMN MASTER KEY cmk2a (realm = 'test2'); +-- fail +ALTER COLUMN MASTER KEY cmk2a (realm = 'test2', realm = 'test2'); +ERROR: conflicting or redundant options +LINE 1: ALTER COLUMN MASTER KEY cmk2a (realm = 'test2', realm = 'tes... + ^ +-- fail +ALTER COLUMN MASTER KEY cmk2a (foo = bar); +ERROR: column master key attribute "foo" not recognized +LINE 1: ALTER COLUMN MASTER KEY cmk2a (foo = bar); + ^ +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + foo = bar +); +ERROR: column encryption key attribute "foo" not recognized +LINE 2: foo = bar + ^ +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + column_master_key = cmk1 +); +ERROR: conflicting or redundant options +LINE 3: column_master_key = cmk1 + ^ +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +ERROR: attribute "column_master_key" must be specified +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + encrypted_value = '\xDEADBEEF' +); +ERROR: attribute "algorithm" must be specified +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1' +); +ERROR: attribute "encrypted_value" must be specified +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + algorithm = 'foo', -- invalid + encrypted_value = '\xDEADBEEF' +); +ERROR: unrecognized encryption algorithm: foo +LINE 3: algorithm = 'foo', -- invalid + ^ +CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +COMMENT ON COLUMN ENCRYPTION KEY cek1 IS 'column encryption key'; +-- duplicate +CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +ERROR: column encryption key "cek1" already exists +ALTER COLUMN ENCRYPTION KEY cek1 ADD VALUE ( + column_master_key = cmk1a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +-- duplicate +ALTER COLUMN ENCRYPTION KEY cek1 ADD VALUE ( + column_master_key = cmk1a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +ERROR: column encryption key "cek1" already has data for master key "cmk1a" +ALTER COLUMN ENCRYPTION KEY fail ADD VALUE ( + column_master_key = cmk1a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +ERROR: column encryption key "fail" does not exist +CREATE COLUMN ENCRYPTION KEY cek2 WITH VALUES ( + column_master_key = cmk2, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +), +( + column_master_key = cmk2a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +CREATE COLUMN ENCRYPTION KEY cek4 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (foo = bar) +); +ERROR: unrecognized column encryption parameter: foo +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (encryption_type = randomized) +); +ERROR: column encryption key must be specified +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = notexist) +); +ERROR: column encryption key "notexist" does not exist +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1, algorithm = 'foo') +); +ERROR: unrecognized encryption algorithm: foo +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = wrong) +); +ERROR: unrecognized encryption type: wrong +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1, column_encryption_key = cek1) +); +ERROR: conflicting or redundant options +CREATE TABLE tbl_29f3 ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1) +); +\d tbl_29f3 + Table "public.tbl_29f3" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+--------- + a | integer | | | + b | text | | | + c | text | | | + +\d+ tbl_29f3 + Table "public.tbl_29f3" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | extended | | | + c | text | | | | external | cek1 | | + +CREATE TABLE tbl_447f ( + a int, + b text +); +ALTER TABLE tbl_447f ADD COLUMN c text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = deterministic); +\d tbl_447f + Table "public.tbl_447f" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+--------- + a | integer | | | + b | text | | | + c | text | | | + +\d+ tbl_447f + Table "public.tbl_447f" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | extended | | | + c | text | | | | external | cek1 | | + +CREATE TABLE tbl_4897 (LIKE tbl_447f); +CREATE TABLE tbl_6978 (LIKE tbl_447f INCLUDING ENCRYPTED); +\d+ tbl_4897 + Table "public.tbl_4897" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | extended | | | + c | text | | | | extended | | | + +\d+ tbl_6978 + Table "public.tbl_6978" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | extended | | | + c | text | | | | external | cek1 | | + +CREATE VIEW view_3bc9 AS SELECT * FROM tbl_29f3; +\d+ view_3bc9 + View "public.view_3bc9" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Description +--------+---------+-----------+----------+---------+----------+------------+------------- + a | integer | | | | plain | | + b | text | | | | extended | | + c | text | | | | external | cek1 | +View definition: + SELECT a, + b, + c + FROM tbl_29f3; + +CREATE TABLE tbl_2386 AS SELECT * FROM tbl_29f3 WITH NO DATA; +\d+ tbl_2386 + Table "public.tbl_2386" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | extended | | | + c | text | | | | external | cek1 | | + +CREATE TABLE tbl_2941 AS SELECT * FROM tbl_29f3 WITH DATA; +ERROR: encrypted columns not yet implemented for this command +\d+ tbl_2941 +-- test partition declarations +CREATE TABLE tbl_13fa ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +) PARTITION BY RANGE (a); +CREATE TABLE tbl_13fa_1 PARTITION OF tbl_13fa FOR VALUES FROM (1) TO (100); +\d+ tbl_13fa + Partitioned table "public.tbl_13fa" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | external | cek1 | | +Partition key: RANGE (a) +Partitions: tbl_13fa_1 FOR VALUES FROM (1) TO (100) + +\d+ tbl_13fa_1 + Table "public.tbl_13fa_1" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | external | cek1 | | +Partition of: tbl_13fa FOR VALUES FROM (1) TO (100) +Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 100)) + +-- test inheritance +CREATE TABLE tbl_36f3_a ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +); +CREATE TABLE tbl_36f3_b ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +); +CREATE TABLE tbl_36f3_c ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek2) +); +CREATE TABLE tbl_36f3_d ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = deterministic) +); +CREATE TABLE tbl_36f3_e ( + a int, + b text +); +-- not implemented (but could be ok) +CREATE TABLE tbl_36f3_ab (c int) INHERITS (tbl_36f3_a, tbl_36f3_b); +NOTICE: merging multiple inherited definitions of column "a" +NOTICE: merging multiple inherited definitions of column "b" +ERROR: multiple inheritance of encrypted columns is not implemented +\d+ tbl_36f3_ab +-- not implemented (but should fail) +CREATE TABLE tbl_36f3_ac (c int) INHERITS (tbl_36f3_a, tbl_36f3_c); +NOTICE: merging multiple inherited definitions of column "a" +NOTICE: merging multiple inherited definitions of column "b" +ERROR: multiple inheritance of encrypted columns is not implemented +CREATE TABLE tbl_36f3_ad (c int) INHERITS (tbl_36f3_a, tbl_36f3_d); +NOTICE: merging multiple inherited definitions of column "a" +NOTICE: merging multiple inherited definitions of column "b" +ERROR: multiple inheritance of encrypted columns is not implemented +-- fail +CREATE TABLE tbl_36f3_ae (c int) INHERITS (tbl_36f3_a, tbl_36f3_e); +NOTICE: merging multiple inherited definitions of column "a" +NOTICE: merging multiple inherited definitions of column "b" +ERROR: column "b" has an encryption specification conflict +-- ok +CREATE TABLE tbl_36f3_a_1 (b text ENCRYPTED WITH (column_encryption_key = cek1), c int) INHERITS (tbl_36f3_a); +NOTICE: moving and merging column "b" with inherited definition +DETAIL: User-specified column moved to the position of the inherited column. +\d+ tbl_36f3_a_1 + Table "public.tbl_36f3_a_1" + Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description +--------+---------+-----------+----------+---------+----------+------------+--------------+------------- + a | integer | | | | plain | | | + b | text | | | | external | cek1 | | + c | integer | | | | plain | | | +Inherits: tbl_36f3_a + +-- fail +CREATE TABLE tbl_36f3_a_2 (b text ENCRYPTED WITH (column_encryption_key = cek2), c int) INHERITS (tbl_36f3_a); +NOTICE: moving and merging column "b" with inherited definition +DETAIL: User-specified column moved to the position of the inherited column. +ERROR: column "b" has an encryption specification conflict +CREATE TABLE tbl_36f3_a_2 (b text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = deterministic), c int) INHERITS (tbl_36f3_a); +NOTICE: moving and merging column "b" with inherited definition +DETAIL: User-specified column moved to the position of the inherited column. +ERROR: column "b" has an encryption specification conflict +CREATE TABLE tbl_36f3_a_2 (b text, c int) INHERITS (tbl_36f3_a); +NOTICE: moving and merging column "b" with inherited definition +DETAIL: User-specified column moved to the position of the inherited column. +ERROR: column "b" has an encryption specification conflict +DROP TABLE tbl_36f3_b, tbl_36f3_c, tbl_36f3_d, tbl_36f3_e; +-- SET SCHEMA +CREATE SCHEMA test_schema_ce; +ALTER COLUMN ENCRYPTION KEY cek1 SET SCHEMA test_schema_ce; +ALTER COLUMN MASTER KEY cmk1 SET SCHEMA test_schema_ce; +ALTER COLUMN ENCRYPTION KEY test_schema_ce.cek1 SET SCHEMA public; +ALTER COLUMN MASTER KEY test_schema_ce.cmk1 SET SCHEMA public; +DROP SCHEMA test_schema_ce; +-- privileges +SET SESSION AUTHORIZATION 'regress_enc_user1'; +-- fail +CREATE COLUMN ENCRYPTION KEY cek10 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +ERROR: permission denied for column master key cmk1 +RESET SESSION AUTHORIZATION; +GRANT USAGE ON COLUMN MASTER KEY cmk1 TO regress_enc_user1; +SET SESSION AUTHORIZATION 'regress_enc_user1'; +-- ok now +CREATE COLUMN ENCRYPTION KEY cek10 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +DROP COLUMN ENCRYPTION KEY cek10; +-- fail +CREATE TABLE tbl_7040 (a int, b text ENCRYPTED WITH (column_encryption_key = cek1)); +ERROR: permission denied for column encryption key cek1 +CREATE TABLE tbl_7040 (a int); +-- fail +ALTER TABLE tbl_7040 ADD COLUMN b text ENCRYPTED WITH (column_encryption_key = cek1); +ERROR: permission denied for column encryption key cek1 +DROP TABLE tbl_7040; +RESET SESSION AUTHORIZATION; +GRANT USAGE ON COLUMN ENCRYPTION KEY cek1 TO regress_enc_user1; +SET SESSION AUTHORIZATION 'regress_enc_user1'; +-- ok now +CREATE TABLE tbl_7040 (a int, b text ENCRYPTED WITH (column_encryption_key = cek1)); +RESET SESSION AUTHORIZATION; +-- has_column_encryption_key_privilege +SELECT has_column_encryption_key_privilege('regress_enc_user1', + (SELECT oid FROM pg_colenckey WHERE cekname='cek1'), 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +SELECT has_column_encryption_key_privilege('regress_enc_user1', 'cek1', 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +SELECT has_column_encryption_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), + (SELECT oid FROM pg_colenckey WHERE cekname='cek1'), 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +SELECT has_column_encryption_key_privilege( + (SELECT oid FROM pg_colenckey WHERE cekname='cek1'), 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +SELECT has_column_encryption_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), 'cek1', 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +SELECT has_column_encryption_key_privilege('cek1', 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +SELECT has_column_encryption_key_privilege('regress_enc_user1', 'cek1', 'USAGE'); + has_column_encryption_key_privilege +------------------------------------- + t +(1 row) + +-- has_column_master_key_privilege +SELECT has_column_master_key_privilege('regress_enc_user1', + (SELECT oid FROM pg_colmasterkey WHERE cmkname='cmk1'), 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +SELECT has_column_master_key_privilege('regress_enc_user1', 'cmk1', 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +SELECT has_column_master_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), + (SELECT oid FROM pg_colmasterkey WHERE cmkname='cmk1'), 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +SELECT has_column_master_key_privilege( + (SELECT oid FROM pg_colmasterkey WHERE cmkname='cmk1'), 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +SELECT has_column_master_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), 'cmk1', 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +SELECT has_column_master_key_privilege('cmk1', 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +SELECT has_column_master_key_privilege('regress_enc_user1', 'cmk1', 'USAGE'); + has_column_master_key_privilege +--------------------------------- + t +(1 row) + +DROP TABLE tbl_7040; +REVOKE USAGE ON COLUMN ENCRYPTION KEY cek1 FROM regress_enc_user1; +REVOKE USAGE ON COLUMN MASTER KEY cmk1 FROM regress_enc_user1; +-- not useful here, just checking that it runs +DISCARD COLUMN ENCRYPTION KEYS; +DROP COLUMN MASTER KEY cmk1 RESTRICT; -- fail +ERROR: cannot drop column master key cmk1 because other objects depend on it +DETAIL: column encryption key data of column encryption key cek1 for column master key cmk1 depends on column master key cmk1 +column encryption key data of column encryption key cek4 for column master key cmk1 depends on column master key cmk1 +HINT: Use DROP ... CASCADE to drop the dependent objects too. +ALTER COLUMN MASTER KEY cmk2 RENAME TO cmk3; +ALTER COLUMN MASTER KEY cmk1 RENAME TO cmk3; -- fail +ERROR: column master key "cmk3" already exists in schema "public" +ALTER COLUMN MASTER KEY cmkx RENAME TO cmky; -- fail +ERROR: column master key "cmkx" does not exist +ALTER COLUMN ENCRYPTION KEY cek2 RENAME TO cek3; +ALTER COLUMN ENCRYPTION KEY cek1 RENAME TO cek3; -- fail +ERROR: column encryption key "cek3" already exists in schema "public" +ALTER COLUMN ENCRYPTION KEY cekx RENAME TO ceky; -- fail +ERROR: column encryption key "cekx" does not exist +SET SESSION AUTHORIZATION 'regress_enc_user1'; +DROP COLUMN ENCRYPTION KEY cek3; -- fail +ERROR: must be owner of column encryption key cek3 +DROP COLUMN MASTER KEY cmk3; -- fail +ERROR: must be owner of column master key cmk3 +RESET SESSION AUTHORIZATION; +ALTER COLUMN MASTER KEY cmk3 OWNER TO regress_enc_user1; +ALTER COLUMN ENCRYPTION KEY cek3 OWNER TO regress_enc_user1; +\dcek cek3 + List of column encryption keys + Schema | Name | Owner | Master key +--------+------+-------------------+------------ + public | cek3 | regress_enc_user1 | cmk2a + public | cek3 | regress_enc_user1 | cmk3 +(2 rows) + +\dcmk cmk3 + List of column master keys + Schema | Name | Owner | Realm +--------+------+-------------------+------- + public | cmk3 | regress_enc_user1 | +(1 row) + +SET SESSION AUTHORIZATION 'regress_enc_user1'; +DROP COLUMN ENCRYPTION KEY cek3; -- ok now +DROP COLUMN MASTER KEY cmk3; -- ok now +RESET SESSION AUTHORIZATION; +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a); +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a); -- fail +ERROR: column encryption key "cek1" has no data for master key "cmk1a" +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = fail); -- fail +ERROR: column master key "fail" does not exist +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a, algorithm = 'foo'); -- fail +ERROR: attribute "algorithm" must not be specified +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a, encrypted_value = '\xDEADBEEF'); -- fail +ERROR: attribute "encrypted_value" must not be specified +DROP COLUMN ENCRYPTION KEY cek4; +DROP COLUMN ENCRYPTION KEY fail; +ERROR: column encryption key "fail" does not exist +DROP COLUMN ENCRYPTION KEY IF EXISTS nonexistent; +NOTICE: column encryption key "nonexistent" does not exist, skipping +DROP COLUMN MASTER KEY cmk1a; +DROP COLUMN MASTER KEY fail; +ERROR: column master key "fail" does not exist +DROP COLUMN MASTER KEY IF EXISTS nonexistent; +NOTICE: column master key "nonexistent" does not exist, skipping +DROP ROLE regress_enc_user1; diff --git a/src/test/regress/expected/object_address.out b/src/test/regress/expected/object_address.out index fc42d418bf1..8dbb4a847ac 100644 --- a/src/test/regress/expected/object_address.out +++ b/src/test/regress/expected/object_address.out @@ -38,6 +38,8 @@ CREATE SERVER "integer" FOREIGN DATA WRAPPER addr_fdw; CREATE USER MAPPING FOR regress_addr_user SERVER "integer"; ALTER DEFAULT PRIVILEGES FOR ROLE regress_addr_user IN SCHEMA public GRANT ALL ON TABLES TO regress_addr_user; ALTER DEFAULT PRIVILEGES FOR ROLE regress_addr_user REVOKE DELETE ON TABLES FROM regress_addr_user; +CREATE COLUMN MASTER KEY addr_cmk; +CREATE COLUMN ENCRYPTION KEY addr_cek WITH VALUES (column_master_key = addr_cmk, algorithm = 'RSAES_OAEP_SHA_1', encrypted_value = ''); -- this transform would be quite unsafe to leave lying around, -- except that the SQL language pays no attention to transforms: CREATE TRANSFORM FOR int LANGUAGE SQL ( @@ -101,6 +103,7 @@ BEGIN ('materialized view'), ('foreign table'), ('table column'), ('foreign table column'), ('aggregate'), ('function'), ('procedure'), ('type'), ('cast'), + ('column encryption key'), ('column encryption key data'), ('column master key'), ('table constraint'), ('domain constraint'), ('conversion'), ('default value'), ('operator'), ('operator class'), ('operator family'), ('rule'), ('trigger'), ('text search parser'), ('text search dictionary'), @@ -201,6 +204,24 @@ WARNING: error for cast,{addr_nsp,zwei},{}: name list length must be exactly 1 WARNING: error for cast,{addr_nsp,zwei},{integer}: name list length must be exactly 1 WARNING: error for cast,{eins,zwei,drei},{}: name list length must be exactly 1 WARNING: error for cast,{eins,zwei,drei},{integer}: name list length must be exactly 1 +WARNING: error for column encryption key,{eins},{}: column encryption key "eins" does not exist +WARNING: error for column encryption key,{eins},{integer}: column encryption key "eins" does not exist +WARNING: error for column encryption key,{addr_nsp,zwei},{}: column encryption key "addr_nsp.zwei" does not exist +WARNING: error for column encryption key,{addr_nsp,zwei},{integer}: column encryption key "addr_nsp.zwei" does not exist +WARNING: error for column encryption key,{eins,zwei,drei},{}: cross-database references are not implemented: eins.zwei.drei +WARNING: error for column encryption key,{eins,zwei,drei},{integer}: cross-database references are not implemented: eins.zwei.drei +WARNING: error for column encryption key data,{eins},{}: column encryption key "eins" does not exist +WARNING: error for column encryption key data,{eins},{integer}: column encryption key "eins" does not exist +WARNING: error for column encryption key data,{addr_nsp,zwei},{}: column encryption key "addr_nsp.zwei" does not exist +WARNING: error for column encryption key data,{addr_nsp,zwei},{integer}: column encryption key "addr_nsp.zwei" does not exist +WARNING: error for column encryption key data,{eins,zwei,drei},{}: cross-database references are not implemented: eins.zwei.drei +WARNING: error for column encryption key data,{eins,zwei,drei},{integer}: cross-database references are not implemented: eins.zwei.drei +WARNING: error for column master key,{eins},{}: column master key "eins" does not exist +WARNING: error for column master key,{eins},{integer}: column master key "eins" does not exist +WARNING: error for column master key,{addr_nsp,zwei},{}: column master key "addr_nsp.zwei" does not exist +WARNING: error for column master key,{addr_nsp,zwei},{integer}: column master key "addr_nsp.zwei" does not exist +WARNING: error for column master key,{eins,zwei,drei},{}: cross-database references are not implemented: eins.zwei.drei +WARNING: error for column master key,{eins,zwei,drei},{integer}: cross-database references are not implemented: eins.zwei.drei WARNING: error for table constraint,{eins},{}: must specify relation and object name WARNING: error for table constraint,{eins},{integer}: must specify relation and object name WARNING: error for table constraint,{addr_nsp,zwei},{}: relation "addr_nsp" does not exist @@ -409,6 +430,9 @@ WITH objects (type, name, args) AS (VALUES ('type', '{addr_nsp.genenum}', '{}'), ('cast', '{int8}', '{int4}'), ('collation', '{default}', '{}'), + ('column encryption key', '{addr_cek}', '{}'), + ('column encryption key data', '{addr_cek}', '{addr_cmk}'), + ('column master key', '{addr_cmk}', '{}'), ('table constraint', '{addr_nsp, gentable, a_chk}', '{}'), ('domain constraint', '{addr_nsp.gendomain}', '{domconstr}'), ('conversion', '{pg_catalog, koi8_r_to_mic}', '{}'), @@ -505,6 +529,9 @@ subscription|NULL|regress_addr_sub|regress_addr_sub|t publication|NULL|addr_pub|addr_pub|t publication relation|NULL|NULL|addr_nsp.gentable in publication addr_pub|t publication namespace|NULL|NULL|addr_nsp in publication addr_pub_schema|t +column master key|addr_nsp|addr_cmk|addr_nsp.addr_cmk|t +column encryption key|addr_nsp|addr_cek|addr_nsp.addr_cek|t +column encryption key data|NULL|NULL|of addr_nsp.addr_cek for addr_nsp.addr_cmk|t --- --- Cleanup resources --- @@ -517,6 +544,8 @@ drop cascades to user mapping for regress_addr_user on server integer DROP PUBLICATION addr_pub; DROP PUBLICATION addr_pub_schema; DROP SUBSCRIPTION regress_addr_sub; +DROP COLUMN ENCRYPTION KEY addr_cek; +DROP COLUMN MASTER KEY addr_cmk; DROP SCHEMA addr_nsp CASCADE; NOTICE: drop cascades to 14 other objects DETAIL: drop cascades to text search dictionary addr_ts_dict @@ -547,6 +576,9 @@ WITH objects (classid, objid, objsubid) AS (VALUES ('pg_type'::regclass, 0, 0), -- no type ('pg_cast'::regclass, 0, 0), -- no cast ('pg_collation'::regclass, 0, 0), -- no collation + ('pg_colenckey'::regclass, 0, 0), -- no column encryption key + ('pg_colenckeydata'::regclass, 0, 0), -- no column encryption key data + ('pg_colmasterkey'::regclass, 0, 0), -- no column master key ('pg_constraint'::regclass, 0, 0), -- no constraint ('pg_conversion'::regclass, 0, 0), -- no conversion ('pg_attrdef'::regclass, 0, 0), -- no default attribute @@ -634,5 +666,8 @@ ORDER BY objects.classid, objects.objid, objects.objsubid; ("(""publication relation"",,,)")|("(""publication relation"",,)")|NULL ("(""publication namespace"",,,)")|("(""publication namespace"",,)")|NULL ("(""parameter ACL"",,,)")|("(""parameter ACL"",,)")|NULL +("(""column master key"",,,)")|("(""column master key"",,)")|NULL +("(""column encryption key"",,,)")|("(""column encryption key"",,)")|NULL +("(""column encryption key data"",,,)")|("(""column encryption key data"",,)")|NULL -- restore normal output mode \a\t diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out index 215eb899be3..63d3081e67e 100644 --- a/src/test/regress/expected/oidjoins.out +++ b/src/test/regress/expected/oidjoins.out @@ -74,6 +74,8 @@ NOTICE: checking pg_type {typcollation} => pg_collation {oid} NOTICE: checking pg_attribute {attrelid} => pg_class {oid} NOTICE: checking pg_attribute {atttypid} => pg_type {oid} NOTICE: checking pg_attribute {attcollation} => pg_collation {oid} +NOTICE: checking pg_attribute {attcek} => pg_colenckey {oid} +NOTICE: checking pg_attribute {attusertypid} => pg_type {oid} NOTICE: checking pg_class {relnamespace} => pg_namespace {oid} NOTICE: checking pg_class {reltype} => pg_type {oid} NOTICE: checking pg_class {reloftype} => pg_type {oid} @@ -266,3 +268,9 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid} NOTICE: checking pg_subscription {subowner} => pg_authid {oid} NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid} NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid} +NOTICE: checking pg_colmasterkey {cmknamespace} => pg_namespace {oid} +NOTICE: checking pg_colmasterkey {cmkowner} => pg_authid {oid} +NOTICE: checking pg_colenckey {ceknamespace} => pg_namespace {oid} +NOTICE: checking pg_colenckey {cekowner} => pg_authid {oid} +NOTICE: checking pg_colenckeydata {ckdcekid} => pg_colenckey {oid} +NOTICE: checking pg_colenckeydata {ckdcmkid} => pg_colmasterkey {oid} diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 9d047b21b88..996628b7aae 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -159,7 +159,8 @@ ORDER BY 1, 2; text | character varying timestamp without time zone | timestamp with time zone txid_snapshot | pg_snapshot -(4 rows) + pg_encrypted_det | pg_encrypted_rnd +(5 rows) SELECT DISTINCT p1.proargtypes[0]::regtype, p2.proargtypes[0]::regtype FROM pg_proc AS p1, pg_proc AS p2 @@ -175,13 +176,15 @@ WHERE p1.oid != p2.oid AND ORDER BY 1, 2; proargtypes | proargtypes -----------------------------+-------------------------- + bytea | pg_encrypted_det bigint | xid8 text | character text | character varying timestamp without time zone | timestamp with time zone bit | bit varying txid_snapshot | pg_snapshot -(6 rows) + pg_encrypted_det | pg_encrypted_rnd +(8 rows) SELECT DISTINCT p1.proargtypes[1]::regtype, p2.proargtypes[1]::regtype FROM pg_proc AS p1, pg_proc AS p2 @@ -197,12 +200,13 @@ WHERE p1.oid != p2.oid AND ORDER BY 1, 2; proargtypes | proargtypes -----------------------------+-------------------------- + bytea | pg_encrypted_det integer | xid timestamp without time zone | timestamp with time zone bit | bit varying txid_snapshot | pg_snapshot anyrange | anymultirange -(5 rows) +(6 rows) SELECT DISTINCT p1.proargtypes[2]::regtype, p2.proargtypes[2]::regtype FROM pg_proc AS p1, pg_proc AS p2 @@ -872,6 +876,8 @@ xid8ge(xid8,xid8) xid8eq(xid8,xid8) xid8ne(xid8,xid8) xid8cmp(xid8,xid8) +pg_encrypted_det_eq(pg_encrypted_det,pg_encrypted_det) +pg_encrypted_det_ne(pg_encrypted_det,pg_encrypted_det) uuid_extract_timestamp(uuid) uuid_extract_version(uuid) -- restore normal output mode diff --git a/src/test/regress/expected/type_sanity.out b/src/test/regress/expected/type_sanity.out index 88d8f6c32d6..faa5d79e42d 100644 --- a/src/test/regress/expected/type_sanity.out +++ b/src/test/regress/expected/type_sanity.out @@ -75,7 +75,9 @@ ORDER BY t1.oid; 4600 | pg_brin_bloom_summary 4601 | pg_brin_minmax_multi_summary 5017 | pg_mcv_list -(6 rows) + 8243 | pg_encrypted_det + 8244 | pg_encrypted_rnd +(8 rows) -- Make sure typarray points to a "true" array type of our own base SELECT t1.oid, t1.typname as basetype, t2.typname as arraytype, @@ -718,6 +720,8 @@ SELECT oid, typname, typtype, typelem, typarray WHERE oid < 16384 AND -- Exclude pseudotypes and composite types. typtype NOT IN ('p', 'c') AND + -- Exclude encryption internal types. + oid != ALL(ARRAY['pg_encrypted_det', 'pg_encrypted_rnd']::regtype[]) AND -- These reg* types cannot be pg_upgraded, so discard them. oid != ALL(ARRAY['regproc', 'regprocedure', 'regoper', 'regoperator', 'regconfig', 'regdictionary', diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index 675c5676171..eb6a0c6ac48 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -98,7 +98,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass column_encryption # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c index 8aeed97be1a..a3dba8109b4 100644 --- a/src/test/regress/pg_regress_main.c +++ b/src/test/regress/pg_regress_main.c @@ -76,7 +76,7 @@ psql_start_test(const char *testname, bindir ? bindir : "", bindir ? "/" : "", dblist->str, - "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on", + "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on -v HIDE_COLUMN_ENCRYPTION=on", infile, outfile); diff --git a/src/test/regress/sql/column_encryption.sql b/src/test/regress/sql/column_encryption.sql new file mode 100644 index 00000000000..0c5f2af2da5 --- /dev/null +++ b/src/test/regress/sql/column_encryption.sql @@ -0,0 +1,371 @@ +\set HIDE_COLUMN_ENCRYPTION false + +CREATE ROLE regress_enc_user1; + +CREATE COLUMN MASTER KEY fail WITH ( + foo = bar +); + +CREATE COLUMN MASTER KEY fail WITH ( + realm = 'test', + realm = 'test' +); + +CREATE COLUMN MASTER KEY cmk1 WITH ( + realm = 'test' +); + +COMMENT ON COLUMN MASTER KEY cmk1 IS 'column master key'; + +-- duplicate +CREATE COLUMN MASTER KEY cmk1 WITH ( + realm = 'test' +); + +CREATE COLUMN MASTER KEY cmk1a WITH ( + realm = 'test' +); + +CREATE COLUMN MASTER KEY cmk2; + +CREATE COLUMN MASTER KEY cmk2a WITH ( + realm = 'testx' +); + +ALTER COLUMN MASTER KEY cmk2a (realm = 'test2'); + +-- fail +ALTER COLUMN MASTER KEY cmk2a (realm = 'test2', realm = 'test2'); + +-- fail +ALTER COLUMN MASTER KEY cmk2a (foo = bar); + +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + foo = bar +); + +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + column_master_key = cmk1 +); + +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + encrypted_value = '\xDEADBEEF' +); + +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1' +); + +CREATE COLUMN ENCRYPTION KEY fail WITH VALUES ( + column_master_key = cmk1, + algorithm = 'foo', -- invalid + encrypted_value = '\xDEADBEEF' +); + +CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +COMMENT ON COLUMN ENCRYPTION KEY cek1 IS 'column encryption key'; + +-- duplicate +CREATE COLUMN ENCRYPTION KEY cek1 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +ALTER COLUMN ENCRYPTION KEY cek1 ADD VALUE ( + column_master_key = cmk1a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +-- duplicate +ALTER COLUMN ENCRYPTION KEY cek1 ADD VALUE ( + column_master_key = cmk1a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +ALTER COLUMN ENCRYPTION KEY fail ADD VALUE ( + column_master_key = cmk1a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +CREATE COLUMN ENCRYPTION KEY cek2 WITH VALUES ( + column_master_key = cmk2, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +), +( + column_master_key = cmk2a, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +CREATE COLUMN ENCRYPTION KEY cek4 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); + +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (foo = bar) +); + +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (encryption_type = randomized) +); + +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = notexist) +); + +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1, algorithm = 'foo') +); + +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = wrong) +); + +CREATE TABLE tbl_fail ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1, column_encryption_key = cek1) +); + +CREATE TABLE tbl_29f3 ( + a int, + b text, + c text ENCRYPTED WITH (column_encryption_key = cek1) +); + +\d tbl_29f3 +\d+ tbl_29f3 + +CREATE TABLE tbl_447f ( + a int, + b text +); + +ALTER TABLE tbl_447f ADD COLUMN c text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = deterministic); + +\d tbl_447f +\d+ tbl_447f + +CREATE TABLE tbl_4897 (LIKE tbl_447f); +CREATE TABLE tbl_6978 (LIKE tbl_447f INCLUDING ENCRYPTED); + +\d+ tbl_4897 +\d+ tbl_6978 + +CREATE VIEW view_3bc9 AS SELECT * FROM tbl_29f3; + +\d+ view_3bc9 + +CREATE TABLE tbl_2386 AS SELECT * FROM tbl_29f3 WITH NO DATA; + +\d+ tbl_2386 + +CREATE TABLE tbl_2941 AS SELECT * FROM tbl_29f3 WITH DATA; + +\d+ tbl_2941 + +-- test partition declarations + +CREATE TABLE tbl_13fa ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +) PARTITION BY RANGE (a); +CREATE TABLE tbl_13fa_1 PARTITION OF tbl_13fa FOR VALUES FROM (1) TO (100); + +\d+ tbl_13fa +\d+ tbl_13fa_1 + + +-- test inheritance + +CREATE TABLE tbl_36f3_a ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +); + +CREATE TABLE tbl_36f3_b ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1) +); + +CREATE TABLE tbl_36f3_c ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek2) +); + +CREATE TABLE tbl_36f3_d ( + a int, + b text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = deterministic) +); + +CREATE TABLE tbl_36f3_e ( + a int, + b text +); + +-- not implemented (but could be ok) +CREATE TABLE tbl_36f3_ab (c int) INHERITS (tbl_36f3_a, tbl_36f3_b); +\d+ tbl_36f3_ab +-- not implemented (but should fail) +CREATE TABLE tbl_36f3_ac (c int) INHERITS (tbl_36f3_a, tbl_36f3_c); +CREATE TABLE tbl_36f3_ad (c int) INHERITS (tbl_36f3_a, tbl_36f3_d); +-- fail +CREATE TABLE tbl_36f3_ae (c int) INHERITS (tbl_36f3_a, tbl_36f3_e); + +-- ok +CREATE TABLE tbl_36f3_a_1 (b text ENCRYPTED WITH (column_encryption_key = cek1), c int) INHERITS (tbl_36f3_a); +\d+ tbl_36f3_a_1 +-- fail +CREATE TABLE tbl_36f3_a_2 (b text ENCRYPTED WITH (column_encryption_key = cek2), c int) INHERITS (tbl_36f3_a); +CREATE TABLE tbl_36f3_a_2 (b text ENCRYPTED WITH (column_encryption_key = cek1, encryption_type = deterministic), c int) INHERITS (tbl_36f3_a); +CREATE TABLE tbl_36f3_a_2 (b text, c int) INHERITS (tbl_36f3_a); + +DROP TABLE tbl_36f3_b, tbl_36f3_c, tbl_36f3_d, tbl_36f3_e; + + +-- SET SCHEMA +CREATE SCHEMA test_schema_ce; +ALTER COLUMN ENCRYPTION KEY cek1 SET SCHEMA test_schema_ce; +ALTER COLUMN MASTER KEY cmk1 SET SCHEMA test_schema_ce; +ALTER COLUMN ENCRYPTION KEY test_schema_ce.cek1 SET SCHEMA public; +ALTER COLUMN MASTER KEY test_schema_ce.cmk1 SET SCHEMA public; +DROP SCHEMA test_schema_ce; + + +-- privileges +SET SESSION AUTHORIZATION 'regress_enc_user1'; +-- fail +CREATE COLUMN ENCRYPTION KEY cek10 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +RESET SESSION AUTHORIZATION; +GRANT USAGE ON COLUMN MASTER KEY cmk1 TO regress_enc_user1; +SET SESSION AUTHORIZATION 'regress_enc_user1'; +-- ok now +CREATE COLUMN ENCRYPTION KEY cek10 WITH VALUES ( + column_master_key = cmk1, + algorithm = 'RSAES_OAEP_SHA_1', + encrypted_value = '\xDEADBEEF' +); +DROP COLUMN ENCRYPTION KEY cek10; + +-- fail +CREATE TABLE tbl_7040 (a int, b text ENCRYPTED WITH (column_encryption_key = cek1)); +CREATE TABLE tbl_7040 (a int); +-- fail +ALTER TABLE tbl_7040 ADD COLUMN b text ENCRYPTED WITH (column_encryption_key = cek1); +DROP TABLE tbl_7040; +RESET SESSION AUTHORIZATION; +GRANT USAGE ON COLUMN ENCRYPTION KEY cek1 TO regress_enc_user1; +SET SESSION AUTHORIZATION 'regress_enc_user1'; +-- ok now +CREATE TABLE tbl_7040 (a int, b text ENCRYPTED WITH (column_encryption_key = cek1)); +RESET SESSION AUTHORIZATION; + +-- has_column_encryption_key_privilege +SELECT has_column_encryption_key_privilege('regress_enc_user1', + (SELECT oid FROM pg_colenckey WHERE cekname='cek1'), 'USAGE'); +SELECT has_column_encryption_key_privilege('regress_enc_user1', 'cek1', 'USAGE'); +SELECT has_column_encryption_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), + (SELECT oid FROM pg_colenckey WHERE cekname='cek1'), 'USAGE'); +SELECT has_column_encryption_key_privilege( + (SELECT oid FROM pg_colenckey WHERE cekname='cek1'), 'USAGE'); +SELECT has_column_encryption_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), 'cek1', 'USAGE'); +SELECT has_column_encryption_key_privilege('cek1', 'USAGE'); +SELECT has_column_encryption_key_privilege('regress_enc_user1', 'cek1', 'USAGE'); + +-- has_column_master_key_privilege +SELECT has_column_master_key_privilege('regress_enc_user1', + (SELECT oid FROM pg_colmasterkey WHERE cmkname='cmk1'), 'USAGE'); +SELECT has_column_master_key_privilege('regress_enc_user1', 'cmk1', 'USAGE'); +SELECT has_column_master_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), + (SELECT oid FROM pg_colmasterkey WHERE cmkname='cmk1'), 'USAGE'); +SELECT has_column_master_key_privilege( + (SELECT oid FROM pg_colmasterkey WHERE cmkname='cmk1'), 'USAGE'); +SELECT has_column_master_key_privilege( + (SELECT oid FROM pg_roles WHERE rolname='regress_enc_user1'), 'cmk1', 'USAGE'); +SELECT has_column_master_key_privilege('cmk1', 'USAGE'); +SELECT has_column_master_key_privilege('regress_enc_user1', 'cmk1', 'USAGE'); + +DROP TABLE tbl_7040; +REVOKE USAGE ON COLUMN ENCRYPTION KEY cek1 FROM regress_enc_user1; +REVOKE USAGE ON COLUMN MASTER KEY cmk1 FROM regress_enc_user1; + + +-- not useful here, just checking that it runs +DISCARD COLUMN ENCRYPTION KEYS; + + +DROP COLUMN MASTER KEY cmk1 RESTRICT; -- fail + +ALTER COLUMN MASTER KEY cmk2 RENAME TO cmk3; +ALTER COLUMN MASTER KEY cmk1 RENAME TO cmk3; -- fail +ALTER COLUMN MASTER KEY cmkx RENAME TO cmky; -- fail + +ALTER COLUMN ENCRYPTION KEY cek2 RENAME TO cek3; +ALTER COLUMN ENCRYPTION KEY cek1 RENAME TO cek3; -- fail +ALTER COLUMN ENCRYPTION KEY cekx RENAME TO ceky; -- fail + +SET SESSION AUTHORIZATION 'regress_enc_user1'; +DROP COLUMN ENCRYPTION KEY cek3; -- fail +DROP COLUMN MASTER KEY cmk3; -- fail +RESET SESSION AUTHORIZATION; +ALTER COLUMN MASTER KEY cmk3 OWNER TO regress_enc_user1; +ALTER COLUMN ENCRYPTION KEY cek3 OWNER TO regress_enc_user1; +\dcek cek3 +\dcmk cmk3 +SET SESSION AUTHORIZATION 'regress_enc_user1'; +DROP COLUMN ENCRYPTION KEY cek3; -- ok now +DROP COLUMN MASTER KEY cmk3; -- ok now +RESET SESSION AUTHORIZATION; + +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a); +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a); -- fail +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = fail); -- fail +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a, algorithm = 'foo'); -- fail +ALTER COLUMN ENCRYPTION KEY cek1 DROP VALUE (column_master_key = cmk1a, encrypted_value = '\xDEADBEEF'); -- fail + +DROP COLUMN ENCRYPTION KEY cek4; +DROP COLUMN ENCRYPTION KEY fail; +DROP COLUMN ENCRYPTION KEY IF EXISTS nonexistent; + +DROP COLUMN MASTER KEY cmk1a; +DROP COLUMN MASTER KEY fail; +DROP COLUMN MASTER KEY IF EXISTS nonexistent; + +DROP ROLE regress_enc_user1; diff --git a/src/test/regress/sql/object_address.sql b/src/test/regress/sql/object_address.sql index 1a6c61f49d5..61828613d9a 100644 --- a/src/test/regress/sql/object_address.sql +++ b/src/test/regress/sql/object_address.sql @@ -41,6 +41,8 @@ CREATE SERVER "integer" FOREIGN DATA WRAPPER addr_fdw; CREATE USER MAPPING FOR regress_addr_user SERVER "integer"; ALTER DEFAULT PRIVILEGES FOR ROLE regress_addr_user IN SCHEMA public GRANT ALL ON TABLES TO regress_addr_user; ALTER DEFAULT PRIVILEGES FOR ROLE regress_addr_user REVOKE DELETE ON TABLES FROM regress_addr_user; +CREATE COLUMN MASTER KEY addr_cmk; +CREATE COLUMN ENCRYPTION KEY addr_cek WITH VALUES (column_master_key = addr_cmk, algorithm = 'RSAES_OAEP_SHA_1', encrypted_value = ''); -- this transform would be quite unsafe to leave lying around, -- except that the SQL language pays no attention to transforms: CREATE TRANSFORM FOR int LANGUAGE SQL ( @@ -93,6 +95,7 @@ CREATE STATISTICS addr_nsp.gentable_stat ON a, b FROM addr_nsp.gentable; ('materialized view'), ('foreign table'), ('table column'), ('foreign table column'), ('aggregate'), ('function'), ('procedure'), ('type'), ('cast'), + ('column encryption key'), ('column encryption key data'), ('column master key'), ('table constraint'), ('domain constraint'), ('conversion'), ('default value'), ('operator'), ('operator class'), ('operator family'), ('rule'), ('trigger'), ('text search parser'), ('text search dictionary'), @@ -174,6 +177,9 @@ CREATE STATISTICS addr_nsp.gentable_stat ON a, b FROM addr_nsp.gentable; ('type', '{addr_nsp.genenum}', '{}'), ('cast', '{int8}', '{int4}'), ('collation', '{default}', '{}'), + ('column encryption key', '{addr_cek}', '{}'), + ('column encryption key data', '{addr_cek}', '{addr_cmk}'), + ('column master key', '{addr_cmk}', '{}'), ('table constraint', '{addr_nsp, gentable, a_chk}', '{}'), ('domain constraint', '{addr_nsp.gendomain}', '{domconstr}'), ('conversion', '{pg_catalog, koi8_r_to_mic}', '{}'), @@ -228,6 +234,8 @@ CREATE STATISTICS addr_nsp.gentable_stat ON a, b FROM addr_nsp.gentable; DROP PUBLICATION addr_pub; DROP PUBLICATION addr_pub_schema; DROP SUBSCRIPTION regress_addr_sub; +DROP COLUMN ENCRYPTION KEY addr_cek; +DROP COLUMN MASTER KEY addr_cmk; DROP SCHEMA addr_nsp CASCADE; @@ -247,6 +255,9 @@ CREATE STATISTICS addr_nsp.gentable_stat ON a, b FROM addr_nsp.gentable; ('pg_type'::regclass, 0, 0), -- no type ('pg_cast'::regclass, 0, 0), -- no cast ('pg_collation'::regclass, 0, 0), -- no collation + ('pg_colenckey'::regclass, 0, 0), -- no column encryption key + ('pg_colenckeydata'::regclass, 0, 0), -- no column encryption key data + ('pg_colmasterkey'::regclass, 0, 0), -- no column master key ('pg_constraint'::regclass, 0, 0), -- no constraint ('pg_conversion'::regclass, 0, 0), -- no conversion ('pg_attrdef'::regclass, 0, 0), -- no default attribute diff --git a/src/test/regress/sql/type_sanity.sql b/src/test/regress/sql/type_sanity.sql index e88d6cbe49d..498ca654c07 100644 --- a/src/test/regress/sql/type_sanity.sql +++ b/src/test/regress/sql/type_sanity.sql @@ -546,6 +546,8 @@ CREATE TABLE tab_core_types AS SELECT WHERE oid < 16384 AND -- Exclude pseudotypes and composite types. typtype NOT IN ('p', 'c') AND + -- Exclude encryption internal types. + oid != ALL(ARRAY['pg_encrypted_det', 'pg_encrypted_rnd']::regtype[]) AND -- These reg* types cannot be pg_upgraded, so discard them. oid != ALL(ARRAY['regproc', 'regprocedure', 'regoper', 'regoperator', 'regconfig', 'regdictionary', diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index c83417ce9d2..c6e9e025023 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -65,6 +65,8 @@ AllocSetFreeList AllocateDesc AllocateDescKind AlterCollationStmt +AlterColumnEncryptionKeyStmt +AlterColumnMasterKeyStmt AlterDatabaseRefreshCollStmt AlterDatabaseSetStmt AlterDatabaseStmt @@ -378,6 +380,7 @@ CatCacheHeader CatalogId CatalogIdMapEntry CatalogIndexState +CekInfo ChangeVarNodes_context ReplaceVarnoContext CheckPoint @@ -403,6 +406,7 @@ ClusterInfo ClusterParams ClusterStmt CmdType +CmkInfo CoalesceExpr CoerceParamHook CoerceToDomain @@ -813,6 +817,9 @@ FormData_pg_authid FormData_pg_cast FormData_pg_class FormData_pg_collation +FormData_pg_colenckey +FormData_pg_colenckeydata +FormData_pg_colmasterkey FormData_pg_constraint FormData_pg_conversion FormData_pg_database @@ -1765,6 +1772,8 @@ PGAlignedBlock PGAlignedXLogBlock PGAsyncStatusType PGCALL2 +PGCEK +PGCMK PGChecksummablePage PGContextVisibility PGEvent base-commit: 5105c90796811f62711538155d207e5311eacf9b -- 2.44.0