Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

On Thu, Jun 13, 2019 at 11:07:25AM -0400, Bruce Momjian wrote:
>On Thu, Jun 13, 2019 at 04:26:47PM +0900, Masahiko Sawada wrote:
>> On Thu, Jun 13, 2019 at 3:48 AM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > The big question is how many people will be mixing encrypted and
>> > unencrypted data in the same cluster, and care about performance? Just
>> > because someone might care is not enough of a justification. They can
>> > certainly create separate encrypted and non-encrypted clusters. Can we
>> > implement level 6 and then implement levels 3-5 later if desired?
>>
>> I guess most users are interested in performance. Users don't want to
>> sacrifice performance for security and vice versa. Fine grained
>> control would allow us to seek a compromise point.
>
>Well, what does that add to the argument? Yes, everyone cares about
>performance, but it is the magnitude of the performance impact vs. the
>complexity that is the issue here. Also, by definition, users will
>trade performance for security because encrypting data will slow down
>the database. The open question is how much, and if that overhead is
>reasonable based on the complexity.
>
>What I don't want to do is to design a system that is more complex than
>required, and it might become so complex we might never get it done.
>

IMHO we should implement the simplest system possible, and optimize the
hell out of it without sacrificing any safety/security aspects. No smart
tunables, no extra GUCs to trade security for performance, nothing.

Then once we have this working, we can see what the impact is, and make
informed choices based on that. It's really hard to make good choices
based on speculation, which is all we have at this point. And the danger
is we'll end up with overly complex system with many parameters - which
is pretty bad when the configuration impacts security, because regular
users may not reaslise the consequences (and we'll get blamed for it).

Also, in my experience the deployments that really need this sort of
encryption tend to be quite valuable, and the owners will be happy with
higher hardware costs to compensate for the performance impact, if it
gives them the feature. So even if the performance impact is 20% (worst
case estimate), I'd say that may be acceptable.

>> > How would you configure the WAL to know which key to use if we did #5?
>> > Wouldn't system tables and statistics, and perhaps referential integry
>> > allow for information leakage?
>>
>> We use a something like a map between tablespace oid and encryption
>> key as a separate file (maybe stored in $PGDATA/global), called
>> keyring. Using the keyring we can obtain encryption key by tablespace
>> oid. For WAL, we add a flag to XLogRecord which indicates whether the
>> WAL record is encrypted, and we already have relfilenode in the header
>> data of WAL. So we can obtain the tablespace oid from the part and
>> obtain the corresponding encryption key.
>
>OK.
>
>> > > 2. Encryption Objects.
>> > > Indexes, WAL and TOAST table pertaining to encrypted tables, and
>> > > temporary files must also be encrypted but we need to discuss whether
>> > > we encrypt non-user data as well such as SLRU data, vm and fsm, and
>> > > perhaps even other files such as 2PC state files, backend_label etc.
>> > > Encryption everything is required by some use case but it's also true
>> > > that there are users who wish to encrypt database while minimizing
>> > > performance overheads.
>> >
>> > I don't think we need to encrypt the "status" files like SLRU data, vm
>> > and fsm.
>>
>> I agree.
>
>Good.
>
>> > Good point about pg_waldump. I am a little worried we might open a
>> > security hole making a new API so they work, so maybe we should avoid
>> > it.
>>
>> Yeah, in principle since data key of 2 tier key architecture should
>> not go outside database I think we should not tell data keys to
>> utility commands. So the rearranging WAL format seems to be a better
>> solution but is there any reason why the main data is placed at end of
>> WAL record? I wonder if we can assemble WAL records as following order
>> and encrypt only 3 and 4.
>>
>> 1. Header data (XLogRecord and other headers)
>> 2. Main data (xl_heap_insert, xl_heap_update etc + related data)
>> 3. Block data (Tuple data, FPI)
>> 4. Sub data (e.g tuple data for logical decoding)
>
>Yes, that does sound like a reasonable idea. It is similar to us not
>encrypting the clog --- there is little value. However, if we only
>encrypt the cluster, we don't need to expose the relfilenode and we can
>just encrypt the entire WAL --- I like that simplicity. We might find
>that the complexity of encrypting only certain tablespaces makes the
>system slower than just encrypting the entire cluster.
>

I personally find the idea of encrypting tablespaces rather strange.
Tablespaces are meant to define hysical location for objects, but this
would also use them to "mark" objects as encrypted or not. That just
seems misguided and would make the life harder for many users.

For example, what if I don't have any tablespaces (except for the
default one), but I want to encrypt only some objects? Suddenly I have
to create a tablespace, which will however cause various difficulties
down the road (during pg_basebackup, etc.).

If we really want to allow encrypting of just some objects (and I'm not
saying we need to), we should either allow defining that for individual
objects, or invent some new logical grouping of objects (where each
group is encrypted or not as a whole).

FWIW my impression is that we only really consider tablespace encryption
because Oracle has it, so users may be familiar with it. That however
ignores that Oracle actually allocates data for a tablespace (IIRC there
is a set of files per tablespace, shared by all objects in it), while
PostgreSQL has files per object.

>> > > Also, for system catalog encryption, it could be a hard part. System
>> > > catalogs are initially created at initdb time and created by copying
>> > > from template1 when CREATE DATABASE. Therefore we would need to either
>> > > modify initdb so that it's aware of encryption keys and KMS or modify
>> > > database creation so that it copies database file while encrypting
>> > > them.
>> >
>> > I assume initdb will use the same API that you would use to start the
>> > server itself, e.g., type in a password, or contact a key server.
>>
>> I realized that in XTS encryption mode since we craft the tweak using
>> relfilenode we will need to have the different tweaks for system
>> catalogs in new database would change. So we might need to re-encrypt
>> system catalogs when CREATE DATABASE after all. I suspect that even
>> the cluster-wide encryption has the same problem.
>
>Yes, this is why I want to just do cluster-wide encryption at this
>stage.
>
>In addition, while the 8k blocks would use a block cipher, the WAL would
>likely use a stream cipher, and it will be very hard to use multiple
>stream ciphers in a single WAL file.
>

Umm, why? Why would WAL necessarily use stream cipher instead of a block
cipher with a suitable mode (say, CBC or XTS)? And even if it did use
some stream cipter, why would it be hard to use multiple ciphers?

I mean, we'd probably want/need to start new streams for each WAL
segment anyway, so that tools can read and process each WAL segment
independently. So we wouldn't get very long streams anyway.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services