diff --git a/contrib/pg_tde/documentation/docs/advanced-topics/index.md b/contrib/pg_tde/documentation/docs/advanced-topics/index.md new file mode 100644 index 0000000000000..9df169821bcd3 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/advanced-topics/index.md @@ -0,0 +1,7 @@ +# Technical Reference + +This section covers the internal components and tools that power `pg_tde`. + +Use it to understand how encryption is implemented, fine-tune a configuration, leverage advanced CLI tools and functions for diagnostics and customization. + +[Architecture](../architecture/index.md){.md-button} [GUC Variables](../variables.md){.md-button} [Functions](../functions.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/apt.md b/contrib/pg_tde/documentation/docs/apt.md index 2ecd9eb232f85..0b2414c23569b 100644 --- a/contrib/pg_tde/documentation/docs/apt.md +++ b/contrib/pg_tde/documentation/docs/apt.md @@ -1,55 +1,54 @@ -# Install `pg_tde` on Debian or Ubuntu +# Install pg_tde on Debian or Ubuntu -This tutorial shows how to install `pg_tde` with [Percona Distribution for PostgreSQL :octicons-link-external-16:](https://docs.percona.com/postgresql/latest/index.html). +This page explains how to install `pg_tde` with [Percona Distribution for PostgreSQL :octicons-link-external-16:](https://docs.percona.com/postgresql/latest/index.html). -Check the [list of supported platforms](install.md#__tabbed_1_1). +Check the [list of supported platforms](install.md#__tabbed_1_1) before continuing. ## Preconditions 1. Debian and other systems that use the `apt` package manager include the upstream PostgreSQL server package (`postgresql-{{pgversion17}}`) by default. You need to uninstall this package before you install Percona Server for PostgreSQL and `pg_tde` to avoid conflicts. 2. You need the `percona-release` repository management tool that enables the desired Percona repository for you. - -### Install `percona-release` {.power-number} +### Install percona-release {.power-number} 1. You need the following dependencies to install `percona-release`: - + - `wget` - `gnupg2` - `curl` - `lsb-release` - + Install them with the following command: - + ```{.bash data-prompt="$"} - $ sudo apt-get install -y wget gnupg2 curl lsb-release + $ sudo apt-get install -y wget gnupg2 curl lsb-release ``` - + 2. Fetch the `percona-release` package ```{.bash data-prompt="$"} - $ sudo wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb + $ sudo wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb ``` 3. Install `percona-release` ```{.bash data-prompt="$"} - $ sudo dpkg -i percona-release_latest.generic_all.deb + $ sudo dpkg -i percona-release_latest.generic_all.deb ``` 4. Enable the Percona Distribution for PostgreSQL repository ```{.bash data-prompt="$"} - $ sudo percona-release enable-only ppg-{{pgversion17}} + $ sudo percona-release enable-only ppg-{{pgversion17}} ``` 6. Update the local cache ```bash - sudo apt-get update + sudo apt-get update ``` -## Install `pg_tde` {.power-number} +## Install pg_tde {.power-number} !!! important @@ -60,19 +59,17 @@ Check the [list of supported platforms](install.md#__tabbed_1_1). The use of the `CASCADE` parameter deletes all tables that were created in the database with `pg_tde` enabled and also all dependencies upon the encrypted table (e.g. foreign keys in a non-encrypted table used in the encrypted one). ```sql - DROP EXTENSION pg_tde CASCADE + DROP EXTENSION pg_tde CASCADE; ``` 2. Uninstall the `percona-postgresql-17-pg-tde` package. After all [preconditions](#preconditions) are met, run the following command to install `pg_tde`: - ```{.bash data-prompt="$"} -$ sudo apt-get install -y percona-postgresql-17 + $ sudo apt-get install -y percona-postgresql-17 ``` +## Next steps -## Next step - -[Setup :material-arrow-right:](setup.md){.md-button} +[Configure pg_tde :material-arrow-right:](setup.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/architecture.md b/contrib/pg_tde/documentation/docs/architecture/index.md similarity index 95% rename from contrib/pg_tde/documentation/docs/architecture.md rename to contrib/pg_tde/documentation/docs/architecture/index.md index f7cd70e4805a9..d5ece3f2a8d17 100644 --- a/contrib/pg_tde/documentation/docs/architecture.md +++ b/contrib/pg_tde/documentation/docs/architecture/index.md @@ -12,7 +12,7 @@ Let's break down what it means. * Storing every key on the same key storage, or using different storages for different databases * Handling permissions: who can manage database specific or global permissions, who can create encrypted or not encrypted tables -**Complete** means that `pg_tde` aims to encrypt data at rest. +**Complete** means that `pg_tde` aims to encrypt data at rest. **Data at rest** means everything written to the disk. This includes the following: @@ -35,10 +35,10 @@ The main components of `pg_tde` are the following: [Percona Server for PostgreSQL location](https://github.com/percona/postgres/tree/{{tdebranch}}) * The **`pg_tde` extension itself** implements the encryption code by hooking into the extension points introduced in the core changes, and the already existing extension points in the PostgreSQL server. - + Everything is controllable with GUC variables and SQL statements, similar to other extensions. -* The **keyring API / libraries** implement the key storage logic with different key providers. The API is internal only, the keyring libraries are part of the main library for simplicity. +* The **keyring API / libraries** implement the key storage logic with different key providers. The API is internal only, the keyring libraries are part of the main library for simplicity. In the future these could be extracted into separate shared libraries with an open API, allowing the use of third-party providers. ## Encryption architecture @@ -48,7 +48,7 @@ In the future these could be extracted into separate shared libraries with an op `pg_tde` uses two kinds of keys for encryption: * Internal keys to encrypt the data. They are stored in PostgreSQL's data directory under `$PGDATA/pg_tde``. -* Higher-level keys to encrypt internal keys. These keys are called "principal keys". They are stored externally, in a Key Management System (KMS) using the key provider API. +* Higher-level keys to encrypt internal keys. These keys are called "principal keys". They are stored externally, in a Key Management System (KMS) using the key provider API. `pg_tde` uses one principal key per database. Every internal key for the given database is encrypted using this principal key. @@ -74,7 +74,7 @@ Support for other cipher lengths / algorithms is planned in the future. `pg_tde` makes it possible to encrypt everything or only some tables in some databases. -To support this without metadata changes, encrypted tables are labeled with a `tde_heap` access method marker. +To support this without metadata changes, encrypted tables are labeled with a `tde_heap` access method marker. The `tde_heap` access method is the same as the `heap` one. It uses the same functions internally without any changes, but with the different name and ID. In such a way `pg_tde` knows that `tde_heap` tables are encrypted and `heap` tables are not. @@ -162,11 +162,11 @@ Retreival of the principal key is cached so it only happens when necessary. ### Key provider management -Key provider configuration or location may change. For example, a service is moved to a new address or the principal key must be moved to a different key provider type. `pg_tde` supports both these scenarios enabling you to manage principal keys using simple [SQL functions](functions.md#key-provider-management). +Key provider configuration or location may change. For example, a service is moved to a new address or the principal key must be moved to a different key provider type. `pg_tde` supports both these scenarios enabling you to manage principal keys using simple [SQL functions](../functions.md#key-provider-management). -In certain cases you can't use SQL functions to manage key providers. For example, if the key provider changed while the server wasn't running and is therefore unaware of these changes. The startup can fail if it needs to access the encryption keys. +In certain cases you can't use SQL functions to manage key providers. For example, if the key provider changed while the server wasn't running and is therefore unaware of these changes. The startup can fail if it needs to access the encryption keys. -For such situations, `pg_tde` also provides [command line tools](command-line-tools.md) to recover the database. +For such situations, `pg_tde` also provides [command line tools](../command-line-tools/index.md) to recover the database. ### Sensitive key provider information @@ -242,7 +242,7 @@ This is also the reason why it requires a `dbOid` instead of a name, as it has n ### Deleting providers -Providers can be deleted by the +Providers can be deleted by using the ```sql pg_tde_delete_database_key_provider(provider_name) @@ -255,7 +255,7 @@ For database specific providers, the function first checks if the provider is us For global providers, the function checks if the provider is used anywhere, WAL or any specific database, and returns an error if it is. -This somewhat goes against the principle that `pg_tde` shouldn't interact with other databases than the one the user is connected to, but on the other hand, it only does this lookup in the internal `pg_tde` metadata, not in postgres catalogs, so it is a gray zone. Making this check makes more sense than potentially making some databases inaccessible. +This somewhat goes against the principle that `pg_tde` should not interact with other databases than the one the user is connected to, but on the other hand, it only does this lookup in the internal `pg_tde` metadata, not in postgres catalogs, so it is a gray zone. Making this check makes more sense than potentially making some databases inaccessible. ### Listing/querying providers @@ -365,7 +365,7 @@ encryption is managed by the admins, normal users only have to create tables wit 2. `CREATE EXTENSION pg_tde;` in `template1` 3. Adding a global key provider 4. Changing the WAL encryption to use the proper global key provider -5. Giving users that are expected to manage database keys permissions for database specific key management, but not database specific key provider management: +5. Giving users that are expected to manage database keys permissions for database specific key management, but not database specific key provider management: specific databases HAVE to use the global key provider Note: setting the `default_table_access_method` to `tde_heap` is possible, but instead of `ALTER SYSTEM` only per database using `ALTER DATABASE`, after a principal key is configured for that specific database. @@ -376,8 +376,8 @@ Alternatively `ALTER SYSTEM` is possible, but table creation in the database wil 1. Installing the extension: `shared_preload_libraries` + `pg_tde.wal_encrypt` (that's not multi tenant currently) 2. `CREATE EXTENSION pg_tde;` in any database -2. Adding a global key provider for WAL -3. Changing the WAL encryption to use the proper global key provider +3. Adding a global key provider for WAL +4. Changing the WAL encryption to use the proper global key provider No default configuration: key providers / principal keys are configured as a per database level, permissions are managed per database diff --git a/contrib/pg_tde/documentation/docs/command-line-tools.md b/contrib/pg_tde/documentation/docs/command-line-tools.md deleted file mode 100644 index a737505c4861c..0000000000000 --- a/contrib/pg_tde/documentation/docs/command-line-tools.md +++ /dev/null @@ -1,60 +0,0 @@ -# Command line tools - -The `pg_tde` extension provides new command line tools and modifies some existing tools to work with encrypted WAL and tables. - -## pg_tde_change_key_provider - -A tool for modifying the configuration of a key provider, possibly also changing its type. - -This tool edits the configuration files directly, ignoring permissions or running `postgres` processes. - -Its only intended use is to fix servers that can't start up because of inaccessible key providers. - -For example, you restore from an old backup and the address of the key provider changed in the meantime. You can use this tool to correct the configuration, allowing the server to start up. - -Use this tool **only when the server is offline.** To modify the key provider configuration when the server is up and running, use the [`pg_tde_change_(global/database)_key_provider_`](functions.md#change-an-existing-provider) SQL functions. - -### Usage - -To modify the key provider configuration, specify all parameters depending on the provider type in the same way as you do when using the [`pg_tde_change_(global/database)_key_provider_`](functions.md#change-an-existing-provider) SQL functions. - -The general syntax is as follows: - -``` -pg_tde_change_key_provider [-D ] -``` - -where : - -* [optional] `` is the data directory. When not specified, `pg_tde` uses the `$PGDATA` value. -* `` is the name you assigned to the key provider -* `` can be a `file`, `vault` or `kmip`. -* `` - - -Depending on the provider type, the additional parameters are: - -``` -pg_tde_change_key_provider [-D ] file -pg_tde_change_key_provider [-D ] vault [] -pg_tde_change_key_provider [-D ] kmip [] -``` - -## pg_waldump - -[`pg_waldump` :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the write-ahead log of a PostgreSQL database cluster. - -To read encrypted WAL records, `pg_waldump` supports the following additional arguments: - -* `keyring_path`: the directory where keyring config files for WAL are stored. These files are: - - * `pg_tde.map`, - * `pg_tde.dat`, - * `pg_tde_keyrings` - -`pg_waldump` will not try to decrypt WAL if the `keyring_path` is not set. - -## pg_checksums - -[`pg_checksums` :octicons-link-external-16:](https://www.postgresql.org/docs/current/app-pgchecksums.html) cannot calculate checksums for encrypted files. -It skips encrypted files and reports this in the output. diff --git a/contrib/pg_tde/documentation/docs/command-line-tools/index.md b/contrib/pg_tde/documentation/docs/command-line-tools/index.md new file mode 100644 index 0000000000000..b854eab1b304a --- /dev/null +++ b/contrib/pg_tde/documentation/docs/command-line-tools/index.md @@ -0,0 +1,7 @@ +# pg_tde CLI Tools + +The `pg_tde` extension introduces new command-line utilities and extends some existing PostgreSQL tools to support encrypted WAL and tables. These include: + +* [pg_tde_change_key_provider](../command-line-tools/pg-tde-change-key-provider.md): change encryption key provider for a database +* [pg_waldump](../command-line-tools/pg-waldump.md): inspect and decrypt WAL files +* [pg_checksums](../command-line-tools/pg-tde-checksums.md): verify data checksums (non-encrypted files only) diff --git a/contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-change-key-provider.md b/contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-change-key-provider.md new file mode 100644 index 0000000000000..a897634c41c37 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-change-key-provider.md @@ -0,0 +1,36 @@ +# pg_tde_change_key_provider + +A tool for modifying the configuration of a key provider, possibly also changing its type. + +This tool edits the configuration files directly, ignoring permissions or running `postgres` processes. + +Its only intended use is to fix servers that can't start up because of inaccessible key providers. + +For example, you restore from an old backup and the address of the key provider changed in the meantime. You can use this tool to correct the configuration, allowing the server to start up. + +:material-information: Warning: Use this tool **only when the server is offline.** To modify the key provider configuration when the server is up and running, use the [`pg_tde_change_(global/database)_key_provider_`](../functions.md#change-an-existing-provider) SQL functions. + +## Example usage + +To modify the key provider configuration, specify all parameters depending on the provider type in the same way as you do when using the [`pg_tde_change_(global/database)_key_provider_`](../functions.md#change-an-existing-provider) SQL functions. + +The general syntax is as follows: + +```bash +pg_tde_change_key_provider [-D ] +``` + +## Parameter description + +* [optional] `` is the data directory.`pg_tde` uses the `$PGDATA` environment variable if this is not specified +* `` is the name you assigned to the key provider +* `` can be a `file`, `vault` or `kmip` +* `` + +Depending on the provider type, the additional parameters are: + +```bash +pg_tde_change_key_provider [-D ] file +pg_tde_change_key_provider [-D ] vault [] +pg_tde_change_key_provider [-D ] kmip [] +``` diff --git a/contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-checksums.md b/contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-checksums.md new file mode 100644 index 0000000000000..391182dcd59c6 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/command-line-tools/pg-tde-checksums.md @@ -0,0 +1,5 @@ +# pg_checksums + +[`pg_checksums` :octicons-link-external-16:](https://www.postgresql.org/docs/current/app-pgchecksums.html) is a PostgreSQL command-line utility used to enable, disable, or verify data checksums on a PostgreSQL data directory. However, it cannot calculate checksums for encrypted files. + +Encrypted files are skipped, and this is reported in the output. diff --git a/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md b/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md new file mode 100644 index 0000000000000..6396249342840 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md @@ -0,0 +1,14 @@ +# pg_waldump + +[`pg_waldump` :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the Write-Ahead Log (WAL) of a PostgreSQL database cluster. + +To read encrypted WAL records, `pg_waldump` supports the following additional arguments: + +* `keyring_path`: the directory where keyring configuration files for WAL are stored. These files include: + * `pg_tde.map` + * `pg_tde.dat` + * `pg_tde_keyrings` + +!!! note + + `pg_waldump` will not decrypt WAL unless the `keyring_path` is set. diff --git a/contrib/pg_tde/documentation/docs/contribute.md b/contrib/pg_tde/documentation/docs/contribute.md index c728f333f49d0..e626488ea1599 100644 --- a/contrib/pg_tde/documentation/docs/contribute.md +++ b/contrib/pg_tde/documentation/docs/contribute.md @@ -1,4 +1,4 @@ -# Contributing guide +# Contributing Guide Welcome to `pg_tde` - the Transparent Data Encryption extension for PostgreSQL! diff --git a/contrib/pg_tde/documentation/docs/faq.md b/contrib/pg_tde/documentation/docs/faq.md index a4842bdd306cd..902e0a18b49fc 100644 --- a/contrib/pg_tde/documentation/docs/faq.md +++ b/contrib/pg_tde/documentation/docs/faq.md @@ -2,16 +2,18 @@ ## Why do I need TDE? +Using TDE provides the following benefits: + - Compliance to security and legal regulations like General Data Protection Regulation (GDPR), Payment Card Industry Data Security Standard (PCI DSS), California Consumer Privacy Act (CCPA), Data Protection Act 2018 (DPA 2018) and others - Encryption of backups. Even when an authorized person gets physical access to a backup, encryption ensures that the data remains unreadable and secure. -- Granular encryption of specific data sets and reducing the performance overhead that encryption brings. +- Granular encryption of specific data sets and reducing the performance overhead that encryption brings. - Additional layer of security to existing security measures ## When and how should I use TDE? If you are dealing with Personally Identifiable Information (PII), data encryption is crucial. Especially if you are involved in areas with strict regulations like: -* financial services where TDE helps to comply with PCI DSS +* financial services where TDE helps to comply with PCI DSS * healthcare and insurance - compliance with HIPAA, HITECH, CCPA * telecommunications, government and education to ensure data confidentiality. @@ -21,18 +23,17 @@ Using TDE helps you avoid the following risks: * Identity theft that may lead to financial fraud and other crimes * Reputation damage leading to loss of customer trust and business * Legal consequences and financial losses for non-compliance with data protection regulations -* Internal threats by misusing unencrypted sensitive data +* Internal threats by misusing unencrypted sensitive data If to translate sensitive data to files stored in your database, these are user data in tables, temporary files, WAL files. TDE has you covered encrypting all these files. -`pg_tde` does not encrypt system catalogs yet. This means that statistics data and database metadata are not encrypted. The encryption of system catalogs is planned for future releases. - +`pg_tde` does not encrypt system catalogs yet. This means that statistics data and database metadata are not encrypted. The encryption of system catalogs is planned for future releases. ## I use disk-level encryption. Why should I care about TDE? Encrypting a hard drive encrypts all data, including system, application, and temporary files. -Full disk encryption protects your data from people who have physical access to your device and even if it is lost or stolen. However, it doesn't protect the data after system boot-up: the data is automatically decrypted when the system runs or when an authorized user requests it. +Full disk encryption protects your data from people who have physical access to your device and even if it is lost or stolen. However, it doesn't protect the data after system boot-up: the data is automatically decrypted when the system runs or when an authorized user requests it. Another point to consider is PCI DSS compliance for Personal Account Numbers (PAN) encryption. @@ -44,7 +45,7 @@ Another point to consider is PCI DSS compliance for Personal Account Numbers (PA Note that PCI DSS 3.4.1 is retiring on March 31, 2025. Therefore, consider switching to PCI DSS 4.0. -* **PCI DSS 4.0** standards consider using only disk and partition-level encryption not enough to ensure PAN protection. It requires an additional layer of security that `pg_tde` can provide. +* **PCI DSS 4.0** standards consider using only disk and partition-level encryption not enough to ensure PAN protection. It requires an additional layer of security that `pg_tde` can provide. `pg_tde` focuses specifically on data files and offers more granular control over encrypted data. The data remains encrypted on disk during runtime and when you move it to another directory, another system or storage. An example of such data is backups. They remain encrypted when moved to the backup storage. @@ -52,7 +53,7 @@ Thus, to protect your sensitive data, consider using TDE to encrypt it at the ta ## Is TDE enough to ensure data security? -No. TDE is an additional layer to ensure data security. It protects data at rest. Consider introducing also these measures: +**No.** Transparent Data Encryption (TDE) adds an extra layer of security for data at rest. You should also consider implementing the following additional security features: * Access control and authentication * Strong network security like TLS @@ -65,29 +66,29 @@ No. TDE is an additional layer to ensure data security. It protects data at rest `pg_tde` uses two keys to encrypt data: * Internal encryption keys to encrypt the data. These keys are stored internally in an encrypted format, in a single `$PGDATA/pg_tde` directory. -* Principal keys to encrypt internal encryption keys. These keys are stored externally, in the Key Management System (KMS). +* Principal keys to encrypt internal encryption keys. These keys are stored externally, in the Key Management System (KMS). You can use the following KMSs: -* [HashiCorp Vault](https://developer.hashicorp.com/vault/docs/what-is-vault). `pg_tde` supports the KV secrets engine v2 of Vault. +* [HashiCorp Vault](https://developer.hashicorp.com/vault/docs/what-is-vault). `pg_tde` supports the KV secrets engine v2 of Vault. * [OpenBao](https://openbao.org/) implementation of Vault * KMIP-compatible server. KMIP is a standardized protocol for handling cryptographic workloads and secrets management -HashiCorp Vault can also act as the KMIP server, managing cryptographic keys for clients that use the KMIP protocol. +HashiCorp Vault can also act as the KMIP server, managing cryptographic keys for clients that use the KMIP protocol. Let's break the encryption into two parts: -### Encryption of data files +### Encryption of data files -First, data files are encrypted with internal keys. Each file that has a different [Object Identifier (OID)](https://www.postgresql.org/docs/current/datatype-oid.html) has an internal key. For example, a table with 4 indexes will have 5 internal keys - one for the table and one for each index. +First, data files are encrypted with internal keys. Each file that has a different [Object Identifier (OID)](https://www.postgresql.org/docs/current/datatype-oid.html) has an internal key. For example, a table with 4 indexes will have 5 internal keys - one for the table and one for each index. -The initial decision on what file to encrypt is based on the table access method in PostgreSQL. When you run a `CREATE` or `ALTER TABLE` statement with the `USING tde_heap` clause, the newly created data files are marked as encrypted, and then file operations encrypt or decrypt the data. Later, if an initial file is re-created as a result of a `TRUNCATE` or `VACUUM FULL` command, the newly created file inherits the encryption information and is either encrypted or not. +The initial decision on what file to encrypt is based on the table access method in PostgreSQL. When you run a `CREATE` or `ALTER TABLE` statement with the `USING tde_heap` clause, the newly created data files are marked as encrypted, and then file operations encrypt or decrypt the data. Later, if an initial file is re-created as a result of a `TRUNCATE` or `VACUUM FULL` command, the newly created file inherits the encryption information and is either encrypted or not. The principal key is used to encrypt the internal keys. The principal key is stored in the key management store. When you query the table, the principal key is retrieved from the key store to decrypt the table. Then the internal key for that table is used to decrypt the data. ### WAL encryption -WAL encryption is done globally for the entire database cluster. All modifications to any database within a PostgreSQL cluster are written to the same WAL to maintain data consistency and integrity and ensure that PostgreSQL cluster can be restored to a consistent state. Therefore, WAL is encrypted globally. +WAL encryption is done globally for the entire database cluster. All modifications to any database within a PostgreSQL cluster are written to the same WAL to maintain data consistency and integrity and ensure that PostgreSQL cluster can be restored to a consistent state. Therefore, WAL is encrypted globally. When you turn on WAL encryption, `pg_tde` encrypts entire WAL files starting from the first WAL write after the server was started with the encryption turned on. @@ -97,12 +98,11 @@ You can turn WAL encryption on and off so WAL can contain both encrypted and une Whenever the WAL is being read (by the recovery process or tools), the decision on what should be decrypted is based solely on the metadata of WAL encryption keys. - ## Should I encrypt all my data? -It depends on your business requirements and the sensitivity of your data. Encrypting all data is a good practice but it can have a performance impact. +It depends on your business requirements and the sensitivity of your data. Encrypting all data is a good practice but it can have a performance impact. -Consider encrypting only tables that store sensitive data. You can decide what tables to encrypt and with what key. The [Set up multi-tenancy](multi-tenant-setup.md) section in the documentation focuses on this approach. +Consider encrypting only tables that store sensitive data. You can decide what tables to encrypt and with what key. The [Set up multi-tenancy](how-to/multi-tenant-setup.md) section in the documentation focuses on this approach. We advise encrypting the whole database only if all your data is sensitive, like PII, or if there is no other way to comply with data safety requirements. @@ -122,7 +122,7 @@ No, it's not yet supported. In our implementation we reply on OpenSSL libraries Yes, you can encrypt an existing table. Run the `ALTER TABLE` command as follows: -``` +```sql ALTER TABLE table_name SET ACCESS METHOD tde_heap; ``` @@ -135,7 +135,7 @@ You must restart the database in the following cases to apply the changes: * after you enabled the `pg_tde` extension * to turn on / off the WAL encryption -After that, no database restart is required. When you create or alter the table using the `tde_heap` access method, the files are marked as those that require encryption. The encryption happens at the storage manager level, before a transaction is written to disk. Read more about [how tde_heap works](table-access-method.md#how-tde_heap-works). +After that, no database restart is required. When you create or alter the table using the `tde_heap` access method, the files are marked as those that require encryption. The encryption happens at the storage manager level, before a transaction is written to disk. Read more about [how tde_heap works](index/table-access-method.md#how-tde_heap-works). ## What happens to my data if I lose a principal key? @@ -143,7 +143,7 @@ If you lose encryption keys, especially, the principal key, the data is lost. Th ## Can I use `pg_tde` in a multi-tenant setup? -Multi-tenancy is the type of architecture where multiple users, or tenants, share the same resource. It can be a database, a schema or an entire cluster. +Multi-tenancy is the type of architecture where multiple users, or tenants, share the same resource. It can be a database, a schema or an entire cluster. In `pg_tde`, multi-tenancy is supported via a separate principal key per database. This means that a database owner can decide what tables to encrypt within a database. The same database can have both encrypted and non-encrypted tables. diff --git a/contrib/pg_tde/documentation/docs/features.md b/contrib/pg_tde/documentation/docs/features.md index 3548c6deb22df..8edd99fbc991c 100644 --- a/contrib/pg_tde/documentation/docs/features.md +++ b/contrib/pg_tde/documentation/docs/features.md @@ -1,20 +1,24 @@ # Features -We provide `pg_tde` in two versions for both PostgreSQL Community and [Percona Server for PostgreSQL](https://docs.percona.com/postgresql/17/). The difference between the versions is in the set of included features which in its turn depends on the Storage Manager API. While PostgreSQL Community uses the default Storage Manager API, Percona Server for PostgreSQL extends the Storage Manager API enabling to integrate custom storage managers. +`pg_tde` is available for [Percona Server for PostgreSQL](https://docs.percona.com/postgresql/17/) +The Percona Server for PostgreSQL provides an extended Storage Manager API that allows integration with custom storage managers. -The following table provides features available for each version: +The following features are available for the extension: -| Percona Server for PostgreSQL version | PostgreSQL Community version (deprecated) | -|-------------------------------|----------------------| -| Table encryption:
- data tables,
- **Index data for encrypted tables**,
- TOAST tables,
- temporary tables created during the database operation.

Metadata of those tables is not encrypted. | Table encryption:
- data tables,
- TOAST tables
- temporary tables created during the database operation.

Metadata of those tables is not encrypted. | -| **Global** Write-Ahead Log (WAL) encryption: for data in encrypted and non-encrypted tables | Write-Ahead Log (WAL) encryption of data in encrypted tables | -| Single-tenancy support via global keyring provider | | -| Multi-tenancy support | Multi-tenancy support | -| Table-level granularity | Table-level granularity | -| Key management via:
- HashiCorp Vault;
- KMIP server;
- Local keyfile | Key management via:
- HashiCorp Vault;
- Local keyfile | -| Logical replication of encrypted tables | | +* Table encryption, including: + * Data tables + * Index data for encrypted tables + * TOAST tables + * Temporary tables created during database operations +!!! note + Metadata of those tables is not encrypted. +* Global Write-Ahead Log (WAL) encryption for data in both encrypted and non-encrypted tables +* Single-tenancy support via a global keyring provider +* Multi-tenancy support +* Table-level granularity for encryption and access control +* Multiple Key management options +* Logical replication of encrypted tables - -[Get started](install.md){.md-button} \ No newline at end of file +[Overview](index/index.md){.md-button} [Get started](install.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/functions.md b/contrib/pg_tde/documentation/docs/functions.md index 45463fb2b12ff..f7bc4647b8990 100644 --- a/contrib/pg_tde/documentation/docs/functions.md +++ b/contrib/pg_tde/documentation/docs/functions.md @@ -36,16 +36,16 @@ A key provider is a system or service responsible for managing encryption keys. Key provider management includes the following operations: -* creating a new key provider, -* changing an existing key provider, -* deleting a key provider, +* creating a new key provider, +* changing an existing key provider, +* deleting a key provider, * listing key providers. ### Add a provider You can add a new key provider using the provided functions, which are implemented for each provider type. -There are two functions to add a key provider: one function adds it for the current database and another one - for the global scope. +There are two functions to add a key provider: one function adds it for the current database and another one - for the global scope. * `pg_tde_add_database_key_provider_('provider-name', )` * `pg_tde_add_global_key_provider_('provider-name', )` @@ -75,14 +75,14 @@ The Vault provider connects to a HashiCorp Vault or an OpenBao server, and store Use the following functions to add the Vault provider: -``` +```sql SELECT pg_tde_add_database_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path'); SELECT pg_tde_add_global_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path'); ``` These functions change the Vault provider: -``` +```sql SELECT pg_tde_change_database_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path'); SELECT pg_tde_change_global_key_provider_vault_v2('provider-name','secret_token','url','mount','ca_path'); ``` @@ -95,25 +95,24 @@ where: * `secret_token` is an access token with read and write access to the above mount point * [optional] `ca_path` is the path of the CA file used for SSL verification -All parameters can be either strings, or JSON objects [referencing remote parameters](external-parameters.md). +All parameters can be either strings, or JSON objects [referencing remote parameters](how-to/external-parameters.md). **Never specify the secret token directly, use a remote parameter instead.** - #### Adding or modifying KMIP providers The KMIP provider uses a remote KMIP server. -Use these functions to add a KMIP provider: +Use these functions to add a KMIP provider: -``` +```sql SELECT pg_tde_add_database_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem'); SELECT pg_tde_add_global_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem'); ``` These functions change the KMIP provider: -``` +```sql SELECT pg_tde_change_database_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem'); SELECT pg_tde_change_global_key_provider_kmip('provider-name','kmip-addr', `port`, '/path_to/server_certificate.pem', '/path_to/client_key.pem'); ``` @@ -129,7 +128,7 @@ where: The specified access parameters require permission to read and write keys at the server. -All parameters can be either strings, or JSON objects [referencing remote parameters](external-parameters.md). +All parameters can be either strings, or JSON objects [referencing remote parameters](how-to/external-parameters.md). ### Adding or modifying local keyfile providers @@ -141,14 +140,14 @@ This function is intended for development or quick testing, and stores the keys Add a local keyfile provider: -``` +```sql SELECT pg_tde_add_database_key_provider_file('provider-name','/path/to/the/key/provider/data.file'); SELECT pg_tde_add_global_key_provider_file('provider-name','/path/to/the/key/provider/data.file'); ``` Change a local keyfile provider: -``` +```sql SELECT pg_tde_change_database_key_provider_file('provider-name','/path/to/the/key/provider/data.file'); SELECT pg_tde_change_global_key_provider_file('provider-name','/path/to/the/key/provider/data.file'); ``` @@ -158,7 +157,7 @@ where: * `provider-name` is the name of the provider. You can specify any name, it's for you to identify the provider. * `/path/to/the/key/provider/data.file` is the path to the key provider file. -All parameters can be either strings, or JSON objects [referencing remote parameters](external-parameters.md). +All parameters can be either strings, or JSON objects [referencing remote parameters](how-to/external-parameters.md). ### Delete a provider @@ -180,7 +179,6 @@ These functions list the details of all key providers for the current database o **All configuration values include possibly sensitive values, such as passwords. Never specify these directly, use the remote configuration option instead.** - ## Principal key management Use these functions to create a new principal key for a specific scope such as a current database, a global or default scope. You can also use them to start using a different existing key for a specific scope. @@ -191,7 +189,7 @@ Princial keys are stored on key providers by the name specified in this function Creates or rotates the principal key for the current database using the specified database key provider and key name. -``` +```sql SELECT pg_tde_set_key_using_database_key_provider('name-of-the-key','provider-name','ensure_new_key'); ``` @@ -205,30 +203,29 @@ SELECT pg_tde_set_key_using_database_key_provider('name-of-the-key','provider-na Creates or rotates the global principal key using the specified global key provider and the key name. This key is used for global settings like WAL encryption. -``` -SELECT pg_tde_set_key_using_global_key_provider('name-of-the-key','provider-name','ensure_new_key'); +```sql + SELECT pg_tde_set_key_using_global_key_provider('name-of-the-key','provider-name','ensure_new_key'); ``` The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation: -* If set to `true` (default), a new key must be unique. +* If set to `true`, a new key must be unique. If the provider already stores a key by that name, the function returns an error. -* If set to `false`, an existing principal key may be reused. +* If set to `false` (default), an existing principal key may be reused. ### pg_tde_set_server_key_using_global_key_provider Creates or rotates the server principal key using the specified global key provider. Use this function to set a principal key for WAL encryption. -``` +```sql SELECT pg_tde_set_server_key_using_global_key_provider('name-of-the-key','provider-name','ensure_new_key'); ``` The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation: -* If set to `true` (default), a new key must be unique. +* If set to `true`, a new key must be unique. If the provider already stores a key by that name, the function returns an error. -* If set to `false`, an existing principal key may be reused. - +* If set to `false` (default), an existing principal key may be reused. ### pg_tde_set_default_key_using_global_key_provider @@ -236,15 +233,15 @@ Creates or rotates the default principal key for the server using the specified The default key is automatically used as a principal key by any database that doesn't have an individual key provider and key configuration. -``` +```sql SELECT pg_tde_set_default_key_using_global_key_provider('name-of-the-key','provider-name','ensure_new_key'); ``` The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation: -* If set to `true` (default), a new key must be unique. +* If set to `true`, a new key must be unique. If the provider already stores a key by that name, the function returns an error. -* If set to `false`, an existing principal key may be reused. +* If set to `false` (default), an existing principal key may be reused. ## Encryption status check @@ -256,13 +253,13 @@ tables and indexes. To verify that a table is encrypted, run the following statement: -``` +```sql SELECT pg_tde_is_encrypted('table_name'); ``` You can also verify if the table in a custom schema is encrypted. Pass the schema name for the function as follows: -``` +```sql SELECT pg_tde_is_encrypted('schema.table_name'); ``` @@ -272,7 +269,7 @@ This can additionally be used to verify that indexes and sequences are encrypted Displays information about the principal key for the current database, if it exists. -``` +```sql SELECT pg_tde_key_info() ``` @@ -280,7 +277,7 @@ SELECT pg_tde_key_info() Displays information about the principal key for the server scope, if exists. -``` +```sql SELECT pg_tde_server_key_info() ``` @@ -288,7 +285,7 @@ SELECT pg_tde_server_key_info() Displays the information about the default principal key, if it exists. -``` +```sql SELECT pg_tde_default_key_info() ``` @@ -304,7 +301,7 @@ This function checks that the current database has a properly functional encrypt If any of the above checks fail, the function reports an error. -``` +```sql SELECT pg_tde_verify_key() ``` @@ -320,7 +317,7 @@ This function checks that the server scope has a properly functional encryption If any of the above checks fail, the function reports an error. -``` +```sql SELECT pg_tde_verify_server_key() ``` @@ -336,6 +333,6 @@ This function checks that the default key is properly configured, which means: If any of the above checks fail, the function reports an error. -``` +```sql SELECT pg_tde_verify_default_key() ``` diff --git a/contrib/pg_tde/documentation/docs/get-help.md b/contrib/pg_tde/documentation/docs/get-help.md index 81ee0d8b001e9..56bbac165e47a 100644 --- a/contrib/pg_tde/documentation/docs/get-help.md +++ b/contrib/pg_tde/documentation/docs/get-help.md @@ -1,4 +1,4 @@ -# Get help from Percona +# Get Help from Percona Our documentation guides are packed with information, but they can't cover everything you need to know about `pg_tde`. They also won't cover every scenario you might come across. Don't be afraid to try things out and ask questions when you get stuck. @@ -17,7 +17,6 @@ Percona experts bring years of experience in tackling tough database performance
We understand your challenges when managing complex database environments. That's why we offer various services to help you simplify your operations and achieve your goals. - | Service | Description | |----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 24/7 Expert Support | Our dedicated team of database experts is available 24/7 to assist you with any database issues. We provide flexible support plans tailored to your specific needs. | @@ -25,4 +24,4 @@ We understand your challenges when managing complex database environments. That' | Expert Consulting | Our experienced consultants provide guidance on database topics like architecture design, migration planning, performance optimization, and security best practices. | | Comprehensive Training | Our training programs help your team develop skills to manage databases effectively, offering virtual and in-person courses. | -We're here to help you every step of the way. Whether you need a quick fix or a long-term partnership, we're ready to provide your expertise and support. \ No newline at end of file +We're here to help you every step of the way. Whether you need a quick fix or a long-term partnership, we're ready to provide your expertise and support. diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/index.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/index.md new file mode 100644 index 0000000000000..f8fbf49d5ab29 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/index.md @@ -0,0 +1,16 @@ +# Configure Key Management (KMS) + +In production environments, storing encryption keys locally on the PostgreSQL server can introduce security risks. To enhance security, `pg_tde` supports integration with external Key Management Systems (KMS) through a Global Key Provider interface. + +This section describes how you can configure `pg_tde` to use the local and external key providers. +To use an external KMS with `pg_tde`, follow these two steps: + +1. Configure a Key Provider +2. Set the [Global Principal Key](set-principal-key.md) + +!!! note + While keyfiles may be acceptable for **local** or **testing environments**, KMS integration is the recommended approach for production deployments. + +Select your prefered configuration from the links below: + +[KMIP Configuration :material-arrow-right:](kmip-server.md){.md-button} [Vault Configuration :material-arrow-right:](vault.md){.md-button} [Keyring File Configuration (not recommended) :material-arrow-right:](keyring.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/keyring.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/keyring.md new file mode 100644 index 0000000000000..d36f915f713fd --- /dev/null +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/keyring.md @@ -0,0 +1,27 @@ +# Keyring File Configuration + +This setup is intended for development and stores the keys unencrypted in the specified data file. See [how to use external reference to parameters](../how-to/external-parameters.md) to add an extra security layer to your setup. + +!!! note + + While keyfiles may be acceptable for **local** or **testing environments**, KMS integration is the recommended approach for production deployments. + +```sql + SELECT pg_tde_add_global_key_provider_file( + 'provider-name', + '/path/to/the/keyring/data.file' + ); +``` + +The following example is used for testing purposes only: + +```sql + SELECT pg_tde_add_global_key_provider_file( + 'file-keyring', + '/tmp/pg_tde_test_local_keyring.per' + ); +``` + +## Next steps + +[Global Principal Key Configuration :material-arrow-right:](set-principal-key.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/kmip-server.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/kmip-server.md new file mode 100644 index 0000000000000..3127f89a29267 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/kmip-server.md @@ -0,0 +1,55 @@ +# KMIP Configuration + +To use a Key Management Interoperability Protocol (KMIP) server with `pg_tde`, you must configure it as a global key provider. This setup enables `pg_tde` to securely fetch and manage encryption keys from a centralized key management appliance. + +!!! note + + You need the root certificate of the KMIP server and a client key/certificate pair with permissions to create and read keys on the server. + +It is recommended to review the [configuration guidelines for the HashiCorp Vault Enterprise KMIP Secrets Engine](https://developer.hashicorp.com/vault/tutorials/enterprise/kmip-engine) if you're using Vault. + +For testing purposes, you can use a lightweight PyKMIP server, which enables easy certificate generation and basic KMIP behavior. If you're using a production-grade KMIP server, ensure you obtain valid, trusted certificates from the key management appliance. + +## Example usage + + ```sql + SELECT pg_tde_add_global_key_provider_kmip( + 'provider-name', + 'kmip-IP', + 5696, + '/path_to/server_certificate.pem', + '/path_to/client_cert.pem', + '/path_to/client_key.pem' + ); + ``` + +## Parameter descriptions + +* `provider-name` is the name of the provider. You can specify any name, it's for you to identify the provider +* `kmip-IP` is the IP address of a domain name of the KMIP server +* `port` is the port to communicate with the KMIP server. Typically used port is 5696 +* `server-certificate` is the path to the certificate file for the KMIP server +* `client_cert` is the path to the client certificate. +* `client_key` (optional) is the path to the client key. If not specified, the certificate key has to contain both the certifcate and the key. + +:material-information: Warning: `pg_tde_add_global_key_provider_kmip` currently accepts only a combined client key and a client certificate for its final parameter, reffered to as `client key`. + +The following example is for testing purposes only. + + ```sql + SELECT pg_tde_add_global_key_provider_kmip( + 'kmip','127.0.0.1', + 5696, + '/tmp/server_certificate.pem', + '/tmp/client_cert_jane_doe.pem', + '/tmp/client_key_jane_doe.pem' + ); + ``` + +For more information on related functions, see the link below: + +[Percona pg_tde function reference](../functions.md){.md-button} + +## Next steps + +[Global Principal Key Configuration :material-arrow-right:](set-principal-key.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md new file mode 100644 index 0000000000000..8a56645595510 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/set-principal-key.md @@ -0,0 +1,37 @@ +# Global Principal Key Configuration + +You can configure a default principal key using a global key provider. This key will be used by all databases that do not have their own encryption keys configured. + +## Create a default principal key + +Run the following command: + +```sql + SELECT pg_tde_set_default_key_using_global_key_provider( + 'name-of-the-key', + 'provider-name', + 'ensure_new_key' + ); +``` + +## Parameter description + +* `name-of-the-key` is the name of the principal key. You will use this name to identify the key. +* `provider-name` is the name of the key provider you added before. The principal key will be associated with this provider. +* `ensure_new_key` defines if a principal key must be unique. The default value `true` means that you must speficy a unique key during key rotation. The `false` value allows reusing an existing principal key. + +This example is for testing purposes only. Replace the key name and provider name with your values: + +```sql + SELECT pg_tde_set_key_using_global_key_provider('test-db-master-key','file-vault','ensure_new_key'); +``` + +!!! note + + The key is auto-generated. + +After this, all databases that do not have something else configured will use this newly generated principal key. + +## Next steps + +[Validate Encryption with pg_tde :material-arrow-right:](../test.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md new file mode 100644 index 0000000000000..f6877bf02ed70 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/global-key-provider-configuration/vault.md @@ -0,0 +1,47 @@ +# Vault Configuration + +You can configure `pg_tde` to use HashiCorp Vault as a global key provider for managing encryption keys securely. + +!!! note + + This guide assumes that your Vault server is already set up and accessible. Vault configuration is outside the scope of this document, see [Vault's official documentation](https://developer.hashicorp.com/vault/docs) for more information. + +## Example usage + +```sql + SELECT pg_tde_add_global_key_provider_vault_v2( + 'provider-name', + 'secret_token', + 'url', + 'mount', + 'ca_path' + ); +``` + +## Parameter descriptions + +* `provider-name` is the name to identify this key provider +* `secret_token` is an access token with read and write access to the above mount point +* `url` is the URL of the Vault server +* `mount` is the mount point where the keyring should store the keys +* [optional] `ca_path` is the path of the CA file used for SSL verification + +The following example is for testing purposes only. Use secure tokens and proper SSL validation in production environments: + +```sql + SELECT pg_tde_add_global_key_provider_vault_v2( + 'my-vault', + 'hvs.zPuyktykA...example...ewUEnIRVaKoBzs2', + 'http://vault.vault.svc.cluster.local:8200', + 'secret/data', + NULL + ); +``` + +For more information on related functions, see the link below: + +[Percona pg_tde function reference](../functions.md){.md-button} + +## Next steps + +[Global Principal Key Configuration :material-arrow-right:](set-principal-key.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/decrypt.md b/contrib/pg_tde/documentation/docs/how-to/decrypt.md similarity index 75% rename from contrib/pg_tde/documentation/docs/decrypt.md rename to contrib/pg_tde/documentation/docs/how-to/decrypt.md index 1aafcca361e35..21e9d10fd28bb 100644 --- a/contrib/pg_tde/documentation/docs/decrypt.md +++ b/contrib/pg_tde/documentation/docs/how-to/decrypt.md @@ -1,42 +1,40 @@ -# Remove encryption from an encrypted table +# Remove Encryption from an Encrypted Table ## Method 1. Change the access method If you encrypted a table with the `tde_heap` access method and need to remove the encryption from it, run the following command against the desired table (`mytable` in the example below): -``` -ALTER TABLE mytable SET ACCESS METHOD heap; +```sql + ALTER TABLE mytable SET ACCESS METHOD heap; ``` Note that the `SET ACCESS METHOD` command drops hint bits and this may affect performance. Running a plain `SELECT count(*)` or `VACUUM` command on the entire table will check every tuple for visibility and set its hint bits. Therefore, after executing the `ALTER TABLE` command, run a simple `count(*)` on your tables: -``` -SELECT count(*) FROM mytable; +```sql + SELECT count(*) FROM mytable; ``` Check that the table is not encrypted: +```sql + SELECT pg_tde_is_encrypted('mytable'); ``` -SELECT pg_tde_is_encrypted('mytable'); -``` - -The output returns `f` meaning that the table is no longer encrypted. - +The output returns `f` meaning that the table is no longer encrypted. ## Method 2. Create a new not encrypted table on the base of the encrypted one -Alternatively, you can create a new not encrypted table with the same structure and data as the initial table. For example, the original encrypted table is `EncryptedCustomers`. Use the following command to create a new table `Customers`: +Alternatively, you can create a new not encrypted table with the same structure and data as the initial table. For example, the original encrypted table is `EncryptedCustomers`. Use the following command to create a new table `Customers`: -``` -CREATE TABLE Customers AS -SELECT * FROM EncryptedCustomers; +```sql + CREATE TABLE Customers AS + SELECT * FROM EncryptedCustomers; ``` The new table `Customers` inherits the structure and the data from `EncryptedCustomers`. (Optional) If you no longer need the `EncryptedCustomers` table, you can delete it. -``` -DROP TABLE EncryptedCustomers; +```sql + DROP TABLE EncryptedCustomers; ``` diff --git a/contrib/pg_tde/documentation/docs/external-parameters.md b/contrib/pg_tde/documentation/docs/how-to/external-parameters.md similarity index 82% rename from contrib/pg_tde/documentation/docs/external-parameters.md rename to contrib/pg_tde/documentation/docs/how-to/external-parameters.md index f68aee653e011..86334dfb6afcc 100644 --- a/contrib/pg_tde/documentation/docs/external-parameters.md +++ b/contrib/pg_tde/documentation/docs/how-to/external-parameters.md @@ -1,9 +1,9 @@ -# Use external reference to parameters +# Use External Reference to Parameters To allow storing secrets or any other parameters in a more secure, external location, `pg_tde` allows users to specify an external reference instead of hardcoded parameters. -In Alpha1 version, `pg_tde` supports the following external storage methods: +In the Alpha1 version, `pg_tde` supports the following external storage methods: * `file`, which just stores the data in a simple file specified by a `path`. The file should be readable to the postgres process. @@ -14,7 +14,7 @@ readable to the postgres process. To use the file provider with a file location specified by the `remote` method, use the following command: -``` +```sql SELECT pg_tde_add_database_key_provider_file( 'file-provider', json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/hello' ) @@ -23,12 +23,11 @@ SELECT pg_tde_add_database_key_provider_file( Or to use the `file` method, use the following command: -``` +```sql SELECT pg_tde_add_database_key_provider_file( 'file-provider', json_object( 'type' VALUE 'remote', 'path' VALUE '/tmp/datafile-location' ) );" ``` -Any parameter specified to the `add_key_provider` function can be a `json_object` instead of the string, -similar to the above examples. \ No newline at end of file +Any parameter specified to the `add_key_provider` function can be a `json_object` instead of the string, similar to the above examples. diff --git a/contrib/pg_tde/documentation/docs/multi-tenant-setup.md b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md similarity index 93% rename from contrib/pg_tde/documentation/docs/multi-tenant-setup.md rename to contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md index 5a6558208da43..abcf3c4779144 100644 --- a/contrib/pg_tde/documentation/docs/multi-tenant-setup.md +++ b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md @@ -1,10 +1,10 @@ -# Set up multi-tenancy +# Configure Multi-tenancy The steps below describe how to set up multi-tenancy with `pg_tde`. Multi-tenancy allows you to encrypt different databases with different keys. This provides granular control over data and enables you to introduce different security policies and access controls for each database so that only authorized users of specific databases have access to the data. -If you don't need multi-tenancy, use the global key provider. See the configuration steps from the [Setup](setup.md) section. +If you don't need multi-tenancy, use the global key provider. See the configuration steps from the [Setup](../setup.md) section. -For how to enable WAL encryption, refer to the [WAL encryption](wal-encryption.md) section. +For how to enable WAL encryption, refer to the [WAL encryption](../wal-encryption.md) section. --8<-- "kms-considerations.md" @@ -12,7 +12,7 @@ For how to enable WAL encryption, refer to the [WAL encryption](wal-encryption.m Load the `pg_tde` at startup time. The extension requires additional shared memory; therefore, add the `pg_tde` value for the `shared_preload_libraries` parameter and restart the `postgresql` cluster. -1. Use the [ALTER SYSTEM :octicons-link-external-16:](https://www.postgresql.org/docs/current/sql-altersystem.html) command from `psql` terminal to modify the `shared_preload_libraries` parameter. This requires superuser privileges. +1. Use the [ALTER SYSTEM :octicons-link-external-16:](https://www.postgresql.org/docs/current/sql-altersystem.html) command from `psql` terminal to modify the `shared_preload_libraries` parameter. This requires superuser privileges. ``` ALTER SYSTEM SET shared_preload_libraries = 'pg_tde'; @@ -20,12 +20,12 @@ Load the `pg_tde` at startup time. The extension requires additional shared memo 2. Start or restart the `postgresql` cluster to apply the changes. - * On Debian and Ubuntu: + * On Debian and Ubuntu: ```sh sudo systemctl restart postgresql-17 ``` - + * On RHEL and derivatives ```sh @@ -37,8 +37,8 @@ Load the `pg_tde` at startup time. The extension requires additional shared memo ``` CREATE EXTENSION pg_tde; ``` - - The `pg_tde` extension is created for the currently used database. To enable data encryption in other databases, you must explicitly run the `CREATE EXTENSION` command against them. + + The `pg_tde` extension is created for the currently used database. To enable data encryption in other databases, you must explicitly run the `CREATE EXTENSION` command against them. !!! tip @@ -60,7 +60,7 @@ You must do these steps for every database where you have created the extension. For testing purposes, you can use the PyKMIP server which enables you to set up required certificates. To use a real KMIP server, make sure to obtain the valid certificates issued by the key management appliance. - ``` + ```sql SELECT pg_tde_add_database_key_provider_kmip('provider-name','kmip-addr', 5696, '/path_to/server_certificate.pem', '/path_to/client_key.pem'); ``` @@ -99,9 +99,9 @@ You must do these steps for every database where you have created the extension. SELECT pg_tde_add_database_key_provider_file_vault_v2('my-vault','http://vault.vault.svc.cluster.local:8200,'secret/data','hvs.zPuyktykA...example...ewUEnIRVaKoBzs2', NULL); ``` - === "With a keyring file" + === "With a keyring file (not recommended)" - This setup is intended for development and stores the keys unencrypted in the specified data file. + This setup is intended for development and stores the keys unencrypted in the specified data file. ```sql SELECT pg_tde_add_database_key_provider_file('provider-name','/path/to/the/keyring/data.file'); @@ -112,8 +112,7 @@ You must do these steps for every database where you have created the extension. ```sql SELECT pg_tde_add_database_key_provider_file('file-keyring','/tmp/pg_tde_test_local_keyring.per'); ``` - - + 2. Add a principal key ```sql @@ -134,5 +133,4 @@ You must do these steps for every database where you have created the extension. The key is auto-generated. - - :material-information: Info: The key provider configuration is stored in the database catalog in an unencrypted table. See [how to use external reference to parameters](external-parameters.md) to add an extra security layer to your setup. \ No newline at end of file + :material-information: Info: The key provider configuration is stored in the database catalog in an unencrypted table. See [how to use external reference to parameters](external-parameters.md) to add an extra security layer to your setup. diff --git a/contrib/pg_tde/documentation/docs/uninstall.md b/contrib/pg_tde/documentation/docs/how-to/uninstall.md similarity index 91% rename from contrib/pg_tde/documentation/docs/uninstall.md rename to contrib/pg_tde/documentation/docs/how-to/uninstall.md index 3005d249416e2..7901f778fd8f4 100644 --- a/contrib/pg_tde/documentation/docs/uninstall.md +++ b/contrib/pg_tde/documentation/docs/how-to/uninstall.md @@ -1,4 +1,4 @@ -# Uninstall `pg_tde` +# Uninstall pg_tde If you no longer wish to use TDE in your deployment, you can remove the `pg_tde` extension. To do so, your user must have the superuser privileges, or a database owner privileges in case you only want to remove it from a single database. @@ -6,13 +6,13 @@ Here's how to do it: 1. Drop the extension using the `DROP EXTENSION` command: - ``` + ```sql DROP EXTENSION pg_tde; ``` - This command will fail if there are still encrypted tables in the database. + This command will fail if there are still encrypted tables in the database. - In this case, you must drop the dependent objects manually. Alternatively, you can run the `DROP EXTENSION ... CASCADE` command to drop all dependent objects automatically. + In this case, you must drop the dependent objects manually. Alternatively, you can run the `DROP EXTENSION ... CASCADE` command to drop all dependent objects automatically. Note that the `DROP EXTENSION` command does not delete the `pg_tde` data files related to the database. @@ -22,7 +22,7 @@ Here's how to do it: 4. Modify the `shared_preload_libraries` and remove the 'pg_tde' from it. Use the `ALTER SYSTEM` command for this purpose, or edit the configuration file. - !!! warning + !!! warning Once `pg_tde` is removed from the `shared_preload_libraries`, reading any leftover encrypted files will fail. Removing the extension from the `shared_preload_libraries` is also possible if the extension is still installed in some databases. @@ -30,12 +30,12 @@ Here's how to do it: 5. Start or restart the `postgresql` cluster to apply the changes. - * On Debian and Ubuntu: + * On Debian and Ubuntu: ```sh sudo systemctl restart postgresql ``` - + * On RHEL and derivatives ```sh diff --git a/contrib/pg_tde/documentation/docs/index.md b/contrib/pg_tde/documentation/docs/index.md index 4bf8a3b9e0e7a..b2a0d1a2cdfa3 100644 --- a/contrib/pg_tde/documentation/docs/index.md +++ b/contrib/pg_tde/documentation/docs/index.md @@ -1,51 +1,11 @@ -# `pg_tde` documentation +# `pg_tde` Documentation -`pg_tde` is the open source PostgreSQL extension that provides Transparent Data Encryption (TDE) to protect data at rest. This ensures that the data stored on disk is encrypted, and no one can read it without the proper encryption keys, even if they gain access to the physical storage media. +`pg_tde` is the open source, community driven and futureproof PostgreSQL extension that provides Transparent Data Encryption (TDE) to protect data at rest. `pg_tde` ensures that the data stored on disk is encrypted, and that no one can read it without the proper encryption keys, even if they gain access to the physical storage media. -Learn more [what Transparent Data Encryption is](tde.md#how-does-it-work) and [why you need it](tde.md#why-do-you-need-tde). - -!!! important +!!! important This is the {{release}} version of the extension and it is not meant for production use yet. We encourage you to use it in testing environments and [provide your feedback](https://forums.percona.com/c/postgresql/pg-tde-transparent-data-encryption-tde/82). - + +[What is Transparent Data Encryption (TDE)?](index/index.md){.md-button} [Get started](install.md){.md-button} [What's new in pg_tde {{release}}](release-notes/release-notes.md){.md-button} - -## What's encrypted: - -* User data in tables, including TOAST tables, that are created using the extension. Metadata of those tables is not encrypted. -* Temporary tables created during the database operation for data tables created using the extension -* Write-Ahead Log (WAL) data for the entire database cluster. This includes WAL data in encrypted and non-encrypted tables -* Indexes on encrypted tables -* Logical replication on encrypted tables - -[Check the full feature list](features.md){.md-button} - -## Known limitations - -* Keys in the local keyfile are stored unencrypted. For better security we recommend using the Key management storage. -* System tables are currently not encrypted. This means that statistics data and database metadata are currently not encrypted. - -* `pg_rewind` doesn't work with encrypted WAL for now. We plan to fix it in future releases. -* `pb_tde` Release candidate is incompatible with `pg_tde`Beta2 due to significant changes in code. There is no direct upgrade flow from one version to another. You must [uninstall](uninstall.md) `pg_tde` Beta2 first and then [install](install.md) and configure the new Release Candidate version. - - - -## Versions and supported PostgreSQL deployments - -The `pg_tde` extension comes in two distinct versions with specific access methods to encrypt the data. These versions are database-specific and differ in terms of what they encrypt and with what access method. Each version is characterized by the database it supports, the access method it provides, and the scope of encryption it offers. - -* **Version for Percona Server for PostgreSQL** - - This `pg_tde` version is based on and supported for [Percona Server for PostgreSQL 17.x :octicons-link-external-16:](https://docs.percona.com/postgresql/17/postgresql-server.html) - an open source binary drop-in replacement for PostgreSQL Community. It provides the `tde_heap` access method and offers [full encryption capabilities](features.md). - -* **Community version** (deprecated) - - This version is available with PostgreSQL Community 16 and 17, and Percona Distribution for PostgreSQL 16. It provides the `tde_heap_basic` access method, offering limited encryption features. The limitations are in encrypting WAL data only for tables created using the extension and no support of index encryption nor logical replication. - -### Which version to choose? - -Enjoy full encryption with the Percona Server for PostgreSQL version and the `tde_heap` access method. The Community version is deprecated and is planned to be removed in future releases. - -Still not sure? [Contact our experts](https://www.percona.com/about/contact) to find the best solution for you. - diff --git a/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md b/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md new file mode 100644 index 0000000000000..e21bdffbccfe2 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/index/how-does-tde-work.md @@ -0,0 +1,27 @@ +# How TDE Works + +To encrypt the data, two types of keys are used: + +* **Internal encryption keys** to encrypt user data. They are stored internally, near the data that they encrypt. +* The **principal key** to encrypt database keys. It is kept separately from the database keys and is managed externally in the key management store. + +!!! note + + For more information on managing and storing principal keys externally, see [Configure Global Key Provider](../global-key-provider-configuration/index.md). + +You have the following options to store and manage principal keys externally: + +* Use the HashiCorp Vault server. Only the back end KV Secrets Engine - Version 2 (API) is supported. +* Use the KMIP-compatible server. `pg_tde` has been tested with the [PyKMIP](https://pykmip.readthedocs.io/en/latest/server.html) server and [the HashiCorp Vault Enterprise KMIP Secrets Engine](https://www.vaultproject.io/docs/secrets/kmip). + +The encryption process is the following: + +![image](../_images/tde-flow.png) + +When a user creates an encrypted table using `pg_tde`, a new random key is generated internally for that table and is encrypted using the AES-CBC cipher algorithm. This key is used to encrypt all data the user inserts in that table. Eventually the encrypted data gets stored in the underlying storage. + +The internal key itself is encrypted using the principal key. The principal key is stored externally in the key management store. + +Similarly when the user queries the encrypted table, the principal key is retrieved from the key store to decrypt the internal key. Then the same unique internal key for that table is used to decrypt the data, and unencrypted data gets returned to the user. So, effectively, every TDE table has a unique key, and each table key is encrypted using the principal key. + +[Encrypted Data Scope](tde-encrypts.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/index/how-tde-helps.md b/contrib/pg_tde/documentation/docs/index/how-tde-helps.md new file mode 100644 index 0000000000000..d4397dc330043 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/index/how-tde-helps.md @@ -0,0 +1,20 @@ +# TDE Benefits + +## Benefits for organizations + +* **Data safety:** Prevents unauthorized access to stored data, even if backup files or storage devices are stolen or leaked. +* **Enterprise-ready Architecture:** Supports both single and multi-tenancy, giving flexibility for SaaS providers or internal multi-user systems. + +## Benefits for DBAs and engineers + +* **Granular control:** Encrypt specific tables or databases instead of the entire system, reducing performance overhead. +* **Operational simplicity:** Works transparently without requiring major application changes. +* **Defense in depth:** Adds another layer of protection to existing controls like TLS (encryption in transit), access control, and role-based permissions. + +When combined with the external Key Management Systems (KMS), TDE enables centralized control, auditing, and rotation of encryption keys—critical for secure production environments. + +!!! admonition "See also" + + Percona Blog: [Transparent Data Encryption (TDE)](https://www.percona.com/blog/transparent-data-encryption-tde/) + +[How TDE works](how-does-tde-work.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/index/index.md b/contrib/pg_tde/documentation/docs/index/index.md new file mode 100644 index 0000000000000..94830bd9e32dc --- /dev/null +++ b/contrib/pg_tde/documentation/docs/index/index.md @@ -0,0 +1,7 @@ +# What is Transparent Data Encryption (TDE)? + +Transparent Data Encryption (TDE) protects your data at rest by ensuring that even if the underlying storage is compromised, the data remains unreadable without the correct encryption keys. This is especially critical for environments handling sensitive, regulated, or high-value information. + +Encryption happens transparently in the background, with minimal impact on database operations. + +[TDE Benefits](how-tde-helps.md){.md-button} [Check the full feature list](../features.md){.md-button} [Get started](../install.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/index/supported-versions.md b/contrib/pg_tde/documentation/docs/index/supported-versions.md new file mode 100644 index 0000000000000..ed4758d0b671c --- /dev/null +++ b/contrib/pg_tde/documentation/docs/index/supported-versions.md @@ -0,0 +1,21 @@ +# Versions and Supported PostgreSQL Deployments + +The `pg_tde` extension is available for [Percona Server for PostgreSQL 17.x](https://docs.percona.com/postgresql/17/postgresql-server.html), an open source, drop-in replacement for PostgreSQL Community. This version provides the `tde_heap` access method and offers [full encryption capabilities](../features.md), including encryption of tables, indexes, WAL data, and support for logical replication. + +The extension is tightly integrated with Percona Server for PostgreSQL to deliver enhanced encryption functionality that is not available in community builds. + +## Why choose Percona Server for PostgreSQL? + +By using our PostgreSQL distribution, you get: + +- **Full encryption support** through the `tde_heap` access method, including tables, indexes, WAL data, and logical replication. +- **Enhanced performance and enterprise-ready features** not available in community builds. +- **Regular updates and security patches** backed by Percona’s expert support team. +- **Professional support** and guidance for secure PostgreSQL deployments. + +!!! note + Support for earlier or limited versions of `pg_tde` (such as `tde_heap_basic`) has been deprecated. + +Still unsure which deployment fits your needs? [Contact our experts](https://www.percona.com/about/contact) to find the best solution for your environment. + +[Get Started: Install pg_tde :material-arrow-right:](../install.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/table-access-method.md b/contrib/pg_tde/documentation/docs/index/table-access-method.md similarity index 86% rename from contrib/pg_tde/documentation/docs/table-access-method.md rename to contrib/pg_tde/documentation/docs/index/table-access-method.md index 818ac7f479f4f..bff2655f0014a 100644 --- a/contrib/pg_tde/documentation/docs/table-access-method.md +++ b/contrib/pg_tde/documentation/docs/index/table-access-method.md @@ -1,8 +1,8 @@ -# Table access method +# Table Access Methods and TDE A table access method is the way how PostgreSQL stores the data in a table. The default table access method is `heap`. PostgreSQL organizes data in a heap structure, meaning there is no particular order to the rows in the table. Each row is stored independently and identified by its unique row identifier (TID). -## How the `heap` access method works +## How the heap access method works **Insertion**: When a new row is inserted, PostgreSQL finds a free space in the tablespace and stores the row there. @@ -16,7 +16,7 @@ Custom access methods allow you to implement and define your own way of organizi Custom access methods are typically available with PostgreSQL extensions. When you install an extension and enable it in PostgreSQL, a custom access method is created. -An example of such an approach is the `tde_heap` access method. It is automatically created **only** for the databases where you [enabled the `pg_tde` extension](setup.md) and configured the key provider, enabling you to encrypt the data. +An example of such an approach is the `tde_heap` access method. It is automatically created **only** for the databases where you [enabled the `pg_tde` extension](../setup.md) and configured the key provider, enabling you to encrypt the data. To use a custom access method, specify the `USING` clause for the `CREATE TABLE` command: @@ -28,30 +28,29 @@ CREATE TABLE table_name ( ) USING tde_heap; ``` -### How `tde_heap` works +### How tde_heap works -The `tde_heap` access method works on top of the default `heap` access method and is a marker to point which tables require encryption. It uses the custom storage manager TDE SMGR, which becomes active only after you installed the `pg_tde` extension. +The `tde_heap` access method works on top of the default `heap` access method and is a marker to point which tables require encryption. It uses the custom storage manager TDE SMGR, which becomes active only after you installed the `pg_tde` extension. -Every data modification operation is first sent to the Buffer Manager, which updates the buffer cache. Then, it is passed to the storage manager, which then writes it to disk. When a table requires encryption, the data is sent to the TDE storage manager, where it is encrypted before written to disk. +Every data modification operation is first sent to the Buffer Manager, which updates the buffer cache. Then, it is passed to the storage manager, which then writes it to disk. When a table requires encryption, the data is sent to the TDE storage manager, where it is encrypted before written to disk. -Similarly, when a client queries the database, the PostgreSQL core sends the request to the Buffer Manager which checks if the requested data is already in the buffer cache. If it’s not there, the Buffer Manager requests the data from the storage manager. The TDE storage manager reads the encrypted data from disk, decrypts it and loads it to the buffer cache. The Buffer Manager sends the requested data to the PostgreSQL core and then to the client. +Similarly, when a client queries the database, the PostgreSQL core sends the request to the Buffer Manager which checks if the requested data is already in the buffer cache. If it’s not there, the Buffer Manager requests the data from the storage manager. The TDE storage manager reads the encrypted data from disk, decrypts it and loads it to the buffer cache. The Buffer Manager sends the requested data to the PostgreSQL core and then to the client. - -Thus, the encryption is done at the storage manager level. +Thus, the encryption is done at the storage manager level. ## Changing the default table access method -You can change the default table access method so that every table in the entire database cluster is created using the custom access method. For example, you can enable data encryption by default by defining the `tde_heap` as the default table access method. +You can change the default table access method so that every table in the entire database cluster is created using the custom access method. For example, you can enable data encryption by default by defining the `tde_heap` as the default table access method. However, consider the following before making this change: -* This is a global setting and applies across the entire database cluster and not just a single database. -We recommend setting it with caution because all tables and materialized views created without an explicit access method in their `CREATE` statement will default to the specified table access method. +* This is a global setting and applies across the entire database cluster and not just a single database. +We recommend setting it with caution because all tables and materialized views created without an explicit access method in their `CREATE` statement will default to the specified table access method. * You must create the `pg_tde` extension and configure the key provider for all databases before you modify the configuration. Otherwise PostgreSQL won't find the specified access method and will throw an error. -Here's how you can set the new default table access method: +Here is how you can set the new default table access method: -1. Add the access method to the `default_table_access_method` parameter. +1. Add the access method to the `default_table_access_method` parameter: === "via the SQL statement" @@ -80,12 +79,13 @@ Here's how you can set the new default table access method: You can run the SET command anytime during the session. ```sql - SET default_table_access_method = tde_heap; + SET default_table_access_method = tde_heap; ``` 2. Reload the configuration to apply the changes: ```sql - SELECT pg_reload_conf(); + SELECT pg_reload_conf(); ``` +[Limitations of TDE](tde-limitations.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/index/tde-encrypts.md b/contrib/pg_tde/documentation/docs/index/tde-encrypts.md new file mode 100644 index 0000000000000..d7cd64386537a --- /dev/null +++ b/contrib/pg_tde/documentation/docs/index/tde-encrypts.md @@ -0,0 +1,11 @@ +# Encrypted Data Scope + +`pg_tde` encrypts the following components: + +* **User data** in tables using the extension, including associated TOAST data. The table metadata (column names, data types, etc.) is not encrypted. +* **Temporary tables** created during the query execution, for data tables created using the extension. +* **Write-Ahead Log (WAL) data** for the entire database cluster. This includes WAL data in encrypted and non-encrypted tables. +* **Indexes** associated encrypted tables. +* **Logical replication data** for encrypted tables (ensures encrypted content is preserved across replicas). + +[Table access methods and TDE](table-access-method.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md new file mode 100644 index 0000000000000..20b760a431026 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -0,0 +1,9 @@ +# Limitations of pg_tde + +* Keys in the local keyfile are stored unencrypted. For better security we recommend using the Key management storage. +* System tables are currently not encrypted. This means that statistics data and database metadata are currently not encrypted. + +* `pg_rewind` doesn't work with encrypted WAL for now. We plan to fix it in future releases. +* `pb_tde` Release candidate is incompatible with `pg_tde`Beta2 due to significant changes in code. There is no direct upgrade flow from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. + +[Versions and supported PostgreSQL deployments](supported-versions.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/install.md b/contrib/pg_tde/documentation/docs/install.md index e4a0cfe78ef25..01ada03b178a1 100644 --- a/contrib/pg_tde/documentation/docs/install.md +++ b/contrib/pg_tde/documentation/docs/install.md @@ -1,8 +1,8 @@ -# Installation +# Install pg_tde -Install `pg_tde` using one of the available installation methods: +To install `pg_tde`, use one of the following methods: -=== ":octicons-terminal-16: Package manager" +=== ":octicons-terminal-16: Package manager" The packages are available for the following operating systems: @@ -23,13 +23,16 @@ Install `pg_tde` using one of the available installation methods: [Run in Docker :material-arrow-right:](https://docs.percona.com/postgresql/latest/docker.html){.md-button} -=== ":octicons-download-16: Manual download" +=== ":octicons-download-16: Tar download" - `pg_tde` is included in the Percona Distribution for PostgreSQL tarball. Check below to get access to a detailed step-by-step guide. + `pg_tde` is included in the Percona Distribution for PostgreSQL tarball. Select the below link to access the step-by-step guide. [Install from tarballs :material-arrow-right:](https://docs.percona.com/postgresql/17/tarball.html){.md-button} +Follow the configuration steps below to continue: -## Next steps +[Configure pg_tde :material-arrow-right:](setup.md){.md-button} -[Setup :material-arrow-right:](setup.md){.md-button} +If you’ve already completed these steps, feel free to skip ahead to a later section: + + [Configure Key Management (KMS)](global-key-provider-configuration/index.md){.md-button} [Validate Encryption with pg_tde](test.md){.md-button} [Configure WAL encryption](wal-encryption.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/release-notes/alpha1.md b/contrib/pg_tde/documentation/docs/release-notes/alpha1.md index aad9d43c5ef47..766c9e28b136e 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/alpha1.md +++ b/contrib/pg_tde/documentation/docs/release-notes/alpha1.md @@ -1,10 +1,9 @@ # pg_tde Alpha 1 (2024-03-28) -`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../tde.md) to PostgreSQL and enables you to keep sensitive data safe and secure. +`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../index/index.md) to PostgreSQL and enables you to keep sensitive data safe and secure. [Get started](../install.md){.md-button} - !!! important This version of Percona Transparent Data Encryption extension **is @@ -12,7 +11,6 @@ This will help us improve the product and make it production-ready faster. - ## Release Highlights The Alpha1 version of the extension introduces the following key features: @@ -28,7 +26,7 @@ The Alpha1 version of the extension introduces the following key features: * The keyring configuration is now stored in a catalog separately for each database, instead of a configuration file * Avoid storing secrets in the unencrypted catalog by configuring keyring parameters to be read from external sources (file, http(s) request) -## Improvements +## Improvements * Renamed the repository and Docker image from `postgres-tde-ext` to `pg_tde`. The extension name remains unchanged * Changed the Initialization Vector (IV) calculation of both the data and internal keys @@ -36,9 +34,8 @@ The Alpha1 version of the extension introduces the following key features: ## Bugs fixed * Fixed toast related crashes -* Fixed a crash with the DELETE statement +* Fixed a crash with the DELETE statement * Fixed performance-related issues * Fixed a bug where `pg_tde` sent many 404 requests to the Vault server * Fixed сompatibility issues with old OpenSSL versions -* Fixed сompatibility with old Curl versions - +* Fixed сompatibility with old Curl versions diff --git a/contrib/pg_tde/documentation/docs/release-notes/beta.md b/contrib/pg_tde/documentation/docs/release-notes/beta.md index c8c540a4f8935..11985962c0b20 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/beta.md +++ b/contrib/pg_tde/documentation/docs/release-notes/beta.md @@ -1,6 +1,6 @@ # pg_tde Beta (2024-06-30) -`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../tde.md) to PostgreSQL and enables you to keep sensitive data safe and secure. +`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../index/index.md) to PostgreSQL and enables you to keep sensitive data safe and secure. [Get started](../install.md){.md-button} @@ -22,4 +22,3 @@ Starting with `pg_tde` Beta, the access method for `pg_tde` extension is renamed * Fixed the issue with adjusting a current position in a file by using raw file descriptor for the `lseek` function. (Thanks to user _rainhard_ for providing the fix) * Enhanced the init script to consider a custom superuser for the POSTGRES_USER parameter when `pg_tde` is running via Docker (Thanks to _Alejandro Paredero_ for reporting the issue) - diff --git a/contrib/pg_tde/documentation/docs/release-notes/beta2.md b/contrib/pg_tde/documentation/docs/release-notes/beta2.md index 8c0e8203d439f..f134f6874b772 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/beta2.md +++ b/contrib/pg_tde/documentation/docs/release-notes/beta2.md @@ -1,10 +1,9 @@ # pg_tde Beta 2 (2024-12-16) -`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../tde.md) to PostgreSQL and enables you to keep sensitive data safe and secure. +`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../index/index.md) to PostgreSQL and enables you to keep sensitive data safe and secure. [Get started](../install.md){.md-button} - !!! important This version of Percona Transparent Data Encryption extension **is @@ -26,8 +25,8 @@ The Beta 2 version introduces the following features and improvements: ### New Features * Added the `tde_heap` access method with which you can now enable index encryption for encrypted tables and global WAL data encryption. To use this access method, you must install Percona Server for PostgreSQL. Check the [installation guide](../install.md) -* Added event triggers to identify index creation operations on encrypted tables and store those in a custom storage. -* Added support for secure transfer of keys using the [OASIS Key Management Interoperability Protocol (KMIP)](https://docs.oasis-open.org/kmip/kmip-spec/v2.0/os/kmip-spec-v2.0-os.html). The KMIP implementation was tested with the PyKMIP server and the HashiCorp Vault Enterprise KMIP Secrets Engine. +* Added event triggers to identify index creation operations on encrypted tables and store those in a custom storage. +* Added support for secure transfer of keys using the [OASIS Key Management Interoperability Protocol (KMIP)](https://docs.oasis-open.org/kmip/kmip-spec/v2.0/os/kmip-spec-v2.0-os.html). The KMIP implementation was tested with the PyKMIP server and the HashiCorp Vault Enterprise KMIP Secrets Engine. ### Improvements @@ -53,6 +52,3 @@ The Beta 2 version introduces the following features and improvements: * Fixed multiple bugs with `tde_heap_basic` and TOAST records * Fixed various memory leaks - - - diff --git a/contrib/pg_tde/documentation/docs/release-notes/mvp.md b/contrib/pg_tde/documentation/docs/release-notes/mvp.md index 9f84d24bf46c6..47e52b3bc019b 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/mvp.md +++ b/contrib/pg_tde/documentation/docs/release-notes/mvp.md @@ -3,6 +3,6 @@ The Minimum Viable Product (MVP) version of `pg_tde` introduces the following functionality: * Encryption of heap tables, including TOAST -* Encryption keys are stored either in Hashicorp Vault server or in local keyring file (for development) +* Encryption keys are stored either in Hashicorp Vault server or in local keyring file (for development) * The key storage is configurable via separate JSON configuration files -* Replication support \ No newline at end of file +* Replication support diff --git a/contrib/pg_tde/documentation/docs/release-notes/rc.md b/contrib/pg_tde/documentation/docs/release-notes/rc.md index 9ee5fc16d73e5..792663a6da00e 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/rc.md +++ b/contrib/pg_tde/documentation/docs/release-notes/rc.md @@ -1,19 +1,18 @@ # pg_tde Release Candidate ({{date.RC}}) -`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../tde.md) to PostgreSQL and enables you to keep sensitive data safe and secure. +`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../index/index.md) to PostgreSQL and enables you to keep sensitive data safe and secure. [Get started](../install.md){.md-button} - ## Release Highlights This release provides the following features and improvements: -* **Improved performance with redesigned WAL encryption**. +* **Improved performance with redesigned WAL encryption**. The approach to WAL encryption has been redesigned. Now, `pg_tde` encrypts entire WAL files starting from the first WAL write after the server was started with the encryption turned on. The information about what is encrypted is stored in the internal key metadata. This change improves WAL encryption flow with native replication and increases performance for large scale databases. -* **Default encryption key for single-tenancy**. +* **Default encryption key for single-tenancy**. The new functionality allows you to set a default principal key for the entire database cluster. This key is used to encrypt all databases and tables that do not have a custom principal key set. This feature simplifies encryption configuration and management in single-tenant environments where each user has their own database instance. @@ -35,90 +34,86 @@ This release provides the following features and improvements: ## Upgrade considerations -`pg_tde` Release Candidate is not backward compatible with `pg_tde` Beta2 due to significant changes in code. This means you cannot directly upgrade from one version to another. You must [uninstall](../uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. +`pg_tde` Release Candidate is not backward compatible with `pg_tde` Beta2 due to significant changes in code. This means you cannot directly upgrade from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. ## Known issues -* The default `mlock` limit on Rocky Linux 8 for ARM64-based architectures equals the memory page size and is 64 Kb. This results in the child process with `pg_tde` failing to allocate another memory page because the max memory limit is reached by the parent process. +* The default `mlock` limit on Rocky Linux 8 for ARM64-based architectures equals the memory page size and is 64 Kb. This results in the child process with `pg_tde` failing to allocate another memory page because the max memory limit is reached by the parent process. - To prevent this, you can change the `mlock` limit to be at least twice bigger than the memory page size: + To prevent this, you can change the `mlock` limit to be at least twice bigger than the memory page size: - * temporarily for the current session using the `ulimit -l ` command. - * set a new hard limit in the `/etc/security/limits.conf` file. To do so, you require the superuser privileges. + * temporarily for the current session using the `ulimit -l ` command. + * set a new hard limit in the `/etc/security/limits.conf` file. To do so, you require the superuser privileges. Adjust the limits with caution since it affects other processes running in your system. * You can now delete global key providers even when their associated principal key is still in use. This known issue will be fixed in the next release. For now, avoid deleting global key providers. - - ## Changelog ### New Features -* [PG-1234](https://perconadev.atlassian.net/browse/PG-1234) - Added functions for separate global and database key management permissions +* [PG-1234](https://perconadev.atlassian.net/browse/PG-1234) - Added functions for separate global and database key management permissions. -* [PG-1255](https://perconadev.atlassian.net/browse/PG-1255) - Added functionality to delete key providers +* [PG-1255](https://perconadev.atlassian.net/browse/PG-1255) - Added functionality to delete key providers. -* [PG-1256](https://perconadev.atlassian.net/browse/PG-1256) - Added single-tenant support via the default principal key functionality +* [PG-1256](https://perconadev.atlassian.net/browse/PG-1256) - Added single-tenant support via the default principal key functionality. -* [PG-1258](https://perconadev.atlassian.net/browse/PG-1258) - Added functions to display additional information about principal keys / providers +* [PG-1258](https://perconadev.atlassian.net/browse/PG-1258) - Added functions to display additional information about principal keys / providers. -* [PG-1294](https://perconadev.atlassian.net/browse/PG-1294) - Redesigned WAL encryption +* [PG-1294](https://perconadev.atlassian.net/browse/PG-1294) - Redesigned WAL encryption. -* [PG-1303](https://perconadev.atlassian.net/browse/PG-1303) - Deprecated tde_heap_basic access method +* [PG-1303](https://perconadev.atlassian.net/browse/PG-1303) - Deprecated tde_heap_basic access method. ## Improvements -* [PG-858](https://perconadev.atlassian.net/browse/PG-858) - Refactored internal/principal key LWLocks to make local databases inherit a global key provider +* [PG-858](https://perconadev.atlassian.net/browse/PG-858) - Refactored internal/principal key LWLocks to make local databases inherit a global key provider. -* [PG-1243](https://perconadev.atlassian.net/browse/PG-1243) - Investigated performance issues at a specific threshold and large databases and updated documentation about handling hint bits +* [PG-1243](https://perconadev.atlassian.net/browse/PG-1243) - Investigated performance issues at a specific threshold and large databases and updated documentation about handling hint bits. -* [PG-1310](https://perconadev.atlassian.net/browse/PG-1310) - Added access method enforcement via the GUC variable +* [PG-1310](https://perconadev.atlassian.net/browse/PG-1310) - Added access method enforcement via the GUC variable. -* [PG-1361](https://perconadev.atlassian.net/browse/PG-1361) - Fixed pg_tde relocatability +* [PG-1361](https://perconadev.atlassian.net/browse/PG-1361) - Fixed pg_tde relocatability. -* [PG-1380](https://perconadev.atlassian.net/browse/PG-1380) - Added support for `pg_tde_is_encrypted()` function on indexes and sequences +* [PG-1380](https://perconadev.atlassian.net/browse/PG-1380) - Added support for `pg_tde_is_encrypted()` function on indexes and sequences. ### Bugs Fixed +* [PG-821](https://perconadev.atlassian.net/browse/PG-821) - Fixed the issue with `pg_basebackup` failing when configuring replication. -* [PG-821](https://perconadev.atlassian.net/browse/PG-821) - Fixed the issue with `pg_basebackup` failing when configuring replication +* [PG-847](https://perconadev.atlassian.net/browse/PG-847) - Fixed the issue with `pg_basebackup` and `pg_checksum` throwing an error on files created by `pg_tde` when the checksum is enabled on the database cluster. -* [PG-847](https://perconadev.atlassian.net/browse/PG-847) - Fixed the issue with `pg_basebackup` and `pg_checksum` throwing an error on files created by `pg_tde` when the checksum is enabled on the database cluster +* [PG-1004](https://perconadev.atlassian.net/browse/PG-1004) - Fixed the issue with `pg_checksums` utility failing during checksum verification on `pg_tde` tables. Now `pg_checksum` skips encrypted relations by looking if the relation has a custom storage manager (SMGR) key. -* [PG-1004](https://perconadev.atlassian.net/browse/PG-1004) - Fixed the issue with `pg_checksums` utility failing during checksum verification on `pg_tde` tables. Now `pg_checksum` skips encrypted relations by looking if the relation has a custom storage manager (SMGR) key +* [PG-1373](https://perconadev.atlassian.net/browse/PG-1373) - Fixed the issue with potential unterminated strings by using the `memcpy()` or `strlcpy()` instead of the `strncpy()` function. -* [PG-1373](https://perconadev.atlassian.net/browse/PG-1373) - Fixed the issue with potential unterminated strings by using the `memcpy()` or `strlcpy()` instead of the `strncpy()` function +* [PG-1378](https://perconadev.atlassian.net/browse/PG-1378) - Fixed the issue with toast tables created by the `ALTER TABLE` command not being encrypted. -* [PG-1378](https://perconadev.atlassian.net/browse/PG-1378) - Fixed the issue with toast tables created by the `ALTER TABLE` command not being encrypted +* [PG-1379](https://perconadev.atlassian.net/browse/PG-1379) - Fixed sequence and alter table handling in the event trigger. -* [PG-1379](https://perconadev.atlassian.net/browse/PG-1379) - Fixed sequence and alter table handling in the event trigger +* [PG-1222](https://perconadev.atlassian.net/browse/PG-1222) - Fixed the bug with confused relations with the same `RelFileNumber` in different databases. -* [PG-1222](https://perconadev.atlassian.net/browse/PG-1222) - Fixed the bug with confused relations with the same `RelFileNumber` in different databases +* [PG-1400](https://perconadev.atlassian.net/browse/PG-1400) - Corrected the pg_tde_change_key_provider naming in help. -* [PG-1400](https://perconadev.atlassian.net/browse/PG-1400) - Corrected the pg_tde_change_key_provider naming in help +* [PG-1401](https://perconadev.atlassian.net/browse/PG-1401) - Fixed the issue with inheriting an encryption status during the ALTER TABLE SET access method command execution by basing a new encryption status only on the new encryption setting. -* [PG-1401](https://perconadev.atlassian.net/browse/PG-1401) - Fixed the issue with inheriting an encryption status during the ALTER TABLE SET access method command execution by basing a new encryption status only on the new encryption setting - -* [PG-1414](https://perconadev.atlassian.net/browse/PG-1414) - Fixed the error message wording when configuring WAL encryption by referencing to a correct function +* [PG-1414](https://perconadev.atlassian.net/browse/PG-1414) - Fixed the error message wording when configuring WAL encryption by referencing to a correct function. * [PG-1450](https://perconadev.atlassian.net/browse/PG-1450) - Fixed the `pg_tde_delete_key_provider()` function behavior when called multiple times by ignoring already deleted records. * [PG-1451](https://perconadev.atlassian.net/browse/PG-1451) -Fixed the issue with the repeating error message about inability to retrieve a principal key even when a user creates non-encrypted tables by checking the current transaction ID in both the event trigger start function and during a file creation. If the transaction changed during the setup of the current event trigger data, the event trigger is reset. - -* [PG-1473](https://perconadev.atlassian.net/browse/PG-1473) - Allowed only users with key viewer privileges to execute `pg_tde_verify_principal_key()` and `pg_tde_verify_global_principal_key()` functions -* [PG-1474](https://perconadev.atlassian.net/browse/PG-1474) - Fixed the issue with the principal key reference corruption when reassigning it to a key provider with the same name by setting the key name in vault/kmip getters +* [PG-1473](https://perconadev.atlassian.net/browse/PG-1473) - Allowed only users with key viewer privileges to execute `pg_tde_verify_principal_key()` and `pg_tde_verify_global_principal_key()` functions. + +* [PG-1474](https://perconadev.atlassian.net/browse/PG-1474) - Fixed the issue with the principal key reference corruption when reassigning it to a key provider with the same name by setting the key name in vault/kmip getters. -* [PG-1476](https://perconadev.atlassian.net/browse/PG-1476) - Fixed the issue with the server failing to start when WAL encryption is enabled by creating a new principal key for WAL in case only one default key exists in the database +* [PG-1476](https://perconadev.atlassian.net/browse/PG-1476) - Fixed the issue with the server failing to start when WAL encryption is enabled by creating a new principal key for WAL in case only one default key exists in the database. -* [PG-1479](https://perconadev.atlassian.net/browse/PG-1479), [PG-1480](https://perconadev.atlassian.net/browse/PG-1480) - Fixed the issue with the lost access to data after the global key provider change and the server restart by fixing the incorrect parameter order in default key rotation +* [PG-1479](https://perconadev.atlassian.net/browse/PG-1479), [PG-1480](https://perconadev.atlassian.net/browse/PG-1480) - Fixed the issue with the lost access to data after the global key provider change and the server restart by fixing the incorrect parameter order in default key rotation. * [PG-1489](https://perconadev.atlassian.net/browse/PG-1489) - Fixed the issue with replicating the keys and key provider configuration by creating the `pg_tde` directory on the replica server. -/browse/PG-1476) - Fixed the issue with the server failing to start when WAL encryption is enabled by creating a new principal key for WAL in case only one default key exists in the database +/browse/PG-1476) - Fixed the issue with the server failing to start when WAL encryption is enabled by creating a new principal key for WAL in case only one default key exists in the database. -* [PG-1479](https://perconadev.atlassian.net/browse/PG-1479), [PG-1480](https://perconadev.atlassian.net/browse/PG-1480) - Fixed the issue with the lost access to data after the global key provider change and the server restart by fixing the incorrect parameter order in default key rotation +* [PG-1479](https://perconadev.atlassian.net/browse/PG-1479), [PG-1480](https://perconadev.atlassian.net/browse/PG-1480) - Fixed the issue with the lost access to data after the global key provider change and the server restart by fixing the incorrect parameter order in default key rotation. * [PG-1489](https://perconadev.atlassian.net/browse/PG-1489) - Fixed the issue with replicating the keys and key provider configuration by creating the `pg_tde` directory on the replica server. - diff --git a/contrib/pg_tde/documentation/docs/release-notes/rc2.md b/contrib/pg_tde/documentation/docs/release-notes/rc2.md new file mode 100644 index 0000000000000..3de65ba1311fa --- /dev/null +++ b/contrib/pg_tde/documentation/docs/release-notes/rc2.md @@ -0,0 +1,119 @@ +# pg_tde Release Candidate ({{date.RC2}}) + +`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../index/index.md) to PostgreSQL and enables you to keep sensitive data safe and secure. + +[Get started](../install.md){.md-button} + +## Release Highlights + +This release provides the following features and improvements: + +* **Improved performance with redesigned WAL encryption**. + + The approach to WAL encryption has been redesigned. Now, `pg_tde` encrypts entire WAL files starting from the first WAL write after the server was started with the encryption turned on. The information about what is encrypted is stored in the internal key metadata. This change improves WAL encryption flow with native replication and increases performance for large scale databases. + +* **Default encryption key for single-tenancy**. + + The new functionality allows you to set a default principal key for the entire database cluster. This key is used to encrypt all databases and tables that do not have a custom principal key set. This feature simplifies encryption configuration and management in single-tenant environments where each user has their own database instance. + +* **Ability to change key provider configuration** + + You no longer need to configure a new key provider and set a new principal key if the provider's configuration changed. Now can change the key provider configuration both for the current database and the entire PostgreSQL cluster using [functions](../functions.md#key-provider-management). This enhancement lifts existing limitations and is a native and common way to operate in PostgreSQL. + +* **Key management permissions** + + The new functions allow you to manage permissions for global and database key management separately. This feature provides more granular control over key management operations and allows you to delegate key management tasks to different roles. + +* **Additional information about principal keys and providers** + + The new functions allow you to display additional information about principal keys and providers. This feature helps you to understand the current key configuration and troubleshoot issues related to key management. + +* **`tde_heap_basic` access method deprecation** + + The `tde_heap_basic` access method has limitations in encryption capabilities and affects performance. Also, it poses a potential security risk when used in production environments due to indexes remaining unencrypted. Considering all the above, we decided to deprecate this access method and remove it in future releases. Use the `tde_heap` access method instead that is available with Percona Server for PostgreSQL 17 - a drop-in replacement for PostgreSQL Community. + +## Upgrade considerations + +`pg_tde` Release Candidate is not backward compatible with `pg_tde` Beta2 due to significant changes in code. This means you cannot directly upgrade from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. + +## Known issues + +* The default `mlock` limit on Rocky Linux 8 for ARM64-based architectures equals the memory page size and is 64 Kb. This results in the child process with `pg_tde` failing to allocate another memory page because the max memory limit is reached by the parent process. + + To prevent this, you can change the `mlock` limit to be at least twice bigger than the memory page size: + + * temporarily for the current session using the `ulimit -l ` command. + * set a new hard limit in the `/etc/security/limits.conf` file. To do so, you require the superuser privileges. + + Adjust the limits with caution since it affects other processes running in your system. + +* You can now delete global key providers even when their associated principal key is still in use. This known issue will be fixed in the next release. For now, avoid deleting global key providers. + +## Changelog + +### New Features + +* [PG-1234](https://perconadev.atlassian.net/browse/PG-1234) - Added functions for separate global and database key management permissions. + +* [PG-1255](https://perconadev.atlassian.net/browse/PG-1255) - Added functionality to delete key providers. + +* [PG-1256](https://perconadev.atlassian.net/browse/PG-1256) - Added single-tenant support via the default principal key functionality. + +* [PG-1258](https://perconadev.atlassian.net/browse/PG-1258) - Added functions to display additional information about principal keys / providers. + +* [PG-1294](https://perconadev.atlassian.net/browse/PG-1294) - Redesigned WAL encryption. + +* [PG-1303](https://perconadev.atlassian.net/browse/PG-1303) - Deprecated tde_heap_basic access method. + +## Improvements + +* [PG-858](https://perconadev.atlassian.net/browse/PG-858) - Refactored internal/principal key LWLocks to make local databases inherit a global key provider. + +* [PG-1243](https://perconadev.atlassian.net/browse/PG-1243) - Investigated performance issues at a specific threshold and large databases and updated documentation about handling hint bits. + +* [PG-1310](https://perconadev.atlassian.net/browse/PG-1310) - Added access method enforcement via the GUC variable. + +* [PG-1361](https://perconadev.atlassian.net/browse/PG-1361) - Fixed pg_tde relocatability. + +* [PG-1380](https://perconadev.atlassian.net/browse/PG-1380) - Added support for `pg_tde_is_encrypted()` function on indexes and sequences. + +### Bugs Fixed + +* [PG-821](https://perconadev.atlassian.net/browse/PG-821) - Fixed the issue with `pg_basebackup` failing when configuring replication. + +* [PG-847](https://perconadev.atlassian.net/browse/PG-847) - Fixed the issue with `pg_basebackup` and `pg_checksum` throwing an error on files created by `pg_tde` when the checksum is enabled on the database cluster. + +* [PG-1004](https://perconadev.atlassian.net/browse/PG-1004) - Fixed the issue with `pg_checksums` utility failing during checksum verification on `pg_tde` tables. Now `pg_checksum` skips encrypted relations by looking if the relation has a custom storage manager (SMGR) key. + +* [PG-1373](https://perconadev.atlassian.net/browse/PG-1373) - Fixed the issue with potential unterminated strings by using the `memcpy()` or `strlcpy()` instead of the `strncpy()` function. + +* [PG-1378](https://perconadev.atlassian.net/browse/PG-1378) - Fixed the issue with toast tables created by the `ALTER TABLE` command not being encrypted. + +* [PG-1379](https://perconadev.atlassian.net/browse/PG-1379) - Fixed sequence and alter table handling in the event trigger. + +* [PG-1222](https://perconadev.atlassian.net/browse/PG-1222) - Fixed the bug with confused relations with the same `RelFileNumber` in different databases. + +* [PG-1400](https://perconadev.atlassian.net/browse/PG-1400) - Corrected the pg_tde_change_key_provider naming in help. + +* [PG-1401](https://perconadev.atlassian.net/browse/PG-1401) - Fixed the issue with inheriting an encryption status during the ALTER TABLE SET access method command execution by basing a new encryption status only on the new encryption setting. + +* [PG-1414](https://perconadev.atlassian.net/browse/PG-1414) - Fixed the error message wording when configuring WAL encryption by referencing to a correct function. + +* [PG-1450](https://perconadev.atlassian.net/browse/PG-1450) - Fixed the `pg_tde_delete_key_provider()` function behavior when called multiple times by ignoring already deleted records. + +* [PG-1451](https://perconadev.atlassian.net/browse/PG-1451) -Fixed the issue with the repeating error message about inability to retrieve a principal key even when a user creates non-encrypted tables by checking the current transaction ID in both the event trigger start function and during a file creation. If the transaction changed during the setup of the current event trigger data, the event trigger is reset. + +* [PG-1473](https://perconadev.atlassian.net/browse/PG-1473) - Allowed only users with key viewer privileges to execute `pg_tde_verify_principal_key()` and `pg_tde_verify_global_principal_key()` functions. + +* [PG-1474](https://perconadev.atlassian.net/browse/PG-1474) - Fixed the issue with the principal key reference corruption when reassigning it to a key provider with the same name by setting the key name in vault/kmip getters. + +* [PG-1476](https://perconadev.atlassian.net/browse/PG-1476) - Fixed the issue with the server failing to start when WAL encryption is enabled by creating a new principal key for WAL in case only one default key exists in the database. + +* [PG-1479](https://perconadev.atlassian.net/browse/PG-1479), [PG-1480](https://perconadev.atlassian.net/browse/PG-1480) - Fixed the issue with the lost access to data after the global key provider change and the server restart by fixing the incorrect parameter order in default key rotation. + +* [PG-1489](https://perconadev.atlassian.net/browse/PG-1489) - Fixed the issue with replicating the keys and key provider configuration by creating the `pg_tde` directory on the replica server. +/browse/PG-1476) - Fixed the issue with the server failing to start when WAL encryption is enabled by creating a new principal key for WAL in case only one default key exists in the database. + +* [PG-1479](https://perconadev.atlassian.net/browse/PG-1479), [PG-1480](https://perconadev.atlassian.net/browse/PG-1480) - Fixed the issue with the lost access to data after the global key provider change and the server restart by fixing the incorrect parameter order in default key rotation. + +* [PG-1489](https://perconadev.atlassian.net/browse/PG-1489) - Fixed the issue with replicating the keys and key provider configuration by creating the `pg_tde` directory on the replica server. diff --git a/contrib/pg_tde/documentation/docs/release-notes/release-notes.md b/contrib/pg_tde/documentation/docs/release-notes/release-notes.md index d5228f15d3e77..e383f05c3aab9 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/release-notes.md +++ b/contrib/pg_tde/documentation/docs/release-notes/release-notes.md @@ -1,11 +1,9 @@ # `pg_tde` release notes index -`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../tde.md) to PostgreSQL and enables you to keep sensitive data safe and secure. +`pg_tde` extension brings in [Transparent Data Encryption (TDE)](../index/index.md) to PostgreSQL and enables you to keep sensitive data safe and secure. * [pg_tde Release Candidate {{date.RC}}](rc.md) * [pg_tde Beta2 (2024-12-16)](beta2.md) * [pg_tde Beta (2024-06-30)](beta.md) * [pg_tde Alpha1 (2024-03-28)](alpha1.md) -* [pg_tde MVP (2023-12-12)](mvp.md) - - +* [pg_tde MVP (2023-12-12)](mvp.md) diff --git a/contrib/pg_tde/documentation/docs/replication.md b/contrib/pg_tde/documentation/docs/replication.md new file mode 100644 index 0000000000000..b745107a2f222 --- /dev/null +++ b/contrib/pg_tde/documentation/docs/replication.md @@ -0,0 +1,4 @@ +# Replication + +