percona · Andriciuc · May 8, 2025 · Apr 17, 2025 · Apr 17, 2025 · Apr 22, 2025
@@ -0,0 +1,7 @@
+# Technical Reference
+
+This section covers the internal components and tools that power `pg_tde`.
+
+Use it to understand how encryption is implemented, fine-tune a configuration, leverage advanced CLI tools and functions for diagnostics and customization.
+
+[Architecture](../architecture/index.md){.md-button} [GUC Variables](../variables.md){.md-button} [Functions](../functions.md){.md-button}
@@ -1,55 +1,54 @@
-# Install `pg_tde` on Debian or Ubuntu
+# Install pg_tde on Debian or Ubuntu
 
-This tutorial shows how to install `pg_tde` with [Percona Distribution for PostgreSQL :octicons-link-external-16:](https://docs.percona.com/postgresql/latest/index.html).
+This page explains how to install `pg_tde` with [Percona Distribution for PostgreSQL :octicons-link-external-16:](https://docs.percona.com/postgresql/latest/index.html).
 
-Check the [list of supported platforms](install.md#__tabbed_1_1).
+Check the [list of supported platforms](install.md#__tabbed_1_1) before continuing.
 
 ## Preconditions
 
 1. Debian and other systems that use the `apt` package manager include the upstream PostgreSQL server package (`postgresql-{{pgversion17}}`) by default. You need to uninstall this package before you install Percona Server for PostgreSQL and `pg_tde` to avoid conflicts.
 2. You need the `percona-release` repository management tool that enables the desired Percona repository for you.
 
-
-### Install `percona-release` {.power-number}
+### Install percona-release {.power-number}
 
 1. You need the following dependencies to install `percona-release`:
-    
+
     - `wget`
     - `gnupg2`
     - `curl`
     - `lsb-release`
-    
+
     Install them with the following command:
-    
+
     ```{.bash data-prompt="$"}
-    $ sudo apt-get install -y wget gnupg2 curl lsb-release
+        $ sudo apt-get install -y wget gnupg2 curl lsb-release
     ```
-    
+
 2. Fetch the `percona-release` package
 
     ```{.bash data-prompt="$"}
-    $ sudo wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb
+        $ sudo wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb
     ```
 
 3. Install `percona-release`
 
     ```{.bash data-prompt="$"}
-    $ sudo dpkg -i percona-release_latest.generic_all.deb
+        $ sudo dpkg -i percona-release_latest.generic_all.deb
     ```
 
 4. Enable the Percona Distribution for PostgreSQL repository
 
     ```{.bash data-prompt="$"}
-    $ sudo percona-release enable-only ppg-{{pgversion17}} 
+        $ sudo percona-release enable-only ppg-{{pgversion17}} 
     ```
 
 6. Update the local cache
 
     ```bash
-    sudo apt-get update
+        sudo apt-get update
     ```
 
-## Install `pg_tde` {.power-number}
+## Install pg_tde {.power-number}
 
 !!! important
 
@@ -60,19 +59,17 @@ Check the [list of supported platforms](install.md#__tabbed_1_1).
         The use of the `CASCADE` parameter deletes all tables that were created in the database with `pg_tde` enabled and also all dependencies upon the encrypted table (e.g. foreign keys in a non-encrypted table used in the encrypted one).     
 
         ```sql
-        DROP EXTENSION pg_tde CASCADE
+            DROP EXTENSION pg_tde CASCADE;
         ```
 
     2. Uninstall the `percona-postgresql-17-pg-tde` package.  
 
 After all [preconditions](#preconditions) are met, run the following command to install `pg_tde`:
 
-
 ```{.bash data-prompt="$"}
-$ sudo apt-get install -y percona-postgresql-17 
+    $ sudo apt-get install -y percona-postgresql-17 
 ```
 
+## Next steps
 
-## Next step 
-
-[Setup :material-arrow-right:](setup.md){.md-button}
+[Configure pg_tde :material-arrow-right:](setup.md){.md-button}
@@ -12,7 +12,7 @@ Let's break down what it means.
 * Storing every key on the same key storage, or using different storages for different databases
 * Handling permissions: who can manage database specific or global permissions, who can create encrypted or not encrypted tables
 
-**Complete** means that `pg_tde` aims to encrypt data at rest. 
+**Complete** means that `pg_tde` aims to encrypt data at rest.
 
 **Data at rest** means everything written to the disk. This includes the following:
 
@@ -35,10 +35,10 @@ The main components of `pg_tde` are the following:
     [Percona Server for PostgreSQL location](https://github.com/percona/postgres/tree/{{tdebranch}})
 
 * The **`pg_tde` extension itself** implements the encryption code by hooking into the extension points introduced in the core changes, and the already existing extension points in the PostgreSQL server.
-   
+
     Everything is controllable with GUC variables and SQL statements, similar to other extensions.
 
-* The **keyring API / libraries** implement the key storage logic with different key providers. The API is internal only, the keyring libraries are part of the main library for simplicity. 
+* The **keyring API / libraries** implement the key storage logic with different key providers. The API is internal only, the keyring libraries are part of the main library for simplicity.
 In the future these could be extracted into separate shared libraries with an open API, allowing the use of third-party providers.
 
 ## Encryption architecture
@@ -48,7 +48,7 @@ In the future these could be extracted into separate shared libraries with an op
 `pg_tde` uses two kinds of keys for encryption:
 
 * Internal keys to encrypt the data. They are stored in PostgreSQL's data directory under `$PGDATA/pg_tde``.
-* Higher-level keys to encrypt internal keys. These keys are called "principal keys". They are stored externally, in a Key Management System (KMS) using the key provider API. 
+* Higher-level keys to encrypt internal keys. These keys are called "principal keys". They are stored externally, in a Key Management System (KMS) using the key provider API.
 
 `pg_tde` uses one principal key per database. Every internal key for the given database is encrypted using this principal key.
 
@@ -74,7 +74,7 @@ Support for other cipher lengths / algorithms is planned in the future.
 
 `pg_tde` makes it possible to encrypt everything or only some tables in some databases.
 
-To support this without metadata changes, encrypted tables are labeled with a `tde_heap` access method marker. 
+To support this without metadata changes, encrypted tables are labeled with a `tde_heap` access method marker.
 
 The `tde_heap` access method is the same as the `heap` one. It uses the same functions internally without any changes, but with the different name and ID. In such a way `pg_tde` knows that `tde_heap` tables are encrypted and `heap` tables are not.
 
@@ -162,11 +162,11 @@ Retreival of the principal key is cached so it only happens when necessary.
 
 ### Key provider management
 
-Key provider configuration or location may change. For example, a service is moved to a new address or the principal key must be moved to a different key provider type. `pg_tde` supports both these scenarios enabling you to manage principal keys using simple [SQL functions](functions.md#key-provider-management).
+Key provider configuration or location may change. For example, a service is moved to a new address or the principal key must be moved to a different key provider type. `pg_tde` supports both these scenarios enabling you to manage principal keys using simple [SQL functions](../functions.md#key-provider-management).
 
-In certain cases you can't use SQL functions to manage key providers. For example, if the key provider changed while the server wasn't running and is therefore unaware of these changes. The startup can fail if it needs to access the encryption keys. 
+In certain cases you can't use SQL functions to manage key providers. For example, if the key provider changed while the server wasn't running and is therefore unaware of these changes. The startup can fail if it needs to access the encryption keys.
 
-For such situations, `pg_tde` also provides [command line tools](command-line-tools.md) to recover the database.
+For such situations, `pg_tde` also provides [command line tools](../command-line-tools/index.md) to recover the database.
 
 ### Sensitive key provider information
 
@@ -242,7 +242,7 @@ This is also the reason why it requires a `dbOid` instead of a name, as it has n
 
 ### Deleting providers
 
-Providers can be deleted by the 
+Providers can be deleted by using the
 
 ```sql
 pg_tde_delete_database_key_provider(provider_name)
@@ -255,7 +255,7 @@ For database specific providers, the function first checks if the provider is us
 
 For global providers, the function checks if the provider is used anywhere, WAL or any specific database, and returns an error if it is.
 
-This somewhat goes against the principle that `pg_tde` shouldn't interact with other databases than the one the user is connected to, but on the other hand, it only does this lookup in the internal `pg_tde` metadata, not in postgres catalogs, so it is a gray zone. Making this check makes more sense than potentially making some databases inaccessible.
+This somewhat goes against the principle that `pg_tde` should not interact with other databases than the one the user is connected to, but on the other hand, it only does this lookup in the internal `pg_tde` metadata, not in postgres catalogs, so it is a gray zone. Making this check makes more sense than potentially making some databases inaccessible.
 
 ### Listing/querying providers
 
@@ -365,7 +365,7 @@ encryption is managed by the admins, normal users only have to create tables wit
 2. `CREATE EXTENSION pg_tde;` in `template1`
 3. Adding a global key provider
 4. Changing the WAL encryption to use the proper global key provider
-5. Giving users that are expected to manage database keys permissions for database specific key management, but not database specific key provider management: 
+5. Giving users that are expected to manage database keys permissions for database specific key management, but not database specific key provider management:
    specific databases HAVE to use the global key provider
 
 Note: setting the `default_table_access_method` to `tde_heap` is possible, but instead of `ALTER SYSTEM` only per database using `ALTER DATABASE`, after a principal key is configured for that specific database.
@@ -376,8 +376,8 @@ Alternatively `ALTER SYSTEM` is possible, but table creation in the database wil
 
 1. Installing the extension: `shared_preload_libraries` + `pg_tde.wal_encrypt` (that's not multi tenant currently)
 2. `CREATE EXTENSION pg_tde;` in any database
-2. Adding a global key provider for WAL
-3. Changing the WAL encryption to use the proper global key provider
+3. Adding a global key provider for WAL
+4. Changing the WAL encryption to use the proper global key provider
 
 No default configuration: key providers / principal keys are configured as a per database level, permissions are managed per database
 

@@ -0,0 +1,7 @@
+# pg_tde CLI Tools
+
+The `pg_tde` extension introduces new command-line utilities and extends some existing PostgreSQL tools to support encrypted WAL and tables. These include:
+
+* [pg_tde_change_key_provider](../command-line-tools/pg-tde-change-key-provider.md): change encryption key provider for a database
+* [pg_waldump](../command-line-tools/pg-waldump.md): inspect and decrypt WAL files
+* [pg_checksums](../command-line-tools/pg-tde-checksums.md): verify data checksums (non-encrypted files only)
@@ -0,0 +1,36 @@
+# pg_tde_change_key_provider
+
+A tool for modifying the configuration of a key provider, possibly also changing its type.
+
+This tool edits the configuration files directly, ignoring permissions or running `postgres` processes.
+
+Its only intended use is to fix servers that can't start up because of inaccessible key providers.
+
+For example, you restore from an old backup and the address of the key provider changed in the meantime. You can use this tool to correct the configuration, allowing the server to start up.
+
+<i warning>:material-information: Warning:</i> Use this tool **only when the server is offline.** To modify the key provider configuration when the server is up and running, use the [`pg_tde_change_(global/database)_key_provider_<type>`](../functions.md#change-an-existing-provider) SQL functions.
+
+## Example usage
+
+To modify the key provider configuration, specify all parameters depending on the provider type in the same way as you do when using the [`pg_tde_change_(global/database)_key_provider_<type>`](../functions.md#change-an-existing-provider) SQL functions.
+
+The general syntax is as follows:
+
+```bash
+pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> <new_provider_type> <provider_parameters...>
+```
+
+## Parameter description
+
+* [optional] `<datadir>` is the data directory.`pg_tde` uses the `$PGDATA` environment variable if this is not specified
+* `<provider_name>` is the name you assigned to the key provider
+* `<new_provider_type>` can be a `file`, `vault` or `kmip`
+* `<dbOid>`
+
+Depending on the provider type, the additional parameters are:
+
+```bash
+pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> file <filename>
+pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> vault <token> <url> <mount_path> [<ca_path>]
+pg_tde_change_key_provider [-D <datadir>] <dbOid> <provider_name> kmip <host> <port> <cert_path> <key_path> [<ca_path>] 
+```
@@ -0,0 +1,5 @@
+# pg_checksums
+
+[`pg_checksums` :octicons-link-external-16:](https://www.postgresql.org/docs/current/app-pgchecksums.html) is a PostgreSQL command-line utility used to enable, disable, or verify data checksums on a PostgreSQL data directory. However, it cannot calculate checksums for encrypted files.
+
+Encrypted files are skipped, and this is reported in the output.
@@ -0,0 +1,14 @@
+# pg_waldump
+
+[`pg_waldump` :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the Write-Ahead Log (WAL) of a PostgreSQL database cluster.
+
+To read encrypted WAL records, `pg_waldump` supports the following additional arguments:
+
+* `keyring_path`: the directory where keyring configuration files for WAL are stored. These files include:
+  * `pg_tde.map`
+  * `pg_tde.dat`
+  * `pg_tde_keyrings`
+
+!!! note
+
+    `pg_waldump` will not decrypt WAL unless the `keyring_path` is set.
@@ -1,4 +1,4 @@
-# Contributing guide
+# Contributing Guide
 
 Welcome to `pg_tde` - the Transparent Data Encryption extension for PostgreSQL!