Merge pull request #6130 from EnterpriseDB/TDE/refresh

TDE refresh

Author: gvasquezvargas

product_docs/docs/postgres_distributed_for_kubernetes/1/tde.mdx

@@ -33,7 +33,7 @@ We show now how to use TDE with a passphrase stored in a Kubernetes Secret,
 which will be used to encrypt the EPAS binary key.
 
 !!! Seealso "EPAS documentation"
-    Please refer to [the EPAS documentation](https://www.enterprisedb.com/docs/tde/latest/key_stores/)
+    Please refer to [the EPAS documentation](https://www.enterprisedb.com/docs/tde/latest/secure_key/)
     for details on the EPAS encryption key.
 
 TDE on EDB Postgres Distributed for Kubernetes relies on the PG4K

product_docs/docs/postgres_for_kubernetes/1/tde.mdx

@@ -41,7 +41,7 @@ The basic approach is to store the passphrase in a Kubernetes secret. Such a
 passphrase will be used to encrypt the EPAS binary key.
 
 !!! Seealso "EPAS documentation"
-    Please refer to [the EPAS documentation](/tde/latest/key_stores/)
+    Please refer to [the EPAS documentation](/tde/latest/secure_key/)
     for details on the EPAS encryption key.
 
 Activating TDE on the operator is simple. In the `epas` section of the manifest,

product_docs/docs/tde/15/about/how.mdx

@@ -0,0 +1,51 @@
+---
+title: How does TDE encrypt data?
+description: How does the encryption of data work when TDE is enabled?
+---
+
+TDE prevents unauthorized viewing of data in operating system files on the database server and on backup storage. Data becomes unintelligible for unauthorized users if it's stolen or misplaced.
+
+Data encryption and decryption is managed by the database and doesn't require application changes or updated client drivers.
+
+EDB Postgres Advanced Server and EDB Postgres Extended Server provide hooks to key management that's external to the database. These hooks allow for simple passphrase encrypt/decrypt or integration with enterprise key management solutions. See [Securing the data encryption key](../secure_key/) for more information.
+
+## How does TDE encrypt data?
+
+EDB TDE uses [OpenSSL](https://openssl-library.org/) to encrypt data files with the AES encryption algorithm. In Windows systems, TDE uses [OpenSSL 3](https://docs.openssl.org/3.0/). In Linux systems, TDE uses the OpenSSL version installed in the host operating system. To check the installed version, run `openssl version`. For more information, see the [OpenSSL documentation](https://docs.openssl.org/master/). If you're using a custom build not provided by the OpenSSL community, consult your vendor's documentation.
+
+Starting with version 16, EDB TDE introduces the option to choose between AES-128 and AES-256 encryption algorithms during the initialization of the Postgres cluster. The choice between AES-128 and AES-256 hinges on balancing performance and security requirements. AES-128 is commonly advised for environments where performance efficiency and lower power consumption are pivotal, making it suitable for most applications. Conversely, AES-256 is recommended for scenarios demanding the highest level of security, often driven by regulatory mandates.
+
+TDE uses AES-128-XTS or AES-256-XTS algorithms for encrypting data files. XTS uses a second value, known as the *tweak value*, to enhance the encryption. The XTS tweak value with TDE uses the database OID, the relfilenode, and the block number. 
+
+For write-ahead log (WAL) files, TDE uses AES-128-CTR or AES-256-CTR, incorporating the WAL's log sequence number (LSN) as the counter component. 
+
+Temporary files that are accessed by block are also encrypted using AES-128-XTS or AES-256-XTS. Other temporary files are encrypted using AES-128-CBC or AES-256-CBC.
+
+## How is data stored on disk with TDE?
+
+In this example, the data in the `tbfoo` table is encrypted. The `pg_relation_filepath` function locates the data file corresponding to the `tbfoo` table.
+
+```sql
+insert into tbfoo values ('abc','123');
+INSERT 0 1
+
+select pg_relation_filepath('tbfoo');
+
+ pg_relation_filepath
+----------------------
+ base/5/16416
+```
+
+Grepping the data looking for characters doesn't return anything. Viewing the last five lines returns the encrypted data:
+
+```shell
+$ hexdump -C 16416 | grep abc
+$
+
+$ hexdump -C 16416 | tail -5
+00001fc0  c8 0f 1d c8 9a 63 3d dc  7d 4e 68 98 b8 f2 5e 0a  |.....c=.}Nh...^.|
+00001fd0  9a eb 20 1d 59 ad be 94  6e fd d5 6e ed 0a 72 8c  |.. .Y...n..n..r.|
+00001fe0  7b 14 7f de 5b 63 e3 84  ba 6c e7 b0 a3 86 aa b9  |{...[c...l......|
+00001ff0  fe 4f 07 50 06 b7 ef 6a  cd f9 84 96 b2 4b 25 12  |.O.P...j.....K%.|
+00002000
+```

product_docs/docs/tde/15/about/index.mdx

@@ -0,0 +1,9 @@
+---
+title: About TDE
+description: Learn about TDE, how it works, what it encrypts and why to use it.
+indexCards: simple
+---
+
+Transparent data encryption (TDE) is an optional feature supported by EDB Postgres Advanced Server and EDB Postgres Extended Server from version 15.
+
+It encrypts user data stored in the database system.

product_docs/docs/tde/15/about/what.mdx

@@ -0,0 +1,37 @@
+---
+title: What's encrypted with TDE?
+description: Which data is encrypted when databases are initialized with TDE?
+---
+
+TDE encrypts:
+
+- The files underlying tables, sequences, indexes, including TOAST tables and system catalogs, and including all forks. These files are known as *data files*.
+
+- The write-ahead log (WAL). 
+
+- Various temporary files that are used during query processing and database system operation.
+
+!!! Note Implications
+
+    - Any WAL fetched from a server using TDE, including by streaming replication and archiving, is encrypted.
+
+    - A physical replica is necessarily encrypted (or not encrypted) in the same way and using the same keys as its primary server.
+
+    - If a server uses TDE, a base backup is automatically encrypted. 
+
+
+The following aren't encrypted or otherwise disguised by TDE:
+
+- Metadata internal to operating the database system that doesn't contain user data, such as the transaction status (for example, pg_subtrans and pg_xact).
+
+- The file names and file system structure in the data directory. That means that the overall size of the database system, the number of databases, the number of tables, their relative sizes, as well as file system metadata such as last access time are all visible without decryption.
+
+- Data in foreign tables.
+
+- The server diagnostics log.
+
+- Configuration files.
+
+!!! Note Implications
+
+    Logical replication isn't affected by TDE. Publisher and subscriber can have different encryption settings. The payload of the logical replication protocol isn't encrypted. (You can use SSL.)

product_docs/docs/tde/15/about/why.mdx

@@ -0,0 +1,22 @@
+---
+title: Why should you use TDE?
+description: Learn about some of the use cases for TDE encryption.
+---
+
+TDE encryption ensures that user data remains protected from unauthorized access.
+
+When configured with a [data encryption key securing mechanism](../secure_key/), data stored on the database server and in backup is only accessible by users and processes with decryption keys.
+
+Some use cases include: 
+
+- **Protection of sensitive personal data:** Industries like finance, e-commerce, healthcare, and government organizations often deal with personally identifiable information that must be protected to comply with data privacy regulations such as GDPR, HIPPA, PCI DSS.
+
+- **Compliance with government standards:** Government institutions must comply with information processing standards like FIPS to ensure computer security and interoperability.
+
+- **Protection of transactional data:** Financial institutions deal with transaction, account, and payment data that must be protected to prevent fraud and financial losses.
+
+- **Protection of intellectual property:** Organizations safeguard proprietary information, designs, and plans to keep their competitive advantage, support brand value, and foster innovation.
+
+- **Protection of data in cloud-based deployments and public web applications:** Encrypting a database's data provides an additional layer of security when infrastructure is shared, or when vulnerabilities could potentially infiltrate in an application's API.
+
+When your data is encrypted, it becomes unintelligible if it's stolen or misplaced.

product_docs/docs/tde/15/affected_commands.mdx

@@ -1,16 +1,13 @@
 ---
 title: "Commands affected by TDE"
-navTitle: Affected commands
+description: Some commands work differently when TDE is enabled. Learn about differences.
 ---
 
-
-
 When TDE is enabled, the following commands have TDE-specific options or read TDE settings in environment variables or configuration files:
 
-- [pg_waldump](/tde/latest/troubleshooting/#dumping-a-tde-encrypted-wal-file) 
-- [pg_resetwal](/tde/latest/troubleshooting/#resetting-a-corrupt-tde-encrypted-wal-file)
-- [pg_verifybackup](/tde/latest/backups/#verify-a-backup-of-a-tde-system)
-- [pg_rewind](/tde/latest/backups/#resynchronize-timelines-in-a-tde-system)
-- [pg_upgrade](pg_upgrade_arguments)
-- [postgres](/tde/latest/single_user/)
-
+- [initdb](./initdb_tde_options/)
+- [pg_waldump](././encrypted_files/wal_files/#dumping-a-tde-encrypted-wal-file)
+- [pg_resetwal](././encrypted_files/wal_files/#resetting-a-corrupt-tde-encrypted-wal-file)
+- [pg_verifybackup](././encrypted_files/backup_files/#verify-a-backup-of-a-tde-system)
+- [pg_rewind](././encrypted_files/backup_files/#resynchronize-timelines-in-a-tde-system)
+- [pg_upgrade](pg_upgrade_arguments)

product_docs/docs/tde/15/enabling/enabling_tde.mdx

@@ -0,0 +1,49 @@
+---
+title: "Creating a database with TDE" 
+description: Create a database server with TDE enabled.
+---
+
+Create a new EDB Postgres Advanced Server cluster with TDE enabled. 
+
+- Set the environment variables to export the `wrap` and `unwrap` commands for encryption. 
+- Initialize a server with encryption enabled.
+- Start the database server.
+- Verify TDE is enabled.
+
+## Worked example
+
+This example uses EDB Postgres Advanced Server 15 running on a Linux platform. It uses openssl to define the passphrase to wrap and unwrap the generated data encryption key.
+
+1. Set the data encryption key (wrap) and decryption (unwrap) environment variables:
+
+   ```shell
+   export PGDATAKEYWRAPCMD='openssl enc -e -aes-128-cbc -pass pass:ok -out %p'
+   export PGDATAKEYUNWRAPCMD='openssl enc -d -aes-128-cbc -pass pass:ok -in %p'
+   ```
+   !!!note
+      - If you are on Windows you don't need the single quotes around the variable value.
+      
+      - Ensure you replace `ok` with the passphrase you want to use to wrap the data encryption key.
+   !!!
+
+1. Initialize the cluster using `initdb` with encryption enabled. This command sets the `data_encryption_key_unwrap_command` parameter in the postgresql.conf file.
+
+   ```shell
+   /usr/edb/as15/bin/initdb --data-encryption -D /var/lib/edb/as15/data 
+   ```
+   
+1. Start the cluster:
+
+   ```shell
+   /usr/edb/as15/bin/pg_ctl -D /var/lib/edb/as15/data start
+   ```
+
+1. Run grep on postgresql.conf to verify the setting of `data_encryption_key_unwrap_command`: 
+
+   ```shell
+   grep data_encryption_key_unwrap_command /var/lib/edb/as15/data/postgresql.conf
+   __OUTPUT__
+   data_encryption_key_unwrap_command = 'openssl enc -d -aes-128-cbc -pass pass:ok -in %p'
+   ```
+
+1.  [Verify that data encryption is enabled](verifying_tde).

product_docs/docs/tde/15/enabling_tde_epas.mdx renamed to product_docs/docs/tde/15/enabling/enabling_tde_epas.mdx

@@ -1,11 +1,11 @@
 ---
-title: "Enabling TDE on an existing EDB Postgres Advanced Server cluster"
-navTitle: Enabling TDE on an existing EDB Postgres Advanced Server cluster
+title: "Enabling TDE on an existing EDB Postgres Advanced Server"
+description: Migrate your existing EDB Postgres Advanced Server to a new TDE-enabled database server.
 deepToC: true
+redirects:
+  - /tde/latest/enabling_tde_epas/ #generated for TDE/refresh
 ---
 
-## Enabling TDE on an EDB Postgres Advanced Server cluster
-
 Create a new EDB Postgres Advanced Server cluster with TDE enabled 
 and use `pg_upgrade` to transfer data from the existing source cluster to the new encrypted cluster. 
 
@@ -61,7 +61,7 @@ Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [
     !!!note
        Alternatively, use the `--key-unwrap-command=<command>` and `--key-wrap-command=<command>` arguments when initializing the encrypted server to include the `wrap` and `unwrap` commands.
 
-       See [Using initdb TDE options](enabling_tde/#using-initdb-tde-options) for more information on possible configurations.
+       See [Using initdb TDE options](../initdb_tde_options/) for more information on possible configurations.
 
 1.  Initialize the new server with encryption: 
 
@@ -92,7 +92,7 @@ Use [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html), [
     !!!note
        If you're using two different Postgres versions, use the psql utility of the encrypted server. Otherwise, the system will attempt to use psql from the previous instance.
 
-1.  To ensure the new server is encrypted, [check for TDE presence](enabling_tde/#checking-for-tde-presence-using-sql). 
+1.  To ensure the new server is encrypted, [check for TDE presence](verifying_tde). 
 
 ### Upgrading to the encrypted server
 

product_docs/docs/tde/15/enabling/index.mdx

@@ -0,0 +1,12 @@
+---
+title: Tutorials
+description: Review some examples of how to create a TDE-enabled database server.
+indexCards: simple
+navigation: 
+ - enabling_tde
+ - enabling_tde_epas
+---
+
+Create a TDE-enabled database server using `initdb`.
+
+Or migrate an existing database instance by creating a TDE-enabled database server with `initdb` and then migrating data with `pg_upgrade`.

product_docs/docs/tde/15/upgrade_use_cases/postgres_to_extended.mdx renamed to product_docs/docs/tde/15/enabling/postgres_to_extended.mdx

@@ -1,7 +1,10 @@
 ---
 title: "Upgrading PostgreSQL to EDB Postgres Extended Server while enabling TDE"
 navTitle: Upgrading PostgreSQL to EDB Postgres Extended Server
+description: Use pg_upgrade to upgrade the database version, change the Postgres distribution, or migrate to a TDE-enabled database.
 deepToC: true
+redirects:
+  - /tde/latest/upgrade_use_cases/postgres_to_extended/ #generated for TDE/refresh
 ---
 
 Create a new EDB Postgres Extended Server cluster with TDE enabled and use pg_upgrade to transfer data from the existing PostgreSQL cluster to the new encrypted cluster. 
@@ -58,7 +61,7 @@ This example upgrades a PostgreSQL 16 instance to EDB Postgres Extended Server 1
     !!!note
        Alternatively, use the `--key-unwrap-command=<command>` and `--key-wrap-command=<command>` arguments when initializing the encrypted server to include the `wrap` and `unwrap` commands.
 
-       See [Using initdb TDE options](../enabling_tde/#using-initdb-tde-options) for more information on possible configurations.
+       See [Using initdb TDE options](../initdb_tde_options/) for more information on possible configurations.
 
 1.  Initialize the new server with encryption: 
 
@@ -89,7 +92,7 @@ This example upgrades a PostgreSQL 16 instance to EDB Postgres Extended Server 1
     !!!note
        If you're using two different Postgres versions, use the psql utility of the encrypted server. Otherwise, the system attempts to use psql from the previous instance.
 
-1.  To ensure the new server is encrypted, [check for TDE presence](../enabling_tde/#checking-for-tde-presence-using-sql). 
+1.  To ensure the new server is encrypted, [check for TDE presence](../enabling/verifying_tde/). 
 
 ### Upgrading to the encrypted server
 

product_docs/docs/tde/15/enabling/verifying_tde.mdx

@@ -0,0 +1,18 @@
+---
+title: "Verifying TDE is enabled" 
+description: Verify TDE is enabled after creating a database server.
+---
+
+You can find out whether TDE is present on a server by querying the `data_encryption_version` column of the `pg_control_init` table.
+
+A value of 0 means TDE isn't enabled. Any nonzero value reflects the version of TDE in use. Currently, when TDE is enabled, this value is 1.
+
+```sql
+select data_encryption_version from pg_control_init();
+__OUTPUT__
+ data_encryption_version
+-------------------------
+                       1
+(1 row)
+```
+