Why Postgres Lacks Transparent Data Encryption

If you've ever compared database feature matrices, you may have noticed something a bit peculiar. Oracle has Transparent Data Encryption. SQL Server has it. MySQL has it. Even MariaDB has it. But Postgres, which we all consider the best database engine? Conspicuously absent.

It’s not that nobody wants TDE. Compliance frameworks like PCI DSS and HIPAA practically demand encryption at rest. Cloud deployments make the “stolen disk” threat model more tangible than ever. And the question comes up constantly on mailing lists, at conferences, and in every database evaluation checklist ever assembled by a procurement department. So what gives?

It’s complicated. The real answer involves nearly a decade of mailing list threads, competing proposals, fundamental disagreements about threat models, and a problem scope so vast it makes most contributors quietly back away. Let’s trace the history and find out why the elephant in the room is still unencrypted.

The Grand Promise of TDE

The concept of TDE is straightforward: encrypt all data when it’s written to disk, decrypt it when it’s read back into memory. The “transparent” part means the encryption layer sits below the SQL engine so client applications operate normally without special tooling or requirements.

TDE protects against an attacker who gains read access to the physical storage: a stolen or improperly decommissioned drive, a backup tape that fell off a truck, or perhaps a misconfigured cloud storage volume. TDE does not protect against a compromised application, a malicious DBA, those with superuser privileges, or SQL injection. The contents of the database can be fully decrypted while the server is running, which means anyone who can connect and query can read everything.

This is the crux of a debate that has raged in the Postgres community for years. If TDE only protects against disk theft, and filesystem encryption (LUKS, ZFS native encryption, dm-crypt) already does the same thing, why should Postgres spend enormous engineering effort to reinvent it inside the database?

The answers, depending on who you ask, include:

• Compliance checkboxes are a real business requirement.

• It’s genuinely more convenient to manage encryption at the database layer.

• Doing so is a waste of time; just use LUKS.

These three positions play out repeatedly.

Four Forks in Search of a Standard

Perhaps in an effort to circumvent the ongoing debate, five separate companies have independently invented TDE for Postgres, each using its own particular approach. And of course, not a single one of them is compatible with the community release, and most are proprietary. One of these (Crunchy Hardened PostgreSQL) appears to no longer exist, bringing the list down to four.

Here’s what remains:

Percona’s pg_tde is the only open-source option, but its full capabilities (index encryption, complete WAL encryption) require Percona Server for PostgreSQL, their custom fork. It uses a two-tier key architecture with AES-256-CBC for data and AES-256-CTR for WAL, and supports external key management through HashiCorp Vault and KMIP. Temporary files remain unencrypted, which is a notable gap for result sets that spill to disk.

EnterpriseDB offers TDE in their proprietary EDB Postgres Advanced Server from version 15 onward. Their implementation is arguably the most thorough, encrypting data files (AES-XTS), WAL (AES-CTR), and temporary files (AES-CBC). The catch, beyond requiring their proprietary server, is that TDE must be enabled at initdb time. It’s not possible to retrofit encryption onto an existing cluster.

CyberTec includes TDE in their PostgreSQL Enterprise Edition (PGEE). Their approach encrypts the entire cluster using AES-CTR with no option for selective per-table encryption. It’s all or nothing. CyberTec has been vocal about their TDE work influencing subsequent community discussions, and their original implementation dates back to around Postgres 12.

Fujitsu bundles TDE in their Enterprise Postgres product, using hardware-accelerated AES-256 with FIPS 140-2 compliance. Their approach encrypts at the tablespace level, meaning encrypted and unencrypted tablespaces can coexist. It’s a pragmatic design, but documentation is largely confined to whitepapers and sales materials.

Five companies, four remaining implementations, zero availability in community PostgreSQL. That should illustrate the difficulty of getting this feature into core. It’s not a lack of motivation or engineering talent. Every one of these companies did the work to make TDE a reality in their flavor of Postgres.

“What Threats Do You Want to Defend Against?”

The most instructive chapter in this saga began in May 2018, when Insung Moon from NTT proposed a comprehensive TDE design on the pgsql-hackers mailing list. The proposal was ambitious: per-table encryption granularity, a two-tier key management system with KMS plugin architecture, WAL encryption with separate keys, and buffer-level encrypt/decrypt during disk I/O. Masahiko Sawada co-designed the approach.

The community response was… educational.

Tomas Vondra got straight to the point:

This is the fundamental problem that haunts every TDE proposal for Postgres. Before designing the solution, it’s necessary to agree on the problem being solved. And the Postgres community has never reached that agreement.

Sawada himself pivoted the discussion toward threat modeling, asking what specific attacks TDE should defend against. The answers diverged immediately. Some contributors saw compliance requirements as sufficient justification. Others wanted protection against curious DBAs. Still others pointed out that if threat models involve a malicious DBA, server-side encryption can’t help because the DBA has access to the running server. The same server that transparently decrypts data during queries.

Then Nico Williams delivered what might be the most devastating critique in the entire thread. He pointed out that the proposal didn’t encrypt pg_catalog, the internal metadata tables Postgres uses to manage everything. An attacker with storage access could manipulate catalog entries to escalate privileges and extract the TDE keys themselves. Per-table encryption without catalog protection was, in his analysis, fundamentally incomplete:

Williams went further, arguing that any threat model where DBAs are not the threat “is just not that interesting to address with crypto within postgres itself.” Filesystem encryption already handles the disk-theft scenario. Cryptography inside the database should protect against someone with database access. And that problem, he noted, requires far more sophisticated protections (Merkle hash trees, MAC verification) that would be “performance killing.”

The conversation branched into competing proposals: per-table vs. per-tablespace vs. per-column encryption, with Joe Conway from Crunchy Data proposing transparent column-level transforms. Debates erupted about cipher modes (CBC vs. XTS, with Fujitsu’s Tsunakawa advocating for XTS per NIST SP 800-38E). Arguments about key management mechanisms went unresolved. None of these branches converged. The thread effectively ended in January 2020 with no path forward.

The Scope of the Surgery

Part of what makes TDE so daunting for Postgres is the sheer number of subsystems it touches. The PostgreSQL TDE Wiki page details the work required, and it’s harrowing. Reading it feels less like a feature specification and more like a dive into The Money Pit.

The wiki establishes a reasonable core design philosophy: TDE must be secure, but it must also have minimal impact on the rest of the codebase. Since only a small fraction of users would enable TDE, the less code added, the less testing is required, and the less likely TDE is to break as Postgres evolves.

Fair enough. Now here’s what “minimal impact” actually entails.

Buffer management needs to encrypt on every write to disk and decrypt on every read. Shared buffers stay in plaintext (that’s the “transparent” part), but the transitions need to be airtight.

WAL is where things get genuinely hairy. WAL buffers are plaintext in memory but must be encrypted when flushed to disk, using a separate encryption key and CTR-mode cipher. Key rotation during streaming replication gets complicated fast. Does the WAL sender decrypt before transmitting? Can a primary and standby use different keys? What happens to key state during a standby promotion?

Temporary files need a unified I/O API so encryption can be applied consistently. Parallel query workers must share the temporary key. And because temp files are created and destroyed constantly during query execution, performance overhead here is especially visible.

Every backup tool in the Postgres ecosystem needs modification: pg_basebackup, pg_rewind, pg_waldump. Each needs a new passphrase command option. Only pg_dump escapes, since it reads data through SQL (already decrypted in shared memory).

Then there are the many edge cases. Hint-bit updates, which normally flip a single bit, produce entirely different ciphertext under block encryption, so wal_log_hints=on becomes mandatory. CREATE DATABASE must potentially decrypt and re-encrypt all copied files if the database OID is part of the initialization vector. Some GiST index pages have fixed LSNs assigned at build time, creating IV reuse scenarios. The page checksum question (encrypt the CRC or not?) remains unresolved. Encrypting it provides a weak form of integrity checking but forces every command-line tool that verifies checksums to also possess the encryption key.

The wiki lists dozens of specific on-disk files requiring encryption, spanning table data, pg_internal.init caches, dynamic shared memory segments, replication slot snapshots, statistics files, and all WAL segments. In the face of all this, the idea of “minimal impact” becomes a pipe dream.

Torn Arguments and Pages

One of the earliest attempts to get TDE into Postgres core came from Ants Aasma back in 2016. He submitted a cluster-wide encryption patch that proposed AES-128-XTS encryption of all data files with a single master key. The design was deliberately simple: one encryption_library GUC, a narrow API (one key-setup function, two encrypt/decrypt functions), and key delivery via environment variable.

Robert Haas raised the most consequential technical objection. Encrypting WAL a block at a time breaks Postgres’s torn-page safety model. If a crash tears an encrypted block mid-write, it becomes impossible to decrypt the partial result, and WAL replay stops dead. This undermines synchronous_commit guarantees, which is to say, it undermines one of the most fundamental promises Postgres makes about data.

Peter Eisentraut challenged the entire premise, noting that filesystem-level encryption already exists and is well-tested. He then raised the topic of tool integration burden and pointed out that environment variables are visible to other processes, making the key-entry mechanism insecure on its face.

Bruce Momjian was characteristically blunt:

Hans-Jurgen Schonig from CyberTec pushed back, arguing the demand was real, especially in financial services where regulations explicitly required database-level encryption. Stephen Frost emphasized backup portability as a genuine advantage: encrypted backups can be restored anywhere with the key, unlike filesystem encryption tied to a specific mount point.

But the arguments for TDE never quite overcame the arguments against. Aasma himself seemed to sense this, calling TDE a “checkbox feature” and openly asking whether the community really thought it was worth the effort. Early benchmarks showed a 2x raw slowdown, and further performance optimization was promised but never delivered. The thread faded out with no consensus.

Keys to the Kingdom

By 2020, Sawada took a different tack. Rather than proposing full TDE, he submitted an internal key management system (KMS) as a foundation that future TDE work could build upon. The design centered on a master encryption key protected by a user-provided passphrase, with SQL functions pg_kmgr_wrap() and pg_kmgr_unwrap() to encrypt and decrypt internal keys. Sehrope Sarkuni contributed a detailed cryptographic specification using a 64-byte master key split into separate encryption and MAC keys, backed by OpenSSL.

The pragmatic “build the foundation first” approach made sense. Unfortunately, the foundation had cracks.

Fabien Coelho spotted the first problem: if pg_kmgr_unwrap() is a regular SQL function, any database user can simply call SELECT pg_kmgr_unwrap('xxxx') and extract the key. That rather defeats the stated goal of hiding keys from users.

Robert Haas identified the second, arguably fatal problem. Passing encryption keys as SQL function arguments means they show up in query logs whenever log_statement = all is configured, or whenever a query throws an error. His verdict was unsparing:

Then Tom Lane weighed in with a position that surprised many:

This legal and policy argument had a chilling effect on the discussion. Andres Freund pushed back, arguing that crypto export restrictions are largely anachronistic in 2020, but the point was never fully resolved. Freund also took the opportunity to warn against building on pgcrypto, calling its code quality “well below our standards” with “very outdated copies of cryptography algorithm implementations.” That’s not a vote of confidence for the only crypto extension currently shipping with Postgres.

Chris Travers and Robert Haas both proposed alternatives that bypassed the SQL layer entirely, using new protocol-level messages to exchange key material without ever touching the query parser or the log infrastructure. These designs were architecturally cleaner but also far more invasive, requiring changes to the client/server protocol itself.

All of these observations apparently proved insurmountable, and the thread eventually ended in October 2020 with no resolution. The patch is still listed as “Returned with feedback.”

Meanwhile, in a Different Room

While the TDE discussion sputtered out, Peter Eisentraut started work on something related but fundamentally different. In December 2021, he proposed Transparent Column Encryption (TCE), a client-side feature that encrypts individual columns before data ever reaches the server. The server only sees ciphertext. It can store it, replicate it, back it up, but it cannot read it.

Eisentraut was explicit about the distinction:

The threat model is completely different from TDE. Where TDE protects against physical disk theft, column encryption protects against the DBA. The server never has the plaintext, so even a malicious administrator can’t read the protected columns. The canonical use case is PCI DSS compliance for credit card numbers, social security numbers, and similarly sensitive data.

The design was both clever and intricate, requiring a boatload of core modifications:

• Protocol-level changes.

• Prepared statements for encrypted inserts, since the client must intercept parameter values to encrypt them.

• A two-tier key hierarchy of Column Encryption Keys wrapped by Column Master Keys stored in an external KMS.

• New DDL commands such as CREATE COLUMN ENCRYPTION KEY and CREATE COLUMN MASTER KEY.

Notice something that’s missing? Since Postgres can’t access the raw data values, it becomes impossible to compute on encrypted columns server-side. That means no indexing, sorting, or WHERE clauses against encrypted values. That’s more than slightly inconvenient.

Despite that, the patch went through dozens of iterations over two and a half years. Jacob Champion raised thorny questions about NULL handling (unencrypted NULLs leak information about which rows have absent data), cryptographic algorithm selection, and AEAD integration. Tomas Vondra pragmatically argued for shipping a basic version that protected against passive attackers first, deferring stronger protections to future work. This patch also remains in “Returned with feedback” status since the thread ended in August 2024.

Even the alternative approach to database-layer encryption hit the same wall: perfect becomes the enemy of good.

A Method to the Madness

Step back from the individual threads and a clear pattern emerges. Every TDE proposal for Postgres has died for some combination of the same five reasons.

There is no agreed threat model. This is the root cause, and everything else flows from it. Some contributors want protection against disk theft, but then LUKS works fine. Some want protection against DBAs, but server-side encryption can’t deliver that because the data is decrypted in memory. Some want a compliance checkbox, but is that worth the engineering cost of touching every subsystem in the database? Without consensus on what problem TDE solves, nobody can evaluate whether any particular design is the right one. Vondra’s observation from 2018 remains perfectly accurate: TDE keeps dying because the community can’t agree on what it’s for.

The scope is enormous. TDE touches buffer management, WAL, temporary files, replication, every backup tool, dozens of on-disk file types, and creates subtle interactions with checksums, hint bits, index construction, and database creation. This isn’t a trivial bolt-on feature. Properly adding TDE is a multi-year project plan, which can’t happen until everyone agrees on an approach.

Key management inside a database is a paradox. The database needs the key to function, but storing the key near the data undermines the encryption. Passing keys through SQL exposes them in query logs. Environment variables are visible to other OS processes. External KMS integration adds deployment complexity and latency. Protocol-level key exchange requires invasive client/server changes. Can any approach address all of these concerns?

Core committers are genuinely skeptical. Bruce Momjian’s “unimpressed” comment, Tom Lane’s “never crypto in core” position, and Peter Eisentraut’s “filesystem encryption already exists” argument reflect real convictions from people who’ve been maintaining Postgres for decades. These are the perspectives of engineers who know exactly how much maintenance burden a feature like this creates, and who have watched the same proposal cycle through the mailing list multiple times without resolution.

The “just use LUKS” argument is hard to beat. As Robert Haas put it:

This is technically correct, but try explaining that to an auditor who expects to see “TDE” in a feature checklist for compliance purposes.

None of this means TDE is pointless, of course. Even Vondra, one of the more vocal skeptics about the threat model confusion, acknowledged that:

There’s real value in a single initdb flag that encrypts everything, manages key rotation, and produces encrypted backups by default. But is that enough to justify the work required to reach that point?

Five years have passed since the last serious attempt at TDE in Postgres core. The Sawada KMS thread died in 2020. Eisentraut’s column encryption (a different feature entirely) stalled in 2024. No new proposals have appeared on the horizon. That’s a long silence for a feature with this much demand.

The Self Service Lane

It’s possible to have encryption at rest for Postgres today without adopting a proprietary fork. The options are well-established if somewhat mundane.

Filesystem overlay encryption with LUKS (Linux Unified Key Setup) is the most common approach. Create an encrypted volume, put the Postgres data directory on it, done. Performance overhead is modest (typically 5-10% with AES-NI hardware), key management is handled by the OS, and it’s battle-tested across millions of deployments.

Native filesystem encryption is available in ZFS, Ceph, and a handful of others. These don’t require the additional encryption layer, and often encrypt at the volume level. This enables tablespace-level granularity if necessary, without a lot of extra setup. These filesystems make transparent encryption practically “free”.

Cloud provider encryption such as AWS EBS encryption, GCP persistent disk encryption, and Azure disk encryption, are also available. They’re transparent, often enabled by default, and satisfy most compliance frameworks. It’s likely that Postgres cloud deployments already have encryption at rest without having done anything special.

For column-level protection of specific sensitive values, pgcrypto remains available as a contrib extension. While it’s not “transparent” in any sense, it does keep ciphertext in the actual column storage, which satisfies certain compliance requirements.

The Debate Continues

Postgres is a remarkable database. It handles replication, partitioning, JSON, full-text search, geospatial data, and a hundred other things with a degree of sophistication that rightfully threatens many commercial offerings. But TDE remains the feature that “got away”.

The community’s caution is understandable. Bad encryption is worse than no encryption because it creates a false sense of security. The scope of the change is genuinely massive. The threat model question has never been answered to everyone’s satisfaction. And the “just use LUKS” argument, while unsatisfying to procurement departments, is technically sound.

But the gap is real. Every year that Postgres lacks native TDE, the four proprietary forks get a little more entrenched with customers who can’t fight the compliance battle anymore. Every abandoned mailing list thread makes the next proposal a harder sell. And every database evaluation matrix with a blank cell next to “Encryption at Rest” costs the Postgres ecosystem mindshare it didn’t have to lose. Many in the Postgres community like to poke fun at MySQL, but even they have it.

Maybe the path forward is something nobody has tried yet: a modular, hook-based approach that lets extensions provide encryption without requiring crypto in core (neatly sidestepping Tom Lane’s objection). Maybe Percona’s pg_tde, imperfect as it is, eventually matures enough to become a de facto standard. Or maybe some future contributor will finally write the patch that threads the needle between skeptics and scope.

Until then, format a LUKS partition, click the cloud provider’s disk encryption checkbox, keep an eye on the mailing lists, and move on with life. Given the community’s devotion to correctness in engineering, any solution that meets their standards is likely superior to competing implementations. When TDE finally arrives, it’s bound to be—like Postgres—the best there is.