Base64 Is Not Encryption. Vault Is Not Custody. What Kubernetes Secrets Actually Need.

TL;DR

"Base64 is not encryption" is the canonical Kubernetes-community meme. Seth Vargo (Google, ex-HashiCorp) has a talk by that exact name on O'Reilly Velocity. The official Kubernetes docs themselves admit secrets are stored unencrypted by default in etcd.

The standard answer — HashiCorp Vault, External Secrets Operator, KMS encryption-at-rest, Sealed Secrets, Mozilla SOPS, AWS / Azure / GCP Secrets Manager — closes one part of the problem: storage outside the cluster.

But common delivery patterns leave a runtime exposure gap: they typically expose plaintext as a standing cluster-level artifact — a k8s Secret resource, an environment variable, or a mounted file inside the pod.

kubectl get secret -o yaml returns plaintext. kubectl exec + printenv returns plaintext. Node-root reads mounted volumes. Backups include Secret resources.

The goal is not to claim that plaintext never exists. Plaintext must exist somewhere to be used. The goal is to prevent plaintext from becoming a persistent cluster-level artifact and to limit its lifetime to the application operation that actually needs it.

The question in 2026 is not "which secrets manager to choose." The question is:

Where does the key live, and how is plaintext recovered for the operation that needs it?

This post is about that gap, why common delivery patterns leave it open, and the architectural pattern that closes it.

The ASTIS answer, in short: the secret travels as a sealed envelope — never plaintext as a cluster artifact — and the key is released only after a runtime workload check. Before a pod can unwrap, it must clear four independent gates: a workload-bound API key, the pod's projected ServiceAccount JWT (bound to its own name and UID), a RAM-only proof-of-possession key, and a live read of your own Kubernetes API confirming every running image — app, init, sidecar, ephemeral — is on an approved allowlist (an injected debug container is refused). The application then decrypts in RAM; the application ciphertext never leaves your cluster. Full architecture on the Workload Secrets page.

---

The problem is real — documented from three independent sources

This is not a marketing narrative. "Base64 is not encryption" is the canonical meme of the K8s community:

1. Seth Vargo, O'Reilly Velocity — Base64 Is Not Encryption

Vargo (Google, ex-HashiCorp, ex-Chef) — a recognized authority. From the talk:

"By default all Kubernetes secrets are base64 encoded and stored as plaintext in etcd."

As his demo shows, anyone with access to the etcd cluster then reads every secret — service accounts and API keys included.

This is not theoretical. It is the core threat model the K8s community starts the security conversation from.

2. Kubernetes official docs — open admission

From the official Kubernetes documentation:

"Kubernetes Secrets are, by default, stored unencrypted in the API server's underlying data store (etcd). Anyone with API access can retrieve or modify a Secret, and so can anyone with access to etcd."

Plus the caution warning:

"Anyone who is authorized to create a Pod in a namespace can use that access to read any Secret in that namespace; this includes indirect access such as the ability to create a Deployment."

The docs themselves state: built-in Secrets are not suitable for high-security production.

3. DEV.to — Why Built-in Secrets Fail in Production (Ciro Veldran)

The article walks through three structural failure modes:

• Base64 ≠ encryption. c3VwZXItc2VjcmV0 decodes in under one second.
• RBAC misconfiguration can expose Secrets broadly. The default view role does not grant Secret reads, but broader roles such as edit, custom bindings, and — critically — the ability to create Pods can expose secret material indirectly.
• Unencrypted etcd. Backups become a "complete credential dump."

The article itself concedes:

"The complexity of proper secrets management is not a reason to use inadequate tools"

Even the author acknowledges no current solution is perfect.

The documented facts are sufficient for the rest of this post:

• Kubernetes Secrets are stored unencrypted in etcd by default.
• Base64 encoding is not encryption.
• A user who can create a Pod in a namespace can indirectly read Secrets in that namespace.

---

What all three sources agree on — and the question they leave open

The shared thesis:

Built-in K8s Secrets do not solve production security. You need Vault / ESO / KMS encryption-at-rest / an external secrets provider.

The question that is rarely addressed directly:

Where does plaintext show up inside the cluster after Vault / ESO / CSI driver has delivered it?

In the common delivery patterns, plaintext typically reaches the Pod as:

• a Kubernetes Secret resource (kubectl get secret ... -o yaml returns plaintext);
• an environment variable (printenv inside the pod returns plaintext);
• a mounted volume (cat /var/run/secrets/... returns plaintext);
• pod memory accessible via kubectl exec + process inspection.

These are standing cluster-level artifacts — exposure surfaces that exist for the lifetime of the pod, not just for the operation that needs the credential.

---

Common Kubernetes secret-delivery patterns leave a runtime exposure gap

The key finding:

These solutions answer "how to store and rotate secrets safely outside the cluster." Many of them still expose recoverable plaintext outside the application boundary — as a Secret resource, env var, or mounted file — for the lifetime of the pod.

---

The question a CISO asks in 2026

The real procurement question:

If an attacker gets kubectl exec into a production pod — what do they see?

The standard answer: "the same thing the application sees." And the application sees everything in plaintext.

This is not a theoretical concern. Several regulatory and audit frameworks are increasingly examined in this exact dimension:

• NIS2 Article 21(2)(h) asks for documented cryptographic policies and procedures, including how keys are managed.
• DORA Article 9 addresses confidentiality of customer data flowing through ICT systems and third-party providers.
• Schrems II / Cloud Act make it material who can be compelled to disclose data. A vendor-held or cloud-KMS-held key is in a different position than a customer-held key.
• SOC 2 CC6.7 covers encryption controls over data in transit and at rest, including how reviewers examine data flow.
• HIPAA §164.312 requires technical safeguards over PHI, including controls over where it can exist.

None of these explicitly require HYOK or a specific architectural pattern. But HYOK and application-boundary unwrap can help implement and document stronger cryptographic controls for regulated environments. "We use Vault" tells reviewers about credential management. It does not, by itself, answer architectural questions about where plaintext exists across the cluster.

---

The architectural pattern that closes the runtime exposure gap

The pattern that closes the gap is not a new secrets manager. It is an architecture with six properties that common solutions do not provide at the same time:

1. The customer holds the Key Encryption Key (KEK) — HYOK. The vendor / cluster operator / cloud provider has no access.
2. The secret is wrapped in a sealed envelope — a PGP / hybrid encrypted blob, opaque to anyone but the KEK holder.
3. The sealed envelope is stored anywhere — Git, ConfigMap, S3, the customer's choice. It is not plaintext.
4. Unwrap happens at the application boundary through the SDK. The SDK recovers plaintext only for the operation that requires it, uses a short-lived buffer, and clears that buffer immediately after use where the runtime permits explicit zeroization.
5. No standing cluster-level artifact carries plaintext. Plaintext is not exposed as a Kubernetes Secret resource, environment variable, or mounted file. It exists only inside the application process at the moment of use.
6. Every unwrap is an audited event in the customer's evidence chain, not a silent operation.

Concrete examples of the application-boundary flow:

• unwrap → connect to DB → clear buffer
• unwrap → sign JWT → clear buffer
• unwrap → call external API → clear buffer

What this pattern closes:

• kubectl get secret returns a sealed envelope (not useful as-is to an attacker).
• kubectl exec + printenv shows opaque blobs, not credential strings.
• Node root reads sealed material from storage and volumes; plaintext only exists in process memory during an unwrap, requiring live memory inspection rather than a file read.
• Backups contain sealed envelopes, not a credential dump.
• Under HYOK, the vendor / cluster operator / cloud KMS cannot disclose plaintext under subpoena — they do not have the KEK. Under Standard tier, ASTIS handles the wrapped DEK in transit, never the decrypted secret.

This is a structural difference vs. Vault / ESO / Sealed Secrets / SOPS — not because those are wrong, but because they answer a different part of the problem.

---

Why this is architecturally different from Vault

This is not "Vault, but better." This is a different architectural category.

The table shows the HYOK custody model, where the key never leaves the customer. In the default managed tier, ASTIS still performs the key-rewrap step — but it handles only the transient wrapped DEK in memory, never the secret itself, and the application ciphertext never leaves the cluster. Two tiers, the same property: plaintext does not materialize as a standing cluster artifact.

---

How ASTIS implements this pattern

The ASTIS trust layer is a single control plane that operates across four product surfaces:

1. Mail + Calendar — sealed envelopes for business communication.
2. API Platform — application-layer trust boundary for sensitive customer data (API Trust Layer for Sensitive Data).
3. AI workflows / MCP — scope-gated agent operations through AZK (AZK: Agentic Zero-Knowledge).
4. Infrastructure trust layer — sealed envelope + HYOK + application-boundary unwrap for K8s workloads (this post).

All four surfaces share the same architectural pattern: a customer-held key, a sealed envelope as transport, plaintext recovered only inside the application process for a specific operation, and audit on every reveal.

For customer data inside the application path (PII / PHI / payment / contracts) this is a shipped product:

• sealed envelope architecture wraps sensitive payloads before they enter downstream services;
• HYOK support — a customer-controlled CVS pod inside the customer's K8s cluster holds the key material; ASTIS provides routing and audit, not key access;
• application-boundary unwrap — plaintext is recovered through the SDK only at the moment of an operation (decrypt, sign, verify); it is not exposed as a Kubernetes Secret, env var, or mounted file;
• AI agents inside the cluster receive scoped operations through ASTIS MCP, not broad plaintext access by default;
• audit chain — every reveal is a hash-chained event in the customer's audit pipeline.

For service secrets (DB passwords, API keys) — the same architectural pattern, with one addition specific to Kubernetes: before any key is released, ASTIS verifies the live workload at runtime. A pod must clear four independent gates —

1. a workload-bound API key (cluster + namespace + ServiceAccount);
2. the pod's projected ServiceAccount JWT, verified against the cluster's pinned JWKS and bound to the pod's own name and UID;
3. a RAM-only proof-of-possession key, with a single active credential pinned per live pod;
4. and a check, read live from your own Kubernetes API, that every running image — app, init, sidecar, ephemeral — matches an approved digest (strict and fail-closed; an unapproved or injected debug container is refused).

Only a pod that is what it claims to be gets the DEK rewrapped to its ephemeral key; the application ciphertext never leaves the cluster. The full trust boundary — and exactly what ASTIS does and does not see — is on the Workload Secrets architecture page.

The same pattern carries two custody tiers. In the default managed tier, ASTIS performs the rewrap step in memory: it handles the wrapped DEK transiently and never the secret itself. Under HYOK, that step runs inside a customer-controlled CVS pod in your own cluster, so key custody — and the unwrap — stay entirely on your side. Either way, ASTIS provides the sealed-envelope formats, the runtime release policy, and the audit chain; pod-side integration today is applied through the customer application SDK, not a turn-key operator out of the box.

ASTIS and Vault are complementary

ASTIS and Vault can be complementary. Vault can remain the upstream system for credential generation, rotation, TTLs, and access policies. ASTIS adds a sealed delivery and application-boundary usage pattern so that selected high-sensitivity credentials do not need to appear as Kubernetes Secrets, environment variables, or mounted files inside the cluster.

Customers running Vault for service-credential management can keep doing so. Vault handles its category well. ASTIS adds the layer Vault does not address directly: a customer-held key + application-boundary unwrap pattern that avoids the runtime exposure gap inside the cluster.

Concretely, you do not replace anything — you integrate. Vault keeps issuing, rotating, and governing credentials exactly as it does today. For the subset of secrets that warrant a stricter boundary, you wrap them once in an ASTIS sealed envelope and let the application recover plaintext at the SDK level, inside the process — unwrap → use → clear — instead of receiving it as a Kubernetes Secret, environment variable, or mounted file. Vault stays the system of record for the credential lifecycle; ASTIS changes only where and how plaintext is recovered for the operation that needs it. The two run side by side: Vault for lifecycle, ASTIS for the in-application reveal.

---

Type 1 vs Type 2 — same pattern, two categories

A bonus: the same pattern works for two different categories of secrets:

• Type 1 — infrastructure secrets: DB passwords, API keys, service certs. The standard solution is Vault. The ASTIS pattern applies when the customer wants HYOK.
• Type 2 — customer plaintext passing through the pod: PII, PHI, payment data, contracts, AI context. There is no standard solution — Vault does not see this layer. ASTIS API Platform implements this today.

One pattern, two categories. Vault does not fully close either.

---

Q&A

Q: Is this a Vault replacement? A: No. Vault is a robust solution for service-credential management within its category (vendor-held or cluster-held key). The ASTIS pattern is a different category: customer-held key, ephemeral RAM, no plaintext materialization in the cluster. Many customers run both.

Q: Can I use this pattern for DB passwords? A: Yes. The architectural pattern works. ASTIS provides sealed envelope formats + HYOK key custody + an audit layer. Pod-side integration code (wrap on initialize, unwrap on connect) is customer-side. It is not a "secrets manager out of the box" — it is a pattern the customer applies.

Q: What if I do not want to run my own KMS? A: Then HYOK is not for you. The default ASTIS deployment is Managed CVS (ASTIS-operated key custody). The trust model is less strict than HYOK but still removes plaintext materialization from the customer's cluster.

Q: What about Confidential Containers / TEEs (Intel TDX, AMD SEV, Nitro Enclaves)? A: TEEs solve a similar problem differently — in hardware, not in protocol. They are a strong solution for compute trust, but they require a TEE-compatible cluster and often a rewritten application. The ASTIS pattern is software-only and works on any K8s. For high threat models, customers combine both: TEE for compute trust + HYOK for key trust.

Q: What if an attacker gets kubectl exec into my pod? A: This is the main subject. In the common stack the attacker sees env vars with secrets, mounted volumes, plaintext as standing artifacts. In this pattern those artifacts are not present; the attacker sees sealed envelopes (opaque blobs). The attack window is reduced from persistent, easily discoverable cluster-level exposure to a short-lived application-memory window during a specific operation. This is not a guarantee against memory inspection of a live process; it is a meaningful reduction in the exposure surface.

Q: What about performance overhead? A: One KMS call on pod startup (~10–50ms), then plaintext in RAM. Unlike PKCS#11 / TEE-based solutions where every crypto call goes through a secure module. For service secrets (set once at boot) the overhead is trivial.

Q: Can I combine this with Vault? A: Yes. Vault for service-credential management in its category. The ASTIS pattern for customer data and any service secret that requires HYOK. These are not competing layers — they are complementary.

---

Get in touch

If your conversation with the CISO has shifted from "we use Vault" to "where does the key live, and where does plaintext materialize" — the second conversation is the one we have with customers.

Book a 30-minute architecture review — bring one production scenario: a service secret with a HYOK requirement, customer data flowing through the cluster, or an AI agent doing runtime crypto. We map the pattern and show where sealed envelope + ephemeral RAM closes the gap that existing solutions leave open.

This is not a "buy our secrets manager" pitch. It is the architectural conversation the K8s community is not closing — because the problem is not the choice of secrets manager, it is where the key lives and where plaintext is allowed to exist.

The goal is not to claim that plaintext never exists. The goal is to prevent plaintext from becoming a persistent cluster-level artifact, and to limit its lifetime to the application operation that actually needs it.

---

References

Canonical sources on the problem:

• Seth Vargo — Base64 is Not Encryption (O'Reilly Velocity)
• Seth Vargo — base64-is-not-encryption (GitHub)
• Kubernetes Secrets — official docs
• DEV.to — Why Built-in Secrets Fail in Production (Ciro Veldran)

Solutions analyzed in this post:

• HashiCorp Vault — Kubernetes integration
• External Secrets Operator
• Sealed Secrets (Bitnami)
• Mozilla SOPS
• Secrets Store CSI Driver
• Confidential Containers

---

— Ievgen Bobliev, Founder of ASTIS. ASTIS — cryptographic trust layer.

Companion to: Infrastructure Access Should Not Mean Data Access, API Trust Layer for Sensitive Data, and AZK — Agentic Zero-Knowledge.