Here is the fact that surprises people every time: Terraform state stores every attribute in plaintext JSON, including the database password you carefully marked `sensitive`. The `sensitive` flag only hides the value in CLI output. In the state file it sits in the clear, which means anyone with read access to your state bucket has your secrets. We have found production credentials in state more times than we would like to count.
The first rule: do not let Terraform generate or hold the secret
The cleanest pattern is for the secret to never pass through Terraform as a value at all. Instead of generating a password in HCL and writing it to the database, we have Terraform create an empty secret container - an AWS Secrets Manager secret or a Vault path - and let the application or a controlled rotation process populate it out of band. Terraform manages the existence and the access policy; it never sees the value.
- Reference, do not store. Apps read secrets at runtime from Secrets Manager or Vault; the value never enters Terraform's graph.
- If Terraform must create a secret, let a rotation Lambda set the real value immediately and use `ignore_changes` on the secret string.
- Never put secrets in `.tfvars` checked into Git. This is still the most common leak we find.
- Treat the state backend itself as a secret store: encryption at rest, tight bucket policy, access logging. Most people secure the repo and forget the state.
Marking a variable `sensitive` hides it from your terminal, not from your state file. Those are very different threats.
When you genuinely need a secret at plan time
Sometimes you cannot avoid it - a third-party provider needs an API key to even authenticate. For those, we inject via environment variables (`TF_VAR_` or the provider's own env var) from the CI secret store, never as a file and never as a default. The secret lives in the pipeline's vault, gets injected for the duration of the run, and is gone. It still lands in state if you assign it to a resource attribute, so the discipline is to use it for authentication only, not to store it in a resource.
data "aws_secretsmanager_secret_version" "db" {
secret_id = "prod/orders/db"
}
# Read at apply, used to configure - but note: this value
# WILL be written to state. Prefer runtime lookup by the app.
Assume the state will leak, then make it boring
The mindset that keeps us out of trouble is to design as if the state file will eventually end up somewhere it should not. If that day comes and the worst thing in it is some resource IDs and ARNs, you have an awkward conversation. If it contains live database passwords and a cloud provider's root key, you have an incident and a disclosure. The whole game is making sure a leaked state is boring. Rotate anything that ever touched state as part of cleanup, lock the backend down hard, and keep the genuinely secret values in a system built to hold them.