Skip to content

Secrets

TL;DR — Service secrets (DB creds, AWS creds, Kafka creds) live in Kubernetes Secrets per environment, never in git. The shared NPM token for @securitize/* packages lives in 4 locations and has a dedicated rotation runbook.

Overview

Two secrets systems coexist:

  1. K8s Secrets — per-environment, per-service sensitive values consumed by microservices at runtime.
  2. NPM token — single shared read token (SECURITIZE_READ_NPM_TOKEN) that every repo uses to install @securitize/* private packages. Stored in 4 locations to cover all contexts (local dev, CI, CD, Docker builds).

Kubernetes Secrets (per environment)

Secrets live as K8s Secret objects in the cluster, per environment. Secret values are never committed to git — what lives in ops-scripts are references (secretRef / env.valueFrom.secretKeyRef) in each service's deployment.yaml.

Two levels of secrets exist:

Global secrets (shared)

K8s Secret objects consumed by many microservices. One set per environment:

K8s Secret Contains
aws-secret AWS credentials for services that need direct SDK access
mongo-secret MongoDB Atlas connection string
mysql-secret RDS MySQL endpoint + credentials
kafka-secret Confluent Cloud API keys per cluster (see databases-and-services.md)

Env vars exposed by each global Secret

Every global Secret is injected via envFrom: [- secretRef: { name: <secret> }] in the service's deployment.yaml — all keys in the Secret become env vars with the same name in the container. The canonical env var names each global Secret exposes:

mysql-secret (MySQL / TypeORM):

Env var Type Description
TYPEORM_HOST string MySQL host address
TYPEORM_PORT number MySQL port (typically 3306)
TYPEORM_TYPE string Always mysql
TYPEORM_USERNAME string DB username
TYPEORM_PASSWORD string DB password

mongo-secret (MongoDB Atlas):

Env var Type Description
MONGODB_HOST string Base connection URI — e.g. mongodb+srv://user:pass@host.mongodb.net. Does NOT include the database name. The database name is service-specific — provide it separately via the dbName option in MongooseModule.forRoot, or as a service-owned env var (e.g. MONGODB_DATABASE).

aws-secret:

Env var Type Description
AWS_ACCESS_KEY_ID string AWS Access Key ID
AWS_SECRET_ACCESS_KEY string AWS Secret Access Key

kafka-secret (Confluent Cloud):

Env var Type Description
KAFKA_BROKERS string Comma-separated broker addresses, no spaces (e.g. pkc-pgq85.us-west-2.aws.confluent.cloud:9092,pkc-pgq82.us-west-2.aws.confluent.cloud:9092)
KAFKA_API_KEY string Kafka API key
KAFKA_API_SECRET string Kafka API secret

Service configuration (e.g. CustomConfigService in NestJS services) must expect env vars with these exact names — they come from K8s via the Secret references, not from .env.default.

Per-service secrets

Each service has its own K8s Secret object. The values are created in the cluster (manual step when scaffolding a new service — see service-creation.md) and are not committed to git. The references (which secrets a service consumes) live in its deployment.yaml under ops-scripts/k8s/<service>/yamls/ — see ops-repos.md.

To inspect which secrets (global or per-service) a given service uses, open that service's deployment.yaml and look for secretRef / secretKeyRef entries.

NPM Token — SECURITIZE_READ_NPM_TOKEN

All repos use this private NPM read token to install internal packages (@securitize/*). The token is always consumed as an environment variable — never hardcoded.

How it works

Every repo has an .npmrc that references the env var: 

//registry.npmjs.org/:_authToken=${SECURITIZE_READ_NPM_TOKEN}

Where the token lives (4 locations)

Context Location
Jenkins CD (all environments) AWS Parameter Store /secrets/global/ops → SECURITIZE_READ_NPM_TOKEN. DEV account covers dev+rc; PROD account covers sandbox+production.
Bitbucket CI (PR pipelines) Bitbucket workspace variable (securitize_dev)
Local development Developer's shell profile (~/.zshrc or ~/.bashrc)
Docker builds / kops servers Passed as --build-arg SECURITIZE_READ_NPM_TOKEN by Jenkins, sourced from Parameter Store

AWS account IDs are in aws-infrastructure.md.

Getting the token for the first time

New developers can fetch the current NPM token (read-only) from a SendSafely Dev Workspace. The workspace contains:

  • npm-read-only-token.txt — for developers (what you need for local setup).
  • npm-write-token.txt — for publishing (reserved for package publishers).

Finding the workspace link

The SendSafely Dev Workspace link (with keycode) is pinned in the #dev Slack channel. Ask there if you can't find it.

Rotation runbook — places to update

When the token is rotated, update these 4 locations:

  1. AWS Parameter Store DEV — /secrets/global/ops → covers dev, rc. (Account ID in aws-infrastructure.md.)
  2. AWS Parameter Store PROD — /secrets/global/ops → covers sandbox, production. (Account ID in aws-infrastructure.md.)
  3. Bitbucket workspace variable — covers all CI pipelines.
  4. Notify all developers — each dev must update their local shell profile (~/.zshrc / ~/.bashrc).

No full redeploy needed

You do not need to redeploy all services at once — each service picks up the new token on its next natural deploy. CI/CD pulls from Parameter Store/workspace variable at build time.

Official documentation

Full developer setup guide and rotation runbook also lives in Confluence (the canonical version): - NPM Token — Confluence

Common failure modes

Symptom Fix
404 npm install for @securitize/* locally Token missing or expired in ~/.zshrc. Update and reload shell.
Jenkins pipeline fails on npm ci Parameter Store token may be stale. Check Parameter Store, rotate if needed.
Bitbucket pipeline fails on npm ci Workspace variable stale. Update in Bitbucket settings.
Docker build fails on npm ci --build-arg not reaching the image. Verify Jenkinsfile and ECR push logs.

See also

Tags

secrets #npm #token #rotation #kubernetes #parameter-store #runbook