Managing API Keys Across Dev, Staging, and Production

Every developer has experienced the nightmare: production API keys leaked in a Git repository, staging credentials used in production, or test data polluting live databases. Environment-based secret management is deceptively complex, and mistakes can be catastrophic.

This guide covers modern best practices for managing API keys, OAuth credentials, database passwords, and other secrets across multiple environments—without compromising security or developer productivity.

The Problem with Traditional Approaches

What Doesn't Work

Here are approaches that seem convenient but create serious risks:

• Single .env file with all environments: Developers often have production credentials on laptops
• .env files in Git: "We'll add it to .gitignore later" famous last words before a breach
• Shared credentials across environments: Testing in staging affects production
• Credentials in CI/CD as plain env vars: Visible in logs and accessible to anyone with repo access
• No rotation strategy: Keys remain unchanged for years, increasing breach impact

Environment Separation Principles

The Three Environments

Development: Local developer machines, frequent changes, lowest security requirements. Uses test accounts and sandbox APIs.

Staging: Production-like environment for testing. Uses separate accounts but production-like data volumes and configurations.

Production: Live customer-facing environment. Strictest security, change control, and audit requirements.

Modern Secret Management Strategies

1. Environment-Specific Secrets

Each environment must have completely separate secrets:

# Development environment
STRIPE_API_KEY=sk_test_abc123...
DATABASE_URL=postgres://localhost:5432/myapp_dev
SENDGRID_API_KEY=SG.dev_key...

# Staging environment
STRIPE_API_KEY=sk_test_staging_def456...
DATABASE_URL=postgres://staging-db.internal:5432/myapp_staging
SENDGRID_API_KEY=SG.staging_key...

# Production environment
STRIPE_API_KEY=sk_live_xyz789...
DATABASE_URL=postgres://prod-db-1.internal:5432/myapp_prod
SENDGRID_API_KEY=SG.prod_key...

Critical rules:

• Production secrets NEVER exist in development environments
• API keys use test vs live modes when available (Stripe, Twilio, etc.)
• Different AWS accounts or GCP projects per environment
• Separate OAuth applications per environment

2. Secrets Management Systems

Never store secrets in files or environment variables. Use dedicated secret management:

Option A: Cloud Provider Solutions

• AWS Secrets Manager: Integrated with IAM, automatic rotation, audit logs
• GCP Secret Manager: Versioned secrets, IAM integration, KMS encryption
• Azure Key Vault: Centralized secrets, keys, and certificates

// Fetching secrets from AWS Secrets Manager
const AWS = require('aws-sdk');
const secretsManager = new AWS.SecretsManager({ region: 'us-east-1' });

async function getSecret(secretName) {
  const data = await secretsManager.getSecretValue({
    SecretId: secretName
  }).promise();

  return JSON.parse(data.SecretString);
}

// Usage in application
const stripeKey = await getSecret('production/stripe-api-key');
const dbConfig = await getSecret('production/database-credentials');

Option B: Third-Party Solutions

• HashiCorp Vault: Industry standard, supports dynamic secrets, powerful policy engine
• Doppler: Developer-friendly, great UI, supports all major platforms
• KnoxCall: API-first secrets management with environment-based organization

3. Environment Detection

Your application needs to automatically detect which environment it's running in:

// Environment detection
function getEnvironment() {
  // Priority order: explicit env var, hostname detection, default to dev
  if (process.env.APP_ENV) {
    return process.env.APP_ENV;  // Explicitly set
  }

  const hostname = require('os').hostname();

  // Check hostname patterns
  if (hostname.includes('prod') || hostname.includes('production')) {
    return 'production';
  }

  if (hostname.includes('staging') || hostname.includes('stg')) {
    return 'staging';
  }

  return 'development';
}

// Load environment-specific config
const environment = getEnvironment();
const config = require(`./config/${environment}.js`);

4. Deployment-Time Secret Injection

Secrets should be injected during deployment, not baked into containers:

# Kubernetes deployment with secret injection
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
  namespace: production
spec:
  template:
    spec:
      containers:
      - name: api
        image: myapp:latest
        env:
        - name: STRIPE_API_KEY
          valueFrom:
            secretKeyRef:
              name: api-secrets
              key: stripe-api-key
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: api-secrets
              key: db-password
---
# Separate secret object (created via sealed-secrets or external-secrets)
apiVersion: v1
kind: Secret
metadata:
  name: api-secrets
  namespace: production
type: Opaque
data:
  stripe-api-key: 
  db-password: 

CI/CD Pipeline Secret Management

GitHub Actions Example

# .github/workflows/deploy.yml
name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production  # Requires approval for production

    steps:
    - uses: actions/checkout@v3

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
        aws-region: us-east-1

    - name: Deploy application
      env:
        # Secrets from GitHub Secrets (environment-specific)
        STRIPE_API_KEY: ${{ secrets.STRIPE_API_KEY }}
        DATABASE_URL: ${{ secrets.DATABASE_URL }}
      run: |
        ./deploy.sh production

Local Development Secret Management

The .env.example Pattern

Provide a template that developers copy and populate:

# .env.example (committed to Git)
# Copy to .env and fill in your values
# NEVER commit .env to Git

STRIPE_API_KEY=sk_test_your_test_key_here
DATABASE_URL=postgres://localhost:5432/myapp_dev
SENDGRID_API_KEY=SG.your_test_key_here
JWT_SECRET=generate_random_string_here

# Optional: use real dev services
# STRIPE_API_KEY=   # Leave blank to fetch from secrets manager

# .gitignore (always include)
.env
.env.local
.env.*.local
*.key
*.pem

Development Secret Fetching

Allow developers to fetch development secrets from your secrets manager:

# CLI tool to fetch dev secrets
#!/bin/bash
# scripts/fetch-dev-secrets.sh

if [ "$1" != "development" ]; then
  echo "Error: This script only works for development environment"
  exit 1
fi

# Fetch from secrets manager
aws secretsmanager get-secret-value \
  --secret-id development/api-keys \
  --query SecretString \
  --output text > .env.development

echo "Development secrets fetched successfully"
echo "File: .env.development"
echo "Remember: NEVER commit this file"

Secret Rotation Strategies

Automated Rotation

Implement regular secret rotation without downtime:

// Secret rotation with zero downtime
async function rotateSecret(secretName) {
  // 1. Generate new secret
  const newSecret = generateAPIKey();

  // 2. Store new secret with different version
  await secretsManager.updateSecret({
    SecretId: secretName,
    SecretString: newSecret,
    VersionStages: ['AWSPENDING']
  });

  // 3. Update downstream services (e.g., register new API key)
  await updateDownstreamService(newSecret);

  // 4. Wait for propagation (grace period)
  await sleep(60000);  // 60 seconds

  // 5. Promote to current
  await secretsManager.updateSecretVersionStage({
    SecretId: secretName,
    VersionStage: 'AWSCURRENT',
    MoveToVersionId: getCurrentVersionId()
  });

  // 6. Old version remains accessible as AWSPREVIOUS for rollback
}

Rotation Schedule

• Production API keys: Every 90 days
• Database passwords: Every 180 days
• OAuth client secrets: Every 365 days or on suspected compromise
• JWT signing keys: Every 30-90 days with overlapping validity

Audit and Compliance

Secret Access Logging

Track every secret access for compliance:

// Log secret access
async function getSecretWithAudit(secretName, userId, purpose) {
  const secret = await getSecret(secretName);

  // Audit log
  await auditLog.create({
    event: 'secret_accessed',
    secretName: secretName,
    userId: userId,
    purpose: purpose,
    environment: getEnvironment(),
    timestamp: new Date(),
    ipAddress: getClientIP()
  });

  return secret;
}

Compliance Requirements

• SOC 2: Encrypted storage, access logging, regular rotation
• PCI DSS: Quarterly rotation, encrypted transmission, restricted access
• GDPR: Right to delete requires secret rotation after user deletion

Common Pitfalls and Solutions

How KnoxCall Handles Environment-Based Secrets

KnoxCall provides built-in environment-based secret management:

• Environment organization: Secrets organized by environment (dev/staging/production) out of the box
• Automatic resolution: KnoxCall automatically uses the correct secret based on route environment
• Encrypted storage: All secrets encrypted at rest with per-environment keys
• Validation: Warns if a secret is missing for an environment before deployment
• OAuth2 handling: Automatic token refresh with environment-specific credentials
• Audit trails: Complete logs of secret access per environment
• Rotation support: Easy secret updates without downtime

Instead of managing multiple secret stores and deployment configurations, KnoxCall handles environment separation automatically.

Best Practices Checklist

• ✓ Separate secrets completely: Production secrets never in dev environments
• ✓ Use secrets management system: AWS Secrets Manager, Vault, or KnoxCall—never files
• ✓ Never commit secrets to Git: Use .gitignore and pre-commit hooks
• ✓ Inject secrets at runtime: Not build-time in Docker images
• ✓ Implement rotation: Regular rotation schedules with zero-downtime
• ✓ Audit access: Log every secret access with user and purpose
• ✓ Role-based access: Developers don't need production secrets
• ✓ Environment parity: Staging mirrors production infrastructure

Key Takeaways

• Environment separation is critical—production secrets should never exist in development
• Use dedicated secrets management systems, not files or environment variables
• Implement secret rotation with zero-downtime strategies
• Audit every secret access for compliance and security
• Inject secrets at deployment time, never bake them into containers
• Modern solutions like KnoxCall handle environment-based secrets automatically