Managing secrets and API keys in Python projects (.env guide)

Photo by the author

# An introduction to keeping secrets

Storing sensitive information such as API keys, database passwords, or tokens directly in Python code is unsafe. If these secrets are leaked, attackers can compromise your systems and your organization may suffer a loss of trust, which could result in financial and legal consequences. Instead, you should externalize secrets so that they never appear in your code or version control. A common best practice is to store secrets in environment variables (outside of the code). This way, secrets never appear in the codebase. Although manual environment variables work, for local development it is convenient to store all secrets in one .env file.

This article explains seven practical techniques for managing secrets in Python projectswith code examples and explanations of common pitfalls.

# Technique 1: Using the .env file locally (and loading it securely)

AND .measure the file is a text file KEY=value pairs that you store locally (not in version control). It allows you to define environment-specific settings and programming secrets. For example, the recommended design layout is:

my_project/
  app/
    main.py
    settings.py
  .env              # NOT committed – contains real secrets
  .env.example      # committed – lists keys without real values
  .gitignore
  pyproject.toml

Your real secrets find their way inside .measure locally, e.g.:

# .env (local only, never commit)
OPENAI_API_KEY=your_real_key_here
DATABASE_URL=postgresql://user:pass@localhost:5432/mydb
DEBUG=true

In contrast, .env.example this is a template that you validate so that other developers can see which keys are needed:

# .env.example (commit this)
OPENAI_API_KEY=
DATABASE_URL=
DEBUG=false

Add patterns to ignore these files in Git:

So that your secret .env file is never accidentally registered. In Python, it is common practice to utilize python-dotenv library that will load the file .measure file at runtime. For example in app/main.py you can write:

# app/main.py
import os
from dotenv import load_dotenv

load_dotenv()  # reads variables from .env into os.environ

api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise RuntimeError("Missing OPENAI_API_KEY. Set it in your environment or .env file.")

print("App started (key loaded).")

Here, load_dotenv() will find it automatically .measure in the working directory and sets each key=value down os.environment (unless this variable is already set). This approach avoids common mistakes like committing an .env file or insecurely sharing it, while providing a neat, repeatable development environment. You can switch between machines or developer configurations without changing code, and local secrets remain sheltered.

# Technique 2: Read the secrets of the surroundings

Some developers include placeholders such as API_KEY=”test” in your code or assume that variables are always set in development. This may work on their machine, but it doesn’t work in production. If the secret is missing, the placeholder may be triggered and create a security risk. Instead, always retrieve secrets from environment variables at runtime. In Python you can utilize os.environment Or os.getenv to get the values ​​safely. For example:

def require_env(name: str) -> str:
    value = os.getenv(name)
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value

OPENAI_API_KEY = require_env("OPENAI_API_KEY")

This makes the application fail quickly on startup if the secret is missing, which is much safer than continuing to work with a missing or dummy value.

# Technique 3: Check the configuration using the settings module

As projects evolve, many of them are scattered os.getenv connections become messy and error-prone. Using a settings class such as Pydantic Basic Settings centralizes configuration, checks types, and loads values ​​from .env and the environment. For example:

# app/settings.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env", extra="ignore")

    openai_api_key: str = Field(min_length=1)
    database_url: str = Field(min_length=1)
    debug: bool = False

settings = Settings()

Then in your application:

# app/main.py
from app.settings import settings

if settings.debug:
    print("Debug mode on")
api_key = settings.openai_api_key

This prevents errors such as mistyped keys, incorrect type parsing (“false” vs. false), or duplicate searches in the environment. Using the settings class ensures that your application crashes quickly if secrets are missing and avoids “runs on my computer” issues.

# Technique 4: Using platform/CI secrets for deployments

When deploying to a production environment, do not copy the local file .measure file. Instead, utilize secrets management for your hosting/CI platform. For example, if you utilize GitHub Actions for CI, you can store secrets encrypted in your repository settings and then introduce them to your workflows. This way your CI or cloud platform injects the actual values ​​at runtime and you never see them in your code or logs.

# Technique 5: Docker

In Docker, avoid putting secrets in images or using plain ENV. Docker and Kubernetes provide secrets mechanisms that are more secure than environment variables that can leak through process lists or logs. For local developers .env plus python-dotenv works, but for production containers mount secrets or utilize a docker secret. To avoid ENV API_KEY=… in Dockerfiles or committing Compose files with secrets. Doing so reduces the risk of permanently revealing secrets in images and simplifies rotation.

# Technique 6: Adding a handrail

People make mistakes, so automate covert protection. GitHub’s push protection can block commits containing secrets, and CI/CD secrets scanning tools like TruffleHog or Gitleaks detect pre-connect credential leaks. Beginners often rely on memory or speed, which leads to accidental commits. Guardrails prevent leaks before they reach the repository, making working with .env files and environment variables during development and deployment much safer.

# Technique 7: Using a Real Secrets Manager

For larger applications, it makes sense to utilize an appropriate secrets manager such as HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. These tools control who can access secrets, log all access, and automatically change keys. Without it, teams often reuse passwords or forget to change them, which is risky. Secrets Manager keeps everything under control, eases rotation, and protects production systems, even if it’s a development or local machine .measure the file is exposed.

# Summary

Protecting secrets is more than just following the rules. It’s about building a workflow that makes your projects secure, maintainable, and portable across a variety of environments. To make this easier, I’ve created a checklist you can utilize in your Python projects.

1. .measure is in .gitignore (never provide real credentials)
2. .env.example exists and is validated with empty values
3. The code reads secrets only through environment variables (os.getenv, settings class, etc.)
4. App fails quickly with an explicit error if the required secret is missing
5. You utilize various secrets for developers, staging and prod (never reuse the same key)
6. CI and implementation applications encrypted secrets (GitHub secret actions, AWS parameter store, etc.)
7. Push protection and/or secret scanning is enabled on your repositories
8. You have rotation policy (in case of leakage, turn the keys immediately, otherwise regularly)

Kanwal Mehreen is a machine learning engineer and technical writer with a deep passion for data science and the intersection of artificial intelligence and medicine. She is co-author of the e-book “Maximizing Productivity with ChatGPT”. As a 2022 Google Generation Scholar for APAC, she promotes diversity and academic excellence. She is also recognized as a Teradata Diversity in Tech Scholar, a Mitacs Globalink Research Scholar, and a Harvard WeCode Scholar. Kanwal is a staunch advocate for change and founded FEMCodes to empower women in STEM fields.