!Architecture Overview
Python Best Practices for Production Applications
Introduction
Prerequisites
| Requirement | Details |
|---|---|
| Basic setup and tooling | Basic setup and tooling |
Figure: Code pattern examples for python best practices for production applications—syntax comparison, idiomatic approaches, performance characteristics, and common pitfalls.
Figure: Best practices implementation for python best practices for production applications—error handling, testing strategies, maintainability patterns, and documentation standards.
Figure: Production readiness checklist for python best practices for production applications—logging, monitoring, performance tuning, and security hardening.
Python's simplicity attracts beginners, but production applications require discipline. This guide covers virtual environments for dependency isolation, type hints for code safety, comprehensive error handling, unit testing with pytest, logging strategies, dependency management with Poetry, and packaging best practices for enterprise Python development.
Virtual Environments
Why Virtual Environments?
Problems without virtual environments:
Architecture Overview: Global Python installation (C: Python312):
Solution with virtual environments:
Architecture Overview: ProjectA
Architecture Overview: ProjectB
```text
### Creating Virtual Environments
**Using venv (built-in):**
```bash
# Create virtual environment
python -m venv venv
## Activate (Windows PowerShell)
.\venv\Scripts\Activate.ps1
## Activate (Linux/Mac)
source venv/bin/activate
## Verify activation (should show venv path)
where python
## Output: C:\Projects\MyApp\venv\Scripts\python.exe
## Install packages (isolated to this venv)
pip install requests flask pytest
## Deactivate
deactivate
Using virtualenv (more features):
## Install virtualenv
pip install virtualenv
## Create with specific Python version
virtualenv -p python3.11 venv
## Create without pip (install later)
virtualenv --no-pip venv
Using conda (data science):
## Create environment with specific Python version
conda create -n myapp python=3.11
## Activate
conda activate myapp
## Install packages
conda install numpy pandas scikit-learn
## List environments
conda env list
## Export environment
conda env export > environment.yml
## Recreate environment from file
conda env create -f environment.yml
Requirements Files
requirements.txt (pinned versions):
## requirements.txt
requests==2.31.0
flask==3.0.0
sqlalchemy==2.0.23
pytest==7.4.3
pytest-cov==4.1.0
black==23.12.0
mypy==1.7.1
## Generate from current environment
pip freeze > requirements.txt
## Install from requirements
pip install -r requirements.txt
requirements-dev.txt (development dependencies):
## requirements-dev.txt
-r requirements.txt # Include production dependencies
pytest==7.4.3
pytest-cov==4.1.0
black==23.12.0
flake8==6.1.0
mypy==1.7.1
sphinx==7.2.6
Using Poetry (modern dependency management):
## Install Poetry
pip install poetry
## Initialize project
poetry init
## Add dependencies
poetry add requests flask sqlalchemy
poetry add --group dev pytest black mypy
## Install dependencies
poetry install
## Run command in virtual environment
poetry run python app.py
poetry run pytest
## Update dependencies
poetry update
## Export requirements.txt
poetry export -f requirements.txt --output requirements.txt
pyproject.toml (Poetry configuration):
[tool.poetry]
name = "myapp"
version = "1.0.0"
description = "Production Python application"
authors = ["Vladimir Luis <email@contoso.com>"]
[tool.poetry.dependencies]
python = "^3.11"
requests = "^2.31.0"
flask = "^3.0.0"
sqlalchemy = "^2.0.23"
[tool.poetry.group.dev.dependencies]
pytest = "^7.4.3"
black = "^23.12.0"
mypy = "^1.7.1"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Type Hints
Basic Type Annotations
Function signatures:
def calculate_total(price: float, quantity: int, discount: float = 0.0) -> float:
```sql
"""Calculate total price with optional discount.
Args:
price: Unit price
quantity: Number of items
discount: Discount percentage (0.0 to 1.0)
Returns:
Total price after discount
"""
subtotal = price * quantity
return subtotal * (1 - discount)
Type checker validates calls
total = calculate_total(19.99, 3, 0.1) # ✅ Valid
total = calculate_total("19.99", 3, 0.1) # ❌ Type error: str not compatible with float
**Variable annotations:**
```python
## Explicit type hints
name: str = "John Doe"
age: int = 30
salary: float = 75000.50
is_active: bool = True
## Type inference (type checker infers from assignment)
count = 10 # Inferred as int
message = "Hello" # Inferred as str
## Multiple types with Union
from typing import Union
def process_id(user_id: Union[int, str]) -> str:
```text
"""Accept int or string ID, return string."""
return str(user_id)
Modern syntax (Python 3.10+)
Figure: Python IDE – debugger, variable explorer, and notebook integration.
def process_id(user_id: int | str) -> str:
return str(user_id)
## Advanced Type Hints
**Collections:**
```python
from typing import List, Dict, Set, Tuple, Optional
## List of strings
names: List[str] = ["Alice", "Bob", "Charlie"]
numbers: List[int] = [1, 2, 3, 4, 5]
## Dictionary mapping string keys to int values
scores: Dict[str, int] = {"Alice": 95, "Bob": 87, "Charlie": 92}
## Set of unique emails
emails: Set[str] = {"alice@contoso.com", "bob@contoso.com"}
## Tuple with fixed types
coordinates: Tuple[float, float] = (40.7128, -74.0060)
person: Tuple[str, int, bool] = ("Alice", 30, True)
## Optional (can be None)
middle_name: Optional[str] = None # Same as Union[str, None]
def find_user(user_id: int) -> Optional[Dict[str, str]]:
```text
"""Return user dict or None if not found."""
if user_id in users:
return users[user_id]
return None
**Generic types:**
```python
from typing import TypeVar, Generic, List
T = TypeVar('T')
class Stack(Generic[T]):
```python
"""Generic stack that works with any type."""
def __init__(self) -> None:
self._items: List[T] = []
def push(self, item: T) -> None:
self._items.append(item)
def pop(self) -> T:
return self._items.pop()
def peek(self) -> T:
return self._items[-1]
def is_empty(self) -> bool:
return len(self._items) == 0
Type-safe usage
int_stack: Stack[int] = Stack()
int_stack.push(10)
int_stack.push(20)
value: int = int_stack.pop() # ✅ Type checker knows this is int
str_stack: Stack[str] = Stack()
str_stack.push("hello")
str_stack.push(42) # ❌ Type error: int not compatible with str
**Callable types:**
```python
from typing import Callable
## Function that takes int and returns str
transformer: Callable[[int], str] = str
## Function with multiple parameters
def apply_operation(x: int, y: int, operation: Callable[[int, int], int]) -> int:
```text
return operation(x, y)
def add(a: int, b: int) -> int:
return a + b
def multiply(a: int, b: int) -> int:
return a * b
result = apply_operation(5, 3, add) # 8
result = apply_operation(5, 3, multiply) # 15
**Protocol (structural typing):**
```python
from typing import Protocol
class Drawable(Protocol):
```python
"""Protocol for objects that can be drawn."""
def draw(self) -> None: ...
class Circle:
def draw(self) -> None:
print("Drawing circle")
class Square:
def draw(self) -> None:
print("Drawing square")
def render(shape: Drawable) -> None:
"""Accepts any object with draw() method."""
shape.draw()
Both work without explicit inheritance
render(Circle()) # ✅
render(Square()) # ✅
## Type Checking with mypy
**mypy configuration (.mypy.ini):**
```ini
[mypy]
python_version = 3.11
warn_return_any = True
warn_unused_configs = True
disallow_untyped_defs = True
disallow_incomplete_defs = True
check_untyped_defs = True
disallow_untyped_decorators = True
no_implicit_optional = True
warn_redundant_casts = True
warn_unused_ignores = True
warn_no_return = True
Running mypy:
## Check single file
mypy app.py
## Check entire project
mypy src/
## Strict mode (maximum type safety)
mypy --strict src/
## Generate HTML report
mypy src/ --html-report ./mypy-report
Error Handling
Exception Best Practices
Specific exceptions:
## ❌ Bad: Catch all exceptions
try:
```text
value = int(user_input)```
except:
```text
print("Error occurred")
✅ Good: Catch specific exceptions
Figure: Go IDE – gopls language server, test runner, and module graph.
try:
value = int(user_input)```
except ValueError as e:
```text
print(f"Invalid number format: {e}")```
except TypeError as e:
```text
print(f"Wrong type provided: {e}")
**Custom exceptions:**
```python
class ApplicationError(Exception):
```text
"""Base exception for application errors."""
pass
class ValidationError(ApplicationError):
"""Raised when validation fails."""
def __init__(self, field: str, message: str) -> None:
self.field = field
self.message = message
super().__init__(f"{field}: {message}")
class DatabaseError(ApplicationError):
"""Raised when database operations fail."""
pass
class AuthenticationError(ApplicationError):
"""Raised when authentication fails."""
pass
Usage
Figure: Configuration and management dashboard with status overview.
def create_user(email: str, password: str) -> None:
if not email or "@" not in email:
raise ValidationError("email", "Invalid email address")
if len(password) < 8:
raise ValidationError("password", "Password must be at least 8 characters")
try:
db.save_user(email, password)
except Exception as e:
raise DatabaseError(f"Failed to save user: {e}") from e
**Context managers for cleanup:**
```python
from contextlib import contextmanager
from typing import Generator
@contextmanager
def database_connection(connection_string: str) -> Generator:
```text
"""Context manager for database connections."""
conn = None
try:
conn = connect(connection_string)
yield conn
except DatabaseError as e:
if conn:
conn.rollback()
raise
else:
if conn:
conn.commit()
finally:
if conn:
conn.close()
Usage
Figure: Configuration and management dashboard with status overview.
with database_connection("postgresql://localhost/mydb") as conn:
cursor = conn.cursor()
cursor.execute("INSERT INTO users (name) VALUES (?)", ("Alice",))
## Automatically commits on success, rolls back on error, always closes
**Retry decorator:**
```python
import time
from functools import wraps
from typing import Callable, TypeVar, Any
T = TypeVar('T')
def retry(max_attempts: int = 3, delay: float = 1.0):
```python
"""Retry decorator for flaky operations."""
def decorator(func: Callable[..., T]) -> Callable[..., T]:
@wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> T:
for attempt in range(1, max_attempts + 1):
try:
return func(*args, **kwargs)
except Exception as e:
if attempt == max_attempts:
raise
print(f"Attempt {attempt} failed: {e}. Retrying in {delay}s...")
time.sleep(delay)
raise RuntimeError("Retry logic failed") # Should never reach here
return wrapper
return decorator
@retry(max_attempts=3, delay=2.0)
def fetch_data_from_api(url: str) -> dict:
"""Fetch data with automatic retry on failure."""
response = requests.get(url, timeout=10)
response.raise_for_status()
return response.json()
## Testing with pytest
### Test Structure
**Basic tests:**
```python
## test_calculator.py
import pytest
from calculator import Calculator
def test_addition():
```text
"""Test addition operation."""
calc = Calculator()
assert calc.add(2, 3) == 5
assert calc.add(-1, 1) == 0
assert calc.add(0, 0) == 0
def test_division():
"""Test division operation."""
calc = Calculator()
assert calc.divide(10, 2) == 5
assert calc.divide(7, 2) == 3.5
def test_division_by_zero():
"""Test division by zero raises exception."""
calc = Calculator()
with pytest.raises(ValueError, match="Cannot divide by zero"):
calc.divide(10, 0)
**Fixtures:**
```python
## conftest.py (shared fixtures)
import pytest
from myapp.database import Database
from myapp.models import User
@pytest.fixture
def db():
```text
"""Create test database."""
database = Database(":memory:")
database.create_tables()
yield database
database.close()
@pytest.fixture
def sample_user():
"""Create sample user for testing."""
return User(
id=1,
email="test@contoso.com",
name="Test User",
is_active=True
)
@pytest.fixture
def authenticated_client(client, sample_user):
"""Return authenticated test client."""
client.login(sample_user)
return client
test_user.py
Figure: Test Studio – recorded test cases, assertions, and execution results.
def test_create_user(db):
"""Test user creation with database fixture."""
user = User(email="alice@contoso.com", name="Alice")
db.save(user)
retrieved = db.get_user_by_email("alice@contoso.com")
assert retrieved.name == "Alice"
def test_user_serialization(sample_user):
"""Test user serialization with fixture."""
json_data = sample_user.to_json()
assert json_data["email"] == "test@contoso.com"
assert json_data["is_active"] is True
**Parametrized tests:**
```python
import pytest
@pytest.mark.parametrize("input_value,expected", [
```text
(2, 4),
(3, 9),
(4, 16),
(5, 25),
(-2, 4),```
])
def test_square(input_value, expected):
```javascript
"""Test square function with multiple inputs."""
assert square(input_value) == expected
@pytest.mark.parametrize("email,is_valid", [
("alice@contoso.com", True),
("bob@test.org", True),
("invalid", False),
("@contoso.com", False),
("user@", False),
("", False),```
])
def test_email_validation(email, is_valid):
```sql
"""Test email validation with various inputs."""
assert validate_email(email) == is_valid
**Mocking:**
```python
from unittest.mock import Mock, patch, MagicMock
import pytest
def test_api_call_with_mock():
```javascript
"""Test function that calls external API."""
with patch('requests.get') as mock_get:
# Configure mock response
mock_response = Mock()
mock_response.status_code = 200
mock_response.json.return_value = {"id": 1, "name": "Test"}
mock_get.return_value = mock_response
# Call function under test
result = fetch_user_data(1)
# Verify
assert result["name"] == "Test"
mock_get.assert_called_once_with("https://api.contoso.com/users/1", timeout=10)
def test_database_operation_with_mock(mocker):
"""Test database operation with pytest-mock."""
mock_db = mocker.patch('myapp.database.Database')
mock_db.return_value.get_user.return_value = User(id=1, name="Alice")
service = UserService()
user = service.get_user_by_id(1)
assert user.name == "Alice"
mock_db.return_value.get_user.assert_called_once_with(1)
## Coverage
**pytest-cov configuration:**
```ini
## pytest.ini
[tool:pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
```text
--cov=src
--cov-report=html
--cov-report=term-missing
--cov-fail-under=80
**Running with coverage:**
```bash
## Run tests with coverage
pytest --cov=src --cov-report=html
## View coverage report
## Open htmlcov/index.html in browser
## Generate terminal report
pytest --cov=src --cov-report=term-missing
## Fail if coverage below 80%
pytest --cov=src --cov-fail-under=80
Logging
Structured Logging
Basic configuration:
import logging
from logging.handlers import RotatingFileHandler
import sys
def setup_logging(log_level: str = "INFO") -> None:
```sql
"""Configure application logging."""
## Create logger
logger = logging.getLogger("myapp")
logger.setLevel(getattr(logging, log_level.upper()))
## Console handler
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(logging.INFO)
console_formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
console_handler.setFormatter(console_formatter)
## File handler with rotation
file_handler = RotatingFileHandler(
'logs/app.log',
maxBytes=10_000_000, # 10 MB
backupCount=5
)
file_handler.setLevel(logging.DEBUG)
file_formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(pathname)s:%(lineno)d - %(message)s'
)
file_handler.setFormatter(file_formatter)
## Add handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)
Usage
Figure: Configuration and management dashboard with status overview.
setup_logging("DEBUG")
logger = logging.getLogger("myapp")
logger.debug("Detailed debug information")
logger.info("Application started")
logger.warning("Configuration file not found, using defaults")
logger.error("Failed to connect to database", exc_info=True)
logger.critical("System shutting down due to critical error")
**Structured logging with contextvars:**
```python
import logging
from contextvars import ContextVar
from typing import Optional
import uuid
## Context variables for request tracking
request_id_var: ContextVar[Optional[str]] = ContextVar('request_id', default=None)
user_id_var: ContextVar[Optional[int]] = ContextVar('user_id', default=None)
class ContextFilter(logging.Filter):
```python
"""Add context variables to log records."""
def filter(self, record: logging.LogRecord) -> bool:
record.request_id = request_id_var.get() or "N/A"
record.user_id = user_id_var.get() or "N/A"
return True
Configure logger with context
Figure: Configuration and management dashboard with status overview.
logger = logging.getLogger("myapp")
handler = logging.StreamHandler()
handler.addFilter(ContextFilter())
formatter = logging.Formatter(
'%(asctime)s - [%(request_id)s] [User:%(user_id)s] - %(levelname)s - %(message)s'```
)
handler.setFormatter(formatter)
logger.addHandler(handler)
## Middleware to set context
def process_request(request):
```text
request_id_var.set(str(uuid.uuid4()))
user_id_var.set(request.user.id if request.user else None)
logger.info(f"Processing request: {request.path}")
## All logs in this context will include request_id and user_id
## Packaging
### Project Structure
> **Architecture Overview:** myapp
**pyproject.toml (modern standard):**
```toml
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "myapp"
version = "1.0.0"
description = "Production Python application"
readme = "README.md"
authors = [
```text
{name = "Vladimir Luis", email = "email@contoso.com"}```
]
license = {text = "MIT"}
classifiers = [
```text
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",```
]
keywords = ["example", "production", "application"]
dependencies = [
```text
"requests>=2.31.0",
"flask>=3.0.0",
"sqlalchemy>=2.0.0",```
]
requires-python = ">=3.11"
[project.optional-dependencies]
dev = [
```text
"pytest>=7.4.0",
"black>=23.0.0",
"mypy>=1.7.0",```
]
[project.scripts]
myapp = "myapp.main:main"
[project.urls]
Homepage = "https://github.com/username/myapp"
Documentation = "https://myapp.readthedocs.io"
Repository = "https://github.com/username/myapp"
Building and distributing:
## Build package
python -m build
## Creates:
## dist/myapp-1.0.0.tar.gz (source distribution)
## dist/myapp-1.0.0-py3-none-any.whl (wheel)
## Install locally
pip install -e . # Editable install for development
## Upload to PyPI
pip install twine
twine upload dist/*
## Install from PyPI
pip install myapp
Best Practices Summary
- Virtual Environments: Always use venv/virtualenv, never install globally
- Type Hints: Add type annotations for all functions and class methods
- Error Handling: Use specific exceptions, create custom exception hierarchy
- Testing: Aim for 80%+ coverage, use fixtures and parametrization
- Logging: Use structured logging with context, rotate log files
- Dependencies: Pin versions in requirements.txt, use Poetry for complex projects
- Code Quality: Run black (formatting), mypy (type checking), pytest (testing)
Architecture Decision and Tradeoffs
When designing software development solutions with Programming Languages, consider these key architectural trade-offs:
| Approach | Best For | Tradeoff |
|---|---|---|
| Managed / platform service | Rapid delivery, reduced ops burden | Less customisation, potential vendor lock-in |
| Custom / self-hosted | Full control, advanced tuning | Higher operational overhead and cost |
Recommendation: Start with the managed approach for most workloads and move to custom only when specific requirements demand it.
Validation and Versioning
- Last validated: April 2026
- Validate examples against your tenant, region, and SKU constraints before production rollout.
- Keep module, CLI, and SDK versions pinned in automation pipelines and review quarterly.
Security and Governance Considerations
- Apply least-privilege access using RBAC roles and just-in-time elevation for admin tasks.
- Store secrets in managed secret stores and avoid embedding credentials in scripts or source files.
- Enable audit logging, data protection policies, and periodic access reviews for regulated workloads.
Cost and Performance Notes
- Define budgets and alerts, then monitor usage and cost trends continuously after go-live.
- Baseline performance with synthetic and real-user checks before and after major changes.
- Scale resources with measured thresholds and revisit sizing after usage pattern changes.
Official Microsoft References
- https://learn.microsoft.com/
- https://learn.microsoft.com/azure/
- https://learn.microsoft.com/power-platform/
- https://learn.microsoft.com/microsoft-365/
Public Examples from Official Sources
- These examples are sourced from official public Microsoft documentation and sample repositories.
- Documentation examples: https://learn.microsoft.com/training/
- Sample repositories: https://github.com/microsoft
- Prefer adapting these examples to your tenant, subscriptions, and governance requirements before production use.
Key Takeaways
- Virtual environments isolate dependencies and prevent conflicts
- Type hints catch errors before runtime with mypy static analysis
- Custom exceptions provide clear error handling and debugging
- pytest fixtures and parametrization reduce test boilerplate
- Structured logging with context enables production troubleshooting
- Modern packaging with pyproject.toml simplifies distribution
Next Steps
- Implement CI/CD pipeline with GitHub Actions running tests and type checks
- Add pre-commit hooks to enforce code quality standards
- Configure Dependabot for automatic dependency updates
- Set up code coverage reporting with Codecov
Additional Resources
Write Python like you mean it.
