api-design

Define Your Public API Surface and Defend It

The public API is the contract between your library and every downstream user. Once a symbol is public, removing or changing it is a breaking change. Getting the boundary right -- what is exported, what is private, how errors are communicated, how complexity is layered -- is the most consequential design decision in a Python library. Get it wrong and you end up like Pydantic v1: an unintentionally wide API surface that required a full rewrite to fix.

httpx is the gold standard. Every public symbol is re-exported from __init__.py. Every implementation module is underscore-prefixed (_client.py, _config.py). The exception hierarchy is a carefully designed tree. Sync and async clients share all non-I/O logic in a _BaseClient. Study httpx before designing your own API.

Public API Surface: __all__ and Underscore Modules

Every module that exports symbols must define __all__. This documents intent, controls from module import *, and tells tools (mypy, pyright, mkdocstrings) what is public.

Rule	Rationale
Define __all__ in every public module	Documents the contract; tools rely on it
Re-export everything from __init__.py	Users write from mylib import Client, never from mylib._client import Client
Prefix all implementation modules with _	_client.py, _config.py, _exceptions.py make the private boundary unambiguous
Sort __all__ alphabetically	Easy to review in diffs, easy to check for completeness
Include type aliases and protocols in __all__	Users need them for type-checking; easy to forget

Bad	Good
from mylib.client import Client	from mylib import Client
No __all__, everything implicitly public	Explicit __all__ listing every public name
utils.py with mixed public/private helpers	_utils.py for internal, re-export public helpers from __init__.py

Expose less than you think you need. You can always make something public later; you cannot easily make it private. If it is in __all__, it is a contract. If it starts with an underscore, you can change it freely.

Progressive Disclosure

Design APIs that serve beginners and experts simultaneously. Simple things should be simple; complex things should be possible.

# Layer 1: Module-level convenience (simplest)
response = httpx.get("https://api.example.com/users")

# Layer 2: Configured client (intermediate)
client = httpx.Client(base_url="https://api.example.com", timeout=30.0)
response = client.get("/users")

# Layer 3: Full customization (advanced)
transport = httpx.HTTPTransport(retries=3)
client = httpx.Client(transport=transport, timeout=httpx.Timeout(5.0, connect=10.0))

Rules for progressive disclosure:

1. The simplest call requires the fewest arguments. Use sensible defaults for everything.
2. Complexity is opt-in, not opt-out. A user who needs no auth should never see auth parameters.
3. Use keyword-only arguments (* in the signature) to prevent positional footguns as signatures grow.
4. Module-level functions for the common case, classes for the configured case. This is the httpx/requests pattern.
5. Config objects instead of dozens of kwargs. httpx.Timeout(5.0, connect=10.0) is more discoverable than timeout=5.0, connect_timeout=10.0.

Exception Hierarchy

A well-designed exception hierarchy lets users catch errors at exactly the right granularity. It is part of __all__ and must be as carefully designed as your classes.

class MyLibError(Exception):
    """Base exception. `except MyLibError` catches everything."""

class ConfigurationError(MyLibError):
    """Raised when configuration is invalid."""

class ConnectionError(MyLibError):
    """Raised when a connection fails."""

class TimeoutError(ConnectionError):
    """Subclasses ConnectionError -- timeout is a type of connection failure."""

class AuthenticationError(MyLibError):
    """Raised when authentication fails."""

class ValidationError(MyLibError):
    """Carries structured error data, not just a string."""
    def __init__(self, errors: list[dict[str, Any]]) -> None:
        self.errors = errors
        super().__init__(f"{len(errors)} validation error(s)")

Rule	Example
Always provide a base exception class	except MyLibError as catch-all
Carry structured data, not just strings	httpx's HTTPStatusError.response, Pydantic's .errors()
Use inheritance to group related errors	except TransportError catches all network issues
Never raise bare Exception or ValueError	Users cannot distinguish your errors from others
Name exceptions as nouns	TimeoutError, not TimedOut
Document which methods raise which exceptions	Part of the API contract

Async/Sync Dual API

Follow httpx's _BaseClient pattern: share all non-I/O logic in a base class, implement genuinely separate sync and async I/O paths.

class _BaseClient:
    """Shared logic: URL merging, headers, cookies, auth -- no I/O."""
    def _build_request(self, method: str, url: str, **kwargs) -> Request:
        ...

class Client(_BaseClient):
    """Synchronous client with blocking transport."""
    def send(self, request: Request) -> Response:
        return self._transport.handle_request(request)

class AsyncClient(_BaseClient):
    """Asynchronous client with async transport."""
    async def send(self, request: Request) -> Response:
        return await self._transport.handle_async_request(request)

Bad	Good
Wrap async with asyncio.run() in sync methods	Separate sync/async transport implementations
Duplicate all non-I/O logic in both clients	Share logic in _BaseClient
Only provide async API	Always provide sync; add async if doing I/O

Never use asyncio.run() as a sync wrapper. It fails if an event loop is already running (Jupyter, async frameworks) and prevents connection pooling.

Plugin Architecture

Choose the right extensibility pattern based on your scale.

Pattern	Complexity	When to Use	Exemplar
pluggy	High	Full plugin ecosystem with hooks	pytest, tox
Entry points	Medium	Installed packages register themselves	pytest plugin discovery
Protocols	Low	Third-party opt-in without importing your lib	Rich (__rich_repr__)
Decorator registry	Low	Internal extensibility within your package	Click, Flask

For protocols, use dunder names (__mylib_serialize__), make them @runtime_checkable, keep them to one method, and always provide a fallback for objects that do not implement the protocol. The key insight from Rich: objects do not need to import or subclass anything from your library to participate.

For entry points, define them in pyproject.toml:

[project.entry-points."mylib.plugins"]
my_plugin = "my_plugin_package:MyPlugin"

Discover them at runtime with importlib.metadata.entry_points(group="mylib.plugins").

Method Naming and Signatures

Use consistent verb-noun naming across the entire API. Pydantic v2 learned from v1's inconsistency by adopting a uniform model_ prefix.

Verb	Meaning	Example
get	Retrieve (may raise if missing)	client.get()
create	Make a new resource	Session.create()
build	Construct from parts	Request.build()
validate	Check and convert	model_validate()
dump	Serialize to format	model_dump(), model_dump_json()
load	Deserialize from format	json.load()

Force keyword-only arguments after the first positional with * in the signature. Use sensible, secure defaults (verify=True, follow_redirects=False). Return rich objects for complex operations (httpx's Response carries status, headers, content, and raise_for_status()), primitives for simple queries, and self for builder/configuration methods.

Configuration Patterns

Use frozen dataclasses for configuration objects. Validate in __post_init__. Provide a from_env() classmethod for environment variable integration without requiring it.

@dataclass(frozen=True)
class ClientConfig:
    base_url: str
    timeout: float = 30.0
    max_retries: int = 3

    def __post_init__(self) -> None:
        if self.timeout <= 0:
            raise ConfigurationError("timeout must be positive")

    @classmethod
    def from_env(cls, prefix: str = "MYLIB_") -> "ClientConfig":
        return cls(
            base_url=os.environ.get(f"{prefix}BASE_URL", ""),
            timeout=float(os.environ.get(f"{prefix}TIMEOUT", cls.timeout)),
        )

Review Checklist

When reviewing code for API design:

• __all__ is defined in every public module, sorted alphabetically, and includes all public symbols (classes, functions, exceptions, type aliases)
• All implementation modules are underscore-prefixed (_client.py, _config.py) and users never import from them directly
• Public symbols are re-exported from the top-level __init__.py
• The API supports progressive disclosure: module-level functions for simple use, configurable classes for advanced use
• All parameters beyond the first positional are keyword-only (use * separator)
• A base exception class exists and all library exceptions inherit from it
• Exceptions carry structured data (not just string messages) and are included in __all__
• No bare Exception, ValueError, or TypeError is raised from library code
• Async and sync clients share non-I/O logic in a base class with separate transport implementations
• No asyncio.run() wrappers are used to bridge async to sync
• Plugin extension points use the appropriate pattern (pluggy for full ecosystems, Protocols for third-party opt-in, entry points for auto-discovery)
• Method naming follows a consistent verb-noun pattern across the entire API surface
• Mutable default arguments are avoided (use None sentinel + factory)
• Importing the library produces no side effects (no network calls, no logging config, no global state mutation)
• Internal state is not exposed through public attributes; use read-only properties that return values, not mutable objects