Optimizing mypy.ini for Large Codebases: Performance & Precision Tuning

Scaling static type checking across massive Python repositories requires precise configuration of Mypy Configuration & Strictness parameters. This guide details exact tuning strategies for incremental caching, per-module strictness scoping, and third-party import routing. You will eliminate CI bottlenecks while maintaining rigorous type safety within broader Static Analysis Tools & CI Integration workflows.

Incremental Caching & SQLite Backend Tuning

Enable persistent caching to bypass cold-start overhead. Set cache_dir to a volume that survives CI job restarts. For repositories exceeding 100k LOC, consider the SQLite backend to reduce filesystem inode pressure during concurrent worker execution.

[mypy]
python_version = 3.11
incremental = true
sqlite_cache = true
cache_dir = .mypy_cache
show_error_codes = true
warn_return_any = true
warn_unused_ignores = true
follow_imports = silent

This configuration enables persistent cache routing and strict error reporting. It silences untyped third-party traversal to prevent AST bloat. Note that sqlite_cache requires mypy >=0.900. Python 3.10+ is recommended for stable PEP 604 union syntax. Run mypy --cache-fine-grained to enable finer-grained invalidation that only rebuilds affected modules. Run mypy --cache-dir /dev/null (or --no-incremental) only during major interpreter upgrades to force a clean baseline.

Under the hood, the default cache writes two files per module into .mypy_cache/<major>.<minor>/: a <module>.meta.json holding the source mtime, size, a content hash, the module’s dependency list, and an interface hash; and a <module>.data.json holding the serialized, fully-resolved types. On each run mypy first compares mtime and size as a cheap staleness check, then falls back to the content hash if those differ. This is why a git checkout that rewrites timestamps but not content can still hit the cache — and why --skip-cache-mtime-checks (hash-only) is worthwhile on content-addressable CI where mtimes are never stable.

The decisive property is that a module is re-checked only when its own source changes or a dependency’s interface hash changes — not when a dependency’s implementation changes. Editing a function body leaves that module’s interface hash untouched, so downstream importers stay cached; editing a function signature flips the interface hash and cascades a recheck to everything that imports it. Understanding this makes cache behavior predictable: signature churn is expensive, body churn is nearly free.

sqlite_cache = true collapses those thousands of small JSON files into a single cache.db. The stored data is identical; only the container changes. On a 200k-LOC monorepo the JSON layout can produce tens of thousands of inodes — enough to slow actions/cache restore and, on some overlay filesystems, exhaust inode quotas. The single database restores atomically and tars in one step. cache_fine_grained = true additionally writes the fine-grained dependency map that the daemon consumes.

For the fastest possible warm checks, run the dmypy daemon instead of the one-shot CLI. dmypy keeps the entire program state and fine-grained dependency graph resident in memory between invocations, so a subsequent check re-analyzes only the handful of files you touched — often in well under a second where a cold mypy takes minutes:

dmypy start -- --config-file mypy.ini    # boot the daemon once
dmypy check src/                          # first check: full analysis
dmypy check src/                          # warm check: only changed files
dmypy status                              # inspect the running daemon

The daemon shines on developer laptops and long-lived self-hosted runners. It is not useful on ephemeral CI runners that spin up fresh per job — there is no resident process to reuse, so a persisted on-disk .mypy_cache (ideally sqlite_cache) is the right lever there. Finally, the cache is keyed on the mypy version and the options that affect analysis; bumping mypy or changing python_version transparently invalidates it, and --skip-version-check suppresses the safety re-scan only when you are certain the version is unchanged.

Per-Module Strictness Overrides for Gradual Adoption

Apply granular strictness via per-module overrides instead of global flags. Define base [mypy] defaults first. Then scope exceptions to legacy paths. This prevents strictness leakage while isolating warn_return_any to active development paths.

[mypy-legacy.*]
disallow_untyped_defs = false
ignore_errors = true

[mypy-app.core.*]
disallow_untyped_defs = true
strict_equality = true

This pattern isolates strict checking to active modules while bypassing legacy code. It maintains global coverage metrics without degrading developer velocity. Consider this Python 3.10+ module that leverages the strict config:

# src/app/core/models.py
from typing import Protocol, TypeAlias

class DataProcessor(Protocol):
    def process(self, data: bytes) -> str: ...

HandlerType: TypeAlias = DataProcessor | None

def execute(handler: HandlerType) -> str:
    if handler is None:
        raise ValueError("Handler required")
    return handler.process(b"payload")  # mypy validates strict equality & return types

The mypy.ini form uses [mypy-<glob>] section headers; the pyproject.toml form uses [[tool.mypy.overrides]] blocks with an explicit module = "app.core.*" key (or a list of globs). They are equivalent — pick one file format and keep the whole project in it. Only a subset of options is legal per module: analysis toggles like disallow_untyped_defs, ignore_errors, ignore_missing_imports, follow_imports, warn_return_any, and strict_equality are per-module, whereas global-only settings such as python_version, cache_dir, incremental, and sqlite_cache are not. Putting a global-only option in a per-module section makes mypy exit with Setting "python_version" not allowed per module.

Precedence is the detail teams get wrong. When several sections match one module, mypy applies the most specific one: a concrete name with no wildcard always beats a glob, so [mypy-app.core.models] overrides [mypy-app.*] regardless of order. Among two equally specific globs, the section that appears later in the file wins, which is why a broad lax block placed after a strict one can silently undo it. The practical rule mirrors the ratchet in rolling out disallow_untyped_defs incrementally: keep the lax global baseline first, list the narrowest strict globs last, and prefer exact module names when locking a finished package. Note that ignore_errors = true suppresses all diagnostics for the matched module — it is a blunt instrument for genuinely frozen legacy code, not a substitute for the targeted follow_imports routing covered next, and it is heavier than the per-package approach in enabling strict mode incrementally.

Third-Party Import Routing & Stub Path Optimization

Route untyped dependencies through custom stub directories instead of ignoring imports globally. Configure mypy_path to point to internal type definitions. Use follow_imports = silent for known-untyped packages — this suppresses noise while still reading .pyi stubs.

[mypy]
mypy_path = ./typeshed_custom

[mypy-pandas.*]
ignore_missing_imports = true

[mypy-numpy.*]
ignore_missing_imports = true

This routes internal type definitions to a dedicated folder while explicitly ignoring specific heavy dependencies. It reduces memory footprint without breaking type propagation for first-party code. Avoid ignore_missing_imports = true at the root [mypy] level — it masks legitimate missing hints in first-party code.

The four follow_imports modes are precise, not interchangeable. normal follows the import and type-checks the target, reporting its errors. silent still follows and analyzes the target — so you keep the inferred types across the boundary — but suppresses every diagnostic originating inside it. skip does not read the module at all; every name imported from it becomes Any, which quietly disables checking on all call sites that touch it. error behaves like skip but additionally emits a diagnostic on the import statement itself, useful when you want to forbid pulling in a particular untyped module. Because skip erases type information, reserve it for modules you truly want opaque; for ordinary untyped dependencies silent is almost always the correct choice.

Missing-import handling changed meaningfully in mypy 1.6, which split the old catch-all into two error codes: [import-untyped] (the module was found but ships no type information, and mypy suggests the matching types-* stub package) and [import-not-found] (the module could not be located at all). Prefer the surgical --disable-error-code=import-untyped — or a per-module disable_error_code — over a blanket ignore_missing_imports = true, so a genuinely misspelled first-party import still surfaces as [import-not-found].

Stub resolution follows PEP 561. mypy searches, in order: the paths in MYPYPATH / mypy_path; installed distributions that ship a py.typed marker (inline types); installed stub-only packages named <pkg>-stubs (the shape PyPI stubs such as types-requests and pandas-stubs take); and finally the bundled typeshed for the standard library. mypy_path therefore lets you hand-write partial .pyi stubs for an untyped dependency in ./typeshed_custom/thatpkg.pyi and have them win over the missing upstream types — a far better outcome than ignore_missing_imports, because you regain real checking at the boundary. Keep those directories on a stable MYPYPATH in both local and CI environments so resolution is identical everywhere.

CI Memory Constraints & Parallel Execution

Prevent OOM kills by monitoring RSS and trimming the module graph mypy has to hold in memory. mypy analyzes the whole program in a single process, so peak resident memory scales with the size of the import closure — not with how many files changed. Incremental mode shortens wall time on warm runs but does not lower that peak, so the levers that matter are excluding directories mypy never needs to see and sharding the work across separate processes.

# mypy has no --jobs flag; shard the tree across processes instead
printf '%s\n' app/core app/api app/web \
  | xargs -P 4 -I{} mypy --config-file mypy.ini {}

A common misconception is that mypy --jobs 4 parallelizes a run — mypy has no --jobs/-j flag, and passing one errors with unrecognized arguments. The open-source checker is single-process by design; that is a large part of why pyright is faster on cold runs. To use more than one core you split the codebase into independent invocations — one per top-level package — and fan them out with xargs -P, GNU parallel, or a CI matrix. Each shard type-checks its own import closure, so the split is only sound where packages don’t cross-import in ways that need each other’s inferred types; keep tightly-coupled packages in the same shard.

For everyday speed the daemon beats sharding: dmypy amortizes the expensive full analysis into the first call and makes subsequent checks near-instant, without the memory duplication that N parallel processes incur. Trim the graph with an exclude regex so mypy never parses vendored code, generated protobufs, or virtualenvs:

[mypy]
exclude = (^|/)(\.venv|\.tox|build|.*_pb2\.py)($|/)
show_traceback = false

Use --no-incremental deliberately for a nightly full scan; it discards the cache and re-derives everything, catching the rare cache-drift bug that a warm run would mask without paying that cost on every PR. show_traceback = false keeps failure output lean in CI. Python 3.11+ lowers the interpreter’s own baseline memory, which helps on constrained runners, but the dominant factor remains the size of the import closure you ask mypy to load — so scoping and excluding beat any interpreter upgrade.

Common Mistakes

• Disabling incremental mode globally: Forces full AST rebuilds on every run, increasing CI times from seconds to minutes. It negates mypy’s primary scaling mechanism. If a run seems to ignore the cache, check that cache_dir is on a writable, persisted volume and that the mypy version and python_version haven’t changed between runs — any of those invalidates the cache legitimately.
• Using follow_imports=skip for untyped libraries: Prevents reading .pyi stubs entirely and collapses everything imported from the module to Any, so call sites that touch it stop being checked at all. This causes false negatives and breaks type propagation across module boundaries. Use follow_imports=silent instead, which keeps the inferred types while hiding the dependency’s own errors.
• Applying ignore_missing_imports = true globally: Silences legitimate missing type hints in first-party code and hides genuine [import-not-found] typos. This masks critical integration errors and degrades coverage metrics. Apply it per-module via [mypy-package.*] sections, or better, disable only [import-untyped] for the specific dependency and install the matching types-* stub package.
• Confusing strict = true with per-module reach: strict is a global umbrella that flips on a dozen sub-flags; a lax per-module override placed after it in the file can silently undo pieces of it. Order sections least-specific-first and lock finished packages with exact (wildcard-free) names, as covered in enabling strict mode incrementally.

FAQ

How do I prevent mypy cache corruption in shared CI environments? Isolate cache directories per branch or job ID using environment variables (MYPY_CACHE_DIR). Run mypy --cache-dir /tmp/fresh_cache when you suspect corruption rather than deleting the shared cache.

Should I use follow_imports=skip to speed up large monorepos? No. Use follow_imports=silent instead. It preserves type inference from .pyi stubs while suppressing error output from untyped dependencies. skip silently drops type information from those imports.

How can I enforce strict typing only on newly added files? Combine git diff --name-only with a pre-commit hook that passes a dynamic file list to mypy. Maintain per-module overrides in mypy.ini for legacy paths to avoid false positives.

What is the optimal cache directory location for Dockerized CI runners? Mount a persistent volume or use GitHub Actions cache keys to preserve .mypy_cache across workflow runs. Ensure the volume is writable by the CI user and keyed on the lockfile hash to avoid stale caches after dependency updates.

Related

• Mypy Configuration & Strictness
• Automating Pre-Commit Type Validation
• Pyright vs Mypy: Type Checking Speed
• Static Analysis Tools & CI Integration

Back to Mypy Configuration & Strictness