Skip to content

ADR-0003: Optional-Dependency Extras for Backend Drivers

  • Date: 2026-05-06
  • Authors: Matteo Rizzo
  • Status: Accepted
  • Approval State: Approved (Approved by: Matteo Rizzo on 2026-05-06)
  • Implementation State: Completed

1. Context and Problem Statement

mirai-shared-skills is a meta-package: each downstream client uses some of the skills, none of them uses all of the skills. The catalog includes Neo4j, Azure AI Search, Cohere Rerank, and deepeval integrations — each of which pulls in a heavy SDK (Azure brings the entire azure-core HTTP layer; Cohere brings tokenization tooling; deepeval pulls in PyTorch transitively in some versions).

If every backend SDK were a required dependency of mirai-shared-skills, every client — including ones that only use WeatherSkill and PdfExtractionSkill — would pay the full install cost: a CI pipeline that runs uv sync would download 800 MB of wheels for a deployment that uses 8 MB of code. We need a way to ship only the drivers a given client actually uses, while keeping every provider's import path stable and giving missing-extra failures a typed, debuggable surface.

2. Decision Drivers (Forces)

  • Install footprint: A WeatherSkill-only client should not install neo4j, azure-*, or cohere.
  • Failure mode: A missing extra should fail with a typed exception that names which extra to install — not a bare ModuleNotFoundError deep in a vendor SDK.
  • No runtime probing in user code: Clients should not have to write try: import neo4j themselves.
  • Composability: A client that wants graph + vector should be able to pip install mirai-shared-skills[rag] and get both.
  • Test reproducibility: The dev profile installs every extra so the full test suite runs.
  • Symmetric with mirai-agent-core: agent-core uses the same [project.optional-dependencies] pattern (e.g. [temporal], [chainlit]).

3. Considered Options

  1. Option 1: Hard dependencies. Every backend SDK in [project.dependencies].
  2. Option 2: Vendor extras (chosen). Each backend family is an [project.optional-dependencies] entry: [neo4j], [azure], [reranker], [eval], plus a meta-extra [rag] that combines the three RAG-relevant ones.
  3. Option 3: Soft imports inside provider code. Providers import their SDK lazily and raise on first call, no extras declared.
  4. Option 4: Split each backend into its own package. mirai-shared-skills-neo4j, mirai-shared-skills-azure, etc.

4. Decision Outcome

Chosen option: Option 2 (vendor extras with typed unavailability errors), because it is the standard Python packaging idiom, lets uv lock resolve a coherent dependency graph per extra-set, and lets us raise a typed exception named after the missing extra so error messages are actionable.

The full set of extras shipped in pyproject.toml:

Extra Adds Use when
neo4j neo4j>=5.20 Using Neo4jGraphProvider.
azure azure-search-documents>=11.5 Using AzureSearchProvider.
reranker cohere>=5.13 Using a Cohere or HTTP-based reranker provider. The Qwen3 reranker uses httpx only and doesn't need this extra strictly, but the cohere SDK is listed for clients that want native typing.
rag All three above The full agentic RAG stack.
eval deepeval>=1.4 Running the agentic RAG eval harness.
dev mkdocs, mypy, pytest, ruff, etc. Local development.

Failure-mode contract: every provider that depends on an extra raises a typed exception subclassing ImportError if the SDK is missing — for example:

# mirai_shared_skills/agentic_rag/providers/neo4j_graph.py
class Neo4jUnavailableError(ImportError):
    """Raised when Neo4jGraphProvider is constructed without the `neo4j` extra installed.

    Install with: `pip install mirai-shared-skills[neo4j]` or `[rag]`.
    """

The provider's __init__ checks try: import neo4j and raises Neo4jUnavailableError if it fails, with the install hint embedded in the docstring/message. This converts the worst Python error message ("ModuleNotFoundError: No module named 'neo4j' from line 47 of an unrelated traceback") into a one-line actionable error.

4.1. Validation / Compliance

  • pyproject.toml lists every backend SDK as an extra, never as a direct dependency.
  • Each provider that requires an extra has a corresponding *UnavailableError class that subclasses ImportError.
  • pytest runs in two modes: uv sync --extra dev (no backends — provider construction must raise typed errors) and uv sync --all-extras (full — provider integration tests run).

5. Pros and Cons of the Options

Option 1: Hard dependencies

  • Pros: Simplest install command.
  • Cons: 800 MB install for an 8 MB app. Hostile to lightweight clients. CI cost compounds.

Option 2 (chosen): Vendor extras with typed errors

  • Pros:
  • Idiomatic Python packaging.
  • uv lock resolves cleanly per extra-set.
  • Typed errors give downstream developers an actionable message.
  • Meta-extras (rag) aggregate related backends without duplication.
  • Cons:
  • Authors of new backend providers must remember to add the extra and the typed error class.

Option 3: Soft imports, no extras

  • Pros: No pyproject.toml editing per backend.
  • Cons: Clients have no way to declare "install Neo4j too" — they just get errors at runtime if they forgot the SDK. Worse error messages.

Option 4: Split packages

  • Pros: Most extreme dependency isolation.
  • Cons: N+1 packages to release in lockstep; breaks the "single shared catalog" mental model.

6. Consequences

  • Positive Consequences:
  • pip install mirai-shared-skills is small. pip install mirai-shared-skills[rag] opts into the heavyweight stack.
  • Adding a new backend = one [project.optional-dependencies] entry + one *UnavailableError class.
  • The provider configuration guide can recommend [rag] for full RAG, [neo4j] for graph-only, etc., without exposing every transitive SDK.
  • Negative Consequences / Trade-offs:
  • Two failure surfaces: missing extra (raises *UnavailableError) vs. wrong-version extra (raises whatever the SDK raises). Documentation should emphasize the first; the second is rare in practice given the >= pins.
  • Risks & Mitigations:
  • Risk: A new backend is added without a typed error class, so missing-extra errors regress to ModuleNotFoundError. Mitigation: PR template requires the typed class; lint rule (future) can enforce.

7. Implementation Plan & Status Updates

  • Target Milestone/Release: v0.1.0 (current).
  • Implementation Notes:
  • 2026-05-06: ADR formalizes the existing extras layout. No code changes; this ADR codifies what pyproject.toml and Neo4jUnavailableError already do.