initiative/docs/adr/004-on-demand-bestiary-loading.md

ADR-004: On-Demand Bestiary Loading via Compact Index and IndexedDB Cache

Date: 2026-03-25 Status: accepted

Context

The application integrates a D&D creature bestiary containing 3,300+ creatures from the 5etools dataset. The full bestiary data (stat blocks, traits, actions, spellcasting) is several megabytes of JSON. Bundling it directly into the application would create two problems: a large initial download for every user, and the distribution of copyrighted game content as part of the application bundle.

Decision

The bestiary is split into two tiers:

1. Compact search index (data/bestiary/index.json, ~350KB) — shipped with the application bundle. Contains only the fields needed for search and display in the autocomplete dropdown: name, source, AC, HP, DEX, CR, initiative proficiency, size, and type. Field names are abbreviated (n, s, ac, hp, dx, cr, ip, sz, tp) to minimize file size. Generated offline by scripts/generate-bestiary-index.mjs from a local clone of the 5etools repository.

2. On-demand source data — full creature stat blocks are fetched per-source when a user first needs them (e.g., when viewing a stat block or adding a creature with HP/AC pre-fill). Fetched data is cached in IndexedDB (initiative-bestiary database) via the idb library, with an in-memory Map fallback when IndexedDB is unavailable. Users can also upload source files directly or bulk-import all sources.

The application never bundles or redistributes the full creature data. Users fetch it themselves from their own configured source URLs.

Alternatives Considered

Bundle all bestiary data — simplest approach, used during early development. Eliminated because it would distribute copyrighted content in the application bundle and inflate the initial download by several megabytes. Most users only need a fraction of the available sources.

Server-side API — a backend service could serve creature data on demand. This would keep the client lightweight and solve the bundle size concern, but the copyright issue remains — we would still be distributing copyrighted content, just from a server instead of a bundle. It also contradicts the project's local-first, single-user, no-backend architecture and would require hosting infrastructure and a network dependency for basic functionality.

Service Worker with lazy caching — fetch and cache bestiary data transparently via a Service Worker. More complex to implement and debug than explicit IndexedDB caching. The explicit approach gives users visibility and control over which sources are cached (via the source manager UI).

localStorage for caching — simpler API than IndexedDB, but localStorage has a ~5MB limit per origin, which is insufficient for multiple bestiary sources. IndexedDB has no practical storage limit.

Consequences

Positive:

• The application does not distribute copyrighted game content. Users fetch data from their own sources.
• Initial bundle stays small (~350KB for the search index). The full bestiary data is only downloaded when needed and then cached locally.
• Offline capability: once sources are cached in IndexedDB, creature data is available without network access.
• Users have explicit control over cached sources (import, clear, manage via UI).

Negative:

• First-time use requires fetching source data before full stat blocks are available. The bulk import feature mitigates this but requires an initial download.
• The search index must be regenerated manually when the upstream 5etools dataset changes. In practice this is infrequent (new D&D source books release a few times per year), so a manual process triggered by a new book release is sufficient at this scale.
• Two separate data representations (compact index vs full source) must be kept conceptually in sync. A creature that appears in the index but whose source hasn't been fetched will show limited information until the source is cached.
• IndexedDB adds adapter complexity (async API, database versioning, migration handling) compared to the synchronous localStorage used for encounter persistence.