Add spec, plan, and tasks for 030-bulk-import-sources feature
Defines the "Bulk Import All Sources" feature for the on-demand bestiary system: one-click loading of all ~104 bestiary sources with concurrent fetching, progress feedback, toast notifications, and completion reporting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Lukas
34
specs/030-bulk-import-sources/checklists/requirements.md
Normal file
34
specs/030-bulk-import-sources/checklists/requirements.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: Bulk Import All Sources
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2026-03-10
|
||||
**Feature**: [spec.md](../spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- All items pass. Spec is ready for `/speckit.clarify` or `/speckit.plan`.
|
||||
42
specs/030-bulk-import-sources/data-model.md
Normal file
42
specs/030-bulk-import-sources/data-model.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Data Model: Bulk Import All Sources
|
||||
|
||||
## Entities
|
||||
|
||||
### BulkImportState
|
||||
|
||||
Tracks the progress and outcome of a bulk import operation.
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| status | "idle" / "loading" / "complete" / "partial-failure" | Current phase of the import operation |
|
||||
| total | number | Total number of sources to fetch (excludes already-cached) |
|
||||
| completed | number | Number of sources successfully fetched and cached |
|
||||
| failed | number | Number of sources that failed to fetch |
|
||||
|
||||
**State Transitions**:
|
||||
- `idle` → `loading`: User clicks "Load All"
|
||||
- `loading` → `complete`: All sources fetched successfully (failed === 0)
|
||||
- `loading` → `partial-failure`: Some sources failed (failed > 0)
|
||||
- `complete` / `partial-failure` → `idle`: User dismisses or starts a new import
|
||||
|
||||
### ToastNotification
|
||||
|
||||
Lightweight notification data for the toast component.
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| message | string | Primary text to display (e.g., "Loading sources... 34/102") |
|
||||
| progress | number (0-1) or undefined | Optional progress bar value |
|
||||
| dismissible | boolean | Whether the toast shows a dismiss button |
|
||||
| autoDismissMs | number or undefined | Auto-dismiss delay in ms; undefined means persistent |
|
||||
|
||||
## Relationships
|
||||
|
||||
- **BulkImportState** drives the content of both the side panel progress UI and the toast notification.
|
||||
- **ToastNotification** is derived from BulkImportState when the side panel is closed during an active import.
|
||||
- Both consume state from the `useBulkImport` hook.
|
||||
|
||||
## Existing Entities (unchanged)
|
||||
|
||||
- **CachedSourceRecord** (bestiary-cache.ts): Stores normalized creature data per source in IndexedDB. No schema changes.
|
||||
- **BestiaryIndex** (domain): Read-only index with source codes and compact creature entries. No changes.
|
||||
67
specs/030-bulk-import-sources/plan.md
Normal file
67
specs/030-bulk-import-sources/plan.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Implementation Plan: Bulk Import All Sources
|
||||
|
||||
**Branch**: `030-bulk-import-sources` | **Date**: 2026-03-10 | **Spec**: [spec.md](./spec.md)
|
||||
**Input**: Feature specification from `/specs/030-bulk-import-sources/spec.md`
|
||||
|
||||
## Summary
|
||||
|
||||
Add a "Bulk Import All Sources" button to the top bar that opens the stat block side panel with a bulk import prompt. The user confirms a base URL, and the app fetches all ~104 bestiary source files concurrently, normalizes each, and caches them in IndexedDB. Progress is shown via a counter and progress bar in the side panel; if the panel is closed mid-import, a lightweight toast notification takes over progress display.
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: TypeScript 5.8 (strict mode, verbatimModuleSyntax)
|
||||
**Primary Dependencies**: React 19, Vite 6, Tailwind CSS v4, Lucide React (icons), idb (IndexedDB wrapper)
|
||||
**Storage**: IndexedDB for cached source data (existing via `bestiary-cache.ts`); localStorage for encounter persistence (existing, unchanged)
|
||||
**Testing**: Vitest
|
||||
**Target Platform**: Browser (desktop + mobile)
|
||||
**Project Type**: Web application (React SPA)
|
||||
**Performance Goals**: Non-blocking async import; UI remains responsive during ~12.5 MB download
|
||||
**Constraints**: All fetches fire concurrently (browser connection pooling); no third-party toast library
|
||||
**Scale/Scope**: ~104 sources, ~12.5 MB total data
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
| Principle | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| I. Deterministic Domain Core | PASS | No domain changes. All fetch/cache logic is in the adapter layer. |
|
||||
| II. Layered Architecture | PASS | Bulk import logic lives entirely in the adapter/UI layer (hooks + components). No domain or application layer changes needed. |
|
||||
| III. Agent Boundary | N/A | No agent layer involvement. |
|
||||
| IV. Clarification-First | PASS | Feature description was comprehensive; no ambiguities remain. |
|
||||
| V. Escalation Gates | PASS | All functionality is within spec scope. |
|
||||
| VI. MVP Baseline Language | PASS | No permanent bans introduced. |
|
||||
| VII. No Gameplay Rules | PASS | No gameplay mechanics involved. |
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/030-bulk-import-sources/
|
||||
├── plan.md # This file
|
||||
├── research.md # Phase 0 output
|
||||
├── data-model.md # Phase 1 output
|
||||
├── quickstart.md # Phase 1 output
|
||||
└── tasks.md # Phase 2 output (/speckit.tasks)
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
|
||||
```text
|
||||
apps/web/src/
|
||||
├── adapters/
|
||||
│ ├── bestiary-cache.ts # Existing — add isSourceCached batch check
|
||||
│ └── bestiary-index-adapter.ts # Existing — add getAllSourceCodes()
|
||||
├── components/
|
||||
│ ├── action-bar.tsx # Existing — add Import button
|
||||
│ ├── stat-block-panel.tsx # Existing — add bulk import mode
|
||||
│ ├── bulk-import-prompt.tsx # NEW — bulk import UI (description + URL + progress)
|
||||
│ └── toast.tsx # NEW — lightweight toast notification component
|
||||
├── hooks/
|
||||
│ ├── use-bestiary.ts # Existing — add bulkImport method
|
||||
│ └── use-bulk-import.ts # NEW — bulk import state management hook
|
||||
└── App.tsx # Existing — wire toast + bulk import panel mode
|
||||
```
|
||||
|
||||
**Structure Decision**: Follows existing patterns. New components for bulk import prompt and toast. New hook for import state management. Minimal changes to existing files (button + wiring).
|
||||
44
specs/030-bulk-import-sources/quickstart.md
Normal file
44
specs/030-bulk-import-sources/quickstart.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# Quickstart: Bulk Import All Sources
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20+, pnpm
|
||||
- Feature 029 (on-demand bestiary) merged and working
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
git checkout 030-bulk-import-sources
|
||||
pnpm install
|
||||
pnpm --filter web dev
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `apps/web/src/hooks/use-bulk-import.ts` | NEW — bulk import state + logic |
|
||||
| `apps/web/src/components/bulk-import-prompt.tsx` | NEW — side panel bulk import UI |
|
||||
| `apps/web/src/components/toast.tsx` | NEW — lightweight toast notification |
|
||||
| `apps/web/src/components/action-bar.tsx` | MODIFIED — Import button added |
|
||||
| `apps/web/src/components/stat-block-panel.tsx` | MODIFIED — bulk import mode |
|
||||
| `apps/web/src/App.tsx` | MODIFIED — wiring |
|
||||
| `apps/web/src/hooks/use-bestiary.ts` | MODIFIED — expose fetchAndCacheSource for bulk use |
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
pnpm test # All tests
|
||||
pnpm vitest run apps/web/src # Web app tests only
|
||||
pnpm check # Full merge gate
|
||||
```
|
||||
|
||||
## Manual Verification
|
||||
|
||||
1. Open app at `localhost:5173`
|
||||
2. Click Import button (top bar) — side panel opens with bulk import prompt
|
||||
3. Verify base URL is pre-filled, editable
|
||||
4. Click "Load All" — observe progress counter and bar
|
||||
5. Close side panel mid-import — toast appears at bottom-center
|
||||
6. Wait for completion — toast shows success/failure message
|
||||
7. Search for any creature — stat block displays without fetch prompt
|
||||
70
specs/030-bulk-import-sources/research.md
Normal file
70
specs/030-bulk-import-sources/research.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# Research: Bulk Import All Sources
|
||||
|
||||
## R1: Source Code List Availability
|
||||
|
||||
**Decision**: Use the existing bestiary index's `sources` object keys to enumerate all source codes for bulk fetching.
|
||||
|
||||
**Rationale**: The `loadBestiaryIndex()` function already returns a `BestiaryIndex` with a `sources: Record<string, string>` mapping all ~104 source codes to display names. This is the single source of truth.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Hardcoded source list: Rejected — would drift from index and require manual maintenance.
|
||||
- Fetching a remote manifest: Rejected — adds complexity and an extra network call.
|
||||
|
||||
## R2: URL Construction Pattern
|
||||
|
||||
**Decision**: Construct fetch URLs by appending `bestiary-{sourceCode.toLowerCase()}.json` to a user-provided base URL, matching the existing `getDefaultFetchUrl()` pattern.
|
||||
|
||||
**Rationale**: The `getDefaultFetchUrl` helper in `bestiary-index-adapter.ts` already implements this pattern. The bulk import reuses it with a configurable base URL prefix.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Per-source URL customization: Rejected — too complex for bulk operation; single base URL is sufficient.
|
||||
|
||||
## R3: Concurrent Fetch Strategy
|
||||
|
||||
**Decision**: Fire all fetch requests via `Promise.allSettled()` and let the browser handle HTTP/2 connection multiplexing and connection pooling (typically 6 concurrent connections per origin for HTTP/1.1).
|
||||
|
||||
**Rationale**: `Promise.allSettled()` (not `Promise.all()`) ensures that individual failures don't abort the entire operation. The browser naturally throttles concurrent connections, so no manual batching is needed.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Manual batching (e.g., 10 at a time): Rejected — adds complexity; browser pooling handles this naturally.
|
||||
- Sequential fetching: Rejected — too slow for 104 sources.
|
||||
|
||||
## R4: Progress State Management
|
||||
|
||||
**Decision**: Create a dedicated `useBulkImport` hook that manages import state (total, completed, failed, status) and exposes it to both the side panel component and the toast component.
|
||||
|
||||
**Rationale**: The import state needs to survive the side panel closing (toast takes over). Lifting state to a hook that lives in App.tsx ensures both UI targets can consume the same progress data.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Context provider: Rejected — overkill for a single piece of state consumed by 2 components.
|
||||
- Global state (zustand/jotai): Rejected — project doesn't use external state management; unnecessary dependency.
|
||||
|
||||
## R5: Toast Implementation
|
||||
|
||||
**Decision**: Build a minimal custom toast component using a React portal rendered at `document.body` level, positioned at bottom-center via fixed positioning.
|
||||
|
||||
**Rationale**: The spec requires no third-party toast library. A portal ensures the toast renders above all other content. The component needs only: text, progress bar, optional dismiss button, and auto-dismiss timer.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Third-party library (react-hot-toast, sonner): Rejected — spec explicitly requires custom component.
|
||||
- Non-portal approach: Rejected — would require careful z-index management and DOM nesting.
|
||||
|
||||
## R6: Skip-Already-Cached Strategy
|
||||
|
||||
**Decision**: Before firing fetches, check each source against `isSourceCached()` and build a filtered list of uncached sources. Update the total count to reflect only uncached sources.
|
||||
|
||||
**Rationale**: This avoids unnecessary network requests and gives accurate progress counts. The existing `isSourceCached()` function supports this directly.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Fetch all and overwrite: Rejected — wastes bandwidth and time.
|
||||
- Check during fetch (lazy): Rejected — harder to show accurate total count upfront.
|
||||
|
||||
## R7: Integration with Existing Bestiary Hook
|
||||
|
||||
**Decision**: The `useBulkImport` hook calls the existing `fetchAndCacheSource` from `useBestiary` for each source. After all sources complete, a single `refreshCache()` call reloads the creature map.
|
||||
|
||||
**Rationale**: Reuses the existing normalization + caching pipeline. Calling `refreshCache()` once at the end (instead of after each source) avoids O(N) full map rebuilds.
|
||||
|
||||
**Alternatives considered**:
|
||||
- Inline the fetch/normalize/cache logic in the bulk import hook: Rejected — duplicates code.
|
||||
- Call refreshCache after each source: Rejected — expensive O(N) rebuild on each call.
|
||||
128
specs/030-bulk-import-sources/spec.md
Normal file
128
specs/030-bulk-import-sources/spec.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# Feature Specification: Bulk Import All Sources
|
||||
|
||||
**Feature Branch**: `030-bulk-import-sources`
|
||||
**Created**: 2026-03-10
|
||||
**Status**: Draft
|
||||
**Input**: User description: "Add a Bulk Import All Sources feature to the on-demand bestiary system"
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - Bulk Load All Sources (Priority: P1)
|
||||
|
||||
The user wants to pre-load all 102 bestiary sources at once so that every creature's stat block is instantly available without per-source fetch prompts during gameplay. They click an import button (Import icon) in the top bar, which opens the stat block side panel. The panel shows a description of what will happen, a pre-filled base URL they can edit, and a "Load All" confirmation button. On confirmation, the app fetches all source files concurrently, normalizes them, and caches them in IndexedDB. Already-cached sources are skipped.
|
||||
|
||||
**Why this priority**: This is the core feature — enabling one-click loading of the entire bestiary is the primary user value.
|
||||
|
||||
**Independent Test**: Can be fully tested by clicking the import button, confirming the load, and verifying that all sources become available for stat block lookups.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** no sources are cached, **When** the user clicks the import button in the top bar, **Then** the stat block side panel opens showing a descriptive explanation, an editable pre-filled base URL, and a "Load All" button.
|
||||
2. **Given** the side panel is showing the bulk import prompt, **When** the user clicks "Load All", **Then** the app fires fetch requests for all 102 sources concurrently (appending `bestiary-{sourceCode}.json` to the base URL), normalizes each response, and caches results in IndexedDB.
|
||||
3. **Given** some sources are already cached, **When** the user initiates a bulk import, **Then** already-cached sources are skipped and only uncached sources are fetched.
|
||||
4. **Given** the user has edited the base URL, **When** they click "Load All", **Then** the app uses their custom base URL for all fetches.
|
||||
5. **Given** all fetches complete successfully, **When** the operation finishes, **Then** all creature stat blocks are immediately available for lookup without additional fetch prompts.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - Progress Feedback in Side Panel (Priority: P1)
|
||||
|
||||
While the bulk import is in progress, the user sees a text counter ("Loading sources... 34/102") and a progress bar in the side panel, giving them confidence the operation is proceeding.
|
||||
|
||||
**Why this priority**: Without progress feedback, the user has no way to know if the operation is working or stalled, especially for a ~12.5 MB download.
|
||||
|
||||
**Independent Test**: Can be tested by initiating a bulk import and observing the counter and progress bar update as sources complete.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a bulk import is in progress, **When** the user views the side panel, **Then** they see a text counter showing completed/total (e.g., "Loading sources... 34/102") and a visual progress bar.
|
||||
2. **Given** sources complete at different times, **When** each source finishes loading, **Then** the counter and progress bar update immediately.
|
||||
|
||||
---
|
||||
|
||||
### User Story 3 - Toast Notification on Panel Close (Priority: P2)
|
||||
|
||||
If the user closes the side panel while a bulk import is still in progress, a persistent toast notification appears at the bottom-center of the screen showing the same progress text and progress bar, so the user can continue using the app without losing visibility into the import status.
|
||||
|
||||
**Why this priority**: Allows the user to multitask while the import runs, which is important for a potentially long operation. Lower priority because the side panel already shows progress.
|
||||
|
||||
**Independent Test**: Can be tested by starting a bulk import, closing the side panel, and verifying the toast appears with progress information.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** a bulk import is in progress, **When** the user closes the side panel, **Then** a toast notification appears at the bottom-center of the screen showing the progress counter and progress bar.
|
||||
2. **Given** the toast is visible, **When** all sources finish loading successfully, **Then** the toast shows "All sources loaded" and auto-dismisses after a few seconds.
|
||||
3. **Given** the toast is visible, **When** some sources fail to load, **Then** the toast shows "Loaded 99/102 sources (3 failed)" (with actual counts) and remains visible until the user dismisses it.
|
||||
|
||||
---
|
||||
|
||||
### User Story 4 - Completion and Failure Reporting (Priority: P2)
|
||||
|
||||
On completion, the user sees a clear success or partial-failure message. Partial failures report how many sources succeeded and how many failed, so the user knows if they need to retry.
|
||||
|
||||
**Why this priority**: Essential for the user to know the outcome, but slightly lower than the core loading and progress functionality.
|
||||
|
||||
**Independent Test**: Can be tested by simulating network failures for some sources and verifying the correct counts appear.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** all sources load successfully, **When** the operation completes, **Then** the side panel (if open) or toast (if panel closed) shows "All sources loaded".
|
||||
2. **Given** some sources fail to load, **When** the operation completes, **Then** the message shows "Loaded X/102 sources (Y failed)" with accurate counts.
|
||||
3. **Given** completion message is in the toast, **When** the result is success, **Then** the toast auto-dismisses after a few seconds.
|
||||
4. **Given** completion message is in the toast, **When** the result is partial failure, **Then** the toast stays visible until manually dismissed.
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- What happens when the user clicks "Load All" while a bulk import is already in progress? The button should be disabled during an active import.
|
||||
- What happens when all 102 sources are already cached? The operation should complete immediately and report "All sources loaded" (0 fetches needed).
|
||||
- What happens when the network is completely unavailable? All fetches fail and the result shows "Loaded 0/102 sources (102 failed)".
|
||||
- What happens when the user navigates away or refreshes during import? Partially completed caches persist; the user can re-run to pick up remaining sources.
|
||||
- What happens if the base URL is empty or invalid? The "Load All" button should be disabled when the URL field is empty.
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST display an import button (Lucide Import icon) in the top bar that opens the stat block side panel with the bulk import prompt.
|
||||
- **FR-002**: System MUST show a descriptive text explaining the bulk import operation, including approximate data volume (~12.5 MB) and number of sources (102).
|
||||
- **FR-003**: System MUST pre-fill a base URL (`https://raw.githubusercontent.com/5etools-mirror-3/5etools-src/main/data/bestiary/`) that the user can edit.
|
||||
- **FR-004**: System MUST construct individual fetch URLs by appending `bestiary-{sourceCode}.json` to the base URL for each source.
|
||||
- **FR-005**: System MUST fire all fetch requests concurrently (browser handles connection pooling).
|
||||
- **FR-006**: System MUST skip sources that are already cached in IndexedDB.
|
||||
- **FR-007**: System MUST normalize fetched data using the existing normalization pipeline before caching.
|
||||
- **FR-008**: System MUST show a text counter ("Loading sources... N/T") and progress bar during the operation.
|
||||
- **FR-009**: System MUST show a toast notification with progress when the user closes the side panel during an active import.
|
||||
- **FR-010**: System MUST auto-dismiss the success toast after a few seconds.
|
||||
- **FR-011**: System MUST keep the partial-failure toast visible until the user dismisses it.
|
||||
- **FR-012**: The toast system MUST be a lightweight custom component — no third-party toast library.
|
||||
- **FR-013**: The bulk import MUST run asynchronously and not block the rest of the app.
|
||||
- **FR-014**: The user MUST explicitly provide/confirm the URL before any fetches occur (app never auto-fetches copyrighted content).
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **Bulk Import Operation**: Tracks total sources, completed count, failed count, and current status (idle/loading/complete/partial-failure).
|
||||
- **Toast Notification**: Lightweight UI element at bottom-center of screen with text, optional progress bar, and optional dismiss button.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: Users can load all 102 bestiary sources with a single confirmation action.
|
||||
- **SC-002**: Users see real-time progress during the bulk import (counter updates as each source completes).
|
||||
- **SC-003**: Users can close the side panel during import without losing progress visibility (toast appears).
|
||||
- **SC-004**: Already-cached sources are skipped, reducing redundant data transfer on repeat imports.
|
||||
- **SC-005**: The rest of the app remains fully interactive during the import operation.
|
||||
- **SC-006**: Users receive clear outcome reporting distinguishing full success from partial failure.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- The existing bestiary index contains all 102 source codes needed to construct fetch URLs.
|
||||
- The existing normalization pipeline handles all source file formats without modification.
|
||||
- The existing per-source fetch-and-cache logic serves as the reference implementation for individual source loading.
|
||||
- The base URL default matches the pattern already used for single-source fetches.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Feature 029 (on-demand bestiary): bestiary cache, bestiary index adapter, normalization pipeline, bestiary hook.
|
||||
171
specs/030-bulk-import-sources/tasks.md
Normal file
171
specs/030-bulk-import-sources/tasks.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# Tasks: Bulk Import All Sources
|
||||
|
||||
**Input**: Design documents from `/specs/030-bulk-import-sources/`
|
||||
**Prerequisites**: plan.md (required), spec.md (required), research.md, data-model.md
|
||||
|
||||
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
|
||||
|
||||
## Format: `[ID] [P?] [Story] Description`
|
||||
|
||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
|
||||
- Include exact file paths in descriptions
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Foundational (Blocking Prerequisites)
|
||||
|
||||
**Purpose**: Adapter helpers and core hook that all user stories depend on
|
||||
|
||||
- [ ] T001 Add `getAllSourceCodes()` helper to `apps/web/src/adapters/bestiary-index-adapter.ts` that returns all source codes from the bestiary index's `sources` object as a `string[]`
|
||||
- [ ] T002 Add `getBulkFetchUrl(baseUrl: string, sourceCode: string)` helper to `apps/web/src/adapters/bestiary-index-adapter.ts` that constructs `{baseUrl}bestiary-{sourceCode.toLowerCase()}.json` (ensure trailing slash normalization on baseUrl)
|
||||
- [ ] T003 Create `apps/web/src/hooks/use-bulk-import.ts` — the `useBulkImport` hook managing `BulkImportState` (status, total, completed, failed) with a `startImport(baseUrl: string, fetchAndCacheSource, isSourceCached, refreshCache)` method that: filters out already-cached sources via `isSourceCached`, fires all remaining fetches concurrently via `Promise.allSettled()`, increments completed/failed counters as each settles, calls `refreshCache()` once when all settle, and transitions status to `complete` or `partial-failure`
|
||||
|
||||
**Checkpoint**: Foundation ready — user story implementation can begin
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: User Story 1 — Bulk Load All Sources (Priority: P1)
|
||||
|
||||
**Goal**: User clicks an Import button in the top bar, sees a bulk import prompt in the side panel with editable base URL, and can load all sources with one click.
|
||||
|
||||
**Independent Test**: Click Import button → side panel opens with prompt → click "Load All" → all uncached sources are fetched, normalized, and cached in IndexedDB.
|
||||
|
||||
### Implementation for User Story 1
|
||||
|
||||
- [ ] T004 [US1] Create `apps/web/src/components/bulk-import-prompt.tsx` — component showing descriptive text ("Load stat block data for all 102 sources at once. This will download approximately 12.5 MB..."), an editable Input pre-filled with `https://raw.githubusercontent.com/5etools-mirror-3/5etools-src/main/data/bestiary/`, and a "Load All" Button (disabled when URL is empty or import is active). On click, calls a provided `onStartImport(baseUrl)` callback.
|
||||
- [ ] T005 [US1] Add Import button (Lucide `Import` icon) to `apps/web/src/components/action-bar.tsx` in the top bar area. Clicking it calls a new `onBulkImport` callback prop.
|
||||
- [ ] T006 [US1] Add bulk import mode to `apps/web/src/components/stat-block-panel.tsx` — when a new `bulkImportMode` prop is true, render `BulkImportPrompt` instead of the normal stat block or source fetch prompt content. Pass through `onStartImport` and bulk import state props.
|
||||
- [ ] T007 [US1] Wire bulk import in `apps/web/src/App.tsx` — add `bulkImportMode` state, pass `onBulkImport` to ActionBar (sets mode + opens panel), pass `bulkImportMode` and `useBulkImport` state to StatBlockPanel, call `useBulkImport.startImport()` with `fetchAndCacheSource`, `isSourceCached`, and `refreshCache` from `useBestiary`.
|
||||
|
||||
**Checkpoint**: User Story 1 fully functional — Import button opens panel, "Load All" fetches all sources, cached sources skipped
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 2 — Progress Feedback in Side Panel (Priority: P1)
|
||||
|
||||
**Goal**: During bulk import, the side panel shows a text counter ("Loading sources... 34/102") and a progress bar that updates in real time.
|
||||
|
||||
**Independent Test**: Start bulk import → observe counter and progress bar updating as each source completes.
|
||||
|
||||
### Implementation for User Story 2
|
||||
|
||||
- [ ] T008 [US2] Add progress display to `apps/web/src/components/bulk-import-prompt.tsx` — when import status is `loading`, replace the "Load All" button area with a text counter ("Loading sources... {completed}/{total}") and a Tailwind-styled progress bar (`<div>` with percentage width based on `(completed + failed) / total`). The component receives `BulkImportState` as a prop.
|
||||
|
||||
**Checkpoint**: User Stories 1 AND 2 work together — full import flow with live progress
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 3 — Toast Notification on Panel Close (Priority: P2)
|
||||
|
||||
**Goal**: If the user closes the side panel during an active import, a toast notification appears at bottom-center showing progress counter and bar.
|
||||
|
||||
**Independent Test**: Start bulk import → close side panel → toast appears with progress → toast updates as sources complete.
|
||||
|
||||
### Implementation for User Story 3
|
||||
|
||||
- [ ] T009 [P] [US3] Create `apps/web/src/components/toast.tsx` — lightweight toast component using `ReactDOM.createPortal` to `document.body`. Renders at bottom-center with fixed positioning. Shows: message text, optional progress bar, optional dismiss button (X). Accepts `onDismiss` callback. Styled with Tailwind (dark background, rounded, shadow, z-50).
|
||||
- [ ] T010 [US3] Wire toast visibility in `apps/web/src/App.tsx` — show the toast when bulk import status is `loading` AND the stat block panel is closed (or `bulkImportMode` is false). Derive toast message from `BulkImportState`: "Loading sources... {completed}/{total}" with progress value `(completed + failed) / total`. Hide toast when panel reopens in bulk import mode.
|
||||
|
||||
**Checkpoint**: User Story 3 functional — closing panel during import shows toast with progress
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: User Story 4 — Completion and Failure Reporting (Priority: P2)
|
||||
|
||||
**Goal**: On completion, show "All sources loaded" (auto-dismiss) or "Loaded X/Y sources (Z failed)" (persistent until dismissed).
|
||||
|
||||
**Independent Test**: Complete a successful import → see success message auto-dismiss. Simulate failures → see partial-failure message persist until dismissed.
|
||||
|
||||
### Implementation for User Story 4
|
||||
|
||||
- [ ] T011 [US4] Add completion states to `apps/web/src/components/bulk-import-prompt.tsx` — when status is `complete`, show "All sources loaded" success message. When status is `partial-failure`, show "Loaded {completed}/{total + completed} sources ({failed} failed)" message. Include a "Done" button to reset bulk import mode.
|
||||
- [ ] T012 [US4] Add completion behavior to toast in `apps/web/src/App.tsx` — when status is `complete`, show "All sources loaded" toast with `autoDismissMs` (e.g., 3000ms) and auto-hide via `setTimeout`. When status is `partial-failure`, show count message with dismiss button, no auto-dismiss. On dismiss, reset bulk import state to `idle`.
|
||||
- [ ] T013 [US4] Add auto-dismiss support to `apps/web/src/components/toast.tsx` — accept optional `autoDismissMs` prop. When set, start a `setTimeout` on mount that calls `onDismiss` after the delay. Clear timeout on unmount.
|
||||
|
||||
**Checkpoint**: All user stories complete — full flow with progress, toast, and completion reporting
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Polish & Cross-Cutting Concerns
|
||||
|
||||
**Purpose**: Edge cases and cleanup
|
||||
|
||||
- [ ] T014 Disable Import button in `apps/web/src/components/action-bar.tsx` while bulk import status is `loading` to prevent double-trigger
|
||||
- [ ] T015 Handle all-cached edge case in `apps/web/src/hooks/use-bulk-import.ts` — if all sources are already cached (0 to fetch), immediately transition to `complete` status without firing any fetches
|
||||
- [ ] T016 Run `pnpm check` (knip + format + lint + typecheck + test) and fix any issues
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
### Phase Dependencies
|
||||
|
||||
- **Foundational (Phase 1)**: No dependencies — can start immediately
|
||||
- **US1 (Phase 2)**: Depends on Phase 1 completion
|
||||
- **US2 (Phase 3)**: Depends on Phase 2 (extends `bulk-import-prompt.tsx` from US1)
|
||||
- **US3 (Phase 4)**: Depends on Phase 1 (uses `BulkImportState`); toast component (T009) can be built in parallel with US1/US2
|
||||
- **US4 (Phase 5)**: Depends on Phase 2 and Phase 4 (extends both panel and toast)
|
||||
- **Polish (Phase 6)**: Depends on all story phases complete
|
||||
|
||||
### User Story Dependencies
|
||||
|
||||
- **US1 (P1)**: Depends only on Foundational
|
||||
- **US2 (P1)**: Depends on US1 (extends same component)
|
||||
- **US3 (P2)**: Toast component (T009) is independent; wiring (T010) depends on US1
|
||||
- **US4 (P2)**: Depends on US1 (panel completion) and US3 (toast completion behavior)
|
||||
|
||||
### Parallel Opportunities
|
||||
|
||||
- T001 and T002 can run in parallel (different functions, same file)
|
||||
- T009 (toast component) can run in parallel with T004–T008 (different files)
|
||||
- T011 and T013 can run in parallel (different files)
|
||||
|
||||
---
|
||||
|
||||
## Parallel Example: Phase 1
|
||||
|
||||
```bash
|
||||
# These foundational tasks modify the same file, so run T001+T002 together then T003:
|
||||
Task: T001 + T002 "Add getAllSourceCodes and getBulkFetchUrl helpers"
|
||||
Task: T003 "Create useBulkImport hook" (independent file)
|
||||
```
|
||||
|
||||
## Parallel Example: US1 + Toast
|
||||
|
||||
```bash
|
||||
# Toast component can be built while US1 is in progress:
|
||||
Task: T004 "Create bulk-import-prompt.tsx" (US1)
|
||||
Task: T009 "Create toast.tsx" (US3) — runs in parallel, different file
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### MVP First (User Stories 1 + 2)
|
||||
|
||||
1. Complete Phase 1: Foundational helpers + hook
|
||||
2. Complete Phase 2: US1 — Import button, prompt, fetch logic
|
||||
3. Complete Phase 3: US2 — Progress counter + bar in panel
|
||||
4. **STOP and VALIDATE**: Full import flow works with progress feedback
|
||||
5. This delivers the core user value with 10 tasks (T001–T008)
|
||||
|
||||
### Incremental Delivery
|
||||
|
||||
1. Foundational → helpers and hook ready
|
||||
2. US1 + US2 → Full import with progress (MVP!)
|
||||
3. US3 → Toast notification on panel close
|
||||
4. US4 → Completion/failure reporting
|
||||
5. Polish → Edge cases and merge gate
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- [P] tasks = different files, no dependencies
|
||||
- [Story] label maps task to specific user story
|
||||
- US1 and US2 share `bulk-import-prompt.tsx` — US2 extends the component from US1
|
||||
- US3's toast component (T009) is fully independent and can be built any time
|
||||
- US4 adds completion behavior to both panel (from US1) and toast (from US3)
|
||||
- No test tasks included — not explicitly requested in spec
|
||||
Reference in New Issue
Block a user