dostulata/initiative
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement the 015-add-jscpd-gate feature that adds copy-paste detection to the quality gate using jscpd with a 5% duplication threshold integrated into pnpm check

Browse Source 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Lukas
2026-03-06 00:13:32 +01:00
parent 56bced8481
commit 2793a66672
11 changed files with 1183 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
specs/015-add-jscpd-gate/checklists/requirements.md Normal file
View File

@@ -0,0 +1,34 @@
# Specification Quality Checklist: Copy-Paste Detection Quality Gate
**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: 2026-03-05
**Feature**: [spec.md](../spec.md)
## Content Quality
- [x] No implementation details (languages, frameworks, APIs)
- [x] Focused on user value and business needs
- [x] Written for non-technical stakeholders
- [x] All mandatory sections completed
## Requirement Completeness
- [x] No [NEEDS CLARIFICATION] markers remain
- [x] Requirements are testable and unambiguous
- [x] Success criteria are measurable
- [x] Success criteria are technology-agnostic (no implementation details)
- [x] All acceptance scenarios are defined
- [x] Edge cases are identified
- [x] Scope is clearly bounded
- [x] Dependencies and assumptions identified
## Feature Readiness
- [x] All functional requirements have clear acceptance criteria
- [x] User scenarios cover primary flows
- [x] Feature meets measurable outcomes defined in Success Criteria
- [x] No implementation details leak into specification
## Notes
- All items pass validation. Spec is ready for `/speckit.clarify` or `/speckit.plan`.

16
specs/015-add-jscpd-gate/data-model.md Normal file
View File

@@ -0,0 +1,16 @@
# Data Model: Copy-Paste Detection Quality Gate
This feature introduces no new domain entities, application state, or data persistence. It is a purely tooling/configuration feature.
## Configuration Entities
### jscpd Configuration (`.jscpd.json`)
- **threshold**: Maximum allowed duplication percentage (numeric, 0-100)
- **minLines**: Minimum number of lines for a block to be considered a duplicate (positive integer)
- **minTokens**: Minimum number of tokens for a block to be considered a duplicate (positive integer)
- **pattern**: Glob patterns for files to include in the scan (string array)
- **ignore**: Glob patterns for files/directories to exclude from the scan (string array)
- **reporters**: Output format for duplication reports (string array)
No state transitions, no relationships to existing domain entities.

66
specs/015-add-jscpd-gate/plan.md Normal file
View File

@@ -0,0 +1,66 @@
# Implementation Plan: Copy-Paste Detection Quality Gate
**Branch**: `015-add-jscpd-gate` | **Date**: 2026-03-06 | **Spec**: [spec.md](./spec.md)
**Input**: Feature specification from `/specs/015-add-jscpd-gate/spec.md`
## Summary
Add jscpd as a copy-paste detection tool to the project, configured with appropriate thresholds for TypeScript/TSX source files. Integrate it into the existing pre-commit quality gate (`pnpm check`) via Lefthook, and expose a standalone `pnpm jscpd` command for on-demand scanning. This is a tooling/configuration feature — no domain, application, or adapter code changes are required.
## Technical Context
**Language/Version**: TypeScript 5.8 (strict mode, verbatimModuleSyntax)
**Primary Dependencies**: jscpd (new dev dependency), Lefthook (existing), Biome 2.0 (existing), Knip (existing)
**Storage**: N/A (no storage changes)
**Testing**: Vitest (existing) — verification via manual/script testing of jscpd config and pre-commit hook behavior
**Target Platform**: Node.js / developer workstation (macOS, Linux)
**Project Type**: Monorepo workspace (pnpm) — web app with domain/application/adapter layers
**Performance Goals**: Scan completes in under 10 seconds for the current codebase
**Constraints**: Must not introduce false positives on structurally repeated patterns (imports, type declarations)
**Scale/Scope**: ~3 workspace packages, small codebase
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
| Principle | Status | Notes |
|-----------|--------|-------|
| I. Deterministic Domain Core | N/A | No domain changes |
| II. Layered Architecture | N/A | No layer changes — tooling config only |
| III. Agent Boundary | N/A | No agent layer involvement |
| IV. Clarification-First | PASS | Feature scope is clear from spec |
| V. Escalation Gates | PASS | All work within spec scope |
| VI. MVP Baseline Language | PASS | Scope boundaries use MVP baseline language |
| VII. No Gameplay Rules | N/A | Not applicable |
| Dev Workflow: Merge gate | PASS | Integrates into existing `pnpm check` gate |
| Dev Workflow: Spec-first | PASS | Spec exists at `specs/015-add-jscpd-gate/spec.md` |
All gates pass. No violations to justify.
## Project Structure
### Documentation (this feature)
```text
specs/015-add-jscpd-gate/
├── plan.md
├── research.md
├── data-model.md
├── quickstart.md
└── tasks.md # Created by /speckit.tasks
```
### Source Code (repository root)
```text
(root)
├── package.json # Add jscpd devDependency + "jscpd" script, update "check" script
├── lefthook.yml # Already runs `pnpm check` — no changes needed
└── .jscpd.json # New: jscpd configuration file (thresholds, patterns, ignore)
```
**Structure Decision**: This feature is purely tooling configuration at the workspace root. No changes to `packages/` or `apps/` directories. The jscpd config file (`.jscpd.json`) lives at the repo root alongside existing config files (`biome.json`, `knip.json`). The `pnpm check` script in `package.json` already serves as the Lefthook pre-commit command, so adding jscpd to the check script chain automatically integrates it into the pre-commit gate.
## Complexity Tracking
No constitution violations. Table not applicable.

36
specs/015-add-jscpd-gate/quickstart.md Normal file
View File

@@ -0,0 +1,36 @@
# Quickstart: Copy-Paste Detection Quality Gate
## What This Feature Does
Adds automated copy-paste detection to the project's quality gates. When a developer commits code, jscpd scans for duplicated code blocks and blocks the commit if duplication exceeds the configured threshold. Developers can also run the scan on-demand.
## Key Commands
```bash
# Run copy-paste detection standalone
pnpm jscpd
# Run full quality gate (includes jscpd)
pnpm check
# Commit triggers the check automatically via Lefthook
git commit -m "my changes"
```
## Configuration
The jscpd configuration lives in `.jscpd.json` at the repo root. Key settings:
- **threshold**: Max allowed duplication percentage (default: 5%)
- **minLines**: Minimum lines for a duplicate block (default: 5)
- **minTokens**: Minimum tokens for a duplicate block (default: 50)
- **pattern**: File globs to scan (TypeScript/TSX)
- **ignore**: Directories/files to skip (node_modules, dist, etc.)
## How It Integrates
```
git commit → Lefthook pre-commit → pnpm check → knip + biome + tsc + vitest + jscpd
```
The `pnpm check` script is the single merge gate. jscpd is added to this chain, so it automatically runs as part of pre-commit hooks and any CI that uses `pnpm check`.

49
specs/015-add-jscpd-gate/research.md Normal file
View File

@@ -0,0 +1,49 @@
# Research: Copy-Paste Detection Quality Gate
## jscpd Configuration Strategy
**Decision**: Use `.jscpd.json` at the repo root for configuration.
**Rationale**: jscpd supports a dedicated JSON config file (`.jscpd.json`) which is the conventional approach. This keeps configuration discoverable alongside other tool configs (`biome.json`, `knip.json`). Command-line flags could work but are less maintainable and harder to share across scripts.
**Alternatives considered**:
- CLI flags in the `pnpm jscpd` script: Less discoverable, harder to maintain threshold changes.
- `package.json` `jscpd` key: Supported but clutters package.json; separate file preferred for consistency with project conventions.
## Threshold Configuration
**Decision**: Use jscpd defaults with minor tuning — minimum 5 lines, minimum 50 tokens for duplicate detection. Set a percentage threshold (e.g., 5% max duplication) to fail the check.
**Rationale**: jscpd's defaults (5 lines, 50 tokens) are well-established industry standards that avoid flagging trivially similar code (imports, short utility patterns) while catching meaningful copy-paste blocks. The percentage threshold provides a clear pass/fail gate.
**Alternatives considered**:
- Stricter thresholds (3 lines, 30 tokens): Too aggressive, would flag structural similarities common in TypeScript (type declarations, import blocks).
- No percentage threshold (fail on any duplicate): Too strict for an existing codebase that may have some acceptable duplication.
## File Inclusion/Exclusion Strategy
**Decision**: Scan TypeScript and TSX files (`**/*.ts`, `**/*.tsx`). Exclude `node_modules`, `dist`, `build`, `coverage`, `.specify`, `specs`, and lock files via jscpd's `ignore` configuration.
**Rationale**: The project is TypeScript-only. Excluding build artifacts, vendored dependencies, and non-source files prevents false positives and keeps scan times fast.
**Alternatives considered**:
- Scan all file types: Unnecessary — the project contains no other source languages.
- Exclude test files: Rejected per spec assumption — test duplication is also worth catching.
## Integration with pnpm check
**Decision**: Add `jscpd` to the `pnpm check` script chain in `package.json`, running it alongside knip, biome, typecheck, and vitest.
**Rationale**: The existing `pnpm check` script is already the pre-commit gate via Lefthook (`lefthook.yml` runs `pnpm check`). Adding jscpd to this chain automatically integrates it into the pre-commit workflow with zero Lefthook config changes.
**Alternatives considered**:
- Separate Lefthook job for jscpd: Would work but deviates from the existing pattern where `pnpm check` is the single merge gate.
- Run jscpd only in CI: Misses the pre-commit enforcement requirement from the spec.
## Knip Integration
**Decision**: Ensure jscpd is recognized by Knip as a used dev dependency. Knip may need the binary referenced in a script to avoid being flagged as unused.
**Rationale**: The project runs `knip` as part of `pnpm check`. Adding `jscpd` as a devDependency and referencing it in a package.json script ensures Knip won't report it as unused.
**Alternatives considered**: None — this is a necessary housekeeping step given the project's use of Knip.

99
specs/015-add-jscpd-gate/spec.md Normal file
View File

@@ -0,0 +1,99 @@
# Feature Specification: Copy-Paste Detection Quality Gate
**Feature Branch**: `015-add-jscpd-gate`
**Created**: 2026-03-05
**Status**: Draft
**Input**: User description: "introduce jscpd to detect copy/pasted code and add it as a quality gate for pre commit"
## User Scenarios & Testing *(mandatory)*
### User Story 1 - Automated Copy-Paste Detection on Commit (Priority: P1)
A developer working on the codebase makes changes and attempts to commit. Before the commit completes, the system automatically scans the codebase for duplicated code blocks. If duplication exceeds the configured threshold, the commit is blocked and the developer sees a clear report identifying the duplicated sections, enabling them to refactor before committing.
**Why this priority**: This is the core value proposition — preventing code duplication from entering the codebase by catching it at the earliest possible point in the development workflow.
**Independent Test**: Can be fully tested by introducing a known duplicated code block and attempting to commit. The commit should be blocked with a report showing the duplication.
**Acceptance Scenarios**:
1. **Given** a codebase with no duplicated code above threshold, **When** a developer commits changes that do not introduce duplication, **Then** the commit succeeds without copy-paste warnings.
2. **Given** a developer has staged changes containing duplicated code blocks, **When** they attempt to commit, **Then** the commit is blocked and a report lists the duplicated files and line ranges.
3. **Given** a developer has staged changes with duplication, **When** the pre-commit check fails, **Then** the output clearly identifies which code blocks are duplicated and where.
---
### User Story 2 - On-Demand Duplication Scanning (Priority: P2)
A developer wants to proactively check the codebase for duplicated code without committing. They run a dedicated command that scans the project and produces a duplication report, allowing them to identify and address copy-paste issues at any time.
**Why this priority**: Supports proactive code quality improvement outside the commit workflow, but the pre-commit gate (P1) is the primary enforcement mechanism.
**Independent Test**: Can be tested by running the duplication scan command on a codebase with known duplicated sections and verifying the report output.
**Acceptance Scenarios**:
1. **Given** a codebase with duplicated code, **When** the developer runs the duplication scan command, **Then** a report is produced listing all duplicated blocks with file paths and line numbers.
2. **Given** a codebase with no duplication above threshold, **When** the developer runs the scan, **Then** the command exits successfully with a clean result.
---
### User Story 3 - Integration with Existing Quality Gate (Priority: P3)
The copy-paste detection check is integrated into the existing unified quality gate command so that it runs alongside linting, formatting, type checking, and tests as part of the single merge gate.
**Why this priority**: Ensures consistency with the existing developer workflow and CI expectations, but depends on the core detection (P1) being implemented first.
**Independent Test**: Can be tested by running the unified quality gate and verifying that duplication scanning is included in the output alongside other checks.
**Acceptance Scenarios**:
1. **Given** the existing unified quality gate command, **When** a developer runs it, **Then** copy-paste detection runs as one of the checks.
2. **Given** a codebase with duplication above threshold, **When** the unified quality gate is run, **Then** the overall command fails and the duplication report is visible in the output.
---
### Edge Cases
- What happens when the codebase has zero source files matching the scan pattern? The scan should succeed with a clean result.
- How does the system handle generated code or build artifacts that may contain legitimate repetition? These are excluded from scanning by default via configuration.
- What happens if the detection tool is not installed? The pre-commit hook and quality gate should fail with a clear error message indicating the missing dependency.
- How does the system behave when duplication exists only in test files? Test files are scanned by default, as duplication in tests also indicates refactoring opportunities.
## Requirements *(mandatory)*
### Functional Requirements
- **FR-001**: The system MUST scan source code files for duplicated code blocks before each commit via the pre-commit hook.
- **FR-002**: The system MUST block commits when code duplication exceeds the configured threshold.
- **FR-003**: The system MUST produce a human-readable report identifying duplicated code blocks, including file paths and line ranges.
- **FR-004**: The system MUST provide a standalone command for running the duplication scan independently of the commit workflow.
- **FR-005**: The system MUST integrate the duplication check into the existing unified quality gate command.
- **FR-006**: The system MUST allow configuration of duplication detection thresholds (minimum token count and minimum number of lines to qualify as a duplicate).
- **FR-007**: The system MUST support excluding specific files or directories from duplication scanning (e.g., generated code, build output, lock files).
- **FR-008**: The system MUST scan project source files relevant to the codebase (TypeScript/TSX) and exclude non-source artifacts by default.
## Success Criteria *(mandatory)*
### Measurable Outcomes
- **SC-001**: 100% of commits pass through the copy-paste detection gate before being accepted.
- **SC-002**: Developers receive duplication feedback within 10 seconds for a typical project-sized codebase.
- **SC-003**: Zero false positives on import statements, type declarations, or other structurally necessary repetitions when using default thresholds.
- **SC-004**: The standalone scan command completes successfully and reports all duplicated blocks in the codebase.
- **SC-005**: The unified quality gate includes and enforces the duplication check alongside existing checks.
## Assumptions
- The project already uses Lefthook for pre-commit hooks, so the new check will be added to the existing hook configuration.
- The duplication detection tool will be added as a development dependency.
- Default thresholds will follow industry-standard defaults (typically 5+ lines or 50+ tokens) and can be adjusted via configuration.
- Test files are included in the scan by default, as duplication in tests can also indicate refactoring opportunities.
- Generated files (e.g., build output, lock files) are excluded by default.
## Scope Boundaries
- **MVP baseline does not include**: CI/CD pipeline integration beyond what the unified quality gate already provides.
- **MVP baseline does not include**: Historical duplication trend tracking or reporting dashboards.
- **MVP baseline does not include**: Auto-fix or refactoring suggestions for detected duplications.
- **MVP baseline does not include**: Per-file or per-directory threshold overrides.

126
specs/015-add-jscpd-gate/tasks.md Normal file
View File

@@ -0,0 +1,126 @@
# Tasks: Copy-Paste Detection Quality Gate
**Input**: Design documents from `/specs/015-add-jscpd-gate/`
**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, quickstart.md
**Tests**: Not explicitly requested in the spec. Test tasks are omitted.
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
## Format: `[ID] [P?] [Story] Description`
- **[P]**: Can run in parallel (different files, no dependencies)
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
- Include exact file paths in descriptions
---
## Phase 1: Setup
**Purpose**: Install jscpd and create its configuration file
- [x] T001 Install jscpd as a dev dependency by adding it to `package.json` devDependencies and running `pnpm install`
- [x] T002 Create jscpd configuration file at `.jscpd.json` with threshold (5%), minLines (5), minTokens (50), pattern for TypeScript/TSX files, ignore list for `node_modules`, `dist`, `build`, `coverage`, `.specify`, `specs`, and add a standalone `"jscpd": "jscpd"` script to `package.json`
**Checkpoint**: jscpd is installed and `pnpm jscpd` runs successfully, scanning TypeScript/TSX files and producing a report
---
## Phase 2: User Story 1 - Automated Copy-Paste Detection on Commit (Priority: P1) 🎯 MVP
**Goal**: Pre-commit hook blocks commits when code duplication exceeds the configured threshold
**Independent Test**: Introduce a duplicated code block across two files, attempt to commit, and verify the commit is blocked with a clear duplication report. Remove the duplication and verify the commit succeeds.
### Implementation for User Story 1
- [x] T003 [US1] Add `jscpd` to the `"check"` script chain in `package.json` so it runs as part of the merge gate (e.g., `"check": "knip && biome check . && tsc --build && vitest run && jscpd"`)
**Checkpoint**: Running `pnpm check` now includes jscpd. Since Lefthook runs `pnpm check` as the pre-commit hook, commits are automatically gated by copy-paste detection. A commit with duplicated code above threshold is blocked.
---
## Phase 3: User Story 2 - On-Demand Duplication Scanning (Priority: P2)
**Goal**: Developers can run duplication scanning independently via a standalone command
**Independent Test**: Run `pnpm jscpd` on the codebase and verify it produces a duplication report with file paths and line numbers for any detected duplicates, or exits cleanly if none found.
### Implementation for User Story 2
No additional tasks — `pnpm jscpd` was already configured in T002. This story is satisfied by the setup phase.
**Checkpoint**: `pnpm jscpd` runs independently and produces a readable duplication report
---
## Phase 4: User Story 3 - Integration with Existing Quality Gate (Priority: P3)
**Goal**: Copy-paste detection runs as part of the unified `pnpm check` command alongside all other checks
**Independent Test**: Run `pnpm check` and verify jscpd output appears in the results alongside knip, biome, tsc, and vitest output.
### Implementation for User Story 3
No additional tasks — integration into `pnpm check` was completed in T003. This story is satisfied by the US1 implementation.
**Checkpoint**: `pnpm check` includes jscpd and the full gate passes on a clean codebase
---
## Phase 5: Polish & Cross-Cutting Concerns
**Purpose**: Validate the full workflow end-to-end
- [x] T004 Verify `pnpm check` passes on the current codebase (no existing duplication above threshold), confirm scan completes within 10 seconds (SC-002), verify no false positives on import statements or type declarations (SC-003), and adjust threshold or ignore patterns in `.jscpd.json` if needed
- [x] T005 Verify Knip does not flag jscpd as an unused dependency by running `pnpm knip`
---
## Dependencies & Execution Order
### Phase Dependencies
- **Setup (Phase 1)**: No dependencies — T001 then T002
- **US1 (Phase 2)**: Depends on T002
- **US2 (Phase 3)**: Satisfied by T002 (no additional work)
- **US3 (Phase 4)**: Satisfied by T003 (no additional work)
- **Polish (Phase 5)**: Depends on T003
### Execution Order
```
T001 → T002 → T003 → T004, T005 (parallel)
```
### Parallel Opportunities
- T004 and T005 can run in parallel (independent validations)
- This is a small, linear feature — limited parallelism needed
---
## Implementation Strategy
### MVP First (User Story 1 Only)
1. Complete Phase 1: Install jscpd + configure (T001, T002)
2. Complete Phase 2: Add to check script (T003)
3. **STOP and VALIDATE**: Attempt a commit with duplicated code — verify it's blocked
4. All three user stories are satisfied at this point
### Incremental Delivery
This feature is small enough that all stories are delivered in a single pass:
1. T001-T002: Setup → `pnpm jscpd` works (US2 done)
2. T003: Add to check → pre-commit gate works (US1 done), quality gate works (US3 done)
3. T004-T005: Polish → verify everything integrates cleanly
---
## Notes
- This is a tooling/configuration feature — no domain, application, or adapter code changes
- All changes are at the workspace root (`package.json`, `.jscpd.json`)
- Lefthook config (`lefthook.yml`) does NOT need changes — it already runs `pnpm check`
- US2 and US3 are naturally satisfied by the setup and US1 implementation with no additional tasks