From 7199b9d2d92d5a91987cacfe2c2627a8ece30997 Mon Sep 17 00:00:00 2001 From: Lukas Date: Thu, 26 Mar 2026 20:10:57 +0100 Subject: [PATCH] Add browser-interactive-testing skill and fix Biome/audit config Integrate the rodney/showboat browser automation skill for headless Chrome screenshots and testing. Exclude .rodney and .agent-tests from Biome file scanning. Add picomatch override to resolve high-severity ReDoS vulnerability in knip/jscpd transitive deps. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../browser-interactive-testing/SKILL.md | 206 ++++++++++++++++++ .../scripts/setup.py | 160 ++++++++++++++ biome.json | 4 +- package.json | 3 +- pnpm-lock.yaml | 31 ++- 5 files changed, 384 insertions(+), 20 deletions(-) create mode 100644 .claude/skills/browser-interactive-testing/SKILL.md create mode 100644 .claude/skills/browser-interactive-testing/scripts/setup.py diff --git a/.claude/skills/browser-interactive-testing/SKILL.md b/.claude/skills/browser-interactive-testing/SKILL.md new file mode 100644 index 0000000..0183057 --- /dev/null +++ b/.claude/skills/browser-interactive-testing/SKILL.md @@ -0,0 +1,206 @@ +--- +name: browser-interactive-testing +description: > + This skill should be used when the user asks to "test a web page", + "take a screenshot of a site", "automate browser interaction", + "create a test report", "verify a page works", or mentions + rodney, showboat, headless Chrome testing, or browser automation. +version: 0.1.0 +--- + +# Browser Interactive Testing + +Test web pages interactively using **rodney** (headless Chrome automation) and document results with **showboat** (executable demo reports). + +## Prerequisites + +Ensure `uv` is installed. If missing, instruct the user to run: + +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +Do NOT install rodney or showboat globally. Run them via `uvx`: + +```bash +uvx rodney +uvx showboat +``` + +## Rodney Quick Reference + +### Start a browser session + +```bash +uvx rodney start # Launch headless Chrome +uvx rodney start --show # Launch visible browser (for debugging) +uvx rodney connect host:port # Connect to existing Chrome with remote debugging +``` + +Use `--local` on all commands to scope the session to the current directory. + +### Navigate and inspect + +```bash +uvx rodney open "https://example.com" +uvx rodney waitload +uvx rodney title +uvx rodney url +uvx rodney text "h1" +uvx rodney html "#content" +``` + +### Interact with elements + +```bash +uvx rodney click "#submit-btn" +uvx rodney input "#email" "user@example.com" +uvx rodney select "#country" "US" +uvx rodney js "document.querySelector('#app').dataset.ready" +``` + +### Assert and verify + +```bash +uvx rodney assert "document.title" "My App" -m "Title must match" +uvx rodney exists ".error-banner" +uvx rodney visible "#loading-spinner" +uvx rodney count ".list-item" +``` + +Exit code `0` = pass, `1` = fail, `2` = error. + +### Screenshots and cleanup + +```bash +uvx rodney screenshot -w 1280 -h 720 page.png +uvx rodney screenshot-el "#chart" chart.png +uvx rodney stop +``` + +Run `uvx rodney --help` for the full command list, including tab management, navigation, waiting, accessibility tree inspection, and PDF export. + +## Showboat Quick Reference + +```bash +uvx showboat init report.md "Test Report Title" +uvx showboat note report.md "Description of what we are testing." +uvx showboat exec report.md bash "uvx rodney title --local" +uvx showboat image report.md '![Page screenshot](screenshot.png)' +uvx showboat pop report.md # Remove last entry (fix mistakes) +uvx showboat verify report.md # Re-run all code blocks and diff +uvx showboat extract report.md # Print commands that recreate the document +``` + +Run `uvx showboat --help` for details on `--workdir`, `--output`, `--filename`, and stdin piping. + +## Output Directory + +Save all reports under `.agent-tests/` in the project root: + +``` +.agent-tests/ +└── YYYY-MM-DD-/ + ├── report.md + └── screenshots/ +``` + +Derive the slug from the test subject (e.g., `login-flow`, `homepage-layout`). Keep it lowercase, hyphen-separated, max ~30 chars. If a directory with the same date and slug already exists, append a numeric suffix (e.g., `tetris-game-2`) or choose a more specific slug (e.g., `tetris-controls` instead of reusing `tetris-game`). + +### Setup Script + +Run the bundled `scripts/setup.py` to create the directory, init the report, start the browser, and capture `DIR` in one step. Replace `` with the actual path to the directory containing this skill's files: + +```bash +DIR=$(python3 /scripts/setup.py "" "") +``` + +This single command: +1. Creates `.agent-tests/YYYY-MM-DD-/screenshots/` +2. Adds `.rodney/` to `.gitignore` (if `.gitignore` exists) +3. Runs `showboat init` for the report +4. Starts a browser (connects to existing, launches system Chrome/Chromium, or falls back to rodney's built-in launcher) +5. Prints the directory path to stdout (all status messages go to stderr) + +After setup, `$DIR` is ready for use with all subsequent commands. + +**Important:** The `--local` flag stores session data in `.rodney/` relative to the current working directory. Do NOT `cd` to a different directory during the session, or rodney will lose the connection. Use absolute paths for file arguments instead. + +## Workflow + +1. **Setup** — Run the setup script to create the dir, init the report, start the browser, and set `$DIR` +2. **Describe the test** — `uvx showboat note "$DIR/report.md" "Testing [subject] for [goals]."` so the report has context up front +3. **Open page** — `uvx showboat exec "$DIR/report.md" bash "uvx rodney open --local 'URL' && uvx rodney waitload --local"` +4. **Add a note** before each test group — Use a heading followed by a short explanation of what the tests in this section verify and why it matters. Use unique section titles; avoid duplicating headings within the same report. + ```bash + uvx showboat note "$DIR/report.md" "## Keyboard Controls" + uvx showboat note "$DIR/report.md" "Verify arrow keys move and rotate the active piece, and that soft/hard drop work correctly." + ``` +5. **Run assertions** — Before each assertion, add a short `showboat note` explaining what it checks. Then wrap the `rodney assert` / `rodney js` call in `showboat exec`: + ```bash + uvx showboat note "$DIR/report.md" "The left arrow key should move the piece one cell to the left." + uvx showboat exec "$DIR/report.md" bash "uvx rodney assert --local '...' '...' -m 'Piece moved left'" + ``` +6. **Capture screenshots** — Take the screenshot with `rodney screenshot`, then embed with `showboat image`. **Important:** `showboat image` resolves image paths relative to the current working directory, NOT relative to the report file. Always use absolute paths (`$DIR/screenshots/...`) in the markdown image reference to avoid "image file not found" errors: + ```bash + uvx rodney screenshot --local -w 1280 -h 720 "$DIR/screenshots/01-initial-load.png" + uvx showboat image "$DIR/report.md" "![Initial load]($DIR/screenshots/01-initial-load.png)" + ``` + Number screenshots sequentially (`01-`, `02-`, ...) and use descriptive filenames. +7. **Pop on failure** — If a command fails, run `showboat pop` then retry +8. **Stop browser** — `uvx rodney stop --local` +9. **Write summary** — Add a final `showboat note` with a summary section listing all pass/fail results and any bugs found. Every report must end with a summary. +10. **Verify report** — `uvx showboat verify "$DIR/report.md"` + +### Best Practices + +- Use `uvx rodney waitload` or `uvx rodney wait ` before interacting with page content. +- Run `uvx showboat pop` immediately after a failed `exec` to keep the report clean. +- Prefer `rodney assert` for checks — clear exit codes and self-documenting output. +- Use `rodney js` only for complex checks or state manipulation that `assert` cannot express. +- Take screenshots at key stages (initial load, after interaction, error states) for visual evidence. +- Add a `showboat note` before each logical group of tests with a heading and a short explanation of what the section tests. Use unique heading titles — duplicate headings make the report confusing. +- Always end reports with a summary `showboat note` listing pass/fail results and any bugs found. This is required, not optional. + +## Quoting Rules for `rodney js` + +`rodney js` evaluates a single JS **expression** (not statements). Nested shell quoting with `showboat exec` causes most errors. Follow these rules strictly: + +1. **Wrap multi-statement JS in an IIFE** — bare `const`, `let`, `for` fail at top level: + ```bash + # WRONG + uvx rodney js --local 'const x = 1; x + 2' + # CORRECT + uvx rodney js --local '(function(){ var x = 1; return x + 2; })()' + ``` + +2. **Use `var` instead of `const`/`let`** inside IIFEs to avoid strict-mode eval scoping issues. + +3. **Direct `rodney js` calls** — use single quotes for the outer shell, double quotes inside JS: + ```bash + uvx rodney js --local '(function(){ var el = document.querySelector("#app"); return el.textContent; })()' + ``` + +4. **Inside `showboat exec`** — use a heredoc with a **quoted delimiter** (`<<'JSEOF'`) to prevent all shell expansion (`$`, backticks, etc.): + ```bash + uvx showboat exec "$DIR/report.md" bash "$(cat <<'JSEOF' + uvx rodney js --local ' + (function(){ + var x = score; + hardDrop(); + return "before:" + x + ",after:" + score; + })() + ' + JSEOF + )" + ``` + For simple one-liners, single quotes inside the double-quoted bash arg also work: + ```bash + uvx showboat exec "$DIR/report.md" bash "uvx rodney js --local '(function(){ return String(score); })()'" + ``` + +5. **Avoid without heredoc**: backticks, `$` signs, unescaped double quotes. The heredoc pattern avoids all of these. + +6. **Prefer `rodney assert` over `rodney js`** when possible — separate arguments avoid quoting entirely. + +7. **Pop after syntax errors** — always `showboat pop` before retrying to keep the report clean. diff --git a/.claude/skills/browser-interactive-testing/scripts/setup.py b/.claude/skills/browser-interactive-testing/scripts/setup.py new file mode 100644 index 0000000..4553d30 --- /dev/null +++ b/.claude/skills/browser-interactive-testing/scripts/setup.py @@ -0,0 +1,160 @@ +#!/usr/bin/env python3 +"""Set up a browser-interactive-testing session. + +Creates the output directory, inits the showboat report, starts a browser, +and prints the DIR path. Automatically detects whether rodney can launch +its own Chromium or falls back to a system-installed browser. +""" + +import datetime +import os +import shutil +import socket +import subprocess +import sys +import time + +REMOTE_DEBUG_PORT = 9222 + + +def find_system_browser(): + """Return the path to a system Chrome/Chromium binary, or None.""" + for name in ["chromium", "chromium-browser", "google-chrome", "google-chrome-stable"]: + path = shutil.which(name) + if path: + return path + return None + + +def port_listening(port): + """Check if something is already listening on the given port.""" + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.settimeout(1) + return s.connect_ex(("localhost", port)) == 0 + + +def try_connect(port): + """Try to connect rodney to a browser on the given port. Returns True on success.""" + result = subprocess.run( + ["uvx", "rodney", "connect", "--local", f"localhost:{port}"], + capture_output=True, + text=True, + ) + if result.returncode == 0: + print(f"Connected to existing browser on port {port}", file=sys.stderr) + return True + return False + + +def launch_system_browser(browser_path): + """Launch a system browser with remote debugging and wait for it to be ready.""" + subprocess.Popen( + [ + browser_path, + "--headless", + "--disable-gpu", + f"--remote-debugging-port={REMOTE_DEBUG_PORT}", + "--no-sandbox", + ], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + # Wait for the browser to start listening + for _ in range(20): + if port_listening(REMOTE_DEBUG_PORT): + return True + time.sleep(0.25) + return False + + +def start_browser(): + """Start a headless browser and connect rodney to it. + + Strategy order (fastest path first): + 1. Connect to an already-running browser on the debug port. + 2. Launch a system Chrome/Chromium (avoids rodney's Chromium download, + which fails on some architectures like Linux ARM64). + 3. Let rodney launch its own browser as a last resort. + """ + # Strategy 1: connect to an already-running browser + if port_listening(REMOTE_DEBUG_PORT) and try_connect(REMOTE_DEBUG_PORT): + return + + # Strategy 2: launch a system browser (most reliable on Linux) + browser = find_system_browser() + if browser: + print(f"Launching system browser: {browser}", file=sys.stderr) + if launch_system_browser(browser): + if try_connect(REMOTE_DEBUG_PORT): + return + print("WARNING: system browser started but rodney could not connect", file=sys.stderr) + else: + print("WARNING: system browser did not start in time", file=sys.stderr) + + # Strategy 3: let rodney try its built-in launcher + result = subprocess.run( + ["uvx", "rodney", "start", "--local"], + capture_output=True, + text=True, + ) + if result.returncode == 0: + print("Browser started via rodney", file=sys.stderr) + return + + print( + "ERROR: Could not start a browser. Tried:\n" + f" - Connecting to localhost:{REMOTE_DEBUG_PORT} (no browser found)\n" + f" - System browser: {browser or 'not found'}\n" + " - rodney start (failed)\n" + "Install chromium or google-chrome and try again.", + file=sys.stderr, + ) + sys.exit(1) + + +def ensure_gitignore_entry(entry): + """Add entry to .gitignore if the file exists and the entry is missing.""" + gitignore = ".gitignore" + if not os.path.isfile(gitignore): + return + with open(gitignore, "r") as f: + content = f.read() + # Check if the entry (with or without trailing slash/newline variations) is already present + lines = content.splitlines() + if any(line.strip() == entry or line.strip() == entry.rstrip("/") for line in lines): + return + # Append the entry + with open(gitignore, "a") as f: + if content and not content.endswith("\n"): + f.write("\n") + f.write(f"{entry}\n") + print(f"Added '{entry}' to .gitignore", file=sys.stderr) + + +def main(): + if len(sys.argv) < 3: + print(f"Usage: {sys.argv[0]} ", file=sys.stderr) + sys.exit(1) + + slug = sys.argv[1] + title = sys.argv[2] + + # Create output directory + d = f".agent-tests/{datetime.date.today()}-{slug}" + os.makedirs(f"{d}/screenshots", exist_ok=True) + + # Ensure .rodney/ is in .gitignore (rodney stores session files there) + ensure_gitignore_entry(".rodney/") + + # Init showboat report + subprocess.run(["uvx", "showboat", "init", f"{d}/report.md", title], check=True) + + # Start browser + start_browser() + + # Print the directory path (only real stdout, everything else goes to stderr) + print(d) + + +if __name__ == "__main__": + main() diff --git a/biome.json b/biome.json index 28f7c4a..c063c76 100644 --- a/biome.json +++ b/biome.json @@ -8,7 +8,9 @@ "!.specify", "!specs", "!coverage", - "!.pnpm-store" + "!.pnpm-store", + "!.rodney", + "!.agent-tests" ] }, "assist": { diff --git a/package.json b/package.json index b7f0102..909c47a 100644 --- a/package.json +++ b/package.json @@ -3,7 +3,8 @@ "packageManager": "pnpm@10.32.1+sha512.a706938f0e89ac1456b6563eab4edf1d1faf3368d1191fc5c59790e96dc918e4456ab2e67d613de1043d2e8c81f87303e6b40d4ffeca9df15ef1ad567348f2be", "pnpm": { "overrides": { - "undici": ">=7.24.0" + "undici": ">=7.24.0", + "picomatch": ">=4.0.4" } }, "devDependencies": { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 3a7b055..b6ebfdf 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -6,6 +6,7 @@ settings: overrides: undici: '>=7.24.0' + picomatch: '>=4.0.4' importers: @@ -1082,7 +1083,7 @@ packages: resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} engines: {node: '>=12.0.0'} peerDependencies: - picomatch: ^3 || ^4 + picomatch: '>=4.0.4' peerDependenciesMeta: picomatch: optional: true @@ -1507,12 +1508,8 @@ packages: picocolors@1.1.1: resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==} - picomatch@2.3.1: - resolution: {integrity: sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==} - engines: {node: '>=8.6'} - - picomatch@4.0.3: - resolution: {integrity: sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==} + picomatch@4.0.4: + resolution: {integrity: sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==} engines: {node: '>=12'} postcss@8.5.8: @@ -2663,9 +2660,9 @@ snapshots: dependencies: walk-up-path: 4.0.0 - fdir@6.5.0(picomatch@4.0.3): + fdir@6.5.0(picomatch@4.0.4): optionalDependencies: - picomatch: 4.0.3 + picomatch: 4.0.4 fill-range@7.1.1: dependencies: @@ -2865,7 +2862,7 @@ snapshots: minimist: 1.2.8 oxc-resolver: 11.19.1 picocolors: 1.1.1 - picomatch: 4.0.3 + picomatch: 4.0.4 smol-toml: 1.6.0 strip-json-comments: 5.0.3 typescript: 5.9.3 @@ -3002,7 +2999,7 @@ snapshots: micromatch@4.0.8: dependencies: braces: 3.0.3 - picomatch: 2.3.1 + picomatch: 4.0.4 mimic-fn@2.1.0: {} @@ -3100,9 +3097,7 @@ snapshots: picocolors@1.1.1: {} - picomatch@2.3.1: {} - - picomatch@4.0.3: {} + picomatch@4.0.4: {} postcss@8.5.8: dependencies: @@ -3315,8 +3310,8 @@ snapshots: tinyglobby@0.2.15: dependencies: - fdir: 6.5.0(picomatch@4.0.3) - picomatch: 4.0.3 + fdir: 6.5.0(picomatch@4.0.4) + picomatch: 4.0.4 tinyrainbow@3.1.0: {} @@ -3356,7 +3351,7 @@ snapshots: vite@8.0.1(@types/node@25.3.3)(jiti@2.6.1)(yaml@2.8.3): dependencies: lightningcss: 1.32.0 - picomatch: 4.0.3 + picomatch: 4.0.4 postcss: 8.5.8 rolldown: 1.0.0-rc.10 tinyglobby: 0.2.15 @@ -3380,7 +3375,7 @@ snapshots: magic-string: 0.30.21 obug: 2.1.1 pathe: 2.0.3 - picomatch: 4.0.3 + picomatch: 4.0.4 std-env: 4.0.0 tinybench: 2.9.0 tinyexec: 1.0.4