Skip to content
flaky-tests

flaky-tests

Automatically detect, categorize, and track flaky tests across your CI runs — so intermittent failures stop slipping through the cracks.
Quick Start View on GitHub
HTML report preview — severity-ranked pattern cards with kind breakdown, hot files, and recent runs

Zero config

Add a single preload line to your Bun or Vitest config. Every test run is silently recorded — test names, durations, error messages, and stack traces — with zero changes to your test code.

Any backend

SQLite for local development, Turso or Supabase for shared team visibility, Postgres for existing infrastructure. All backends share the same interface — swap with an environment variable.

Detect patterns

The CLI compares failure rates across sliding time windows (default: 7 days) and flags tests that recently crossed a flakiness threshold. Failures are categorized as assertion errors, timeouts, or uncaught exceptions.

Close the loop

Generate AI-ready investigation prompts with full context, copy them to your clipboard, open GitHub issues automatically (with duplicate detection), or export an HTML report.

How it works

Section titled “How it works”

  1. Install a plugin — add @flaky-tests/plugin-bun or @flaky-tests/plugin-vitest as a preload. It hooks into your test runner and records every run to a local or remote database.

  2. Run your tests as usual — the plugin captures pass/fail results, error categories (assertion, timeout, uncaught), stack traces, git SHA, and timing data. No test code changes needed.

  3. Detect flakiness — run bunx @flaky-tests/core (or npx) to analyze recorded results. It compares the current time window against the prior window and surfaces tests that are newly failing intermittently.

  4. Take action — use --prompt to generate an investigation prompt you can paste into an AI assistant, --create-issue to open a GitHub issue for each flaky pattern, or --html to generate a visual report.