scrutineer

An extensible test framework for CLI programs, REST APIs, GraphQL, gRPC, browsers, and network protocols. Declarative YAML tests. Zero third-party dependencies.

Go 1.26+MIT Licensev0.0.1-dev
Get StartedView Source
example.test.yaml
suite: "User API Tests"
tags: [api, smoke]

tests:
  - name: "Create user returns 201"
    connector: http
    steps:
      - action: request
        method: POST
        path: /users
        body:
          name: "Alice"
          email: "alice@example.com"
        assert:
          - status: 201
          - body.name: {equals: "Alice"}
          - elapsed: {less_than: 2s}
        capture:
          user_id: body.id

  - name: "Verify user exists"
    connector: http
    steps:
      - action: request
        method: GET
        path: /users/${capture.user_id}
        assert:
          - status: 200
          - body.email: {equals: "alice@example.com"}

Features

Everything you need for comprehensive testing, in a single binary with zero dependencies.

Declarative YAML Tests

Define tests as data, not code. Describe what to assert, not how to execute. Familiar to Playwright and assertion-based test users.

Modular Connectors

CLI, HTTP, SSH, gRPC, GraphQL, and browser connectors. Add new protocols by implementing a single Go interface.

Browser Automation

Headless Chromium, Firefox, and WebKit via Chrome DevTools Protocol. Selectors, interactions, screenshots, network interception.

Load Testing

Parallel test execution distributed across nodes via SSH. Configurable concurrency, ramp-up, and duration. Locust-style scaling.

Nanosecond Telemetry

Every test captures timing data automatically. Structured binary TLV logs with nanosecond timestamps for benchmark analysis.

Zero Dependencies

Built with the Go standard library. No node_modules, no pip packages, no dependency hell. One binary, every platform.

Coverage as a Feature

Built-in test coverage measurement with configurable thresholds. Know exactly which tests ran, which steps executed, which assertions fired.

Fuzz Testing

Declarative fuzz targets integrated with Go's built-in fuzzing. Corpus management and automated edge-case discovery.

Rich Assertions

Equality, contains, regex, JSON path, HTTP status, headers, timing, collections. Extensible assertion library with clear error messages.

Protocol Support

Test anything that speaks a protocol.

ProtocolConnectorStatusFeatures
HTTP/1.1, HTTP/2httpv0.0.1TLS 1.2/1.3, self-signed certs, request/response assertions
REST APIshttpv0.0.1CRUD, auth (Bearer, Basic, API key), JSON/XML, pagination, HATEOAS
GraphQLhttpv0.0.1Queries, mutations, subscriptions, introspection, variables
gRPC / Protobufgrpcv0.0.1Unary, client/server/bidi streaming, .proto + reflection
SSHsshv0.0.1Key-based auth, command execution, tunneling
CLI Programscliv0.0.1stdin/stdout/stderr, exit codes, filesystem side-effects
Chromium / Firefox / WebKitbrowserv0.0.1CDP, selectors, interactions, screenshots, network mocking
HTTP/3 (QUIC)httpplannedPending Go stdlib or from-scratch QUIC
SMTPsmtpplannedSend, auth, envelope validation
IMAPimapplannedMailbox access, search, fetch

Install

One command. No dependencies.

$ go install github.com/asymmetric-effort/scrutineer/cmd/scrutineer@latest

Or download a pre-built binary from Releases for Linux, macOS, or Windows (AMD64 / ARM64).

Quick start
# Install scrutineer
$ go install github.com/asymmetric-effort/scrutineer/cmd/scrutineer@latest

# Install browsers (for browser testing)
$ scrutineer browsers install

# Run tests
$ scrutineer run

# Run with JSON output
$ scrutineer run --format json

# Dump binary telemetry logs
$ scrutineer log-dump scrutineer.log

Cross-Platform

Build once, test everywhere.

Linux

AMD64 / ARM64

macOS

AMD64 / ARM64

Windows

AMD64 / ARM64