scrutineer

An extensible test framework for CLI programs, REST APIs, GraphQL, gRPC, browsers, and network protocols. Declarative YAML tests. Zero third-party dependencies.

Go 1.26+MIT Licensev0.0.1-dev
Get StartedView Source
example.test.yaml
suite: "E-Commerce API"
tags: [api, loadtest]

execution:
  mode: weighted
  concurrency: 100
  duration: "10m"
  repeat: 0
  fleet:
    providers:
      - provider: static
        weight: 100
        ttl: 0
        static:
          ssh:
            user: scrutineer
            key_file: ~/.ssh/ed25519
          nodes: [10.0.1.10, 10.0.1.11]

interactions:
  - name: "User session"
    weight: 7
    mode: sequential
    tests:
      - name: "Login"
        connector: http
        steps:
          - action: request
            method: POST
            path: /api/auth/login
            body:
              email: "${fn:concat(random_string(alpha, 5, 8), '@test.com')}"
              password: "${fn:env_or('TEST_PASSWORD', 'secret')}"
            assert:
              - field: status
                operator: equal
                expected: 200
            capture:
              token: body.token

      - name: "Browse catalog"
        connector: http
        steps:
          - action: request
            method: GET
            path: /api/products
            headers:
              Authorization: "Bearer ${capture.token}"
            query:
              page: "${fn:random_int(1, 50)}"
            assert:
              - field: status
                operator: equal
                expected: 200
              - field: elapsed_ms
                operator: less_than
                expected: 3000

  - name: "Checkout"
    weight: 2
    mode: sequential
    tests:
      - name: "Place order"
        connector: http
        steps:
          - action: request
            method: POST
            path: /api/orders
            body:
              product_id: "${fn:random_int(1, 1000)}"
              idempotency_key: "${fn:uuid()}"
            assert:
              - field: status
                operator: status_class
                expected: "2xx"

  - name: "Admin"
    weight: 1
    mode: random
    tests:
      - name: "Pull report"
        connector: http
        steps:
          - action: request
            method: GET
            path: /api/admin/reports
            assert:
              - field: status
                operator: equal
                expected: 200

Features

Everything you need for comprehensive testing, in a single binary with zero dependencies.

Declarative YAML Tests

Define tests as data, not code. Describe what to assert, not how to execute. Familiar to Playwright and assertion-based test users.

Modular Connectors

CLI, HTTP, SSH, gRPC, GraphQL, and browser connectors. Add new protocols by implementing a single Go interface.

Browser Automation

Headless Chromium, Firefox, and WebKit via Chrome DevTools Protocol. Selectors, interactions, screenshots, network interception.

Load Testing

Parallel test execution distributed across nodes via SSH. Configurable concurrency, ramp-up, and duration. Locust-style scaling.

Nanosecond Telemetry

Every test captures timing data automatically. Structured binary TLV logs with nanosecond timestamps for benchmark analysis.

Zero Dependencies

Built with the Go standard library. No node_modules, no pip packages, no dependency hell. One binary, every platform.

Coverage as a Feature

Built-in test coverage measurement with configurable thresholds. Know exactly which tests ran, which steps executed, which assertions fired.

Fuzz Testing

Declarative fuzz targets integrated with Go's built-in fuzzing. Corpus management and automated edge-case discovery.

Rich Assertions

Equality, contains, regex, JSON path, HTTP status, headers, timing, collections. Extensible assertion library with clear error messages.

Supported Configurations

Test anything that speaks a protocol.

FeatureConnectorStatusFeatures
HTTP/1.1, HTTP/2httpv0.0.1TLS 1.2/1.3, self-signed certs, request/response assertions
REST APIshttpv0.0.1CRUD, auth (Bearer, Basic, API key), JSON/XML, pagination, HATEOAS
GraphQLhttpv0.0.1Queries, mutations, subscriptions, introspection, variables
gRPC / Protobufgrpcv0.0.1Unary, client/server/bidi streaming, .proto + reflection
SSHsshv0.0.1Key-based auth, command execution, tunneling
CLI Programscliv0.0.1stdin/stdout/stderr, exit codes, filesystem side-effects
Chromium / Firefox / WebKitbrowserv0.0.1CDP, selectors, interactions, screenshots, network mocking
HTTP/3 (QUIC)httpplannedPending Go stdlib or from-scratch QUIC
SMTPsmtpplannedSend, auth, envelope validation
IMAPimapplannedMailbox access, search, fetch

Install

One command. No dependencies.

$ go install github.com/asymmetric-effort/scrutineer/cmd/scrutineer@latest

Or download a pre-built binary from Releases for Linux, macOS, or Windows (AMD64 / ARM64).

Quick start
# Install scrutineer
$ go install github.com/asymmetric-effort/scrutineer/cmd/scrutineer@latest

# Install browsers (for browser testing)
$ scrutineer browsers install

# Run tests
$ scrutineer run

# Run with JSON output
$ scrutineer run --format json

# Dump binary telemetry logs
$ scrutineer log-dump scrutineer.log

Cross-Platform

Build once, test everywhere.

Linux

AMD64 / ARM64

macOS

AMD64 / ARM64

Windows

AMD64 / ARM64