scrutineer

An extensible test framework for CLI programs, REST APIs, GraphQL, gRPC, browsers, and network protocols. Declarative YAML tests. Zero third-party dependencies.

Go 1.26+MIT Licensev0.0.1-dev
Get StartedView Source
example.test.yaml
suite: "E-Commerce API"
tags: [api, loadtest]

execution:
  mode: weighted
  concurrency: 100
  duration: "10m"
  repeat: 0
  fleet:
    providers:
      - provider: static
        weight: 100
        ttl: 0
        static:
          ssh:
            user: scrutineer
            key_file: ~/.ssh/ed25519
          nodes: [10.0.1.10, 10.0.1.11]

interactions:
  - name: "User session"
    weight: 7
    mode: sequential
    tests:
      - name: "Login"
        connector: http
        steps:
          - action: request
            method: POST
            path: /api/auth/login
            body:
              email: "${fn:concat(random_string(alpha, 5, 8), '@test.com')}"
              password: "${fn:env_or('TEST_PASSWORD', 'secret')}"
            assert:
              - field: status
                operator: equal
                expected: 200
            capture:
              token: body.token

      - name: "Browse catalog"
        connector: http
        steps:
          - action: request
            method: GET
            path: /api/products
            headers:
              Authorization: "Bearer ${capture.token}"
            query:
              page: "${fn:random_int(1, 50)}"
            assert:
              - field: status
                operator: equal
                expected: 200
              - field: elapsed_ms
                operator: less_than
                expected: 3000

  - name: "Checkout"
    weight: 2
    mode: sequential
    tests:
      - name: "Place order"
        connector: http
        steps:
          - action: request
            method: POST
            path: /api/orders
            body:
              product_id: "${fn:random_int(1, 1000)}"
              idempotency_key: "${fn:uuid()}"
            assert:
              - field: status
                operator: status_class
                expected: "2xx"

  - name: "Admin"
    weight: 1
    mode: random
    tests:
      - name: "Pull report"
        connector: http
        steps:
          - action: request
            method: GET
            path: /api/admin/reports
            assert:
              - field: status
                operator: equal
                expected: 200

Features

Everything you need for comprehensive testing, in a single binary with zero dependencies.

▷

Declarative YAML Tests

Define tests as data, not code. Describe what to assert, not how to execute. Familiar to Playwright and assertion-based test users.

⚙

Modular Connectors

CLI, HTTP, SSH, gRPC, GraphQL, and browser connectors. Add new protocols by implementing a single Go interface.

☷

Browser Automation

Headless Chromium, Firefox, and WebKit via Chrome DevTools Protocol. Selectors, interactions, screenshots, network interception.

↱

Load Testing

Parallel test execution distributed across nodes via SSH. Configurable concurrency, ramp-up, and duration. Locust-style scaling.

⏱

Nanosecond Telemetry

Every test captures timing data automatically. Structured binary TLV logs with nanosecond timestamps for benchmark analysis.

■

Zero Dependencies

Built with the Go standard library. No node_modules, no pip packages, no dependency hell. One binary, every platform.

⚖

Coverage as a Feature

Built-in test coverage measurement with configurable thresholds. Know exactly which tests ran, which steps executed, which assertions fired.

✦

Fuzz Testing

Declarative fuzz targets integrated with Go's built-in fuzzing. Corpus management and automated edge-case discovery.

✎

Rich Assertions

Equality, contains, regex, JSON path, HTTP status, headers, timing, collections. Extensible assertion library with clear error messages.

Supported Configurations

Test anything that speaks a protocol.

FeatureConnectorStatusFeatures
HTTP/1.1, HTTP/2httpv0.0.1TLS 1.2/1.3, self-signed certs, request/response assertions
REST APIshttpv0.0.1CRUD, auth (Bearer, Basic, API key), JSON/XML, pagination, HATEOAS
GraphQLhttpv0.0.1Queries, mutations, subscriptions, introspection, variables
gRPC / Protobufgrpcv0.0.1Unary, client/server/bidi streaming, .proto + reflection
SSHsshv0.0.1Key-based auth, command execution, tunneling
CLI Programscliv0.0.1stdin/stdout/stderr, exit codes, filesystem side-effects
Chromium / Firefox / WebKitbrowserv0.0.1CDP, selectors, interactions, screenshots, network mocking
HTTP/3 (QUIC)httpplannedPending Go stdlib or from-scratch QUIC
SMTPsmtpplannedSend, auth, envelope validation
IMAPimapplannedMailbox access, search, fetch

Install

One command. No dependencies.

$ go install github.com/asymmetric-effort/scrutineer/cmd/scrutineer@latest

Or download a pre-built binary from Releases for Linux, macOS, or Windows (AMD64 / ARM64).

Quick start
# Install scrutineer
$ go install github.com/asymmetric-effort/scrutineer/cmd/scrutineer@latest

# Install browsers (for browser testing)
$ scrutineer browsers install

# Run tests
$ scrutineer run

# Run with JSON output
$ scrutineer run --format json

# Dump binary telemetry logs
$ scrutineer log-dump scrutineer.log

Cross-Platform

Build once, test everywhere.

Linux

AMD64 / ARM64

macOS

AMD64 / ARM64

Windows

AMD64 / ARM64