135 KiB
Testing REA
Use this guide when choosing coverage, pruning tests, or running a verification lane. Start with the commands below; use real-toolchain lanes for engine prerequisites and client verification for installed-client and agent runs.
Developer commands
Use source feedback while editing, explicit boundary checks for the changed behavior, and complete CI evidence before merging.
| Command | Scope |
|---|---|
npm run test:local |
Dirty source tests via the import graph, without build; explicit source paths run regardless of Git status |
npm run test:focused -- PATH... |
Exact existing test files; compiled boundaries build first, and unmatched paths fail |
npm run test:changed |
Source tests affected since the merge base with origin/main, including committed and dirty changes |
npm run test:fast |
All domain, service, adapter, composition, conformance, and evaluation tests without build |
npm run test:boundary |
Boundary, CLI boundary, MCP boundary, process boundary, and process-global projects |
npm run test:mcp |
MCP boundary project |
npm run test:acceptance |
Complete compiled CLI and MCP acceptance workflows |
npm run test:python |
LLDB and mitmproxy producer contracts plus analysis-efficiency checks; no real debugger or proxy required |
npm run test:watch |
Dirty source tests in watch mode, without build |
npm run test:watch:all |
Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
npm run check:changed |
Cached typecheck/lint and branch-related source feedback |
npm run check:pr |
Opt-in complete local deterministic gate and generated-file checks |
npm run docs:check |
Generated-document validation and fresh onboarding example consistency checks |
npm run docs:build |
Render VitePress documentation and apply its page-link checks; no live external-link or GitHub fragment verification |
The general source suite above runs on Linux in CI, and Windows CI runs a
curated lane (Native platform baseline in CI).
A Windows source checkout can fail source tests for reasons unrelated to the
change, such as POSIX fixture paths, #!/bin/sh tool stubs, or POSIX rename
semantics. Compare failures against the base revision, or run the suite on
Linux or WSL, before attributing them to a change. See CONTRIBUTING.md →
Windows source checkouts.
The CI documentation lane runs both docs:check and docs:build when the
existing change classifier selects documentation or its producing inputs.
Generated facts and rendered page links have separate checks. VitePress does not
establish that every external URL or GitHub-rendered localized heading exists;
verify those destinations when editing them.
For example:
npm run test:local -- src/config.test.ts
npm run test:focused -- tests/acceptance/applications/runtime.test.ts
npm run test:changed -- --base origin/main
npm run test:changed -- --dry-run
test:focused accepts exact repository-relative test file paths. Source-only
paths do not build; boundary, acceptance, process-global, or unfamiliar tests/
paths build conservatively. Tests use Vitest concurrency and isolated
workspaces; commands do not hold a broad test lock. Build and documentation
writers retain checkout-local locks for their shared output files. Explicit
selections do not use --changed or permit zero-test success.
The dry-run option reports the chosen merge base, scope and build prerequisite
without executing tests or building. A missing Git base reports how to fetch
it or select another revision.
Source typechecking needs no compiled runtime or generated test metadata.
Typecheck, runtime compilation, and Frida binding verification share
scripts/run-typescript.mjs. The pinned tsrs package supplies Linux x64/arm64
(GNU and musl) and macOS arm64 compilers. Windows and other unsupported hosts
retain the existing stable official TypeScript compiler. Its upstream version
can differ from tsrs, so both compiler tasks hash the selected package and version
through scripts/run-turbo.mjs before cache lookup. Outputs from different
compilers occupy different cache entries. Repository npm scripts and focused-test
preparation use that entrypoint; a raw Turbo compiler task without matching
identity fails instead of publishing an ambiguous successful cache entry.
A missing binary on a supported host also fails without silently falling back.
The Rust compiler defaults to at most two checker threads. For --noEmit checks,
--maxMemory 3G also sets a soft memory target; emit builds do not use that target.
Explicit compiler arguments can override the defaults. These controls do not
impose a process-family memory limit. Termux's explicit build, Stryker's compiler
API dependency, and independent fixture compiler oracles keep their existing
TypeScript versions.
npm run build:cached compiles the CLI/MCP runtime and its packaged skill.
The build also records dist/mcp-catalog.json (compact inputs) and
dist/mcp-catalog-full.json from the canonical SDK registrations and binds
each to the runtime with its own SHA-256 digest. These are
runtime artifacts, separate from the test catalog below. Default target-free
MCP startup serves its schemas without constructing analysis validators;
the first tool or guided-workflow request materializes the canonical runtime
on that same SDK server. Explicit compact/full input selections preserve
their advertised schemas throughout materialization. Initial-target,
HTTP and output-schema-off startup keep the eager path; source runs without
recorded artifacts also use eager composition. Compare memory both after discovery and after a completed
request; report diagnostic GC separately from normal resident memory.
Turbo's strict task environment preserves caller-selected GOMAXPROCS,
GOMEMLIMIT, and UV_THREADPOOL_SIZE for resource-constrained builds and
checks. These execution controls do not change artifact content, so changing
their values retains cache reuse. They control individual runtimes; monitor
the complete process family separately when enforcing a total memory budget.
npm run test:prepare also generates the MCP contract test catalog, product
catalog, and portable managed evidence used by the complete suite. Focused
tests prepare those extra outputs only when their selected files consume them.
The MCP test catalog is JSON in .cache/mcp-tool-catalog.json; the tracked
test loader supplies its types without making generation a source-check
prerequisite. Run npm run mcp-catalog:generate to refresh it independently.
Documentation generation and managed conformance run through their separate
docs:generate and evidence:generate task graphs.
Changed selection can miss runtime registration, generated data, shell
entrypoints, bridges, or other relationships absent from the import graph.
An empty changed selection means no tests were selected, not verified
correctness. Select relevant boundary files and real-provider lanes explicitly.
Source projects also contain large capacity regressions; test:fast promises
no compiled-runtime prerequisite, not a fixed time budget.
npm run lint prints errors and a warning summary; npm run jscpd prints
duplication statistics. Complete reports remain in .cache/static-checks/.
The fast gate uploads them as static-check-reports. Full qualification
uploads duplication diagnostics from the deep job as
static-deep-check-reports, including after that gate fails.
Task-graph recording runs check:ci:fast:tasks and check:ci:deep:tasks
without the artifacts lock, so it can record hashes while check:pr or
check:test already holds that lock. The CI jobs still run the locked
check:ci:fast and check:ci:deep commands.
Use npm run lint:details or npm run jscpd:details for all console diagnostics.
Both duplicate commands scan production and test code using the limit in
.jscpd.json.
The summary commands preserve checker failures and restore reports from cached
static tasks. Both lint caches include the website, documentation configuration,
and experiment code scanned by the checker; a regression compares the actual
lint inventory with Turbo's resolved inputs.
Routine iterations and rebases need focused regressions and relevant checks.
Before handing off a PR, run npm run check and generated-document checks when
applicable; CI owns the full suite and coverage. Use the full local gate for
broad changes or diagnosing CI, rather than after every edit. Package/install
changes additionally need package verification; provider changes need actual
provider evidence.
Vitest projects use up to two workers, bounded by host parallelism.
process-boundary runs later with serial files because process-tree sampling
shares host resources. Acceptance and process-global cases use isolated forks
without serial scheduling. Pure domain/service, composition, conformance,
evaluation, CLI boundary, and MCP boundary projects share module graphs. Their
tests avoid process-global state and keep resource cleanup test-scoped. Each
MCP session still owns its resources, and CLI cases retain fresh command
instances or subprocesses. Fixtures using vi.mock or vi.doMock need an
isolated project. CLI/MCP fixtures normally live in tests/process-global/.
When a path must stay, assign it explicitly in scripts/lib/test-projects.mjs,
exclude it from its shared owner, and preserve its source/built group.
Product catalog verification stays with the
isolated filesystem boundary tests because it loads both source and compiled
module graphs. Adapter, other boundary, acceptance, and process-global projects
retain per-file isolation. See vitest.config.ts for current settings.
Inject producer faults through scoped dependencies in shared workers: a module mock cannot replace a consumer that was imported earlier. Check these tests with the consumer preloaded as well as alone. For simulated protocol traces, use an acknowledged event barrier before asserting complete observations; a zero-duration capture does not guarantee queued events have arrived.
Focused contract tests should import the named contract family they exercise; reserve the full tool catalog for catalog-wide assertions. Resolve contracts by operation name rather than array position.
Check advertised schemas after SDK conversion for every supported schema profile. Meta-schema validity does not establish client acceptance or useful rendered declarations. Keep canonical rejection checks separate from the intentionally reduced compact advertisement, and probe actual client renderers when changing schema representations.
The reused CLI boundary workers cap V8 old space at 512 MiB, with or without coverage. Ordinary CI also exhausted the former 384 MiB budget after earlier files loaded the shared source/contract graph, while later command registration and schema/help rendering still needed transient allocation space. This is a measured capacity control, not a suite-wide speedup claim. The worker count remains bounded at two, and spawned REA processes retain their own heap settings. Coverage enablement still controls instrumentation, test deadlines and coverage thresholds; explicitly disabling it or selecting only a provider does not enable instrumentation.
Vitest worker limits do not bound subprocesses launched within a test. Bound those batches separately so repeated CLI validation cannot oversubscribe the runner. Preserve simultaneous launches where concurrency is itself under test.
Coordinate expensive checks across agents sharing a host. If a test unexpectedly consumes heavy CPU, stop its owned process family, retain its log and revision, and profile a bounded representative case before retrying. Separate production work from fixture generation, schema compilation, subprocess startup and cleanup. Choose elapsed, CPU and process-family memory limits before the run; a worker count or test timeout alone does not enforce those limits.
Remove demonstrated repeated work while preserving distinct assertions and resource ownership. Share immutable fixture preparation where safe; keep fresh sessions when lifecycle or concurrency is the behavior under test. Compare the same completed workload before and after, report cache state and measurement limits, and record interrupted runs separately from passes. Increasing a timeout does not explain a CPU regression.
Build/documentation writers hold checkout-local output locks. Tests do not
hold a broad command lock; check:pr finishes tests before document validation.
Turbo caches deterministic preparation and static checks, but test:run always
executes. Repository files do not fully describe installed tool contents or host
state. A previous test pass therefore cannot replace current validation, even
when the source tree is unchanged. check:pr also runs the fast Python producer
suite; CI runs it in the static job for code changes. These producer tests do not
establish real LLDB engine coverage.
Vitest's persistent compile cache remains disabled. The cliTest fixture shares
a temporary Node compile cache among its fresh CLI subprocesses. The worker
removes it after test-scoped processes finish. Explicit child environment
settings take precedence; children requesting NODE_V8_COVERAGE do not receive
the fixture cache. These subprocesses are outside aggregate Vitest coverage;
subprocess coverage attachment remains disabled. Cold-start import checks launch
independently without this fixture cache.
analyzeJavaScriptDirectory stdio MCP cases in one cliTest file share one
server and that compile cache. The first case pays startup; later cases reuse
the process. File teardown waits for startup still in flight, closes a server
registered during that wait, and does not open a server after close. A failed
handshake stays the caller error when its cleanup also fails, including a
rejection without a reason. That cleanup stays owned until a later close or
call releases it. Another server does not open while the cleanup remains, and
a cleanup that already succeeded is not run again. Pass
mcpCall when a case needs its own server, environment, or lifecycle. CLI
cases for that helper share one compiled CLI process per child environment.
The first case pays process startup; later cases reuse the loaded runtime.
The process restarts when the environment changes, after 32 commands, or when
the worker reports a resident set of at least 1 GiB. A fatal worker failure
or a broken session frame retires that process and waits for it to exit before
the next command starts.
cli.run still starts a fresh process, including stdin and explicit deadlines.
runFileCli uses the file session for commands that have neither. Go binary,
IL2CPP metadata, and dylib-resolution commands use that session. File teardown
stops the shared CLI process, including startup still in flight, and does not
open another after close.
exportedFunctionBindingsCli.test.ts uses the shared compiled CLI session for
its file and compares its JSON command results with the MCP session. It selects
20-second startup and command deadlines; other callers retain their existing
deadline behavior. Retired processes stay owned until exit, block replacement,
and remain available for a later close to retry after failed termination.
Shared session cleanup closes stdin so the worker can release JavaScript
resources, waits the bounded grace, then escalates to SIGTERM and SIGKILL only
if needed. Confirmed exit succeeds; an unconfirmed exit reports cleanup failure.
When startup or a command fails and cleanup also fails, the reported aggregate
retains both failures and the original failure as its cause.
applicationWorkflowCli.test.ts keeps one compiled CLI process for its JSON
commands, including a non-zero status whose stdout is still a JSON document,
and closes that process after the file.
The Apple artifact verifier step shares a job-temporary bytecode cache among
its fresh CLI/MCP children via REA_ARTIFACT_NODE_COMPILE_CACHE. The artifact
helper maps this setting to child NODE_COMPILE_CACHE; npm, Turbo, and nested
Vitest runs do not inherit that setting. Explicit NODE_COMPILE_CACHE takes
precedence, and coverage collection or NODE_DISABLE_COMPILE_CACHE=1 disables
the helper's cache. The cache is discarded with the CI job.
To evaluate persistent compile caching for repeated local runs, opt in for both cold and warm measurements with an isolated cache:
NODE_COMPILE_CACHE=.cache/node-compile npm test
Do not report the warm result as a cold-suite improvement, and do not enable the cache in coverage or benchmark CI without first showing that its instrumentation remains equivalent.
Choose verification for the change
test:local is fast source-level feedback. For the strongest practical
regression, prefer a real public CLI/MCP workflow, then production-boundary
integration, then a producer golden; use the table to select quick feedback and
claim-specific additional evidence. Provider-behavior claims require the
matching real-provider lane.
| Change or claim | Fast feedback | Claim-specific coverage |
|---|---|---|
| Source semantics | npm run test:local -- PATH... |
Relevant boundary cases when the behavior crosses a port |
| CLI/MCP workflow | npm run test:focused -- PATH... |
Actual public calls; an injected provider has limited scope |
| Documentation or generated metadata | npm run docs:check |
npm run docs:build for rendered pages and links |
| Engine adapter | Its real-provider lane below | Host, version, target identity, and owned cleanup |
| Client configuration or schema conversion | Native client lane | Distinguish loopback models from live services |
| Agent investigation quality | Agent evaluation | Review transcript and scenario-specific evidence |
Test design and pruning
Prefer real public CLI/MCP workflows, then production-boundary integration, then goldens captured from real producers. Keep module tests for distinct failures or semantics that stronger workflows cannot reliably reproduce. Paths, compiled imports, and suite names do not establish test depth.
Prune tests that mirror implementation details or duplicate a stronger claim. Before deleting a distinct regression, identify the replacement scenario and assertions. Preserve malformed formats, cancellation, cleanup, permissions, capacity, target identity, and combinations with distinct interactions. Trivial helper assertions need no replacement. Remove abandoned production scaffolding only after checking runtime and verifier consumers, including compiled imports.
Use representative scenarios rather than Cartesian matrices of equivalent
cases. Share setup without hiding inputs or expected evidence. For successful
Result values, throw the typed error before asserting the value; a preceding
.ok assertion adds no coverage. Protocol fixtures must reject unmodeled
commands and preserve actual producer reply shapes.
For comparison normalization, include a legitimate equivalent pair and a distinct pair that would collapse if representation tags or scope were lost. For learned identifiers, cover conflicting pairs and reused values in different collections; mappings must remain one-to-one within their declared scope.
For coverage propagation, compare the same returned rows under complete and incomplete examination. If a row exists on the left but is missing from a fully examined compatible right inventory, removal can be established. If the right side examined only eight of ten applicable records, or returns only one page, the same missing row remains unknown unless separate evidence proves absence. Assert the structured coverage and comparison status as well as the explanation.
For format compatibility, identify the producer revision that establishes each rule. Include an accepted boundary case beside the rejected case so a stricter parser cannot pass by refusing both. Keep source-derived behavior, synthetic fixtures, and executed producer or runtime oracles distinct in validation reports. A fixture accepted by REA proves its reader behavior; it does not prove that a real loader accepts the artifact. If multiple producer revisions share a format identifier but differ in behavior, record that ambiguity rather than using one installed version as an oracle for all of them.
Packed bridge validation uses Python's standard-library AST and tar readers
with -I -S: caller-local modules, PYTHONPATH, and site startup hooks must
not replace those readers or execute during validation. Controlled ast.py
and sitecustomize.py fixtures must leave their markers absent and still
reject malformed packed Python with SyntaxError. This checks static package
verification, not real Hopper or Ghidra behavior.
Stateless source-analysis matrices may share a file-owned stdio MCP connection and retain representative compiled CLI parity checks. Every distinct source scenario still runs through the public workflow; session-lifecycle scenarios need their own sessions and cleanup assertions.
When a matrix already holds a named tool definition from live tools/list,
pass that definition through the SDK's toolDefinition call option. This keeps
the selected output-schema validation without eagerly compiling unrelated
catalog schemas. Re-read definitions if the server advertises a changed list.
Artifact verification uses artifactMcpCall for both lifecycle and analysis
calls. A fresh SDK client has no held tool listing, so a bare callTool can
skip output-schema validation. The helper discovers actual advertised
definitions, rejects missing or ambiguous schemas, and validates the requested
tool without compiling unrelated catalog schemas. REA registers process-static
tool contracts; bindings share one discovery per negotiated connection and are
discarded on reconnect or failed discovery. Keep raw artifact calls on this
shared path and retain explicit request options such as timeouts.
Caller-supplied tool definitions are rejected: the helper owns selection from
the advertised catalog so a stale or permissive schema cannot bypass it.
The regression fixture changes only canonical_path in a real stdio response
to a number. The old helper accepts that response; the named schema must reject
it. This client-boundary fault injection does not establish native Apple
behavior, which remains covered by the macOS verification lane.
Report unavailable authority as a named skip. Await owned cleanup before a verifier reports success. Native process inspectors are released at Vitest worker teardown; fork termination does not run Node exit hooks, and per-file teardown would retire inspectors still needed by other files.
Measure slow cases before pruning capacity coverage. Retain inputs beyond the formerly failing size/depth and distinguish cold runs, warm builds, coverage, and toolchain conditions when comparing timings.
Lifecycle regressions
For changes to resource acquisition, readiness, cancellation, or cleanup, select the affected owners and applicable scenarios below. Include temporary snapshots and one-shot commands. Use public CLI/MCP workflows where practical and a controlled production seam for interleavings that cannot be reproduced reliably.
| Scenario | Assertion and control |
|---|---|
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Return typed cancellation with available partial observations and actual cleanup status. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
Shared process-supervision controls separate leader exit from output closure. Cover output during held or failed cleanup, bounded retention and drain timeout, retry, and borrowed stream authority. The POSIX inherited-stderr control uses a separate pipe as a release barrier after leader exit; it does not establish native Windows Job Object behavior.
Synchronize cancellation with the finalization boundary being tested; a fixed delay after dispatch does not establish that phase. Use markers, barriers, or owned cleanup seams while preserving real cleanup where feasible. Check server settlement and retained resources even if the MCP SDK rejects the client promise before the handler finishes. A cancellation during active execution does not cover cancellation during final cleanup. Use explicit abort reasons and assert their identity at admission and after provider completion, alongside retained observations and cleanup state.
Electron fuse regressions cover actual descriptor close denial and retry through the built CLI, same-connection MCP recovery, mixed descriptor outcomes, and server shutdown. Controlled seams cover cancellation during final close and after reader finalization, plus active-provider shutdown during preflight. They preserve real filesystem cleanup where possible; mocked launch boundaries do not establish real Electron teardown or naturally occurring OS failures.
Active Electron cleanup controls retain independent transport and process-family owners, block replacement captures until recovery, and retry only unresolved owners. Cover late close acknowledgement and rejection after a wait deadline, opaque rejection reasons, cancellation during failed finalization, and missing lineage followed by a confirmed passive family census. Windows controls require a fresh lineage observation and do not signal cached descendant PIDs after root termination. Simulated Windows controls do not establish native Windows behavior.
Native process-ownership controls distinguish start identity, zombie lifecycle, current parent/group relation, and token-read evidence. Retain generation-bound candidates through smaller lineage observations, table omissions, reparenting, group changes, and failed cleanup retries. Identityless exit-refresh controls cover a replacement row that is not linked to the prior descendant set: a matching token on its own group is signaled, a foreign or absent token is not, an unreadable live token stays unresolved, a replacement that appears only in the confirmation table is still token-checked, a group discovered during revalidation is signaled before cleanup succeeds, a member that changes groups during confirmation is signaled in its new group, a PID first seen with a foreign token is read again on the confirmation table, a PID whose token was already read is read again even when its parent and group return to the pre-exit values, a pre-existing PID is read again when its parent or group changes, an unchanged pre-existing row that was never token-read stays unread, an unbounded confirmation chain stays unresolved, lineage reads a row again when its launcher-tree membership changes, a cached lineage token is read again on the next refresh, an initial descendant whose token already matched is read again on a later refresh, the launcher is read again on that refresh and a missing, unowned, or relation-changed launcher PID stays unavailable, a descendant absent from the latest refresh is omitted, a group that reappears after it was processed is revalidated once, a further reappearance stays unresolved, lineage publishes one row per PID, and lineage does not publish a verified empty descendant list while that row remains unexamined. A readable token with unavailable identity stays unresolved. Native UID facts, rather than an earlier process-table row, determine whether a token denial concerns another user. Darwin tests compile the helper and exercise snapshot growth, absence, lifecycle, and UID observations; controlled Linux procfs tests do not establish real Linux exit or PID-reuse behavior.
For atomic writes, exercise cancellation on both sides of the documented commit point and preserve any publication receipt. A late interruption does not prove that no output was published.
For successful provider reports, test cancellation immediately before the report reaches the application workflow as well as during provider cleanup. Validate the report before classifying completion: a valid report must retain its Evidence on cancellation, and an invalid report must keep its output error. Exercise both the provider boundary and the shared CLI/MCP workflow; a downstream guard does not repair the provider's own completion contract. When callers share startup, cancel each waiter independently and verify that the last departure awaits owned cleanup while interested peers can still reach readiness. Repeat the same shutdown event after a cleanup failure to verify that its listener can retry. Exercise non-default lifecycle deadlines through queued requests and their consumers; default-only controls cannot detect dropped request options.
For descriptor-backed reads, inject a close failure after a successful read and after a primary read or cancellation failure. Verify that the exact descriptor remains owned, both failures survive, and a later close retries it. Repeat admission attempts while close still fails: no additional descriptor or native command should start. Restore close and verify that the exact owner resolves the originating cleanup marker and allows later work, while the earlier failure remains available as a diagnostic. Retain errors returned by multiple failed retries, including admission and shutdown aggregates, and assert that each becomes complete after recovery. For an aggregate with multiple owners, release one owner while another still fails and assert that it remains incomplete. Cover recovery before the caller-specific error is bound to its retained owner, as another request can retry cleanup concurrently. Confirm that an already released owner resolves a later binding too. Include admission failure after opening the descriptor. If caller cancellation returns before an inspection settles, gate its final read or close and verify that session shutdown waits for the owned work and reports any retained resources. Gate a later read in a multi-read inspection, not only its first read. Overlap operations so one retains a failed descriptor while another reaches its next read; both the operation and shutdown must settle without nested-admission deadlock. Deny child close and verify its containing root survives until retry.
Exercise publication on filesystems without hard links using an unsupported-link seam. Check exclusive destination creation, stat/write/close failures, completed output receipts, and cancellation at the documented cutoff. A streamed fallback has different visibility from an atomic hard-link commit and must report it accurately. For Windows command supervision, keep a descendant writing after its leader exits, deny cleanup observation, and verify a competing writer remains excluded until the exact Job Object closes. Portable doubles cover retry logic; the native Windows lane must establish actual process-tree behavior.
Supervisor tests must observe process exit, output-pipe closure, and IPC
disconnection separately. In the pinned Node runtime, a parent-initiated IPC
disconnect can omit the aggregate ChildProcess close event. Join the actual
resource observations before reporting completion; process exit alone does not
establish pipe or process-tree release.
When changing an internal helper contract, check its JavaScript verifier and subprocess-script consumers alongside TypeScript references. Source typecheck does not check those calls; run their orchestration tests against the built modules, including the verifier's failure and cleanup paths.
Representative regressions include process cleanup-time SIGINT, late authenticated cleanup, LLDB launch and finalization, composed child ownership, and public CLI cleanup denial. These controlled probes do not replace the matching real-provider lane. Record the base failure, passing control, exact host/provider, verified cleanup, and unavailable coverage; a missing engine is not a pass.
Behavioral depths
| Depth | Location | What it proves |
|---|---|---|
| Module | src/**/*.test.ts |
One domain, service, or adapter through its narrow public surface |
| Composition | tests/composition/** |
Provider-neutral session and registry wiring; filesystem access is limited to fixture materialization |
| Boundary | tests/boundary/** |
Exactly one production filesystem, process, network, browser, CLI, or provider boundary |
| MCP boundary | tests/boundary/mcp/** |
MCP transport and tool-contract boundaries with isolated sessions and explicit cleanup |
| Acceptance | tests/acceptance/** |
A compiled CLI or MCP journey; injected providers still make it integration |
| Conformance | tests/conformance/** |
Shared provider contracts, parameterized by declared capabilities and explicit opt-outs |
| Evaluation | tests/evaluation/** |
Deterministic evaluator parsing, scoring, and report generation |
Colocated domain tests may import only their owning domain and inward dependencies. They do not construct sessions or servers and do not start subprocesses or network listeners. Colocated application tests exercise one service through explicit ports and recording adapters, never a CLI or MCP entrypoint. Composition tests may assemble provider-neutral sessions and registries but do not cross production filesystem, process, socket, or browser boundaries. Boundary tests cross one production boundary. Only acceptance tests assemble the complete runtime or invoke the compiled product surface.
Focused immutable builders and recording ports shared by one test family live
beside their production owner as src/**/*.fixture.ts. They are typechecked
with the suite and excluded from package builds; broader runtime and provider
fixtures remain under tests/fixtures/**.
tests/process-global/** is reserved for cases with a demonstrated dependency
on process-global state. Those tests use isolated forks so environment and
exit-status changes cannot leak between files. Serialize a case only when it
demonstrably shares an external resource that cannot be isolated. Reusable,
test-scoped fixtures live under tests/support/**; immutable source artifacts
remain under tests/fixtures/**.
The process-global Vitest configuration contract rejects new direct temporary-root
creation outside the workspace seam and its narrowly documented boundary/package
exceptions.
The shared temporary-directory and workspace helpers preserve caller labels
under an owned rea- prefix when needed. npm run verify:test-temp-hygiene
uses the same ownership rule to find retained fixtures under a private temporary
root. Third-party caches remain outside that ownership rule.
Real Hopper, Ghidra, IDA, browser, package, and managed-code claims belong to their
explicit npm run verify:* lanes. The reconstruction-readiness lane also
checks deterministic rerun, tamper, and stale-input handling; those checks do
not execute extracted JavaScript modules. When application runtime behavior is
needed, exercise the actual target through browser, Electron, or process
capture. Real model trials are manual; Vitest covers deterministic evaluator
logic.
verify:managed runs the portable PE byte-fixture conformance entrypoint under
scripts/verify/managed/, with its byte builder under
scripts/fixtures/managed/. It checks static classification, members,
reconstruction, native-boundary relationships and application graphs without
executing fixture PE files. Operator-local manifests and actual ILSpy oracles
remain optional, separately reported checks; the real Ghidra NativeAOT lane has
its own toolchain prerequisites. See the managed guide
for those configurations. Generated completion-ledger checks use the same owning
entrypoint and include its verifier/fixture files in their cache inputs.
End-to-end, integration and golden evidence
Prefer the public CLI or MCP workflow when it can observe the required behavior. Use production-boundary integration for contracts the complete workflow cannot reliably exercise, and producer goldens for format-specific facts. An acceptance case with an injected provider verifies that integration boundary; it does not establish real-provider behavior.
For installed-client versions, executable selectors, disposable profiles, model fixtures, and detailed receipts, see client verification. Use real-toolchain lanes for engines and agent evaluation for investigation quality. Report unavailable tools or authority explicitly and verify owned cleanup before claiming success.
Coverage and timing
CI owns coverage; aggregate and domain/contract thresholds are maintained in
vitest.config.ts. They are glob-specific and never updated automatically.
Coverage does not replace named boundary or real-provider scenarios.
The reverse-tools commands run the focused tests and report coverage and
mutation results without an additional percentage requirement. Test failures,
invalid mutation configuration, and runner failures still fail the check.
Mutation targets cover reviewed production behavior, including signal-terminated
commands, Cutter descriptor identity, debugger cleanup ownership, GDB command
exit and cancellation, and Rizin startup, timeout Evidence, and process exit.
A second exhaustive pass runs the focused Cutter and Rizin timeout semantics tests against every compilable
selected mutant to avoid relying on per-test mutation coverage associations;
Stryker reports compile-error mutants separately because those cannot execute
tests. Both passes resolve named syntax owners, scoped declarations, guards,
callbacks, and reviewed expressions from the current TypeScript source; no
hand-maintained line ranges select mutations. Missing, ambiguous, or malformed
source targets stop configuration before mutation generation. The selections
retain their review scope rather than expanding to whole source files. The
focused suite also exercises affected MCP routes and the Python Cutter bridge.
Contract, integration, and real-provider lanes complement these focused tests;
simulated providers do not establish real-provider behavior.
Required CI retains fresh reverse-tools coverage and preflights both canonical
Stryker configurations with --dryRunOnly. These preflights resolve the owned
targets, instrument the source, and execute the real initial tests. Invalid
configuration, target resolution, and initial test failures still fail the job.
Preflights execute no mutants and produce no mutation-outcome report. Both fast
feedback and full qualification, including exact-revision release qualification,
require coverage and both preflights. A failed, canceled, skipped or missing
selected quality job still fails the required gate.
The separate Mutation sweep workflow runs both complete configurations daily
and on manual dispatch, in two isolated jobs. It is a maintenance check outside
the PR and release publication gates. ci:full requests full regression,
coverage and provider qualification; it does not request a complete mutation
sweep. Use the Mutation sweep workflow to request that result. Targets,
checkers, concurrency, deadlines and the existing non-blocking mutation-score
threshold are unchanged. npm run test:mutation:reverse-tools also retains both
complete passes for local callers.
The two passes write separate JSON and HTML reports under reports/mutation/:
reverse-tools and reverse-tools-semantics. Required CI retains focused
coverage; the sweep retains each configuration's JSON/HTML reports separately.
Both retain available diagnostics for seven days after failure and skip uploads
on cancellation. Artifact names identify the tier or configuration and run
attempt. A successful sweep requires both jobs and their nonempty reports;
partial diagnostics and preflights do not establish complete mutation outcomes
or qualify the release revision for mutation coverage.
Fast CI starts the source-only projects after classification, independently of
build and static checks, and runs eight built-dependent shards after the build.
Those nine test jobs still select every portable test. Full qualification keeps six
complete Vitest shards with coverage. Neither tier retries tests. Canonical
project ownership lives in scripts/lib/test-projects.mjs, shared by Vitest,
CI group execution, and development-test preparation. A test importing compiled
verifier code belongs to the built group even if its directory is conformance.
Source and built blobs have separate labels to prevent merge collisions.
Each test job emits a blob report; the merge job produces JUnit and JSON diagnostics, a module timing
report, plus aggregate coverage during full qualification. It retains available
reports even when a shard fails, uploads available diagnostics, and writes the
slowest files to the workflow summary. Partial reports are marked diagnostic
only. Failed, cancelled, or missing selected shards still fail the required gate,
and aggregate coverage thresholds remain enforced in full qualification.
Only a successful complete merge on the default branch publishes shard history,
and only from a push, schedule, or manual run. Fast runs publish
test-timings-fast. Full runs publish test-timings-full. Pull requests do not
publish it: the lookup reads one unpaginated artifact page, and pull-request
artifacts would crowd out the default branch. The build job downloads the
artifact that matches its tier, so fast shards use durations measured without
coverage and full shards keep coverage-weighted durations. A tier with no
qualifying artifact leaves history unset. The download is that small
checkout-relative timing file, not the coverage/JUnit bundle. Timing weights come from Vitest module
diagnostics and include tests and hooks, worker preparation, environment,
imports, and setup. Import timings already include transform waits, which are
not added again. The weights describe file costs under the measured worker and
coverage settings; their sum is not workflow elapsed time. The diagnostic
artifact retains the phase breakdown, while shard history keeps only each
file's total. Legacy test-body-only history and missing history fall back to native
sharding. Static checks, documentation, build, package verification, and
real-system lanes remain separate jobs so one kind of evidence cannot stand in
for another. Workflow syntax and expression validation runs inside the required
static-check job when workflow changes select it, avoiding a separate runner.
Native, provider, fixture, SQLite compatibility, and website lanes wait for
the fast static checks (typecheck, built lint, and format) to succeed before
allocating their runners. Knip, duplication, architecture guards, test
discovery, and the Python producer run in the separate full-tier deep static
job and do not gate provider admission. Portable test shards and documentation
checks remain independent of static results, so they can still provide
diagnostic feedback. The required gate rejects skipped selected lanes when
static checks fail; on the full tier it also requires the deep static job.
Passing revisions retain all selected coverage. This ordering reduces peak
demand on invalid revisions but can add static-check waiting to otherwise
passing runs; it is not a measured reduction in end-to-end latency.
Core jobs have execution limits of five minutes for classification, ten minutes for static checks, builds, docs, and report merging, and thirty minutes per portable test shard. These limits leave headroom for cold setup and full coverage while preventing a hung job from holding the active main run for hours. They do not bound runner queue waiting. Timeouts still fail the required gate.
The separate CI timings workflow also diagnoses missing, startup-failed,
queued/running, approval-required, canceled, and failed PR CI through bounded,
read-only API scans. Incomplete pagination or changing identity stays unknown.
It never closes/reopens PRs, reruns workflows, posts comments, or changes gates.
The timing manifest retains each run's event and head branch, including explicit
unknowns, so PR runs cannot consume the eligible-main comparison budget.
Its static failure report compares supported lint/duplication diagnostics with
failed main runs using exact source, ancestry, task hashes, and toolchain proof.
A match describes those diagnostics only; it does not establish that every
failure in a job came from main. Duplication reports retain the actual checker
exit result, so passing duplication checks preserve lint failures and caller
thresholds remain owned by the producer. Missing checker outcomes leave
those duplication diagnostics unclassified. Missing historical receipts remain unknown.
The reporter executes only trusted default-branch code and parses whitelisted
JSON artifact members without executing artifact content. Reports and limits
are retained in ci-health-diagnostics; its ten-minute job limit covers the
existing four-minute timing collection plus the bounded diagnostic stages.
Fast built shards target under ten minutes of uncontended shard-job execution.
That figure is the test-shard job. The job waits for classification and the
build, and the portable-test merge runs after the slowest shard. On 2026-10-20,
run 39451548945 measured those surrounding jobs with runner queue excluded:
classification 15 seconds, build 49 seconds, and the portable-test merge 29
seconds. Shard 1/5 of that run spent 22 seconds before built Vitest and 3
seconds after it, including npm ci at 9 seconds. node_modules is about 674
MiB, so publishing it as a shared artifact would transfer more data than that
install saves. The npm download cache stays the setup cache.
File timings from that run put the built suite at 7944 seconds of module time.
The sequencer keeps each file's full duration on one worker. Extra workers
overlap other files in the same sequence group, and process-boundary runs
after that group. Scheduling the unsplit inventory onto five shards peaks at
820 seconds of tests. That run's slowest built shard job lasted 925 seconds
(shard 4/5), and shard 1/5 spent 874 seconds in built Vitest, so hosted job
time sits above the file schedule. Six shards of the same inventory peak at
684 seconds of tests. Eight shards, after the 578-second loader file is split
by case count (54 cases stay in that file and 46 move into each of three
files), peak at 513 seconds of tests. With the measured 25-second shard setup,
that shard job is about 538 seconds. Adding the measured classification, build,
and merge makes an uncontended portable critical path of about 631 seconds,
still before the gap between this file schedule and hosted Vitest. Runner queue
is separate and grows when more shard jobs compete for runners.
process-boundary stays serial because process-tree sampling shares the host.
Raising the worker cap from two to four on this four-core host cut a six-file
acceptance sample from 96–101 seconds to 71 seconds and did not remove the
single-file floor, so the cap stays two. Full qualification keeps six covered
shards; its longer instrumentation is outside this fast budget.
The separate CI timings workflow batches recent completed CI and Release runs
hourly or on manual dispatch. It uploads ci-job-timings-batch with raw API pages and
a normalized point-in-time report for each collected workflow attempt, plus a
batch manifest describing examined coverage, failures, and limits. Overlapping
lookback windows may collect an attempt again; identify records by run ID and
attempt. Each batch examines at most 100 completed runs per workflow created in
the preceding 24 hours, with a four-minute API collection budget. It collects the
latest attempt of each selected run and does not claim an exhaustive archive.
It runs the reporter from the trusted default branch with read-only permissions,
treating workflow runs as API data. It does not allocate a new reporter job for
every completed, cancelled, or skipped workflow. Reports separate queue
waiting (created_at to started_at) from completed execution and report
percentiles plus matched npm ci and artifact-step durations.
Queue excludes dependency waiting before job creation and does not establish
whether account concurrency or runner availability caused the wait. Queued jobs with
placeholder start times, skipped jobs, and missing or invalid timestamps remain
unknown. The reporter targets each selected run and attempt explicitly.
Summed job elapsed time is parallel work, not workflow wall time, CPU time,
memory usage, or billing. Step names identify samples, not complete transfer
costs. Its queue, execution, and failures are independent of the required gate
and publication. CPU time and peak process-family memory are not measured by this API.
The build job saves portable Turbo outputs across runs. Eight Linux real-system
lanes reuse its current-run artifact; standalone dispatches restore the cross-run cache. The shared action
derives its key from test:prepare --dry=json task hashes, runner OS and
architecture, .npmrc, .nvmrc, and the lockfile; test-only changes can reuse compiled outputs. Every
build and verification command still runs, so Turbo checks task inputs and
rebuilds misses. A restore prefix with the same platform and toolchain permits
partial task reuse when one generator's inputs change; Turbo still invalidates
that generator's entry. Native outputs are not stored in this cache.
Native package jobs, the Windows curated lane, and the Apple artifact lane reuse
the same workflow's Turbo cache for portable compiled JavaScript, skills, and
catalogs. The key parser retains the validated test:prepare task identities in
.cache/ci-build-plan.json; staging reads this plan without starting another
Turbo run that could evict cache entries. The producer stages matching archives,
metadata, and available manifests into .cache/ci-build-cache/.
portable-build-cache.json records availability per task. Evicted or partially
missing entries are omitted as whole entries, and consumers rebuild cache misses,
including when every entry is unavailable. Malformed plans, non-file entries,
and I/O errors other than disappearance still fail publication. After a trusted
static run, superseded cross-run Turbo snapshots for that OS and architecture
are deleted down to the two newest generations. The Windows lane also runs the
official compiler's complete source typecheck with two checker threads,
independently of the accelerated Linux checker. Both lanes retain the normal
build and prepack commands, which restore matching outputs or rebuild on a cache
miss. Native
artifacts, dependency installs, package archives, and runtime verification remain
host-local. Keep platform-dependent outputs out of this shared cache.
The Windows native verifier's private-runtime ACL observer reads the owner and
access rules only in SID form, checks that every identity it compares is a
SecurityIdentifier, and never uses Get-Acl or the convenience Access
property, which resolves account names. Its Windows PowerShell observers call
.NET members directly and build their JSON without cmdlets, pipelines, module
autoloading, or type-table updates. The private-runtime-acl stderr markers
report the powershell.exe process uptime and each completed phase; inspect them
on a timeout before attributing it to startup, ACL reads, or identity lookup.
The 60-second observer deadline bounds a hung observer while covering a cold
powershell.exe start on a starved hosted runner. All ownership, DACL, and
readonly assertions remain in force. A passing retry alone does not explain a
previous timeout.
Record cold and warm preparation separately from fresh test execution when
benchmarking the optional npm run check:pr gate. Compare multiple runs on the
same host, including subprocess and queue costs; a cached pass is not a test
runtime measurement. A PR that touches packaging or real-system
behavior requires the applicable verify:* lanes; packaging and installation
changes also require npm run verify:package. The full-gate benchmark measures
that explicit lane, not the routine iteration requirement.
The package-update registry fixture retains its first proxy/body failure and
aborts the owned updater command. It joins the helper and installer descendants
before workspace removal. Incomplete process cleanup retains the exact owner
and workspace rather than reporting successful removal. The controlled
packageUpdateRegistryFailure regression needs built process modules, uses no
public registry, and checks both the primary failure and live descendant absence.
Real-toolchain verification lanes
Linux Rizin debugger ownership changes require a separate real-provider gate on
Rizin 0.7.4. Run
npm run test:focused -- tests/boundary/providers/RizinInferiorTty.test.ts.
This test verifies the caller-selected TTY against the actual inferior's file
descriptors, confirms dbg.exitkills=false, detaches through the olj/o-
protocol, and checks that the target survives debugger close and writes to the
selected TTY. The suite is included in the reverse-tools Vitest and mutation
typecheck selections; its real-provider assertions skip when Rizin 0.7.4 is
unavailable. Run the focused command above on a host with that provider for the
explicit real-provider verification gate.
Each real-toolchain command must require only the host tools needed to prove its stated claim. Use a host-native fixture for host/provider acceptance, and place optional cross-target formats or platform-specific runners in separate commands. Check prerequisites before starting expensive work and name the missing command, target, and lane in any failure message. A lane must not imply that a host or target is covered when it was skipped.
Provider admission accepts Ghidra 12.1.x and the JDK range declared by that
installation (application.java.min through application.java.max). Current
12.1 releases require JDK 21 or newer and set no maximum. The lanes below still
prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
| Ghidra lane | Supported runner/target | Additional local tools |
|---|---|---|
npm run verify:ghidra |
Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
npm run verify:ghidra:seeds |
POSIX; raw x86 fixture; seed ordering, BSS labels, analyzer readback, not-applicable and unknown analyzer names | Ghidra 12.1.4 and full JDK 21 |
npm run verify:ghidra:swift |
macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
npm run verify:ghidra:riscv |
Linux/macOS x64/arm64 host; source-owned RV32/RV64 ELF through CLI and MCP | RISC-V Clang, LLD, LLVM readelf, Ghidra 12.1.4, and full JDK 21 |
npm run verify:ghidra:switch |
Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
npm run verify:ghidra:aarch64-jump-table |
Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
npm run verify:ghidra:cross-format |
Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | clang, LLD, and lld-link in addition to host-lane prerequisites |
npm run verify:ghidra:windows |
Controlled Windows x64 with native x86-64 PE; -- --x86 selects native x86 PE |
Ghidra 12.1.4, full JDK 21, and the matching native artifact |
The optional Swift lane compiles a source-owned fixture and checks modern and provider-demangled procedure names through real stdio MCP, with and without the dSYM companion. It checks unfiltered, structs, and literal-name filters, same-address alias evidence, deduplication, and explicit unresolved categories. It does not add a Swift prerequisite to the host C fixture lane.
The host lane also verifies namespaced C++ ABI symbols and annotation identity.
That facet requires Ghidra's matching demangler_gnu_v2_41 native component in
GPL/DemanglerGnu/os/<platform>/ or build/os/<platform>/, in addition to the
native decompiler. The namespace verifier reports missing or non-executable
components before testing the fixture. REA does not build or install them.
Windows native conformance runs with npm run verify:windows-native and does
not require Ghidra or Java. It includes Job Object commit-limit controls (a
limited child that reaches the limit, a control that does not, and invalid
limit rejection). An optional independently compiled Windows fixture
adds in-place reparse, breakaway, environment, and token observations.
The separate scripts/verify-windows-native-inspection.mjs lane uses the
.cache/windows-native-inspection-fixture artifact built on Linux with
node scripts/build-windows-native.mjs --inspection-fixture. Its controlled
first deletion failure checks retained NTFS ownership across a same-volume
rename, later-query handle-based recovery, successful-admission caching, and
environment-teardown cleanup. It neither
establishes the cause of the intermittent #1701 host failure nor replaces
verification with the shipped production artifact.
npm run verify:ghidra:windows:package additionally packs and installs REA into
an isolated prefix and runs ordinary-user CLI/MCP operations against their
canonical schemas. Run the controlled fixture generator before that lane, or
supply an installed package root and an explicit fixture as arguments.
The host-native Ghidra lane also verifies native value tracing through the
production CLI and a separate stdio MCP process. It compares complete dependency
graphs, validates Evidence and upstream/workflow profiles, checks capability
discovery, and closes the MCP session. No provider or transport is mocked.
It also validates every advertised input/output JSON Schema and the exercised
MCP outputs, probes address spelling and name/address ambiguity, and checks
direct versus targetless calls, byte-read completeness, invalid input diagnostics,
CLI/MCP parity, atomic annotation rollback, refreshed inventories, unchanged
executable bytes, and discarded edits after reopen. A deliberately long temporary
path exercises private Unix socket allocation and cleanup, including cancellation
after a real headless process launches. Native annotation probes reject NUL and
unpaired Unicode surrogates without partial edits or a broken bridge, preserve
supported Unicode and control text, and check lossless malformed-text diagnostics.
Memory-to-file mapping is checked against original artifact bytes.
The fixture also stores a pointer one byte past a function entry; exact
xrefs, raw procedure references, and CLI/MCP dossiers must retain that data edge.
A valid legacy snapshot reconstructs the former omitted edge under its older
profile; CLI and MCP must reject that binding with a mismatch reason and
recovery advice. The rejected open must preserve the active live session.
Exact external entries must resolve while retaining an empty body; unknown
external addresses remain unresolved and external annotations are rejected.
An adversarial regex over a full 12 KB literal must report stack exhaustion as
a resource constraint, preserve live annotations, and allow complete literal
searches afterward; CLI and MCP must agree on both results and recovery advice.
Real snapshot lifecycle checks retain edited API results as Evidence while
rejecting immutable snapshot saves and imports before and after a repeated
open of the same target. They verify unchanged live annotations and run identity,
absent rejected output files, an unchanged source snapshot, and successful
snapshot import/save after closing and recreating the database. The pristine
snapshot is written by an independent real CLI session.
They also start a real annotation and snapshot close concurrently: the edit
must succeed, the snapshot must be rejected without creating a file, and the
edited session must remain usable until explicitly closed.
Source-admission probes change a caller-owned fixture after open_binary but
before the first Ghidra query. They require an actionable artifact_changed
error preserving both digests and the selected path, unchanged provider
availability, failed-copy cleanup, and successful recovery after reopening.
After import, deleting that source must preserve the captured database identity.
Instruction inspection and containing-function lookup also agree across
hexadecimal case, leading zeros, and encoded default address-space spellings.
The same source-acquisition workflow exercises missing and directory-replaced
inputs, plus real read-permission denial on a host that enforces it. Permission
fixtures must probe the relevant read, write or search operation before relying
on POSIX mode bits: a non-root Linux process may bypass DAC through capabilities.
requirePermissionDenial reports an explicit skip when the host permits the
probe; it restores the probe's permissions before skipping. Such a skip leaves
real permission coverage unverified. Filesystem integration separately checks
selected-platform routing and exclusive creation.
Namespace annotation probes compile a separate host C fixture with C++ ABI
symbols, avoiding a C++ runtime prerequisite. Real Ghidra demangling supplies
duplicate leaf names in two top-level namespaces and a nested namespace. The
workflow verifies leaf and qualified renames, repeated reuse of fully qualified
readback, lookup by the returned name, literal namespace-like leaf names,
rejection of empty qualified leaf names without changing comments, CLI
behavior, and independent CLI/MCP database ownership.
Large-result probes compile initialized host-native data sized from the pinned
MCP SDK receive budget. Real byte reads, annotation edits and function dossiers
exceed that budget while preserving the connection and active analysis run.
Successful-result and oversized-error delivery constraints must identify their
successfully retained Evidence records;
export must recover every source byte, complete annotation and original error
diagnostic, with CLI parity
and an unchanged executable. Formatter integration separately checks unacknowledged Evidence recording.
Long ordinary procedure names and encoded address-space selectors must produce
normal validation errors without exhausting Java's regex stack or losing the
private bridge connection. Each rejection is followed by a real provider lookup;
address-like literal names still resolve exactly after annotation.
Malformed annotation readback, memory completeness, and inventory data remain separate
SDK/provider integration cases; success from a real
provider cannot establish rejection of a contradictory provider response.
The Linux switch lane checks dense, sparse-with-holes, shared-body, nonzero,
negative, and nonexact JSON integer labels plus a comparison-only control.
Independent source labels, ELF file bytes, table slots, and bounds branches
define expected case/default destinations. Production CLI and MCP must agree;
debug labels must retain their signed values. For stripped negative fixtures,
the independently checked 32-bit dispatch permits equivalent unsigned labels
only alongside the reported low-confidence undefined4 parameter; original
source signedness remains unknown and the ABI residual must remain visible.
unsafe labels remain unresolved and every recovered destination is retained.
The compiler oracle also injects malformed records to check that its assertions
reject missing/default-confused labels, wrong destinations, and numeric guesses.
It also invokes the actual bridge methods on detached Ghidra model objects to
check ambiguous dispatches, signed literals, precision bounds, shared targets,
and conflicting labels. This reflection fixture depends on the pinned Ghidra
model and does not claim a compiler produced those synthetic graph shapes.
Pass --entrypoint /path/to/installed/rea-agents/scripts/rea.mjs direct