Scan provider output before retention and keep bounded confirmed token facts through output completion. Exclude identifier prefixes, pre-checkpoint context, and unconfirmed tails; preserve invalid UTF-8 and directional truncation facts. Make confirmed OOME observations authoritative for projection. Retained raw prefixes cannot establish an OOME, and unrelated output cannot supply its resource kind or recovery advice. Incomplete no-OOME output preserves the primary error and explains uncertainty. Current checkpoint coverage remains separate from historical budget loss. Adapt the retention approach from #4774 with independently reproduced scanner and projection counterexamples corrected before integration. Validation: 268 focused tests, npm run check, npm run docs:check, and test discovery (1639 files exactly once) pass. Controlled Node pipes/socket and fake Ghidra JVM only; real Ghidra and native Linux/Windows behavior are unverified. Fixes #4682 Fixes #5220 Co-authored-by: reason-machines[bot] <293529679+reason-machines[bot]@users.noreply.github.com> Co-authored-by: Saim Shafique <msaimshafique4664@gmail.com>
14 KiB
Offline binary diagnostics
inspect_binary_layout / inspect-binary-layout answers what an explicit
file contains: section/segment layout, original symbols/relocations, static
linkage names and mitigation indicators. It works without an active Hopper,
Ghidra or IDA target. CTF is one possible use; crash triage, compatibility and
ordinary binary inspection use the same modular capability.
Bring your own engine
Initial real verification covers Linux x64, ELF64 x86-64 little-endian EXEC/DYN/REL. Provide an absolute Python executable whose environment already contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
rea inspect-binary-layout ./selected.elf --json
{
"name": "inspect_binary_layout",
"arguments"nstructions in Unicorn; its outputs are static candidates.
Exec syscall tracing observes host process launches. Map completeness
remains unknown and original upstream warnings are returned inline.
Canary, PIE, NX, executable-stack and RELRO are static inferences. NX and
executable-stack are separate upstream indicators: on x86-64 an executable
stack can accompany an unknown (null) NX indicator. ET_DYN can be a shared
library, and an absent canary symbol does not prove every function unprotected.
When complete layout Evidence is already retained, inspect_analysis_view /
inspect-analysis-view projects a summary, the mitigations or linkage facet,
one section or symbol, or a stable page without re-running the decoder. CLI
one-shot invocations pass portable inline Evidence JSON; an MCP session uses
the retained evidence_id. Page limit is required and bounded by the
measured MCP stdio budget documented on the tool contract.
Over MCP, inspect_binary_layout with "detail": "summary" retains the
complete layout Evidence in the session and returns only its summary view.
Its normalized_result.parent_evidence_id names the retained layout for later
section, symbol, mitigation or linkage views. The default complete detail and
CLI output are unchanged.
Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics
and 30-second owned command deadline. REA_PWNTOOLS_MAX_INPUT_BYTES raises the
input bound alone, up to 1 GiB; the reply and deadline bounds are unchanged. The
reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
copies and encoding overhead. The
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
a caller-configured StdioClientTransport({ maxBufferSize: ... }) and a matching
server REA_MCP_MAX_RESPONSE_BYTES. See MCP delivery budgets.
These settings affect delivery, not decoder coverage or limits. The large
extended-index verification uses 256 MiB and an explicit five-minute request
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
retained, and the effective values are returned in the result limitations. Virtual address space is not RSS;
Unicorn needs a large virtual map. Limits fail with no partial success, and
owned process/root cleanup completes independently of cancellation.
Core files, GDB session/state/control, optional pwndbg enrichment and instruction
inspection are separate increments tracked by #969. This increment makes no
real support claim for those features or other operating systems/architectures.
Typed decoder failures retain bounded captured_output (stdout, stderr and an
explicit truncation flag) alongside the original failure category and reason.
Successful diagnostics also exposes the supervisor's truncated flag. Output
beyond the complete diagnostic budget is rejected, including when the decoder
writes a valid reply; cleanup and late cancellation reuse the observed flag.
Memory allocation failures use resource_constraint, with effective resource
limits (or an explicit unknown) and memory-specific recovery guidance. A
MemoryError alone does not establish the exact failed allocation or that a
particular budget was exhausted. Invalid undersized symbol entries are rejected
before REA reports source ranges for them.
The bridge reserves 1 MiB to report allocation failures. If even error reporting
or serialization fails with MemoryError, its reserved exit status preserves the
resource classification with unknown effective limits.
Reserved exits require a matching private marker written by the bridge's actual
failure branch. Bare launcher exits or missing/unwritable markers retain process
diagnostics with the resource cause unverified.
Signal termination alone is never classified as observed memory exhaustion.
The small bootstrap also covers catchable MemoryError during adapter imports,
compilation and initialization. Python interpreter startup failures before the
bootstrap runs retain their observed process diagnostics without guessed causes.
SIGXCPU retains a CPU resource diagnostic and CPU-specific recovery advice.
Observed EFBIG writes or SIGXFSZ termination retain a file-size resource
diagnostic and file-size-specific recovery advice. If a tight file limit also
prevents writing the limit record, effective values remain unknown.
The bridge records actual limits in owned storage before analysis; missing or
malformed limit reports remain unknown with their read failure preserved. A
received signal alone does not establish its exact cause. Dynamic tags require
a complete DT_NULL inside PT_DYNAMIC; interpreter names and ranges end at the
first NUL, with any padding still represented by the original segment range.
Ghidra process failures
When output from the failing Ghidra startup or operation reports
java.lang.OutOfMemoryError, REA returns resource_constraint. Java heap or GC
overhead exhaustion includes heap-specific recovery advice. Metaspace and array
size failures do not recommend increasing the heap; native-thread exhaustion is
reported as a capacity constraint with unknown limits.
configured_jvm_max_heap records the explicit -Xmx value selected by REA's
launch builder. It is a configuration fact, not measured memory use or proof of
the effective JVM limit; JVM environment options may affect that limit.
When a Windows script wrapper selects the JVM, this value remains unknown.
Historical stdout and stderr remain available, but classification uses only the
failing operation's failure_output. Retained stdout and stderr share an 8 MiB
budget, but every drained byte is scanned for java.lang.OutOfMemoryError:
failure_output.observed_oome records the latest confirmed token's bounded line per stream from
the failing operation, with its byte offset, even after the budget is spent. These observations own OOME detection; retained text that ends at a class-name prefix cannot confirm a complete token. line_complete distinguishes a known line end from an incomplete trailing context; invalid_utf8 marks replacement-decoded context.
Incomplete operation output without an observed OOME keeps the original failure code and explains that a memory cause cannot be ruled out. Incompleteness alone does not confirm a memory constraint or justify heap-specific advice. Token context stays within the failing operation's checkpoint; a class-name prefix in a different identifier is not OOME evidence. A trailing bare token requires an observed stream EOF to establish its boundary.
Output draining waits at most one second,
subject to earlier cancellation or the startup deadline. output_incomplete
marks a snapshot taken before stream closure; cleanup ownership remains with
the session until release succeeds.
Portable PE resources
inspect_pe_resources / inspect-pe-resources inspects an explicit PE32 or
PE32+ file without an active target, native engine, Python, or Windows SDK:
rea inspect-pe-resources ./selected.exe --json
{
"name"