rewa81/rea
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(artifacts): inspect classic iNES container regions (#2816)

Browse Source 
* feat(artifacts): inspect classic iNES container regions

* fix(artifacts): preserve iNES source disappearance as integrity failure

* fix(artifacts): revalidate iNES source before parsing header

* fix(artifacts): validate admitted roots before classification

* fix(artifacts): normalize source loss during root hashing

* fix(artifacts): bind iNES extraction admission to inventory

* test(artifacts): clarify non-executing inspection wording

* fix: preserve binary plist detection with extended magic probe

* test(browser): wait for popup document loads before capture

* fix(artifacts): honor cancellation across source validation

* fix(artifacts): prefer artifact magic over implicit database suffix

* fix(artifacts): select extraction readers from scanned format

* fix(artifacts): complete prefix reads and share ZIP format admission

* fix(browser): budget scenario cleanup for content drain and shutdown

* test(artifacts): admit valid universal inputs before routing

* test(artifacts): verify iNES workflows on macOS

* test(ci): own iNES macOS coverage and portable fixture paths

* fix: preserve cleanup retries and avoid duplicate inventory parsing

* fix(contracts): share ZIP target formats across MCP lifecycle schemas

* fix(target): preserve typed cancellation during descriptor admission

* refactor(artifacts): use JSON-safe inventory producer output directly

* test(artifacts): share file read setup and satisfy CI checks

* fix(artifacts): classify changed admission as integrity failure

* ci: run Mach-O summary delivery coverage on macOS

* test(javascript): align parameter provenance with main contract

* test(browser): await discoverable fixture pages before runtime verification

* fix(artifacts): pass target metadata to format detection

* fix(artifacts): bind extraction to opened target

* fix: repair the static-check failures on the default branch (#2895)

* fix: repair the static-check failures on the default branch

The default branch fails its own required `Static checks` job three ways, so
every pull request reports that job as failed regardless of its own change.

- `npm run format:check` reports `src/domain/functionComparison.test.ts` and
  `src/ida/IdaOperationRunner.ts`.
- `npm run lint` reports a duplicate `import { join } from "node:path"` in
  `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (lines 4 and 17).
- `npm run typecheck` reports `format: "devin"` missing from the client format
  union in `src/application/SupportedClients.ts`, and an unchecked index in
  `tests/boundary/cli/clientProfileLifecycle.test.ts`, whose pair array is
  inferred as `string[][]`.

Apply the pinned formatter, drop the duplicate import, list the `"devin"` format
that its own definition already uses, and mark the pair array `as const`.

Validation: `npm run check:fast` and `npm run format:check` pass.

* fix(setup): keep Devin on the JSON configuration dialect

---------

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
(cherry picked from commit df3d96da80)

* fix(artifacts): bind session inventories to target identity

* test(browser): bound fixture readiness acknowledgements

---------

Co-authored-by: 64579525 <17373302683@163.com>
Co-authored-by: 吴佳浩Alben <35291372+XiaoBinGan@users.noreply.github.com>
Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
This commit is contained in:
morluto64579525吴佳浩Alben吴佳浩
 authored and GitHub committed 2026-10-31 15:49:16 +08:00
1 parent e669365c3b
commit 22e5aa97af
51 files changed
+3037 -91
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.agents/skills/reverse-engineer-anything/references/native-and-artifacts.md
+10
View File
@@ -585,6 +585,16 @@ REA. Symlinks and encrypted entries are inventory facts, not extractable files.
The result reports the materialized directory; callers do not choose the
destination. No separate permission grant is needed.
For classic iNES cartridges, `open_binary` recognizes the byte signature without
an executable architecture. `inspect_artifact` retains header facts and each
PRG/CHR file unit's offset, length and SHA-256; its nested inventory's
`ines.reconstruction` verifies the actual ordered region bytes against the
whole-file digest. Extraction includes the original header, optional trainer
and uninterpreted trailing bytes. These file units are not mapper/CPU bank
maps. NES 2.0, UNIF, dirty legacy headers and truncated payloads are explicitly
refused. No emulator or 6502 analysis is performed. See the
[iNES guide](https://github.com/morluto/rea/blob/main/docs/ines-containers.md).
Native DMG traversal automatically uses a read-only mount on supported macOS
hosts, with an owned temporary mount directory and cleanup. It needs no approval
flag. Unsupported hosts or mount failures remain explicit limitations; do not
.github/workflows/ci.yml
+7
View File
@@ -779,6 +779,12 @@ jobs:
npm run verify:legacy-simulator
npm run verify:apple-dispatch
npm run verify:interface-builder
- name: Verify portable iNES CLI, MCP and source boundaries
run: >-
npm run test:focused --
tests/acceptance/inesContainer.test.ts
tests/boundary/filesystem/inesArtifact.test.ts
tests/boundary/filesystem/inesCancellation.test.ts
- name: Run native Apple and process boundary regressions
run: >-
npm run test:focused --
@@ -799,6 +805,7 @@ jobs:
tests/boundary/cli/nativeDemangleCli.test.ts
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
tests/boundary/mcp/nativeDemangleMcp.test.ts
tests/boundary/mcp/nativeMachoSummaryMcp.test.ts
tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts
tests/boundary/cli/nativeSignatureCli.test.ts
tests/boundary/cli/nativeMachoArm64_32Cli.test.ts
docs/.vitepress/config.ts
+1
View File
@@ -70,6 +70,7 @@ export default defineConfig({
{ text: "Apple applications", link: "/apple-application-analysis" },
{ text: "Desktop installations", link: "/desktop-applications" },
{ text: "Firmware", link: "/firmware-analysis" },
{ text: "iNES containers", link: "/ines-containers" },
{ text: "SQLite databases", link: "/sqlite-databases" },
{ text: "Packet captures", link: "/packet-captures" },
{ text: "Signal recordings", link: "/signal-recordings" },
docs/ines-containers.md
+72
View File
@@ -0,0 +1,72 @@
# Classic iNES container inspection
REA recognizes classic iNES from the `NES` + `0x1a` byte signature, independently
of the file extension. This read-only container inspection uses portable
Node.js file reads; it needs neither Ghidra nor an emulator.
```bash
rea inspect-artifact ./cartridge.nes --json
rea extract-artifact ./cartridge.nes --json
```
Through MCP, call `open_binary` with the local path, then `inspect_artifact`.
Use `extract_artifact` to materialize the regions in the fresh directory named
by its result. The caller owns that successful extraction directory.
## Header and file regions
Inspection returns `container-header` and `reconstruction` observations inline.
The single inventory Evidence under `substeps[].evidence.normalized_result`
contains the complete graph and an `ines` object:
- `ines.header`: original 16-byte `header_hex`, PRG/CHR file-unit counts, mapper,
raw flags 6/7, trainer and battery flags, and the declared mirroring mode.
- `occurrences[]`: each region's path, `source_location.offset` and
`source_location.length` in the original container. Follow `artifact_id` to
the node's observed SHA-256 and byte count.
- `ines.reconstruction`: ordered region paths, concatenated byte count,
SHA-256 and `verified: true`. REA accumulates the concatenation digest during
the same physical-order region scan that produces node hashes, then compares
it with the original whole-file digest. Final source freshness is also checked.
No reconstructed file is written during inspection.
Regions are `header.bin`, optional `trainer.bin` (512 bytes),
`prg/000.bin` onward (16 KiB each), `chr/000.bin` onward (8 KiB each), and
optional `trailing.bin`. Every input byte is retained, including unused header
fields and uninterpreted trailing data. Zero CHR-ROM units create no CHR region;
REA does not fabricate CHR-RAM contents.
After extraction, concatenate the materialized files in
`ines.reconstruction.region_order` from the inspection inventory. Their bytes
reproduce the input exactly. Graph path order is lexical and is not physical
file order; use the explicit region order or source offsets.
## Supported boundary
This increment supports classic iNES with a complete 16-byte header, nonzero
PRG count, complete declared trainer/PRG/CHR payload, and zero reserved bytes
12–15. Dirty legacy headers are refused because their upper mapper bits are
ambiguous; REA does not silently repair them. NES 2.0 and UNIF signatures,
unsupported header variants and truncated payloads are explicitly refused.
An arbitrary file named `.nes` is not admitted as iNES.
The PRG/CHR units describe file storage, not mapper bank sizes or CPU windows.
The mapper number and flags are header declarations, not proof of cartridge
hardware or runtime behavior. Bank switching, 6502/65816 disassembly, emulator
sessions and interpretation of console-specific trailing data remain outside
this lane. Raw-binary Ghidra analysis is a separate workflow. This container
increment addresses the first milestone of [#2072](https://github.com/morluto/rea/issues/2072);
the related [#1641](https://github.com/morluto/rea/issues/1641) tracks 6502/65816
native analysis with explicit load addresses and provider limitations. Extracted
PRG units still need caller-selected CPU windows before static code analysis;
their physical file offsets do not establish those addresses.
Source-owned byte fixtures verify original offsets and hashes, extraction and
byte-exact reconstruction, CLI/stdio MCP parity, malformed/unsupported inputs,
failed target-switch recovery, cancellation and source identity. These checks
do not establish emulator behavior or compatibility with every historical ROM.
The portable CLI/stdio MCP acceptance and source-identity checks run in Linux
and macOS CI. The same workflows have also been executed on native Windows x64.
Format references: [iNES](https://www.nesdev.org/wiki/INES) and
[NES 2.0](https://www.nesdev.org/wiki/NES_2.0).
scripts/ci/scopes.mjs
+7
View File
@@ -125,6 +125,12 @@ const PATH_SCOPES = {
],
// Apple artifact CLI/MCP lane on macOS, including its focused regressions.
apple: [
"src/artifacts/InesArtifactReader.ts",
"src/domain/inesContainer.ts",
"tests/fixtures/ines.ts",
"tests/acceptance/inesContainer.test.ts",
"tests/boundary/filesystem/inesArtifact.test.ts",
"tests/boundary/filesystem/inesCancellation.test.ts",
"src/hopper/BridgeLauncher.ts",
"src/domain/hopperErrors.ts",
"tests/boundary/providers/hopper/hopperMacLaunchArguments.test.ts",
@@ -152,6 +158,7 @@ const PATH_SCOPES = {
"tests/boundary/cli/nativeDemangleCli.test.ts",
"tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts",
"tests/boundary/mcp/nativeDemangleMcp.test.ts",
"tests/boundary/mcp/nativeMachoSummaryMcp.test.ts",
"tests/boundary/providers/native/nativeSignatureTargetBinding.test.ts",
"tests/boundary/cli/nativeSignatureCli.test.ts",
"tests/boundary/cli/nativeMachoArm64_32Cli.test.ts",
scripts/lib/browser-popup-e2e.mjs
+55 -10
View File
@@ -17,17 +17,28 @@ import { compareBrowserScenarios } from "../../dist/domain/browserScenarioDiff.j
import { browserScenarioCaptureSchema } from "../../dist/domain/browserScenarioCapture.js";
import { parseEvidence } from "../../dist/domain/evidence.js";
// Readiness includes descendant document loads, so complete network capture
// does not depend on whether an inline script runs before an iframe finishes.
const document = (path) => {
const icon = '<link rel="icon" href="data:,">';
if (path.startsWith("/nested"))
return `<script>throw new Error("nested popup failure")</script>
<script>opener.postMessage("nested-ready", "*")</script>`;
return `${icon}<script>throw new Error("nested popup failure")</script>
<script>addEventListener("load", () => opener.postMessage("nested-ready", "*"))</script>`;
if (path.startsWith("/popup"))
return `<script>throw new Error("popup failure")</script>
<script>onmessage = e => {
if (e.data === "nested-ready") opener.postMessage("popup-ready", "*");
}; window.open("/nested");</script><iframe src="/child"></iframe>`;
if (path.startsWith("/child")) return "child frame";
return `<script>onmessage = e => {
return `${icon}<script>throw new Error("popup failure")</script>
<script>
let loaded = false, nested = false;
const ready = () => {
if (loaded && nested) opener.postMessage("popup-ready", "*");
};
addEventListener("load", () => { loaded = true; ready(); });
onmessage = e => {
if (e.data === "nested-ready") { nested = true; ready(); }
};
window.open("/nested");
</script><iframe src="/child"></iframe>`;
if (path.startsWith("/child")) return `${icon}child frame`;
return `${icon}<script>onmessage = e => {
if (e.data === "popup-ready") document.body.dataset.ready = "true";
}</script><button onclick="window.open('/redirect')">Open popup</button>`;
};
@@ -75,7 +86,25 @@ const assertNetwork = (result, origin) => {
1,
`${kind} ${path} must be captured exactly once`,
);
assert.equal(result.completeness.equality_eligible, true);
assert.equal(
result.completeness.equality_eligible,
true,
JSON.stringify({
limitations: result.limitations,
unfinished: result.events.items.filter(
(event) => event.kind === "request-unfinished",
),
}),
);
assert.equal(
result.events.items.filter(
(event) =>
event.kind === "request-finished" &&
event.url?.url === origin + "/child",
).length,
1,
"child document must finish before network capture ends",
);
assert.equal(compareWithSelf(result).overall_status, "unchanged");
};
@@ -180,13 +209,27 @@ const verifyTransports = async (executable, origin) => {
/** Verify popup timing, selection, coverage, and shared-context isolation on Chromium. */
export async function verifyPopupEventCoverage(executable) {
let slowChild = true;
let childTimer;
const childStarted = Promise.withResolvers();
const server = createServer((request, response) => {
if (request.url === "/redirect") {
response.writeHead(302, { Location: "/popup" });
response.end();
} else {
response.setHeader("Content-Type", "text/html");
response.end(document(request.url ?? "/"));
if (slowChild && request.url === "/child") {
// Keep one response open while the nested page can announce readiness.
// This exposes premature capture cutoff even on fast local browsers.
response.writeHead(200);
response.write(document("/child"));
childStarted.resolve();
childTimer = setTimeout(() => response.end(), 1500);
} else if (slowChild && request.url === "/nested") {
void childStarted.promise.then(() => response.end(document("/nested")));
} else {
response.end(document(request.url ?? "/"));
}
}
});
server.listen(0, "127.0.0.1");
@@ -199,6 +242,7 @@ export async function verifyPopupEventCoverage(executable) {
await capture(scenarioFor(executable, origin, ["network"])),
origin,
);
slowChild = false;
const errors = await capture(
scenarioFor(executable, origin, ["page-errors"]),
);
@@ -240,6 +284,7 @@ export async function verifyPopupEventCoverage(executable) {
stdio_mcp: true,
};
} finally {
clearTimeout(childTimer);
server.closeAllConnections();
await new Promise((resolve, reject) =>
server.close((error) =>
scripts/lib/browser-runtime-fixture-lifecycle.mjs
+71 -1
View File
@@ -11,6 +11,8 @@ import { cleanupOwnedProcessGroup } from "../../dist/process/ProcessOwnership.js
import { ArtifactResourceScope } from "../../dist/artifacts/ArtifactResourceScope.js";
import { AnalysisCleanupLease } from "../../dist/artifacts/AnalysisCleanupLease.js";
import { withVerifierCleanup } from "./verifier-cleanup.mjs";
import { discoverCdpEndpoint } from "../../dist/browser/CdpEndpoint.js";
import { BrowserObservationError } from "../../dist/domain/browserObservationError.js";
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
import { importVerifierSource } from "./verifier-source-loader.mjs";
@@ -26,6 +28,7 @@ const { BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS, waitForBrowserDevtoolsPort } =
export async function startRuntimeFixtureBrowser(executable, initialUrl) {
await requireOrdinaryWindowsVerifierCaller();
const profile = await mkdtemp(join(tmpdir(), "rea-real-web-runtime-"));
const deadline = performance.now() + BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS;
let supervisor;
let stopOwnedBrowser;
const resources = new ArtifactResourceScope();
@@ -94,7 +97,13 @@ export async function startRuntimeFixtureBrowser(executable, initialUrl) {
stderr: () => supervisor.snapshot().stderr.text,
// Match the broader real-browser verifier's allowance for cold Chrome
// startup on shared runners; early process failures still reject at once.
timeoutMs: BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS,
timeoutMs: Math.max(1, deadline - performance.now()),
});
await waitForRuntimeFixturePage({
child: owned.process,
port,
origin: initialUrl,
deadline,
});
return { port, profile, process: owned.process, close };
} catch (cause) {
@@ -104,6 +113,67 @@ export async function startRuntimeFixtureBrowser(executable, initialUrl) {
}
}
async function waitForRuntimeFixturePage({ child, port, origin, deadline }) {
const controller = new AbortController();
let lastFailure;
const onExit = () =>
controller.abort(
new Error(
"Owned runtime fixture exited before CDP discovery became ready",
{ cause: lastFailure },
),
);
const onError = (cause) => controller.abort(cause);
const timer = setTimeout(
() =>
controller.abort(
new Error(
"Owned runtime fixture page was not discoverable within its startup budget",
{ cause: lastFailure },
),
),
Math.max(1, deadline - performance.now()),
);
child.once("exit", onExit);
child.once("error", onError);
try {
for (;;) {
if (child.exitCode !== null || child.signalCode !== null) onExit();
controller.signal.throwIfAborted();
try {
const discovery = await discoverCdpEndpoint(
`http://127.0.0.1:${port}`,
"list_browser_targets",
controller.signal,
);
if (
discovery.targets.some(
(target) =>
target.type === "page" &&
(target.url === origin || target.url.startsWith(`${origin}/`)),
)
)
return;
} catch (cause) {
controller.signal.throwIfAborted();
// Only transient startup failures are retryable. Invalid protocol
// responses and payload-limit failures must remain visible immediately.
if (
!(cause instanceof BrowserObservationError) ||
!["timeout", "endpoint_unreachable"].includes(cause.reason)
)
throw cause;
lastFailure = cause;
}
await new Promise((resolve) => setTimeout(resolve, 25));
}
} finally {
clearTimeout(timer);
child.removeListener("exit", onExit);
child.removeListener("error", onError);
}
}
function createOwnedBrowserStop(supervisor, profile) {
let connection;
return async () => {
src/application/BinaryTargetResolver.ts
+48 -16
View File
@@ -4,6 +4,10 @@ import {
} from "../domain/artifactNameSelection.js";
import { readFileHandleRange } from "../filesystem/readFileHandleRange.js";
import { readPeCoffHeader } from "../domain/peCoffHeader.js";
import {
nesContainerFormat,
parseInesLayout,
} from "../domain/inesContainer.js";
import {
regularFileCancellation,
RegularFileCleanupFailure,
@@ -22,18 +26,13 @@ import { jsonValueSchema } from "../domain/jsonValue.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { constants, type Stats } from "node:fs";
import { createHash } from "node:crypto";
import {
access,
lstat,
realpath,
stat,
type FileHandle,
} from "node:fs/promises";
import { access, realpath, stat, type FileHandle } from "node:fs/promises";
import { extname } from "node:path";
import { BinaryTargetError } from "../domain/configurationErrors.js";
import {
sameRegularFileState,
assertStableRegularFileIdentity,
RegularFileChangedError,
type StableRegularFileDescriptor,
} from "../filesystem/RegularFile.js";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
@@ -455,10 +454,13 @@ const readOpenedBinaryTarget = async (
const artifactFormat =
targetKind === "database"
? undefined
: await detectArtifactFormat(classificationPath, source, signal);
: await detectArtifactFormat(classificationPath, source, signal, path);
if (
targetKind === "database" ||
(targetKind === undefined && path.toLowerCase().endsWith(".hop"))
(targetKind === undefined &&
artifactFormat !== "ines" &&
artifactFormat !== "plist" &&
path.toLowerCase().endsWith(".hop"))
)
return ok({
path,
@@ -720,6 +722,7 @@ const detectArtifactFormat = async (
path: string,
source: StableRegularFileDescriptor,
signal?: AbortSignal,
sourcePath = path,
): Promise<
| Exclude<
BinaryTarget["format"],
@@ -740,6 +743,16 @@ const detectArtifactFormat = async (
const lower = path.toLowerCase();
// Fill the prefix so a short host read cannot hide a signature.
const magic = Buffer.from(await readHandleRange(handle, 0, 4096, signal));
try {
await assertStableRegularFileIdentity(handle, sourcePath, initial, {
symlinks: "reject",
...(signal === undefined ? {} : { signal }),
});
} catch (cause: unknown) {
throwIfTargetResolutionCancelled(signal);
throw cause;
}
throwIfTargetResolutionCancelled(signal);
if (
magic.length >= 8 &&
magic.subarray(0, 8).equals(Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]))
@@ -749,6 +762,15 @@ const detectArtifactFormat = async (
"WebAssembly v1 artifacts are not supported by the native opening route",
{ constraint: "wasm_requires_static_inspection" },
);
if (nesContainerFormat(magic) !== undefined) {
const parsed = parseInesLayout(magic, initial.size);
throwIfTargetResolutionCancelled(signal);
if (!parsed.ok)
throw new BinaryTargetError(path, parsed.error, {
constraint: "file_format_rejected",
});
return "ines";
}
if (hasZipSignature(magic)) return zipRootPackageFormat(lower);
if (hasGodotPckHeader(magic)) return "godot-pck";
if (hasUnityFsHeader(magic)) return "unityfs";
@@ -839,6 +861,7 @@ const isArchiveFormat = (
isZipPackageFormat(format) ||
[
"asar",
"ines",
"godot-pck",
"unityfs",
"uimage",
@@ -925,12 +948,21 @@ const sha256Handle = async (
throw new BinaryTargetError(path, "target grew while being read");
}
throwIfTargetResolutionCancelled(signal);
const [opened, currentPath] = await Promise.all([handle.stat(), lstat(path)]);
if (
!sameRegularFileState(initial, opened) ||
!sameRegularFileState(initial, currentPath) ||
position !== initial.size
)
try {
await assertStableRegularFileIdentity(handle, path, initial, {
symlinks: "reject",
...(signal === undefined ? {} : { signal }),
});
} catch (cause: unknown) {
throwIfTargetResolutionCancelled(signal);
if (cause instanceof RegularFileChangedError)
throw new BinaryTargetError(path, "target changed while being read", {
cause,
});
throw cause;
}
throwIfTargetResolutionCancelled(signal);
if (position !== initial.size)
throw new BinaryTargetError(path, "target changed while being read");
return hash.digest("hex");
};
src/application/binary/TargetFormatVocabularies.test.ts
+1
View File
@@ -117,6 +117,7 @@ const TARGETS: TargetFixtures = {
hsp: { ...archive, format: "hsp" },
"app-pack": { ...archive, format: "app-pack" },
asar: { ...archive, format: "asar" },
ines: { ...archive, format: "ines" },
"godot-pck": { ...archive, format: "godot-pck" },
unityfs: { ...archive, format: "unityfs" },
uimage: { ...archive, format: "uimage" },
src/artifacts/ArtifactHash.ts
+5 -1
View File
@@ -26,18 +26,20 @@ export const abortIfNeeded = (signal?: AbortSignal): void => {
);
};
/** Hash every stream byte and retain its bounded classification prefix. */
/** Hash every stream byte, optionally feeding the same bytes to a scan aggregate. */
export const hashReadable = async (
stream: Readable,
signal?: AbortSignal,
integrityAlgorithm?: "sha256" | "md5",
asarHeaderAdmission?: AsarHeaderAdmission,
observeChunk?: (chunk: Buffer) => void,
): Promise<HashResult> => {
const completed = await hashReadableToCompletion(
stream,
signal,
integrityAlgorithm,
asarHeaderAdmission,
observeChunk,
);
abortIfNeeded(signal);
return completed;
@@ -51,6 +53,7 @@ export const hashReadableToCompletion = async (
signal?: AbortSignal,
integrityAlgorithm?: "sha256" | "md5",
asarHeaderAdmission?: AsarHeaderAdmission,
observeChunk?: (chunk: Buffer) => void,
): Promise<HashResult> => {
const hash = createHash("sha256");
const md5 = integrityAlgorithm === "md5" ? createHash("md5") : undefined;
@@ -61,6 +64,7 @@ export const hashReadableToCompletion = async (
const chunk = streamChunkToBuffer(raw);
bytes += chunk.length;
hash.update(chunk);
observeChunk?.(chunk);
md5?.update(chunk);
probe.write(chunk);
}
src/artifacts/ArtifactReader.ts
+1
View File
@@ -193,6 +193,7 @@ export interface ArtifactReader {
| "directory"
| ZipPackageFormat
| "asar"
| "ines"
| "godot-pck"
| "unityfs"
| "uimage"
src/artifacts/InesArtifactReader.ts
+192
View File
@@ -0,0 +1,192 @@
import { readFileHandlePrefix } from "../filesystem/readFileHandlePrefix.js";
import { Readable } from "node:stream";
import {
parseInesLayout,
type InesInventory,
type InesLayout,
} from "../domain/inesContainer.js";
import type { ArtifactCommand } from "../domain/artifactGraph.js";
import {
RegularFileChangedError,
assertStableRegularFileIdentity,
type StableRegularFileDescriptor,
} from "../filesystem/RegularFile.js";
import { readFileHandleChunks } from "../filesystem/readFileHandleChunks.js";
import type { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import {
ArtifactReaderFailure,
copyArtifactEntry,
sameArtifactEntry,
type ArtifactEntry,
type ArtifactReader,
} from "./ArtifactReader.js";
import { abortIfNeeded } from "./ArtifactHash.js";
import { streamChunkToBuffer } from "./StreamBytes.js";
export const INES_LIMITATION =
"Classic iNES file regions are observed; mapper execution, CPU bank mappings, CHR-RAM contents, 6502 analysis and emulator behavior are not observed. Trailing bytes are retained without interpretation. NES 2.0 and UNIF are unsupported.";
/** Actual concatenated bytes observed by the inventory's ordered region scan. */
export type InesRegionScan = Omit<InesInventory["reconstruction"], "verified">;
/** Read positional regions from a descriptor retained by the artifact resource scope. */
export class InesArtifactReader implements ArtifactReader {
readonly format = "ines" as const;
readonly #entries = new Map<string, ArtifactEntry>();
#layout: InesLayout | undefined;
#closed = false;
constructor(
private readonly path: string,
private readonly source: StableRegularFileDescriptor,
private readonly ownedSource?: OwnedFileHandle,
) {}
async *entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry> {
abortIfNeeded(signal);
await this.#verifySource(signal);
const prefix = await readFileHandlePrefix(this.source.handle, 16, () =>
abortIfNeeded(signal),
);
abortIfNeeded(signal);
// Concurrent source changes take precedence over malformed header diagnostics.
await this.#verifySource(signal);
const parsed = parseInesLayout(prefix, this.source.initial.size);
if (!parsed.ok) throw new ArtifactReaderFailure("format", parsed.error);
this.#layout = parsed.value;
this.#entries.clear();
for (const region of parsed.value.regions) {
abortIfNeeded(signal);
const entry: ArtifactEntry = {
path: region.path,
kind: "file",
declaredSize: region.length,
compressedSize: null,
executable: false,
encrypted: false,
byteOffset: region.offset,
declaredIntegrity: null,
unpacked: false,
limitations: [],
adapterKey: region.path,
};
this.#entries.set(entry.adapterKey, copyArtifactEntry(entry));
yield entry;
}
}
async open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable> {
abortIfNeeded(signal);
const produced = this.#entries.get(entry.adapterKey);
if (produced === undefined || !sameArtifactEntry(entry, produced))
throw new ArtifactReaderFailure(
"integrity",
"iNES region does not match this reader's inventory",
);
await this.#verifySource(signal);
const offset = produced.byteOffset;
const size = produced.declaredSize;
if (offset === null || size === null || size <= 0)
throw new ArtifactReaderFailure("integrity", "Invalid iNES region range");
const stream = readFileHandleChunks(this.source.handle, {
start: offset,
end: offset + size - 1,
signal,
});
const verify = () => this.#verifySource(signal);
return Readable.from(
(async function* () {
let bytes = 0;
try {
for await (const raw of stream) {
abortIfNeeded(signal);
const chunk = streamChunkToBuffer(raw);
bytes += chunk.length;
yield chunk;
}
if (bytes !== size)
throw new ArtifactReaderFailure(
"integrity",
"Truncated iNES region",
);
await verify();
} catch (cause: unknown) {
abortIfNeeded(signal);
if (cause instanceof ArtifactReaderFailure) throw cause;
throw new ArtifactReaderFailure("io", "Could not read iNES region", {
cause,
});
}
})(),
{ objectMode: false },
);
}
/** Validate the completed region scan without rereading its payload. */
async verifyReconstruction(
expectedSha256: string,
observed: InesRegionScan,
signal?: AbortSignal,
): Promise<InesInventory> {
abortIfNeeded(signal);
const layout = this.#layout;
if (layout === undefined || this.#entries.size !== layout.regions.length)
throw new ArtifactReaderFailure("integrity", "Incomplete iNES inventory");
await this.#verifySource(signal);
if (
observed.region_order.length !== layout.regions.length ||
layout.regions.some(
(region, index) => observed.region_order[index] !== region.path,
) ||
observed.bytes !== this.source.initial.size ||
observed.sha256 !== expectedSha256
)
throw new ArtifactReaderFailure(
"integrity",
"iNES reconstruction differs from the source",
);
return {
header: layout.header,
reconstruction: {
...observed,
verified: true,
},
};
}
provenance(): readonly ArtifactCommand[] {
return [];
}
close(): Promise<void> {
// Inventory borrows its admitting scope's descriptor. Extraction owns its
// separately opened descriptor and retains this reader if close fails.
this.#closed = true;
return this.ownedSource?.close() ?? Promise.resolve();
}
async #verifySource(signal?: AbortSignal): Promise<void> {
abortIfNeeded(signal);
if (this.#closed)
throw new ArtifactReaderFailure("unavailable", "iNES reader is closed");
try {
await assertStableRegularFileIdentity(
this.source.handle,
this.path,
this.source.initial,
{ symlinks: "reject", ...(signal === undefined ? {} : { signal }) },
);
} catch (cause: unknown) {
abortIfNeeded(signal);
if (cause instanceof RegularFileChangedError)
throw new ArtifactReaderFailure(
"integrity",
"iNES source changed during reading",
{ cause: cause.cause ?? cause },
);
throw cause;
}
// State validation is asynchronous even after the final byte was consumed.
abortIfNeeded(signal);
}
}
src/artifacts/artifactFormats.test.ts
+1
View File
@@ -27,6 +27,7 @@ type FormatsWithReaders =
| "bun-standalone"
| "dmg"
| "asar"
| "ines"
| "godot-pck"
| "unityfs"
| "uimage"
src/artifacts/artifactFormats.ts
+21 -1
View File
@@ -9,7 +9,10 @@ import {
ZIP_PACKAGE_FORMATS,
type ZipPackageFormat,
} from "../domain/zipPackageFormat.js";
import type { ArtifactReader } from "./ArtifactReader.js";
import {
ArtifactReaderFailure,
type ArtifactReader,
} from "./ArtifactReader.js";
import type { ArtifactReaderAdmission } from "./ArtifactReaderAdmission.js";
import { AsarArtifactReader } from "./AsarArtifactReader.js";
import type { BunStandaloneAdmission } from "./bun/BunStandaloneAdmission.js";
@@ -25,6 +28,7 @@ import { EspImageArtifactReader } from "./EspImageArtifactReader.js";
import { FitArtifactReader } from "./FitArtifactReader.js";
import type { GodotPckAdmission } from "./GodotPckLocation.js";
import { GodotPckArtifactReader } from "./GodotPckArtifactReader.js";
import { InesArtifactReader, INES_LIMITATION } from "./InesArtifactReader.js";
import { HexRecordArtifactReader } from "./HexRecordArtifactReader.js";
import { MachOSliceArtifactReader } from "./MachOSliceArtifactReader.js";
import { NativeDmgArtifactReader } from "./NativeDmgArtifactReader.js";
@@ -244,6 +248,22 @@ const NON_ZIP_FORMATS = {
: { selectedPath: options.selectedPath }),
}),
}),
ines: container("ines", {
roleContainer: true,
retainClassifiedSource: true,
extractable: true,
absentInventory: "unavailable",
readerLimitation: INES_LIMITATION,
createReader: (path, options) => {
const source = sourceOf(options);
if (source === undefined)
throw new ArtifactReaderFailure(
"integrity",
"iNES inventory requires its admitted source descriptor",
);
return new InesArtifactReader(path, source);
},
}),
"godot-pck": container("godot-pck", {
roleContainer: true,
retainClassifiedSource: true,
src/artifacts/extraction/ArtifactExtraction.ts
+73 -7
View File
@@ -4,7 +4,11 @@ import {
assertArtifactSelectedSource,
resolveArtifactSourcePath,
} from "../ArtifactSelectedSource.js";
import { hashStableRootArtifactHandle } from "../inventory/hashStableRootArtifact.js";
import { InesArtifactReader } from "../InesArtifactReader.js";
import {
openRootArtifact,
hashStableRootArtifactHandle,
} from "../inventory/hashStableRootArtifact.js";
import { absoluteCallerPath } from "../../filesystem/callerPath.js";
import type { AsarHeaderAdmission } from "../../config/asarHeaderAdmission.js";
import type { EspFlashSelection } from "../../domain/firmware/espFlashLocation.js";
@@ -165,7 +169,19 @@ export const materializeArtifactInventoryInScope = async (
// A directory source must never acquire its own output as a new member on
// the verification traversal. Resolve the selected parent before any write.
if (transferredSource === undefined) {
const canonicalSource = await realpath(sourcePath);
let canonicalSource: string;
try {
canonicalSource = await realpath(sourcePath);
} catch (cause: unknown) {
abortIfNeeded(signal);
if (inventorySourceChanged(cause))
throw new ArtifactReaderFailure(
"integrity",
`Inventoried source changed: ${sourcePath}`,
{ cause },
);
throw cause;
}
if ((await lstat(canonicalSource)).isDirectory()) {
const { canonicalOutput } = await SafeOutputTree.resolveDestination(
input.outputRoot,
@@ -332,11 +348,12 @@ const materializeSelection = async ({
transferredSource.admission.source,
"Artifact extraction",
);
const reader = createReader(
const reader = await createReader(
sourcePath,
input,
inventory.manifest.root_format,
inventory.manifest,
transferredSource,
signal,
);
const readerOwner: ArtifactResourceOwner = {
kind: "reader" as const,
@@ -628,7 +645,7 @@ const requireExtractableFormat = async (
};
/** Reader selection belongs to inventory classification, not the adapter hint. */
const createReader = (
const createReader = async (
path: string,
input: Pick<
ArtifactExtractionInput,
@@ -636,11 +653,53 @@ const createReader = (
| "selectedPath"
| "environment"
| "asarHeaderAdmission"
| "resourceScope"
| "espFlashSelection"
>,
containerFormat: ArtifactGraphManifest["root_format"],
manifest: ArtifactGraphManifest,
transferredSource?: ActiveArtifactInventorySource,
): ArtifactReader => {
signal?: AbortSignal,
): Promise<ArtifactReader> => {
const containerFormat = manifest.root_format;
if (containerFormat === "ines" && transferredSource === undefined) {
let source: Awaited<ReturnType<typeof openRootArtifact>>;
try {
source = await openRootArtifact(path, input.resourceScope, signal);
} catch (cause: unknown) {
abortIfNeeded(signal);
if (
inventorySourceChanged(cause) ||
(cause instanceof ArtifactReaderFailure && cause.reason === "format")
)
throw new ArtifactReaderFailure(
"integrity",
`Inventoried source changed: ${path}`,
{ cause },
);
throw cause;
}
try {
const digest = await hashStableRootArtifactHandle(
path,
source.handle,
source.initial,
signal,
);
assertAdmittedRootDigest(path, manifest.root_sha256, digest.sha256);
return new InesArtifactReader(path, source, source.owner);
} catch (cause: unknown) {
const attempt = await input.resourceScope.release({
kind: "file-handle",
handle: source.owner,
resource: `iNES source ${path}`,
});
if (attempt.kind === "failed")
throw input.resourceScope.cleanupFailure({ kind: "failed", cause }, [
attempt,
]);
throw cause;
}
}
// DMG inventory may mount read-only, but extraction has no DMG authority.
const reader =
containerFormat === "dmg"
@@ -678,3 +737,10 @@ const preflight = (entry: ArtifactEntry): void => {
`Selected artifact entry cannot be read: ${entry.path}`,
);
};
const inventorySourceChanged = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
["ENOENT", "ENOTDIR", "EISDIR", "ENOTFILE", "ESTALE", "ELOOP"].includes(
String(cause.code),
);
src/artifacts/inventory/ArtifactGraphConstruction.ts
+11
View File
@@ -39,6 +39,10 @@ import {
readFatContainerHeader,
readMachoHeaderPrefixFacts,
} from "../../domain/apple/machoContainer.js";
import {
nesContainerFormat,
parseInesLayout,
} from "../../domain/inesContainer.js";
/** Mutable internal occurrence used until root-bound IDs are known. */
export interface MutableOccurrence {
@@ -588,6 +592,13 @@ export const classifyArtifactBytes = (
fileSize = evidence.prefix.length,
): ArtifactNode["format"] => {
const { prefix, mzWindowsHeaderSignature } = evidence;
// A header-only region has the signature but not the declared cartridge
// extent. Keep it as a file instead of inventing another container.
if (
nesContainerFormat(prefix) === "ines" &&
parseInesLayout(prefix, fileSize).ok
)
return "ines";
if (hasZipSignature(prefix)) return "zip";
if (hasGodotPckHeader(prefix)) return "godot-pck";
if (hasUnityFsHeader(prefix)) return "unityfs";
src/artifacts/inventory/classify.ts
+31 -1
View File
@@ -1,3 +1,9 @@
import {
nesContainerFormat,
parseInesLayout,
} from "../../domain/inesContainer.js";
import type { GodotPckAdmission } from "../GodotPckLocation.js";
import type { PyInstallerAdmission } from "../pyinstaller/PyInstallerAdmission.js";
import type {
ArtifactReaderAdmission,
ArtifactFormatAdmission,
@@ -13,7 +19,11 @@ import {
ArtifactEmbeddedFormatProbes,
type ArtifactEmbeddedFormatProbeFacts,
} from "../ArtifactEmbeddedFormatProbes.js";
import { sameRegularFileState } from "../../filesystem/RegularFile.js";
import {
assertStableRegularFileIdentity,
RegularFileChangedError,
sameRegularFileState,
} from "../../filesystem/RegularFile.js";
import type { FileHandle } from "node:fs/promises";
import type { Stats } from "node:fs";
import type { StableRegularFileDescriptor } from "../../filesystem/RegularFile.js";
@@ -224,6 +234,21 @@ const classifyRootFormat = async (
const archiveBytes = (await handle.stat()).size;
const evidence = probe.finish(archiveBytes);
abortIfNeeded(signal);
try {
await assertStableRegularFileIdentity(handle, path, source.initial, {
symlinks: "reject",
...(signal === undefined ? {} : { signal }),
});
} catch (cause: unknown) {
abortIfNeeded(signal);
if (cause instanceof RegularFileChangedError)
throw new ArtifactReaderFailure(
"integrity",
`Root artifact changed during classification: ${path}`,
{ cause: cause.cause ?? cause },
);
throw cause;
}
// Fill each range: a short result is end of file, not a partial host read.
const read = async (position: number, length: number): Promise<Buffer> => {
const bytes = Buffer.alloc(length);
@@ -257,6 +282,11 @@ const classifyRootFormat = async (
);
return { kind: "esp-flash-image", source, location: selected.value };
}
if (nesContainerFormat(evidence.prefix) !== undefined) {
const parsed = parseInesLayout(evidence.prefix, archiveBytes);
if (!parsed.ok) throw new ArtifactReaderFailure("format", parsed.error);
return "ines";
}
const classificationPath = artifactNameForClassification(
selectedPath ?? path,
path,
src/artifacts/inventory/hashStableRootArtifact.ts
+18 -8
View File
@@ -1,5 +1,4 @@
import { regularFileCancellation } from "../../filesystem/OwnedRegularFile.js";
import { lstat } from "node:fs/promises";
import type { Stats } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import { readArtifactFileChunks } from "../readArtifactFileChunks.js";
@@ -7,10 +6,11 @@ import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import { ArtifactReaderFailure } from "../ArtifactReader.js";
import {
assertStableRegularFileIdentity,
RegularFileChangedError,
NonRegularFileReadError,
RegularFileAdmissionFailure,
openRegularFile,
sameRegularFileState,
type StableRegularFileDescriptor,
} from "../../filesystem/RegularFile.js";
import {
@@ -155,12 +155,22 @@ export const hashStableRootArtifactHandle = async (
signal,
);
abortIfNeeded(signal);
const [opened, currentPath] = await Promise.all([handle.stat(), lstat(path)]);
if (
!sameRegularFileState(initial, opened) ||
!sameRegularFileState(initial, currentPath) ||
digest.bytes !== initial.size
)
try {
await assertStableRegularFileIdentity(handle, path, initial, {
symlinks: "reject",
...(signal === undefined ? {} : { signal }),
});
} catch (cause: unknown) {
abortIfNeeded(signal);
if (cause instanceof RegularFileChangedError)
throw new ArtifactReaderFailure(
"integrity",
`Root artifact changed during inventory: ${path}`,
{ cause: cause.cause ?? cause },
);
throw cause;
}
if (digest.bytes !== initial.size)
throw new ArtifactReaderFailure(
"integrity",
`Root artifact changed during inventory: ${path}`,
src/artifacts/inventory/scanCanonical.ts
+24
View File
@@ -1,8 +1,13 @@
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
import type { InesInventory } from "../../domain/inesContainer.js";
import type { ArtifactEmbeddedFormatProbeFacts } from "../ArtifactEmbeddedFormatProbes.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
import { AsarArtifactReader } from "../AsarArtifactReader.js";
import {
InesArtifactReader,
type InesRegionScan,
} from "../InesArtifactReader.js";
import { lstat } from "node:fs/promises";
import type { Stats } from "node:fs";
@@ -139,6 +144,7 @@ const scanCanonical = async (
occurrences,
pendingContradictions,
caseCollisionLimitation,
inesRegionScan,
unknownParentOccurrences,
traversal,
} = await scanReader(
@@ -163,6 +169,7 @@ const scanCanonical = async (
occurrences,
pendingContradictions,
caseCollisionLimitation,
inesRegionScan,
unknownParentOccurrences,
traversal,
}),
@@ -194,6 +201,7 @@ const scanCanonical = async (
occurrences: [],
pendingContradictions: [],
caseCollisionLimitation: undefined,
inesRegionScan: undefined,
unknownParentOccurrences: new Set(),
traversal: {
status: "unknown",
@@ -250,6 +258,7 @@ const scanCanonical = async (
outcome.kind === "completed" &&
rootSource !== undefined &&
(isZipPackageFormat(rootFormat) ||
rootFormat === "ines" ||
(readerAdmission !== undefined && "kind" in readerAdmission))
? "transfer"
: "release",
@@ -393,6 +402,7 @@ interface SnapshotBuildInput {
readonly occurrences: MutableOccurrence[];
readonly pendingContradictions: PendingIntegrityContradiction[];
readonly caseCollisionLimitation: string | undefined;
readonly inesRegionScan: InesRegionScan | undefined;
readonly unknownParentOccurrences: ReadonlySet<MutableOccurrence>;
readonly traversal: ArtifactReaderTraversal;
}
@@ -470,6 +480,19 @@ const buildInventorySnapshot = async (
orderedOccurrences,
orderedEdges,
});
let ines: InesInventory | undefined;
if (reader instanceof InesArtifactReader) {
if (rootDigest === null || input.inesRegionScan === undefined)
throw new ArtifactReaderFailure(
"integrity",
"Missing iNES reconstruction scan",
);
ines = await reader.verifyReconstruction(
rootDigest.sha256,
input.inesRegionScan,
signal,
);
}
const traversal = inventoryCoverage(rootFormat, reader, input.traversal);
const coverage =
edgeResult.limitations.length === 0
@@ -477,6 +500,7 @@ const buildInventorySnapshot = async (
: { ...traversal.coverage, status: "unknown" as const };
return {
manifest,
...(ines === undefined ? {} : { ines }),
nodes: orderedNodes,
occurrences: orderedOccurrences,
edges: orderedEdges,
src/artifacts/inventory/scanReader.ts
+37
View File
@@ -5,6 +5,7 @@ import {
type ArtifactIntegrityPair,
} from "../../domain/artifactIntegrity.js";
import { isAbsolute } from "node:path";
import { createHash, type Hash } from "node:crypto";
import {
annotateEnglishUnicodeCaseCollisions,
@@ -21,6 +22,10 @@ import {
import { ArtifactTraversalObservation } from "../ArtifactTraversalObservation.js";
import { DirectoryArtifactEntryAccessDenied } from "../DirectoryArtifactReader.js";
import { AsarArtifactReader } from "../AsarArtifactReader.js";
import {
InesArtifactReader,
type InesRegionScan,
} from "../InesArtifactReader.js";
import type { ArtifactNode } from "../../domain/artifactGraph.js";
import {
classifyArtifactContent,
@@ -60,6 +65,13 @@ interface ScanContext {
readonly registry: ArtifactPathRegistry;
readonly expandedContainerIds: Set<string>;
readonly ownedReaders: ArtifactReader[];
readonly reconstruction:
| {
readonly hash: Hash;
readonly regionOrder: string[];
bytes: number;
}
| undefined;
readonly traversal: ArtifactTraversalObservation;
}
@@ -74,6 +86,7 @@ export const scanReader = async (
readonly occurrences: MutableOccurrence[];
readonly pendingContradictions: PendingIntegrityContradiction[];
readonly caseCollisionLimitation: string | undefined;
readonly inesRegionScan: InesRegionScan | undefined;
readonly unknownParentOccurrences: ReadonlySet<MutableOccurrence>;
readonly traversal: ArtifactReaderTraversal;
}> => {
@@ -84,6 +97,7 @@ export const scanReader = async (
occurrences,
pendingContradictions,
caseCollisionLimitation: undefined,
inesRegionScan: undefined,
unknownParentOccurrences: new Set(),
traversal: { status: "complete", limitations: [] },
};
@@ -99,6 +113,10 @@ export const scanReader = async (
registry: new ArtifactPathRegistry(),
expandedContainerIds: new Set<string>(),
ownedReaders,
reconstruction:
reader instanceof InesArtifactReader
? { hash: createHash("sha256"), regionOrder: [], bytes: 0 }
: undefined,
traversal: new ArtifactTraversalObservation(),
};
await visitArtifactEntries(context, reader, "");
@@ -120,6 +138,14 @@ export const scanReader = async (
occurrences,
pendingContradictions,
caseCollisionLimitation: annotateEnglishUnicodeCaseCollisions(occurrences),
inesRegionScan:
context.reconstruction === undefined
? undefined
: {
region_order: context.reconstruction.regionOrder,
bytes: context.reconstruction.bytes,
sha256: context.reconstruction.hash.digest("hex"),
},
unknownParentOccurrences: parentResolution.unknownParentOccurrences,
traversal: context.traversal.snapshot(),
};
@@ -316,12 +342,23 @@ const digestArtifactEntry = async (
> => {
if ((entry.kind !== "file" && entry.kind !== "slice") || entry.encrypted)
return undefined;
const reconstruction =
currentReader === context.reader ? context.reconstruction : undefined;
if (reconstruction !== undefined && entry.byteOffset !== reconstruction.bytes)
throw new ArtifactReaderFailure("integrity", "Noncontiguous iNES regions");
const digest = await hashReadable(
await currentReader.open(entry, context.signal),
context.signal,
entry.declaredIntegrity?.algorithm,
context.asarHeaderAdmission,
reconstruction === undefined
? undefined
: (chunk) => {
reconstruction.hash.update(chunk);
reconstruction.bytes += chunk.length;
},
);
reconstruction?.regionOrder.push(entry.path);
const claim = entry.declaredIntegrity;
const observed = claim?.algorithm === "md5" ? digest.md5 : digest.sha256;
if (claim !== null && observed === undefined)
src/browser/PlaywrightExecutionBoundary.ts
+2
View File
@@ -3,6 +3,8 @@ import {
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
export const PLAYWRIGHT_NETWORK_READ_TIMEOUT_MS = 5_000;
const OPERATION = "capture_browser_scenario" as const;
// Chrome shutdown can take several seconds; expiry bounds the response, not ownership.
src/browser/PlaywrightScenarioBrowser.test.ts
+33 -1
View File
@@ -15,6 +15,7 @@ import type {
import { DEFAULT_CDP_COMMAND_TIMEOUT_MS } from "../domain/browserDeadline.js";
import { browserScenarioSchema } from "../domain/browserScenario.js";
import { PLAYWRIGHT_CLEANUP_CONFIRMATION_TIMEOUT_MS } from "./PlaywrightExecutionBoundary.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import {
attachedScenarioLeaseKey,
@@ -179,6 +180,35 @@ it("keeps Termux execution support when launching Chromium on Android", async ()
});
});
it("waits for bounded event finalization and owned browser teardown", async () => {
vi.useFakeTimers();
try {
const closeBrowser = vi.fn(
() => new Promise<void>((resolve) => setTimeout(resolve, 1_500)),
);
const removeProfile = vi.fn(async () => undefined);
const cleanup = new PlaywrightScenarioBrowserCleanupOwner({
closeBrowser,
removeProfile,
});
const finished = cleanup
.close(() => new Promise<void>((resolve) => setTimeout(resolve, 4_500)))
.then(
() => ({ ok: true }),
(cause: unknown) => ({ ok: false, cause }),
);
await vi.advanceTimersByTimeAsync(4_500);
expect(removeProfile).not.toHaveBeenCalled();
await vi.advanceTimersByTimeAsync(1_500);
expect(await finished).toEqual({ ok: true });
expect(cleanup.browserClosed).toBe(true);
expect(closeBrowser).toHaveBeenCalledOnce();
expect(removeProfile).toHaveBeenCalledOnce();
} finally {
vi.useRealTimers();
}
});
it("keeps attached-target ownership after the cleanup response timeout", async () => {
vi.useFakeTimers();
try {
@@ -209,7 +239,9 @@ it("keeps attached-target ownership after the cleanup response timeout", async (
responseSettled = true;
})
.catch(() => {});
await vi.advanceTimersByTimeAsync(29_999);
await vi.advanceTimersByTimeAsync(
PLAYWRIGHT_CLEANUP_CONFIRMATION_TIMEOUT_MS - 1,
);
expect(responseSettled).toBe(false);
await vi.advanceTimersByTimeAsync(1);
await timeoutExpectation;
src/browser/PlaywrightScenarioSession.test.ts
+81 -4
View File
@@ -60,7 +60,8 @@ vi.mock("./CdpTargetSession.js", () => ({
}),
}));
vi.mock("./PlaywrightScenarioBrowser.js", () => ({
vi.mock("./PlaywrightScenarioBrowser.js", async (importOriginal) => ({
...(await importOriginal<typeof import("./PlaywrightScenarioBrowser.js")>()),
openPlaywrightScenarioBrowser: mocks.openBrowser,
failBrowserScenarioOperation: async (
cleanup: { release(): Promise<void> },
@@ -118,6 +119,7 @@ vi.mock("./PlaywrightScenarioEvents.js", () => ({
import { browserScenarioSchema } from "../domain/browserScenario.js";
import { PlaywrightScenarioSession } from "./PlaywrightScenarioSession.js";
import { PlaywrightScenarioBrowserCleanupOwner } from "./PlaywrightScenarioBrowser.js";
const scenario = browserScenarioSchema.parse({
browser: { mode: "launch", executable_path: process.execPath },
@@ -211,13 +213,14 @@ const fixture = () => {
return browserClosed;
},
};
mocks.openBrowser.mockResolvedValue({
const opened = {
page,
context,
browser,
cleanup,
profilePath: "/tmp/profile",
});
};
mocks.openBrowser.mockResolvedValue(opened);
mocks.storage = {
initialize: vi.fn<PlaywrightScenarioStorage["initialize"]>(
async () => undefined,
@@ -230,7 +233,7 @@ const fixture = () => {
}),
),
};
return { cleanup, page, storage: mocks.storage };
return { cleanup, page, context, storage: mocks.storage, opened };
};
const policyResponse = (method: string): unknown =>
@@ -1153,6 +1156,80 @@ it("retains a detached policy connection until the main realm is restored", asyn
expect(cleanup.close).toHaveBeenCalledTimes(1);
});
it("finishes initialization cleanup independently of request cancellation", async () => {
const { context, opened } = fixture();
const controller = new AbortController();
const closeBrowser = vi.fn(async () => undefined);
const removeProfile = vi.fn(async () => undefined);
mocks.openBrowser.mockResolvedValue({
...opened,
cleanup: new PlaywrightScenarioBrowserCleanupOwner({
closeBrowser,
removeProfile,
}),
});
context.addCookies.mockImplementationOnce(async () => {
controller.abort();
});
const retainCleanup = vi.fn();
await expect(
PlaywrightScenarioSession.open(
scenario,
{},
{
signal: controller.signal,
retainCleanup,
},
),
).rejects.toMatchObject({ _tag: "AnalysisCancelledError" });
expect(closeBrowser).toHaveBeenCalledTimes(1);
expect(removeProfile).toHaveBeenCalledTimes(1);
expect(retainCleanup).not.toHaveBeenCalled();
});
it("can retry cleanup of a browser that opened after request cancellation", async () => {
const { opened } = fixture();
const closeBrowser = vi
.fn()
.mockRejectedValueOnce(new Error("transport close failed"))
.mockResolvedValueOnce(undefined);
const removeProfile = vi.fn(async () => undefined);
const lateOpened = {
...opened,
cleanup: new PlaywrightScenarioBrowserCleanupOwner({
closeBrowser,
removeProfile,
}),
};
let resolveOpening: ((value: typeof lateOpened) => void) | undefined;
const opening = new Promise<typeof lateOpened>((resolve) => {
resolveOpening = resolve;
});
mocks.openBrowser.mockReturnValueOnce(opening);
const controller = new AbortController();
const retained: AnalysisCleanupLease[] = [];
const result = PlaywrightScenarioSession.open(
scenario,
{},
{
signal: controller.signal,
retainCleanup: (close) => retained.push(close),
},
);
const cancelled = expect(result).rejects.toMatchObject({
_tag: "AnalysisCancelledError",
});
controller.abort();
await cancelled;
resolveOpening?.(lateOpened);
await vi.waitFor(() => expect(closeBrowser).toHaveBeenCalledTimes(1));
expect(retained).toHaveLength(1);
await expect(retained[0]?.release()).resolves.toBeUndefined();
expect(closeBrowser).toHaveBeenCalledTimes(2);
expect(removeProfile).toHaveBeenCalledTimes(1);
});
it("uses browser disconnect as terminal storage release after detach failure", async () => {
const { cleanup, storage } = fixture();
const detachFailure = new Error("CDP detach rejected");
src/browser/network/PlaywrightScenarioNetwork.ts
+2 -1
View File
@@ -10,6 +10,7 @@ import type {
BrowserScenarioEvent,
} from "../../domain/browserScenarioCaptureValues.js";
import type { BrowserScenarioSecrets } from "../BrowserScenarioSecrets.js";
import { PLAYWRIGHT_NETWORK_READ_TIMEOUT_MS } from "../PlaywrightExecutionBoundary.js";
import {
readBoundedResponseBody,
MAX_SAVED_RESPONSE_BYTES,
@@ -155,7 +156,7 @@ export class PlaywrightScenarioNetwork {
constructor(private readonly options: NetworkCaptureOptions) {
this.limits = options.limits;
this.readOptions = {
timeoutMs: options.readTimeoutMs ?? 5_000,
timeoutMs: options.readTimeoutMs ?? PLAYWRIGHT_NETWORK_READ_TIMEOUT_MS,
maxBodyBytes: this.limits.max_network_body_bytes,
maxBodyAllocationBytes: this.limits.max_network_body_allocation_bytes,
maxHeaderAllocationBytes: this.limits.max_network_header_allocation_bytes,
src/contracts/artifactToolContracts.ts
+1 -1
View File
@@ -83,7 +83,7 @@ const artifact = <
export const ARTIFACT_TOOL_CONTRACTS = [
artifact(
"inspect_artifact",
"Inspect an active archive or application package, including Node.js SEA carriers, in one call. By default returns the complete content-addressed artifact graph once, as the typed nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Bun standalone inventory describes stored module, sourcemap and opaque bytecode regions. For PCK, Bun and large containers prefer detail summary: it retains the complete inspection and returns its derived summary view; read exact paths, batch occurrence/content IDs and selected fields with inspect_analysis_view using parent_evidence_id. This changes delivery, not inventory coverage. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize regular contents of supported formats. DMG extraction is unavailable; use a caller-managed read-only mount on macOS to access its files.",
"Inspect an active archive or application package, including Node.js SEA carriers, in one call. By default returns the complete content-addressed artifact graph once, as the typed nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Bun standalone inventory describes stored module, sourcemap and opaque bytecode regions. For PCK, Bun and large containers prefer detail summary: it retains the complete inspection and returns its derived summary view; read exact paths, batch occurrence/content IDs and selected fields with inspect_analysis_view using parent_evidence_id. This changes delivery, not inventory coverage. Classic iNES includes header facts, PRG/CHR file-region hashes and byte-exact reconstruction evidence; mapper execution and CPU bank mappings remain unobserved. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize regular contents of supported formats. DMG extraction is unavailable; use a caller-managed read-only mount on macOS to access its files.",
artifactInspectionInputSchema,
{ evidenceParameters: "preserve-detail" },
),
src/contracts/toolEffects.test.ts
+1
View File
@@ -17,6 +17,7 @@ const EXECUTION_WORDING =
/\b(?:run|runs|running|launch|launches|launching|execute|executes|executing|execution|start|starts|install|installs|uninstall|uninstalls)\b/iu;
const EXECUTION_WORDING_EXCEPTIONS: Readonly<Record<string, string>> = {
inspect_artifact: "states that mapper execution remains unobserved",
guided_workflows:
"renders existing guidance and states that it does not start analysis",
inspect_analysis_settings:
src/domain/analysisSnapshot.ts
+1
View File
@@ -73,6 +73,7 @@ const formatSchema = z.enum([
"hsp",
"app-pack",
"asar",
"ines",
"godot-pck",
"unityfs",
"dmg",
src/domain/artifactGraph.ts
+4
View File
@@ -6,6 +6,7 @@ import {
import { artifactFormatMetadataSchema } from "./artifactFormatMetadata.js";
import { z } from "zod";
import { prefixedDigestSchema } from "./../domain/digests.js";
import { inesInventorySchema } from "./inesContainer.js";
/** Entry kinds or subtrees deliberately excluded by an artifact traversal. */
export const artifactTraversalOmissionKindSchema = z.enum([
@@ -62,6 +63,7 @@ export const ARTIFACT_FORMAT_IDS = [
"zip",
"pkg",
"asar",
"ines",
"godot-pck",
"unityfs",
"uimage",
@@ -104,6 +106,7 @@ const artifactFormatSchema = z.enum(ARTIFACT_FORMAT_IDS);
/** Formats established from bytes or a virtual directory tree, independent of path roles. */
const artifactContentFormatSchema = artifactFormatSchema.extract([
"ines",
"zip",
"godot-pck",
"unityfs",
@@ -246,6 +249,7 @@ export const artifactInventoryResultSchema = z.strictObject({
occurrences: z.array(artifactOccurrenceSchema),
edges: z.array(artifactEdgeSchema),
provenance: z.array(artifactCommandSchema),
ines: inesInventorySchema.exactOptional(),
integrity_contradictions: z.array(integrityContradictionSchema),
coverage: artifactInventoryCoverageSchema,
limitations: z.array(z.string()),
src/domain/artifactInspection.ts
+22 -1
View File
@@ -31,7 +31,12 @@ const artifactIdSchema = prefixedDigestSchema("art");
const inspectionObservationSchema = z.strictObject({
observation_id: prefixedDigestSchema("aio"),
kind: z.enum(["root-manifest", "integrity"]),
kind: z.enum([
"root-manifest",
"integrity",
"container-header",
"reconstruction",
]),
subject: textSchema,
value: jsonValueSchema,
evidence_id: evidenceIdSchema,
@@ -231,6 +236,22 @@ const allObservations = (
...inventory.integrity_contradictions.map((value) =>
observation("integrity", value.contradiction_id, value, evidenceId),
),
...(inventory.ines === undefined
? []
: [
observation(
"container-header",
inventory.manifest.root_artifact_id,
inventory.ines.header,
evidenceId,
),
observation(
"reconstruction",
inventory.manifest.root_artifact_id,
inventory.ines.reconstruction,
evidenceId,
),
]),
];
const observation = (
src/domain/artifactInventoryOutput.test.ts
+69
View File
@@ -0,0 +1,69 @@
import { expect, it } from "vitest";
import { artifactInventoryResultSchema } from "./artifactGraph.js";
import { jsonValueSchema } from "./jsonValue.js";
const digest = "a".repeat(64);
const inventory = {
manifest: {
manifest_id: `agm_${digest}`,
root_artifact_id: `art_${digest}`,
root_sha256: digest,
root_format: "zip",
graph_sha256: digest,
node_count: 1,
occurrence_count: 1,
edge_count: 0,
},
nodes: [
{
artifact_id: `art_${digest}`,
format: "zip",
sha256: digest,
size: 22,
media_type: null,
architecture: null,
content_state: "materialized",
limitations: [],
},
],
occurrences: [
{
occurrence_id: `occ_${digest}`,
artifact_id: `art_${digest}`,
parent_occurrence_id: null,
logical_path: ".",
entry_kind: "file",
artifact_kind: "container",
artifact_format: "zip",
declared_size: 22,
compressed_size: null,
executable: false,
encrypted: false,
hash_status: "verified",
source_location: null,
limitations: [],
},
],
edges: [],
provenance: [],
integrity_contradictions: [],
limitations: [],
coverage: { scope: "container-entries", status: "complete" },
};
it("keeps absent container facts absent in JSON-safe inventory output", () => {
const output = artifactInventoryResultSchema.parse(inventory);
expect(Object.hasOwn(output, "ines")).toBe(false);
expect(jsonValueSchema.safeParse(output).success).toBe(true);
expect(
artifactInventoryResultSchema.safeParse({ ...inventory, ines: undefined })
.success,
).toBe(false);
});
it("keeps historical coverage omission separate from current output requirements", () => {
const { coverage: _coverage, ...historical } = inventory;
expect(artifactInventoryResultSchema.safeParse(historical).success).toBe(
false,
);
});
src/domain/artifactInventorySnapshot.ts
+1
View File
@@ -15,6 +15,7 @@ export interface ArtifactInventorySnapshot {
ArtifactInventoryResult["provenance"][number]
>;
readonly integrity_contradictions: readonly IntegrityContradiction[];
readonly ines?: ArtifactInventoryResult["ines"];
readonly coverage: ArtifactInventoryCoverage;
readonly limitations: readonly string[];
}
src/domain/binaryTargetTypes.ts
+1
View File
@@ -290,6 +290,7 @@ export type BinaryTarget =
readonly format:
| ZipPackageFormat
| "asar"
| "ines"
| "godot-pck"
| "unityfs"
| "uimage"
src/domain/evidence.test.ts
+36
View File
@@ -66,6 +66,42 @@ it.each(RAW_ONLY_ARCHITECTURES)(
},
);
it("keeps 6502 confined to explicit raw Evidence subjects alongside iNES", () => {
const selected = {
path: TARGET.path,
sha256: TARGET.sha256,
format: "raw-binary" as const,
architecture: "6502" as const,
};
expect(
createEvidence(selected, PROVIDER, {
operation: "inspect",
parameters: {},
result: null,
}).subject,
).toMatchObject({ format: "raw-binary", architecture: "6502" });
expect(
createEvidence(
{
path: selected.path,
sha256: selected.sha256,
format: "ines",
},
PROVIDER,
{ operation: "inspect", parameters: {}, result: null },
).subject,
).toMatchObject({ format: "ines", architecture: null });
for (const format of ["pe", "ines"] as const) {
expect(() =>
createEvidence({ ...selected, format }, PROVIDER, {
operation: "inspect",
parameters: {},
result: null,
}),
).toThrow();
}
});
it("preserves omitted relationship diagnostics and authenticates reported diagnostics", () => {
const observation = {
operation: "procedure_callees",
src/domain/evidenceSubject.ts
+1
View File
@@ -24,6 +24,7 @@ export const evidenceSubjectFormatSchema = z.enum([
"hsp",
"app-pack",
"asar",
"ines",
"godot-pck",
"unityfs",
"uimage",
src/domain/inesContainer.test.ts
+92
View File
@@ -0,0 +1,92 @@
import { expect, it } from "vitest";
import { parseInesLayout } from "./inesContainer.js";
const header = (): Buffer => {
const bytes = Buffer.alloc(16);
bytes.write("NES\x1a", "latin1");
bytes[4] = 1;
return bytes;
};
it("bounds the complete classic header count without inventing RAM regions", () => {
const bytes = header();
bytes[4] = 255;
bytes[5] = 255;
bytes[6] = 0xfe;
bytes[7] = 0xf3;
const parsed = parseInesLayout(bytes, 16 + 512 + 255 * (16_384 + 8_192));
if (!parsed.ok) throw new Error(parsed.error);
expect(parsed.value.header).toMatchObject({
mapper: 255,
trainer_present: true,
battery_flag: true,
mirroring: "four-screen",
});
expect(parsed.value.regions).toHaveLength(512);
expect(parsed.value.regions.at(-1)).toMatchObject({
path: "chr/254.bin",
length: 8_192,
});
});
it.each([
{
label: "short header",
modify: (bytes: Buffer) => bytes.subarray(0, 8),
size: 8,
reason: "truncated iNES header",
},
{
label: "short payload",
modify: (bytes: Buffer) => bytes,
size: 16,
reason: "truncated iNES payload",
},
{
label: "NES 2.0",
modify: (bytes: Buffer) => {
bytes[7] = 8;
return bytes;
},
size: 16_400,
reason: "nes-2.0",
},
{
label: "dirty legacy header",
modify: (bytes: Buffer) => {
bytes[12] = 1;
return bytes;
},
size: 16_400,
reason: "ambiguous iNES mapper",
},
{
label: "empty PRG",
modify: (bytes: Buffer) => {
bytes[4] = 0;
return bytes;
},
size: 16,
reason: "at least one PRG",
},
{
label: "unsupported variant",
modify: (bytes: Buffer) => {
bytes[7] = 4;
return bytes;
},
size: 16_400,
reason: "unsupported iNES header variant",
},
{
label: "unsafe extent",
modify: (bytes: Buffer) => bytes,
size: Number.MAX_SAFE_INTEGER + 1,
reason: "invalid iNES file extent",
},
])("refuses $label before producing regions", ({ modify, size, reason }) => {
expect(parseInesLayout(modify(header()), size)).toMatchObject({
ok: false,
error: expect.stringContaining(reason),
});
});
src/domain/inesContainer.ts
+131
View File
@@ -0,0 +1,131 @@
import { z } from "zod";
import { err, ok, type Result } from "./result.js";
/** Byte signatures select a container family, never an executable architecture. */
export const nesContainerFormat = (
prefix: Buffer,
): "ines" | "nes-2.0" | "unif" | undefined => {
if (prefix.subarray(0, 4).equals(Buffer.from("UNIF"))) return "unif";
if (!prefix.subarray(0, 4).equals(Buffer.from([0x4e, 0x45, 0x53, 0x1a])))
return undefined;
return prefix.length >= 8 && (prefix.readUInt8(7) & 0x0c) === 0x08
? "nes-2.0"
: "ines";
};
const inesHeaderSchema = z.strictObject({
format: z.literal("ines"),
header_hex: z.string().regex(/^[a-f0-9]{32}$/u),
prg_rom_units_16k: z.number().int().min(1).max(255),
chr_rom_units_8k: z.number().int().min(0).max(255),
mapper: z.number().int().min(0).max(255),
flags_6: z.number().int().min(0).max(255),
flags_7: z.number().int().min(0).max(255),
trainer_present: z.boolean(),
battery_flag: z.boolean(),
mirroring: z.enum(["horizontal", "vertical", "four-screen"]),
});
/** Verified concatenation is a byte identity claim, not an emulator claim. */
export const inesInventorySchema = z.strictObject({
header: inesHeaderSchema,
reconstruction: z.strictObject({
region_order: z.array(z.string().min(1)).min(2),
bytes: z.number().int().min(16).max(Number.MAX_SAFE_INTEGER),
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
verified: z.literal(true),
}),
});
export type InesInventory = z.infer<typeof inesInventorySchema>;
export interface InesRegion {
readonly path: string;
readonly offset: number;
readonly length: number;
}
export interface InesLayout {
readonly header: InesInventory["header"];
readonly regions: readonly InesRegion[];
}
/**
* Classic iNES counts 16 KiB PRG and 8 KiB CHR file units. Those units do not
* establish the mapper's runtime windows. Preserve all other bytes verbatim.
*/
export const parseInesLayout = (
prefix: Buffer,
fileSize: number,
): Result<InesLayout, string> => {
const format = nesContainerFormat(prefix);
if (format !== "ines")
return err(
format === undefined
? "missing iNES byte signature"
: `${format} containers are not supported by the classic iNES lane`,
);
if (!Number.isSafeInteger(fileSize) || fileSize < 0)
return err("invalid iNES file extent: expected a nonnegative safe integer");
if (fileSize < 16 || prefix.length < 16)
return err("truncated iNES header: expected 16 bytes");
const prgUnits = prefix.readUInt8(4);
const chrUnits = prefix.readUInt8(5);
const flags6 = prefix.readUInt8(6);
const flags7 = prefix.readUInt8(7);
if ((flags7 & 0x0c) !== 0) return err("unsupported iNES header variant");
// Dirty legacy headers make the upper mapper bits ambiguous. Do not repair
// them or claim a mapper that the observed header cannot establish.
if (prefix.subarray(12, 16).some((byte) => byte !== 0))
return err(
"ambiguous iNES mapper: reserved header bytes 12..15 are nonzero",
);
if (prgUnits === 0) return err("iNES requires at least one PRG-ROM unit");
const trainer = (flags6 & 4) !== 0;
const declaredEnd =
16 + (trainer ? 512 : 0) + prgUnits * 16_384 + chrUnits * 8_192;
if (declaredEnd > fileSize)
return err(
`truncated iNES payload: header declares ${declaredEnd} bytes; file has ${fileSize}`,
);
const regions: InesRegion[] = [{ path: "header.bin", offset: 0, length: 16 }];
let offset = 16;
if (trainer) {
regions.push({ path: "trainer.bin", offset, length: 512 });
offset += 512;
}
for (const [kind, count, length] of [
["prg", prgUnits, 16_384],
["chr", chrUnits, 8_192],
] as const)
for (let index = 0; index < count; index += 1) {
regions.push({
path: `${kind}/${String(index).padStart(3, "0")}.bin`,
offset,
length,
});
offset += length;
}
if (offset < fileSize)
regions.push({ path: "trailing.bin", offset, length: fileSize - offset });
return ok({
header: {
format: "ines",
header_hex: prefix.subarray(0, 16).toString("hex"),
prg_rom_units_16k: prgUnits,
chr_rom_units_8k: chrUnits,
mapper: (flags6 >>> 4) | (flags7 & 0xf0),
flags_6: flags6,
flags_7: flags7,
trainer_present: trainer,
battery_flag: (flags6 & 2) !== 0,
mirroring:
(flags6 & 8) !== 0
? "four-screen"
: (flags6 & 1) !== 0
? "vertical"
: "horizontal",
},
regions,
});
};
src/domain/liveBinaryTargetSummary.ts
+1
View File
@@ -70,6 +70,7 @@ export const liveBinaryTargetSummaryVariants = [
format: z.enum([
...ZIP_PACKAGE_FORMATS,
"asar",
"ines",
"godot-pck",
"unityfs",
"uimage",
src/filesystem/RegularFile.ts
+6 -3
View File
@@ -230,10 +230,13 @@ export const assertStableRegularFileIdentity = async (
throw cause;
}
};
const selected = await named();
assertRegularFileActive(options.signal);
const opened = await file.stat({ bigint });
assertRegularFileActive(options.signal);
if (
!sameRegularFileState(before, await file.stat({ bigint })) ||
!sameRegularFileState(before, await named())
!sameRegularFileState(before, opened) ||
!sameRegularFileState(before, selected)
)
throw new RegularFileChangedError(path);
assertRegularFileActive(options.signal);
};
src/filesystem/readFileHandlePrefix.ts
+30
View File
@@ -0,0 +1,30 @@
import type { FileHandle } from "node:fs/promises";
/** Fill a positional prefix until its bound or true EOF, preserving caller cancellation errors. */
export const readFileHandlePrefix = async (
handle: Pick<FileHandle, "read">,
length: number,
checkCancellation: () => void,
): Promise<Buffer> => {
const prefix = Buffer.alloc(length);
let offset = 0;
while (offset < length) {
checkCancellation();
let bytesRead: number;
try {
({ bytesRead } = await handle.read(
prefix,
offset,
length - offset,
offset,
));
} catch (cause: unknown) {
checkCancellation();
throw cause;
}
checkCancellation();
if (bytesRead === 0) break;
offset += bytesRead;
}
return prefix.subarray(0, offset);
};
tests/acceptance/inesContainer.test.ts
+305
View File
@@ -0,0 +1,305 @@
import { createHash } from "node:crypto";
import { readFile, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect } from "vitest";
import { artifactInspectionResultSchema } from "../../src/domain/artifactInspection.js";
import {
artifactInventoryResultSchema,
artifactExtractionResultSchema,
} from "../../src/domain/artifactGraph.js";
import { analysisViewResultSchema } from "../../src/domain/analysisView/analysisView.js";
import { parseEvidence } from "../../src/domain/evidence.js";
import { cliTest } from "../support/cli/cliFixture.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
import { connectLocalToolsMcp } from "../fixtures/localToolsMcp.js";
import { parseMcpToolError } from "../fixtures/mcpToolError.js";
import { inesFixture } from "../fixtures/ines.js";
import { writeOrderedZip } from "../fixtures/artifactEntryOrder.js";
import { ZIP_PACKAGE_FORMATS } from "../../src/domain/zipPackageFormat.js";
import { toolContract } from "../../src/contracts/toolContracts.js";
import { buildBinary } from "plist";
cliTest(
"refuses extraction after an already open plist is replaced through MCP",
async () => {
const root = await createTestTempDirectory("rea-ines-plist-refresh-");
const path = join(root, "source.bin");
await writeFile(path, buildBinary({ value: "old" }));
const { call } = await connectLocalToolsMcp();
const opened = await call("open_binary", { path });
expect(opened.isError, JSON.stringify(opened.structuredContent)).not.toBe(
true,
);
expect(
toolContract("open_binary").outputSchema.parse(opened.structuredContent)
.result,
).toMatchObject({ format: "plist" });
const bytes = inesFixture();
await writeFile(path, bytes);
const extracted = await call("extract_artifact", {});
expect(extracted.isError).toBe(true);
expect(JSON.stringify(parseMcpToolError(extracted))).toContain(
"Artifact format has no extraction reader: plist",
);
await call("close_binary", {});
},
);
cliTest(
"refreshes an iNES path through MCP for every ZIP package family",
async () => {
const root = await createTestTempDirectory("rea-ines-zip-refresh-");
const { call } = await connectLocalToolsMcp();
for (const format of ZIP_PACKAGE_FORMATS) {
const path = join(
root,
`cartridge.${format === "app-pack" ? "app" : format}`,
);
await writeFile(path, inesFixture());
const first = await call("open_binary", { path });
expect(first.isError, JSON.stringify(first)).not.toBe(true);
await writeOrderedZip(path, ["payload.txt"]);
const refreshed = await call("open_binary", { path });
expect(
refreshed.isError,
JSON.stringify({ format, response: refreshed }),
).not.toBe(true);
expect(
toolContract("open_binary").outputSchema.parse(
refreshed.structuredContent,
).result,
).toMatchObject({ kind: "archive", format });
const session = await call("binary_session", {});
expect(
session.isError,
JSON.stringify(session.structuredContent),
).not.toBe(true);
expect(
toolContract("binary_session").outputSchema.parse(
session.structuredContent,
).result,
).toMatchObject({ open: true, kind: "archive", format });
}
},
);
const inventoryFrom = (input: unknown) => {
const evidence = parseEvidence(input);
expect(evidence.subject).toMatchObject({
format: "ines",
architecture: null,
});
const inspection = artifactInspectionResultSchema.parse(
evidence.normalized_result,
);
expect(inspection.coverage.status).toBe("complete-within-substeps");
expect(inspection.observations.map(({ kind }) => kind)).toEqual([
"root-manifest",
"container-header",
"reconstruction",
]);
const substep = inspection.substeps[0];
if (substep === undefined) throw new Error("Missing inventory substep");
parseEvidence(substep.evidence);
const inventory = artifactInventoryResultSchema.parse(
substep.evidence.normalized_result,
);
expect(inventory.coverage.status).toBe("complete");
expect(inventory.limitations).toContainEqual(
expect.stringContaining("CPU bank mappings"),
);
return inventory;
};
cliTest(
"classic iNES has the same complete byte inventory through compiled CLI and stdio MCP",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-ines-public-");
const path = join(root, "卡带.hop");
const bytes = inesFixture({
trainer: true,
trailing: Buffer.from([0, 255, 3, 0]),
});
await writeFile(path, bytes);
const cliResult = await cli.run({
arguments: ["inspect-artifact", path, "--json"],
});
expect(cliResult.exitCode, cliResult.stderr).toBe(0);
const inventory = inventoryFrom(cliResult.json);
const digest = createHash("sha256").update(bytes).digest("hex");
expect(inventory.ines).toMatchObject({
header: {
mapper: 19,
trainer_present: true,
prg_rom_units_16k: 2,
chr_rom_units_8k: 1,
},
reconstruction: { bytes: bytes.length, sha256: digest, verified: true },
});
const expected = [
["header.bin", 0, 16],
["trainer.bin", 16, 512],
["prg/000.bin", 528, 16_384],
["prg/001.bin", 16_912, 16_384],
["chr/000.bin", 33_296, 8_192],
["trailing.bin", 41_488, 4],
] as const;
const nodes = new Map(
inventory.nodes.map((node) => [node.artifact_id, node]),
);
for (const [logical_path, offset, length] of expected) {
const occurrence = inventory.occurrences.find(
(item) => item.logical_path === logical_path,
);
if (
occurrence?.artifact_id === undefined ||
occurrence.artifact_id === null
)
throw new Error(`Missing region ${logical_path}`);
expect(occurrence.source_location).toEqual({ offset, length });
expect(nodes.get(occurrence.artifact_id)?.sha256).toBe(
createHash("sha256")
.update(bytes.subarray(offset, offset + length))
.digest("hex"),
);
}
const { call } = await connectLocalToolsMcp();
const opened = await call("open_binary", { path });
if (opened.isError) throw new Error(JSON.stringify(opened));
const inspected = await call("inspect_artifact", {});
expect(
inspected.isError,
JSON.stringify(inspected.structuredContent),
).not.toBe(true);
expect(inventoryFrom(inspected.structuredContent)).toEqual(inventory);
const typed = artifactInspectionResultSchema.parse(
parseEvidence(inspected.structuredContent).normalized_result,
);
expect(typed.substeps[0]?.evidence.normalized_result).toMatchObject({
ines: inventory.ines,
});
const withInventory = (normalized_result: unknown) => ({
...typed,
substeps: typed.substeps.map((substep) => ({
...substep,
evidence: { ...substep.evidence, normalized_result },
})),
});
expect(
artifactInspectionResultSchema.safeParse(
withInventory({ ...inventory, ines: undefined }),
).success,
).toBe(false);
const withoutInes = { ...inventory };
delete withoutInes.ines;
expect(
artifactInspectionResultSchema.safeParse(withInventory(withoutInes))
.success,
).toBe(true);
const summary = await call("inspect_artifact", { detail: "summary" });
expect(summary.isError, JSON.stringify(summary.structuredContent)).not.toBe(
true,
);
const view = analysisViewResultSchema.parse(
parseEvidence(summary.structuredContent).normalized_result,
);
expect(view.kind).toBe("summary");
const regions = await call("inspect_analysis_view", {
source: {
kind: "retained-evidence",
evidence_id: view.parent_evidence_id,
},
view: {
kind: "inventory",
collection: "occurrences",
logical_paths: ["prg/000.bin"],
},
});
expect(regions.isError, JSON.stringify(regions.structuredContent)).not.toBe(
true,
);
expect(
parseEvidence(regions.structuredContent).normalized_result,
).toMatchObject({
kind: "inventory",
items: inventory.occurrences.filter(
(item) => item.logical_path === "prg/000.bin",
),
});
const extracted = await call("extract_artifact", {});
expect(
extracted.isError,
JSON.stringify(extracted.structuredContent),
).not.toBe(true);
const extraction = artifactExtractionResultSchema.parse(
parseEvidence(extracted.structuredContent).normalized_result,
);
try {
expect(extraction.artifacts).toHaveLength(expected.length);
const reconstructed = Buffer.concat(
await Promise.all(
expected.map(([region]) =>
readFile(join(extraction.output_root, region)),
),
),
);
expect(reconstructed.equals(bytes)).toBe(true);
} finally {
// The successful extraction transfers this exact output directory to the caller.
await rm(extraction.output_root, { recursive: true, force: true });
}
await call("close_binary", {});
expect((await readFile(path)).equals(bytes)).toBe(true);
},
);
cliTest(
"unsupported NES containers fail explicitly and leave a valid MCP session usable",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-ines-public-refusal-");
const validPath = join(root, "valid.nes");
await writeFile(validPath, inesFixture({ chrUnits: 0 }));
const { call } = await connectLocalToolsMcp();
const opened = await call("open_binary", { path: validPath });
if (opened.isError) throw new Error(JSON.stringify(opened));
for (const [name, bytes, reason] of [
["truncated", inesFixture().subarray(0, 20), "truncated iNES payload"],
[
"nes2",
(() => {
const bytes = inesFixture();
bytes[7] = 8;
return bytes;
})(),
"nes-2.0",
],
["unif", Buffer.from("UNIF"), "unif"],
] as const) {
const path = join(root, `${name}.nes`);
await writeFile(path, bytes);
const refused = await cli.run({
arguments: ["inspect-artifact", path, "--json"],
});
expect(refused.exitCode).not.toBe(0);
expect(JSON.stringify(refused.json)).toContain(reason);
const switched = await call("open_binary", { path });
expect(switched.isError).toBe(true);
expect(JSON.stringify(parseMcpToolError(switched))).toContain(reason);
}
const retained = await call("inspect_artifact", {});
expect(
retained.isError,
JSON.stringify(retained.structuredContent),
).not.toBe(true);
const inventory = inventoryFrom(retained.structuredContent);
expect(inventory.ines?.header.chr_rom_units_8k).toBe(0);
expect(
inventory.occurrences.some(({ logical_path }) =>
logical_path.startsWith("chr/"),
),
).toBe(false);
await call("close_binary", {});
},
);
tests/boundary/browser/browserProcessStartup.test.ts
+6 -13
View File
@@ -157,22 +157,15 @@ describe("browser process startup", () => {
),
"utf8",
);
expect(devtoolsPortDeadlines(source)).toEqual([
["BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS"],
]);
expect(source).toContain(
"const deadline = performance.now() + BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS",
);
expect(source).toContain(
"timeoutMs: Math.max(1, deadline - performance.now())",
);
});
});
const temporaryRoot = async (): Promise<string> => {
return createTestTempDirectory("rea-browser-startup-");
};
const devtoolsPortDeadlines = (source: string): string[][] => {
const calls = /waitForBrowserDevtoolsPort\(\{(?<body>[\s\S]*?)\}\)/gu;
const deadlines = /\btimeoutMs:\s*(?<value>[^\n,]+)/gu;
return [...source.matchAll(calls)].map((call) =>
[...String(call.groups?.body).matchAll(deadlines)].map((deadline) =>
String(deadline.groups?.value).trim(),
),
);
};
tests/boundary/browser/runtimeFixtureCleanup.test.ts
+11 -3
View File
@@ -198,12 +198,20 @@ async function writeFixtureBrowser(root: string): Promise<string> {
const { spawn } = require('node:child_process');
const { writeFileSync } = require('node:fs');
const { join } = require('node:path');
const { createServer } = require('node:http');
const profile = process.argv.find(value => value.startsWith('--user-data-dir=')).split('=').slice(1).join('=');
const child = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)']);
writeFileSync(join(profile, 'descendant'), String(child.pid));
if (!process.argv.at(-1).endsWith('/startup-fail'))
writeFileSync(join(profile, 'DevToolsActivePort'), '1\\n/devtools/browser/fixture');
else setTimeout(() => process.exit(3), 30);
if (!process.argv.at(-1).endsWith('/startup-fail')) {
const origin = process.argv.at(-1);
const server = createServer((req, res) => {
const port = server.address().port;
res.end(JSON.stringify(req.url === '/json/version'
? { Browser: 'Fixture/1', 'Protocol-Version': '1.3', 'User-Agent': 'Fixture', 'V8-Version': '1', 'WebKit-Version': '1', webSocketDebuggerUrl: 'ws://127.0.0.1:' + port + '/devtools/browser/fixture' }
: [{ id: 'fixture', type: 'page', title: 'fixture', url: origin }]));
});
server.listen(0, '127.0.0.1', () => writeFileSync(join(profile, 'DevToolsActivePort'), String(server.address().port) + '\\n/devtools/browser/fixture'));
} else setTimeout(() => process.exit(3), 30);
console.error('owned fixture diagnostics');
setInterval(() => {}, 1000);
`,
tests/boundary/browser/runtimeFixtureReadiness.test.ts
+231
View File
@@ -0,0 +1,231 @@
import { ChildProcess } from "node:child_process";
import { createServer } from "node:http";
import { setImmediate } from "node:timers/promises";
import { afterEach, expect, it, vi } from "vitest";
import { z } from "zod";
import { awaitAcknowledgement } from "../../fixtures/lifecycleAcknowledgement.js";
const mock = vi.hoisted(() => ({ spawn: vi.fn(), port: vi.fn() }));
vi.mock("../../../dist/process/ProviderProcess.js", () => ({
spawnOwnedProviderProcess: mock.spawn,
ProviderProcessSupervisor: class {
constructor(
readonly launch: {
process: ChildProcess;
cleanup: () => Promise<unknown>;
},
) {}
snapshot() {
return { stderr: { text: "" } };
}
async stop() {
await this.launch.cleanup();
return { status: "complete" };
}
},
}));
vi.mock("../../../dist/process/ProcessOwnershipObservation.js", () => ({
createSystemProcessOwnershipHost: () => ({ listProcesses: async () => [] }),
}));
vi.mock("../../../scripts/lib/verifier-source-loader.mjs", () => ({
importVerifierSource: async () => ({
BROWSER_DEVTOOLS_STARTUP_TIMEOUT_MS: 60_000,
waitForBrowserDevtoolsPort: mock.port,
}),
}));
afterEach(() => vi.restoreAllMocks());
interface FixtureLease {
readonly port: number;
close(): Promise<void>;
}
type StartFixture = (
executable: string,
origin: string,
) => Promise<FixtureLease>;
const startFixture = async (): Promise<StartFixture> =>
z
.object({
startRuntimeFixtureBrowser: z.custom<StartFixture>(
(value) => typeof value === "function",
),
})
.parse(
await import(
new URL(
"../../../scripts/lib/browser-runtime-fixture-lifecycle.mjs",
import.meta.url,
).href
),
).startRuntimeFixtureBrowser;
const deferred = () => {
let resolve!: () => void;
const promise = new Promise<void>((complete) => {
resolve = complete;
});
return { promise, resolve };
};
const fixture = async (
mode: "empty targets" | "discovery timeout" | "malformed" | "exited",
) => {
let ready = false;
let requests = 0;
const listed = deferred();
const published = deferred();
const server = createServer((request, response) => {
requests += 1;
if (mode === "exited") {
listed.resolve();
return;
}
if (mode === "malformed") {
response.end("{broken");
return;
}
if (mode === "discovery timeout" && requests === 1) {
response.once("close", () => {
ready = true;
});
return;
}
response.end(
JSON.stringify(
request.url === "/json/version"
? {
Browser: "FixtureChrome/1",
"Protocol-Version": "1.3",
"User-Agent": "FixtureChrome",
"V8-Version": "1",
"WebKit-Version": "1",
webSocketDebuggerUrl: `ws://127.0.0.1:${port}/devtools/browser/fixture`,
}
: ready
? [{ id: "fixture", type: "page", title: "fixture", url: origin }]
: [],
),
);
if (request.url === "/json/list") listed.resolve();
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("Missing fixture address");
const port = address.port;
const origin = `http://127.0.0.1:${port}`;
const child = new ChildProcess();
let signalCode: NodeJS.Signals | null = null;
Object.defineProperty(child, "signalCode", { get: () => signalCode });
vi.spyOn(child, "kill").mockImplementation(() => {
signalCode = "SIGTERM";
child.emit("exit", null, "SIGTERM");
return true;
});
const cleanup = vi.fn(async () => ({ cleaned: true }));
mock.spawn.mockResolvedValue({
process: child,
ownership: { processGroupId: 7 },
cleanup,
});
mock.port.mockImplementation(async () => {
published.resolve();
return port;
});
return {
origin,
listed: listed.promise,
published: published.promise,
cleanup,
exit: () => child.kill("SIGTERM"),
makeReady: () => {
ready = true;
},
requests: () => requests,
close: async () => {
server.closeAllConnections();
await new Promise<void>((resolve) => server.close(() => resolve()));
},
};
};
it.each(["empty targets", "discovery timeout"] as const)(
"does not hand out the owned fixture while CDP has %s",
async (mode) => {
const start = await startFixture();
const server = await fixture(mode);
const pending = start(process.execPath, server.origin);
let settled = false;
void pending.then(
() => {
settled = true;
},
() => {
settled = true;
},
);
try {
await awaitAcknowledgement(
server.published,
pending,
"fixture port publication",
);
await setImmediate();
expect(settled).toBe(false);
if (mode === "empty targets") {
await awaitAcknowledgement(
server.listed,
pending,
"fixture target listing",
);
expect(settled).toBe(false);
server.makeReady();
}
await pending;
expect(server.requests()).toBeGreaterThanOrEqual(3);
} finally {
server.makeReady();
await (await pending.catch(() => undefined))?.close();
await server.close();
}
},
);
it("preserves malformed discovery and closes the owned fixture", async () => {
const start = await startFixture();
const server = await fixture("malformed");
const pending = start(process.execPath, server.origin);
try {
await expect(pending).rejects.toMatchObject({
reason: "invalid_endpoint_response",
});
expect(server.requests()).toBe(1);
expect(server.cleanup).toHaveBeenCalledTimes(1);
} finally {
await (await pending.catch(() => undefined))?.close();
await server.close();
}
});
it("interrupts pending discovery when the owned fixture exits", async () => {
const start = await startFixture();
const server = await fixture("exited");
const pending = start(process.execPath, server.origin);
const rejected = expect(pending).rejects.toThrow(
"exited before CDP discovery",
);
try {
await awaitAcknowledgement(
server.listed,
pending,
"fixture target listing",
);
server.exit();
await rejected;
expect(server.cleanup).toHaveBeenCalledTimes(1);
} finally {
await (await pending.catch(() => undefined))?.close();
await server.close();
}
});
tests/boundary/filesystem/binaryTarget.test.ts
+19 -10
View File
@@ -476,16 +476,19 @@ describe("binary target I/O: package classification", () => {
});
});
it("classifies a binary plist by its magic under any extension", async () => {
const directory = await createTestTempDirectory("rea-bplist-target-");
const path = join(directory, "Model.archive");
await writeFile(path, buildBinary({ $archiver: "NSKeyedArchiver" }));
const result = await parseBinaryTarget(path);
expect(result.ok && result.value).toMatchObject({
kind: "artifact",
format: "plist",
});
});
it.each(["archive", "hop"])(
"classifies binary plist magic under a .%s extension",
async (suffix) => {
const directory = await createTestTempDirectory("rea-bplist-target-");
const path = join(directory, `Model.${suffix}`);
await writeFile(path, buildBinary({ $archiver: "NSKeyedArchiver" }));
const result = await parseBinaryTarget(path);
expect(result.ok && result.value).toMatchObject({
kind: "artifact",
format: "plist",
});
},
);
it("classifies XML plists by content while preserving named formats", async () => {
const directory = await createTestTempDirectory("rea-xml-plist-target-");
@@ -553,7 +556,9 @@ describe("binary target I/O: package classification", () => {
error: { _tag: "BinaryTargetError" },
});
});
});
describe("binary target I/O: image classification", () => {
it("classifies ESP images by bytes and refuses ESP8266 v1 images", async () => {
const directory = await createTestTempDirectory("rea-esp-target-");
const app = join(directory, "firmware.ota");
@@ -742,6 +747,10 @@ describe("binary target I/O: filesystem and app bundle target resolution", () =>
await writeFile(join(directory, "sample.hop"), "database");
await writeFile(join(directory, "text"), "hello");
const database = await parseBinaryTarget("sample.hop", { cwd: directory });
expect(database).toMatchObject({
ok: true,
value: { kind: "database", format: "analysis-database" },
});
expect(database.ok && database.value.sha256).toBe(
"3549b0028b75d981cdda2e573e9cb49dedc200185876df299f912b79f69dabd8",
);
tests/boundary/filesystem/ciChangeScope.test.ts
+24
View File
@@ -861,6 +861,30 @@ it.each([
"tests/boundary/cli/nativePlistCli.test.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"src/artifacts/InesArtifactReader.ts",
{ code: true, apple: true, package: false, runtime: true },
],
[
"src/domain/inesContainer.ts",
{ code: true, apple: true, docs: true, package: false, runtime: true },
],
[
"tests/fixtures/ines.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"tests/acceptance/inesContainer.test.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"tests/boundary/filesystem/inesArtifact.test.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"tests/boundary/filesystem/inesCancellation.test.ts",
{ code: true, apple: true, package: false, runtime: false },
],
[
"scripts/ci/qualified-package-envelope.mjs",
{ code: true, package: true, runtime: true, apple: false },
tests/boundary/filesystem/inesArtifact.test.ts
+781
View File
@@ -0,0 +1,781 @@
import * as rootHash from "../../../src/artifacts/inventory/hashStableRootArtifact.js";
import { readFileHandlePrefix } from "../../../src/filesystem/readFileHandlePrefix.js";
import { abortIfNeeded } from "../../../src/artifacts/ArtifactHash.js";
import { createHash } from "node:crypto";
import {
lstat,
mkdir,
open,
readFile,
unlink,
writeFile,
type FileHandle,
} from "node:fs/promises";
import { join } from "node:path";
import { expect, it, vi } from "vitest";
import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { inesFixture } from "../../fixtures/ines.js";
import {
inventoryArtifact,
scanArtifactInventory,
materializeArtifactInventory,
} from "../../fixtures/artifactInventory.js";
import { InesArtifactReader } from "../../../src/artifacts/InesArtifactReader.js";
import { OwnedFileHandle } from "../../../src/filesystem/OwnedFileHandle.js";
import { ArtifactResourceScope } from "../../../src/artifacts/ArtifactResourceScope.js";
import {
extractArtifact,
materializeArtifactInventoryInScope as materializeInScope,
} from "../../../src/artifacts/extraction/ArtifactExtraction.js";
const sha256 = (bytes: Buffer) =>
createHash("sha256").update(bytes).digest("hex");
const fileReadPrototype = async (
path: string,
): Promise<Pick<FileHandle, "read">> => {
const handle = await open(path, "r");
try {
return Object.getPrototypeOf(handle) as Pick<FileHandle, "read">;
} finally {
await handle.close();
}
};
it("reuses scanned iNES region bytes for reconstruction without another full read", async () => {
const root = await createTestTempDirectory("rea-ines-read-budget-");
const path = join(root, "large-tail.nes");
const bytes = inesFixture({ trailing: Buffer.alloc(2 * 1024 * 1024, 0x5a) });
await writeFile(path, bytes);
const prototype = await fileReadPrototype(path);
const originalRead = prototype.read;
let bytesRead = 0;
const read = vi.spyOn(prototype, "read").mockImplementation(async function (
this: FileHandle,
...args
) {
const result = await originalRead.call(this, ...args);
bytesRead += result.bytesRead;
return result;
});
try {
const result = await inventoryArtifact(path);
expect(result.ines?.reconstruction).toMatchObject({
bytes: bytes.length,
sha256: sha256(bytes),
verified: true,
});
// Initial root hash, region scan, final freshness hash, plus bounded headers.
expect(bytesRead).toBeLessThanOrEqual(3 * bytes.length + 64 * 1024);
} finally {
read.mockRestore();
}
});
it("rejects region bytes that disagree with stable whole-file hashes", async () => {
const root = await createTestTempDirectory("rea-ines-region-digest-");
const path = join(root, "source.nes");
const bytes = inesFixture();
await writeFile(path, bytes);
const prototype = await fileReadPrototype(path);
const originalRead = prototype.read;
let changed = false;
const read = vi.spyOn(prototype, "read").mockImplementation(async function (
this: FileHandle,
...args
) {
const result = await originalRead.call(this, ...args);
const arguments_: readonly unknown[] = args;
if (!changed && arguments_[3] === 16n && result.bytesRead === 16_384) {
// Alter only the region observation, leaving both whole-file passes intact.
if (!Buffer.isBuffer(result.buffer))
throw new Error("Expected byte buffer");
result.buffer.writeUInt8(result.buffer.readUInt8(0) ^ 0xff, 0);
changed = true;
}
return result;
});
try {
await expect(inventoryArtifact(path)).rejects.toMatchObject({
reason: "integrity",
message: "iNES reconstruction differs from the source",
});
expect(changed).toBe(true);
expect((await readFile(path)).equals(bytes)).toBe(true);
} finally {
read.mockRestore();
}
});
it.each(["data", "hop", "HOP"])(
"routes iNES bytes as an archive with a .%s suffix",
async (suffix) => {
const root = await createTestTempDirectory("rea-ines-target-");
const path = join(root, `cartridge.${suffix}`);
await writeFile(path, inesFixture());
const target = await parseBinaryTarget(path);
if (!target.ok) throw target.error;
expect(target.value).toMatchObject({ kind: "archive", format: "ines" });
},
);
it("honors explicit database selection before parsing NES-signature bytes", async () => {
const root = await createTestTempDirectory("rea-ines-database-");
for (const bytes of [inesFixture(), Buffer.from("NES\x1a")]) {
const path = join(root, "selected.hop");
await writeFile(path, bytes);
expect(
await parseBinaryTarget(path, { targetKind: "database" }),
).toMatchObject({
ok: true,
value: { kind: "database", format: "analysis-database" },
});
}
});
it.each([true, false])(
"inventories and reconstructs all bytes with trainer=%s",
async (trainer) => {
const root = await createTestTempDirectory("rea-ines-inventory-");
const path = join(root, "fixture.nes");
const bytes = inesFixture({
trainer,
chrUnits: trainer ? 1 : 0,
trailing: Buffer.from("retained tail"),
});
await writeFile(path, bytes);
const inventory = await inventoryArtifact(path);
expect(inventory.ines?.header).toMatchObject({
header_hex: bytes.subarray(0, 16).toString("hex"),
mapper: 19,
trainer_present: trainer,
chr_rom_units_8k: trainer ? 1 : 0,
});
expect(inventory.ines?.reconstruction).toMatchObject({
bytes: bytes.length,
sha256: sha256(bytes),
verified: true,
});
const children = inventory.occurrences.filter(
({ logical_path }) => logical_path !== ".",
);
const ordered = [...children].sort(
(a, b) =>
(a.source_location?.offset ?? 0) - (b.source_location?.offset ?? 0),
);
const nodes = new Map(
inventory.nodes.map((node) => [node.artifact_id, node]),
);
const headerRegion = children.find(
({ logical_path }) => logical_path === "header.bin",
);
if (
headerRegion?.artifact_id === undefined ||
headerRegion.artifact_id === null
)
throw new Error("Missing header region");
expect(nodes.get(headerRegion.artifact_id)?.format).toBe("file");
let offset = 0;
for (const occurrence of ordered) {
const range = occurrence.source_location;
if (range === null || occurrence.artifact_id === null)
throw new Error("Missing region identity");
expect(range.offset).toBe(offset);
const original = bytes.subarray(
range.offset,
range.offset + range.length,
);
expect(nodes.get(occurrence.artifact_id)).toMatchObject({
sha256: sha256(original),
size: range.length,
});
offset += range.length;
}
expect(offset).toBe(bytes.length);
expect(inventory.ines?.reconstruction.region_order).toEqual(
ordered.map(({ logical_path }) => logical_path),
);
expect(await inventoryArtifact(path)).toEqual(inventory);
const scope = new ArtifactResourceScope();
const outputRoot = join(root, "extracted");
try {
const extraction = await extractArtifact({
inputPath: path,
inputFormat: "ines",
outputRoot,
environment: process.env,
integrityPolicy: "fail",
resourceScope: scope,
});
expect(extraction.artifacts).toHaveLength(children.length);
const reconstructed = Buffer.concat(
await Promise.all(
ordered.map(({ logical_path }) =>
readFile(join(outputRoot, logical_path)),
),
),
);
expect(reconstructed.equals(bytes)).toBe(true);
} finally {
await scope.close();
}
},
);
it.each([
{
name: "NES 2.0",
bytes: () => {
const bytes = inesFixture();
bytes[7] = 8;
return bytes;
},
reason: "nes-2.0",
},
{ name: "UNIF", bytes: () => Buffer.from("UNIF"), reason: "unif" },
{
name: "truncated",
bytes: () => inesFixture().subarray(0, 20),
reason: "truncated iNES payload",
},
])(
"refuses $name explicitly through both file entry points",
async ({ bytes, reason }) => {
const root = await createTestTempDirectory("rea-ines-refusal-");
const path = join(root, "invalid.nes");
await writeFile(path, bytes());
expect(await parseBinaryTarget(path)).toMatchObject({
ok: false,
error: { reason: expect.stringContaining(reason) },
});
await expect(inventoryArtifact(path)).rejects.toMatchObject({
reason: "format",
message: expect.stringContaining(reason),
});
},
);
it("does not select iNES from a .nes suffix", async () => {
const root = await createTestTempDirectory("rea-ines-suffix-");
const path = join(root, "not-a-rom.nes");
await writeFile(path, "ordinary bytes");
expect(await parseBinaryTarget(path)).toMatchObject({ ok: false });
});
it("rejects forged region ranges, cancellation and a changed retained source", async () => {
const root = await createTestTempDirectory("rea-ines-reader-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const handle = await open(path, "r");
const reader = new InesArtifactReader(path, {
handle,
initial: await lstat(path),
});
try {
const iterator = reader.entries()[Symbol.asyncIterator]();
const first = await iterator.next();
if (first.done) throw new Error("No header region");
await expect(
reader.open({ ...first.value, byteOffset: 16 }),
).rejects.toMatchObject({ reason: "integrity" });
await expect(
reader.open(first.value, AbortSignal.abort()),
).rejects.toMatchObject({ reason: "cancelled" });
await writeFile(
path,
Buffer.concat([inesFixture(), Buffer.from("changed")]),
);
await expect(reader.open(first.value)).rejects.toMatchObject({
reason: "integrity",
});
await iterator.return?.();
await reader.close();
expect((await handle.stat()).size).toBeGreaterThan(0);
await expect(reader.open(first.value)).rejects.toMatchObject({
reason: "unavailable",
});
} finally {
await reader.close();
await handle.close();
}
});
it.each(["before", "during"])(
"reports source removal %s a region read as integrity failure",
async (when) => {
const root = await createTestTempDirectory("rea-ines-removed-source-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const handle = await open(path, "r");
const reader = new InesArtifactReader(path, {
handle,
initial: await handle.stat(),
});
try {
const entries = [];
for await (const entry of reader.entries()) entries.push(entry);
const entry = entries[0];
if (entry === undefined) throw new Error("No header region");
if (when === "before") {
await unlink(path);
await expect(reader.open(entry)).rejects.toMatchObject({
reason: "integrity",
cause: { code: "ENOENT" },
});
} else {
const stream = await reader.open(entry);
await unlink(path);
await expect(
(async () => {
for await (const chunk of stream)
expect(Buffer.isBuffer(chunk)).toBe(true);
})(),
).rejects.toMatchObject({
reason: "integrity",
cause: { code: "ENOENT" },
});
}
} finally {
await reader.close();
await handle.close();
}
},
);
it.each(["truncated", "overwritten"])(
"reports a header %s during admission as an integrity failure",
async (change) => {
const root = await createTestTempDirectory("rea-ines-header-race-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const handle = await open(path, "r");
const reader = new InesArtifactReader(path, {
handle,
initial: await handle.stat(),
});
const originalRead = handle.read.bind(handle);
// Mutate the actual file after pre-read validation, at the read boundary.
const read = vi
.spyOn(handle, "read")
.mockImplementationOnce(async (...args) => {
const changed =
change === "truncated"
? Buffer.alloc(0)
: Buffer.alloc(inesFixture().length);
await writeFile(path, changed);
return originalRead(...args);
});
try {
await expect(
reader.entries()[Symbol.asyncIterator]().next(),
).rejects.toMatchObject({
reason: "integrity",
});
} finally {
read.mockRestore();
await reader.close();
await handle.close();
}
},
);
it.each(["inventory", "target"])(
"rejects changed iNES prefixes at the %s entry point",
async (route) => {
const root = await createTestTempDirectory("rea-ines-classification-race-");
const path = join(root, "source.nes");
for (const change of ["truncated", "counts", "signature"] as const) {
const bytes = inesFixture();
await writeFile(path, bytes);
const prototype = await fileReadPrototype(path);
const originalRead = prototype.read;
const read = vi
.spyOn(prototype, "read")
.mockImplementationOnce(async function (this: FileHandle, ...args) {
read.mockRestore();
const changed =
change === "truncated" ? bytes.subarray(0, 16) : Buffer.from(bytes);
if (change === "counts") changed[4] = 255;
if (change === "signature") changed.fill(0, 0, 4);
await writeFile(path, changed);
return originalRead.call(this, ...args);
});
try {
if (route === "inventory") {
await expect(inventoryArtifact(path)).rejects.toMatchObject({
reason: "integrity",
});
} else {
expect(await parseBinaryTarget(path)).toMatchObject({
ok: false,
error: { reason: expect.stringContaining("changed") },
});
}
} finally {
read.mockRestore();
}
}
},
);
it.each([1, 2])(
"reports source loss during inventory root hash pass %i as a changed input",
async (hashPass) => {
const root = await createTestTempDirectory("rea-ines-hash-loss-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const originalHash = rootHash.hashStableRootArtifactHandle;
let observedPasses = 0;
let faultInjected = false;
const hash = vi
.spyOn(rootHash, "hashStableRootArtifactHandle")
.mockImplementation(async (...args) => {
observedPasses += 1;
if (observedPasses !== hashPass) return originalHash(...args);
const handle = args[1];
const originalRead = handle.read.bind(handle);
const read = vi
.spyOn(handle, "read")
.mockImplementation(async (...readArgs) => {
const result = await originalRead(...readArgs);
if (result.bytesRead === 0) {
faultInjected = true;
await unlink(path);
}
return result;
});
try {
return await originalHash(...args);
} finally {
read.mockRestore();
}
});
try {
await expect(inventoryArtifact(path)).rejects.toMatchObject({
reason: "integrity",
cause: { code: "ENOENT" },
});
expect(observedPasses).toBe(hashPass);
expect(faultInjected).toBe(true);
} finally {
hash.mockRestore();
}
},
);
it("reports source loss during target hashing as a changed input", async () => {
const root = await createTestTempDirectory("rea-ines-target-hash-loss-");
const path = join(root, "source.nes");
const bytes = inesFixture();
await writeFile(path, bytes);
const prototype = await fileReadPrototype(path);
const originalRead = prototype.read;
let faultInjected = false;
const read = vi.spyOn(prototype, "read").mockImplementation(async function (
this: FileHandle,
...args
) {
const result = await originalRead.call(this, ...args);
const readArguments: readonly unknown[] = args;
const [buffer, , , position] = readArguments;
if (
Buffer.isBuffer(buffer) &&
buffer.byteLength === 64 * 1024 &&
position === bytes.length &&
result.bytesRead === 0
) {
faultInjected = true;
await unlink(path);
}
return result;
});
try {
expect(await parseBinaryTarget(path)).toMatchObject({
ok: false,
error: { reason: expect.stringContaining("changed") },
});
expect(faultInjected).toBe(true);
} finally {
read.mockRestore();
}
});
it.each(["removed", "directory", "malformed"])(
"rejects a %s source after inventory before extraction",
async (change) => {
const root = await createTestTempDirectory("rea-ines-extraction-change-");
const path = join(root, "source.nes");
const outputRoot = join(root, "output");
await writeFile(path, inesFixture());
const snapshot = await scanArtifactInventory(path);
await unlink(path);
if (change === "directory") await mkdir(path);
if (change === "malformed") await writeFile(path, "changed cartridge");
await expect(
materializeArtifactInventory(
{
inputPath: path,
inputFormat: "ines",
outputRoot,
environment: process.env,
integrityPolicy: "fail",
},
path,
snapshot,
),
).rejects.toMatchObject({ reason: "integrity" });
await expect(lstat(outputRoot)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.each([
"EISDIR",
"ENOTFILE",
"ESTALE",
"EACCES",
"cancelled",
...(process.platform === "win32" ? ["changed metadata"] : []),
])(
"preserves the %s admission meaning while reopening extraction",
async (code) => {
const root = await createTestTempDirectory("rea-ines-reopen-error-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const snapshot = await scanArtifactInventory(path);
const sample = await open(path, "r");
const prototype = Object.getPrototypeOf(sample) as Pick<
typeof sample,
"stat"
>;
await sample.close();
const originalStat = prototype.stat;
const controller = new AbortController();
const stat = vi
.spyOn(prototype, "stat")
.mockImplementationOnce(async function (this: FileHandle, ...args) {
if (code === "changed metadata") {
const metadata = await originalStat.call(this, ...args);
// Change the real file between descriptor stat and admission lstat.
await writeFile(path, "replacement cartridge");
return metadata;
}
if (code !== "cancelled")
throw Object.assign(new Error("injected admission failure"), {
code,
});
const result = await originalStat.call(this, ...args);
controller.abort();
return result;
});
try {
await expect(
materializeArtifactInventory(
{
inputPath: path,
inputFormat: "ines",
outputRoot: join(root, "output"),
environment: process.env,
integrityPolicy: "fail",
},
path,
snapshot,
controller.signal,
),
).rejects.toMatchObject(
code === "EACCES"
? { code }
: { reason: code === "cancelled" ? "cancelled" : "integrity" },
);
} finally {
stat.mockRestore();
}
},
);
it("retains a failed close after rejecting changed extraction bytes", async () => {
const root = await createTestTempDirectory("rea-ines-reopen-cleanup-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const snapshot = await scanArtifactInventory(path);
await writeFile(path, "changed source");
const scope = new ArtifactResourceScope();
const originalClose = OwnedFileHandle.prototype.close;
const owners: OwnedFileHandle[] = [];
const close = vi
.spyOn(OwnedFileHandle.prototype, "close")
.mockImplementation(async function (this: OwnedFileHandle) {
owners.push(this);
if (owners.length === 1) throw new Error("injected close failure");
await originalClose.call(this);
});
try {
await expect(
materializeInScope(
{
inputPath: path,
inputFormat: "ines",
outputRoot: join(root, "output"),
environment: process.env,
integrityPolicy: "fail",
resourceScope: scope,
},
path,
snapshot,
),
).rejects.toMatchObject({
reason: "integrity",
cleanup: { reason: expect.stringContaining("injected close failure") },
});
expect(owners).toHaveLength(1);
await scope.close();
expect(owners).toHaveLength(2);
expect(owners[1]).toBe(owners[0]);
} finally {
close.mockRestore();
await scope.close();
}
});
it("releases an extraction-owned source descriptor on repeated close", async () => {
const root = await createTestTempDirectory("rea-ines-owned-source-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const handle = await open(path, "r");
const owner = new OwnedFileHandle(handle);
const reader = new InesArtifactReader(
path,
{ handle, initial: await handle.stat() },
owner,
);
try {
await reader.close();
await expect(handle.stat()).rejects.toMatchObject({ code: "EBADF" });
await reader.close();
} finally {
await reader.close();
}
});
it.each(["target", "inventory", "reader"])(
"fills short iNES prefix reads at the %s boundary",
async (route) => {
const root = await createTestTempDirectory("rea-ines-short-read-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const sample = await open(path, "r");
const initial = await sample.stat();
const prototype = Object.getPrototypeOf(sample) as {
read(
buffer: Buffer,
offset: number,
length: number,
position: number | bigint,
): Promise<{ bytesRead: number; buffer: Buffer }>;
};
const originalRead = prototype.read;
const read = vi.spyOn(prototype, "read").mockImplementation(async function (
this: FileHandle,
...args
) {
const [buffer, offset, length, position] = args;
return originalRead.call(
this,
buffer,
offset,
typeof position === "number" && position < 16
? Math.min(length, 3)
: length,
position,
);
});
try {
if (route === "target") {
expect(await parseBinaryTarget(path)).toMatchObject({
ok: true,
value: { format: "ines" },
});
} else if (route === "inventory") {
expect(
(await inventoryArtifact(path)).ines?.reconstruction.verified,
).toBe(true);
} else {
const reader = new InesArtifactReader(path, {
handle: sample,
initial,
});
try {
const entries = [];
for await (const entry of reader.entries()) entries.push(entry.path);
expect(entries).toContain("prg/000.bin");
} finally {
await reader.close();
}
}
} finally {
read.mockRestore();
await sample.close();
}
},
);
it.each(["eof", "cancelled", "failed-and-cancelled"])(
"preserves %s while filling a short prefix",
async (mode) => {
const root = await createTestTempDirectory("rea-prefix-boundary-");
const path = join(root, "source");
const bytes = Buffer.from(
mode === "eof" ? "NES\x1a" : "NES\x1a123456789012",
);
await writeFile(path, bytes);
const handle = await open(path, "r");
const prototype = Object.getPrototypeOf(handle) as {
read(
buffer: Buffer,
offset: number,
length: number,
position: number,
): Promise<{ bytesRead: number; buffer: Buffer }>;
};
const original = prototype.read;
const controller = new AbortController();
let reads = 0;
const read = vi.spyOn(prototype, "read").mockImplementation(async function (
this: typeof handle,
buffer,
offset,
length,
position,
) {
reads++;
if (reads === 2 && mode === "failed-and-cancelled") {
controller.abort();
throw new Error("injected read failure");
}
const observed = await original.call(
this,
buffer,
offset,
Math.min(length, 3),
position,
);
if (reads === 2 && mode === "cancelled") controller.abort();
return observed;
});
try {
const result = readFileHandlePrefix(handle, 16, () =>
abortIfNeeded(controller.signal),
);
if (mode === "eof") {
expect((await result).equals(bytes)).toBe(true);
expect(reads).toBe(3);
} else {
await expect(result).rejects.toMatchObject({ reason: "cancelled" });
expect(reads).toBe(2);
}
} finally {
read.mockRestore();
await handle.close();
}
},
);
tests/boundary/filesystem/inesCancellation.test.ts
+320
View File
@@ -0,0 +1,320 @@
import { createHash } from "node:crypto";
import { open, writeFile, type FileHandle } from "node:fs/promises";
import { join } from "node:path";
import { expect, it, vi } from "vitest";
import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js";
import { InesArtifactReader } from "../../../src/artifacts/InesArtifactReader.js";
import { scanReader } from "../../../src/artifacts/inventory/scanReader.js";
import { hashStableRootArtifactHandle } from "../../../src/artifacts/inventory/hashStableRootArtifact.js";
import { inventoryArtifact } from "../../fixtures/artifactInventory.js";
import { inesFixture } from "../../fixtures/ines.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { RetainedRegularFileCleanupFailure } from "../../../src/application/RegularFileRead.js";
// Complete the real I/O, then cancel while its caller is still awaiting it.
const cancelAfterNextStat = (
handle: FileHandle,
controller: AbortController,
mutate?: () => Promise<void>,
) => {
const original = handle.stat.bind(handle);
const stat = vi
.spyOn(handle, "stat")
.mockImplementationOnce(async (...args) => {
await mutate?.();
const observed = await original(...args);
controller.abort();
return observed;
});
return () => stat.mockRestore();
};
const malformed = (format: string) => {
if (format === "UNIF") return Buffer.from("UNIF");
const bytes = inesFixture();
if (format === "NES 2.0") {
bytes[7] = 8;
return bytes;
}
return bytes.subarray(0, 20);
};
it.each(["default", "caller-error", "failed-close"])(
"preserves cancellation and descriptor ownership during %s admission",
async (reasonKind) => {
const root = await createTestTempDirectory("rea-ines-cancel-admission-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const sample = await open(path, "r");
const prototype = Object.getPrototypeOf(sample) as Pick<FileHandle, "stat">;
await sample.close();
const original = prototype.stat;
const controller = new AbortController();
let admitted: FileHandle | undefined;
const cleanupFailure = new Error("descriptor close failed");
const callerReason = new Error("caller stopped admission");
let restoreClose = () => {};
const stat = vi
.spyOn(prototype, "stat")
.mockImplementationOnce(async function (this: FileHandle, ...args) {
admitted = this;
const observed = await original.call(this, ...args);
if (reasonKind === "failed-close") {
const close = vi
.spyOn(this, "close")
.mockRejectedValueOnce(cleanupFailure);
restoreClose = () => close.mockRestore();
}
controller.abort(
reasonKind === "caller-error" ? callerReason : undefined,
);
return observed;
});
try {
const result = parseBinaryTarget(path, { signal: controller.signal });
if (reasonKind === "failed-close") {
const failure = await result;
if (failure.ok) throw new Error("Expected a retained cleanup failure");
expect(failure.error.cleanupIncomplete).toBe(true);
expect(failure.error.cause).toBeInstanceOf(
RetainedRegularFileCleanupFailure,
);
const retained = failure.error
.cause as RetainedRegularFileCleanupFailure;
expect(retained.cleanupCause).toBe(cleanupFailure);
expect(retained.outcome).toMatchObject({ kind: "failed" });
await retained.cleanupLease.release();
expect(failure.error.cleanupIncomplete).toBe(false);
} else {
const cancelled = await result;
expect(cancelled).toMatchObject({
ok: false,
error: {
_tag: "AnalysisCancelledError",
operation: "resolve_binary_target",
},
});
if (cancelled.ok) throw new Error("Expected cancellation");
if (reasonKind === "caller-error")
expect(cancelled.error.cause).toBe(callerReason);
}
if (admitted === undefined)
throw new Error("Admission descriptor was not observed");
await expect(admitted.stat()).rejects.toMatchObject({ code: "EBADF" });
} finally {
restoreClose();
stat.mockRestore();
}
},
);
it.each([
["truncated", "read"],
["NES 2.0", "read"],
["UNIF", "read"],
["truncated", "validation"],
["NES 2.0", "validation"],
["UNIF", "validation"],
])("honors cancellation during %s root prefix %s", async (format, phase) => {
const root = await createTestTempDirectory("rea-ines-cancel-prefix-");
const path = join(root, "source.nes");
await writeFile(path, malformed(format));
const sample = await open(path, "r");
const prototype = Object.getPrototypeOf(sample) as Pick<FileHandle, "read">;
await sample.close();
const original = prototype.read;
const controller = new AbortController();
let admitted: FileHandle | undefined;
let restoreStat = () => {};
const read = vi
.spyOn(prototype, "read")
.mockImplementationOnce(async function (this: FileHandle, ...args) {
admitted = this;
const observed = await original.call(this, ...args);
if (phase === "read") controller.abort();
else restoreStat = cancelAfterNextStat(this, controller);
return observed;
});
try {
await expect(
inventoryArtifact(path, { signal: controller.signal }),
).rejects.toMatchObject({ reason: "cancelled" });
expect(controller.signal.aborted).toBe(true);
if (admitted === undefined)
throw new Error("Prefix descriptor was not observed");
await expect(admitted.stat()).rejects.toMatchObject({ code: "EBADF" });
} finally {
restoreStat();
read.mockRestore();
}
});
it("honors cancellation during malformed reader header validation", async () => {
const root = await createTestTempDirectory("rea-ines-cancel-header-");
const path = join(root, "source.nes");
await writeFile(path, malformed("truncated"));
const handle = await open(path, "r");
const reader = new InesArtifactReader(path, {
handle,
initial: await handle.stat(),
});
const controller = new AbortController();
const original = handle.read.bind(handle);
let restoreStat = () => {};
const read = vi
.spyOn(handle, "read")
.mockImplementationOnce(async (...args) => {
const observed = await original(...args);
restoreStat = cancelAfterNextStat(handle, controller);
return observed;
});
try {
await expect(
reader.entries(controller.signal)[Symbol.asyncIterator]().next(),
).rejects.toMatchObject({ reason: "cancelled" });
expect(controller.signal.aborted).toBe(true);
} finally {
restoreStat();
read.mockRestore();
await reader.close();
await handle.close();
}
});
it.each(["stream", "reconstruction"])(
"does not complete %s after cancellation in the final region validation",
async (operation) => {
const root = await createTestTempDirectory("rea-ines-cancel-region-");
const path = join(root, "source.nes");
const bytes = inesFixture();
await writeFile(path, bytes);
const handle = await open(path, "r");
const reader = new InesArtifactReader(path, {
handle,
initial: await handle.stat(),
});
const controller = new AbortController();
let restoreStat = () => {};
const original = handle.read.bind(handle);
const read = vi
.spyOn(handle, "read")
.mockImplementation(async (...args) => {
const observed = await original(...args);
const arguments_: readonly unknown[] = args;
const position = arguments_[3];
if (
(typeof position === "number" || typeof position === "bigint") &&
observed.bytesRead > 0 &&
Number(position) + observed.bytesRead === bytes.length
)
restoreStat = cancelAfterNextStat(handle, controller);
return observed;
});
try {
const entries = [];
for await (const entry of reader.entries()) entries.push(entry);
const last = entries.at(-1);
if (last === undefined) throw new Error("Missing last iNES region");
const result =
operation === "reconstruction"
? (async () => {
const { inesRegionScan } = await scanReader(
reader,
[reader],
controller.signal,
);
if (inesRegionScan === undefined)
throw new Error("Missing region scan");
return reader.verifyReconstruction(
createHash("sha256").update(bytes).digest("hex"),
inesRegionScan,
controller.signal,
);
})()
: (async () => {
for await (const chunk of await reader.open(
last,
controller.signal,
))
expect(Buffer.isBuffer(chunk)).toBe(true);
})();
await expect(result).rejects.toMatchObject({ reason: "cancelled" });
expect(controller.signal.aborted).toBe(true);
} finally {
restoreStat();
read.mockRestore();
await reader.close();
await handle.close();
}
},
);
it("does not publish a root digest cancelled during its final state check", async () => {
const root = await createTestTempDirectory("rea-ines-cancel-hash-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const handle = await open(path, "r");
const initial = await handle.stat();
const controller = new AbortController();
const restore = cancelAfterNextStat(handle, controller);
try {
await expect(
hashStableRootArtifactHandle(path, handle, initial, controller.signal),
).rejects.toMatchObject({ reason: "cancelled" });
expect(controller.signal.aborted).toBe(true);
} finally {
restore();
await handle.close();
}
});
it.each([
["prefix", false],
["prefix", true],
["hash", false],
["hash", true],
] as const)(
"cancels target %s validation with concurrent change=%s",
async (phase, changed) => {
const root = await createTestTempDirectory("rea-ines-cancel-target-");
const path = join(root, "source.nes");
await writeFile(path, inesFixture());
const sample = await open(path, "r");
const prototype = Object.getPrototypeOf(sample) as Pick<FileHandle, "read">;
await sample.close();
const original = prototype.read;
const controller = new AbortController();
let admitted: FileHandle | undefined;
let restoreStat = () => {};
const read = vi.spyOn(prototype, "read").mockImplementation(async function (
this: FileHandle,
...args
) {
const observed = await original.call(this, ...args);
if (
admitted === undefined &&
(phase === "prefix" || observed.bytesRead === 0)
) {
admitted = this;
restoreStat = cancelAfterNextStat(
this,
controller,
changed ? () => writeFile(path, "changed cartridge") : undefined,
);
}
return observed;
});
try {
expect(
await parseBinaryTarget(path, { signal: controller.signal }),
).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } });
expect(controller.signal.aborted).toBe(true);
if (admitted === undefined)
throw new Error("Hash descriptor was not observed");
await expect(admitted.stat()).rejects.toMatchObject({ code: "EBADF" });
} finally {
restoreStat();
read.mockRestore();
}
},
);
tests/boundary/filesystem/stableFileHashOwnership.test.ts
+8 -6
View File
@@ -62,12 +62,14 @@ it("classifies a selected name removed at final validation as a changed file and
.spyOn(regularFile, "openRegularFile")
.mockImplementationOnce(async (...args) => {
descriptor = await nativeOpen(...args);
const nativeStat = descriptor.handle.stat.bind(descriptor.handle);
vi.spyOn(descriptor.handle, "stat").mockImplementationOnce(async () => {
const metadata = await nativeStat();
await unlink(path);
return metadata;
});
const nativeRead = descriptor.handle.read.bind(descriptor.handle);
vi.spyOn(descriptor.handle, "read").mockImplementation(
async (...args) => {
const result = await nativeRead(...args);
if (result.bytesRead === 0) await unlink(path);
return result;
},
);
return descriptor;
});
try {
tests/boundary/providers/native/artifactProvider.extractionFormat.test.ts
+100 -2
View File
@@ -1,7 +1,11 @@
import { access, readFile, symlink, writeFile } from "node:fs/promises";
import { createPackage } from "@electron/asar";
import { inesFixture } from "../../../fixtures/ines.js";
import { nativeAotPeFixture } from "../../../fixtures/nativeaotPe.js";
import { access, mkdir, readFile, symlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { buildBinary } from "plist";
import { elfExportImage } from "../../../fixtures/binaryDiagnostics/nativeExports.js";
import { writeOrderedZip } from "../../../fixtures/artifactEntryOrder.js";
@@ -22,8 +26,100 @@ import { artifactExtractionExecutionSchema } from "../../../../src/contracts/art
import { artifactExtractionResultSchema } from "../../../../src/domain/artifactGraph.js";
import type { BinaryTarget } from "../../../../src/domain/binaryTargetTypes.js";
it.each(["plist", "pe"] as const)(
"refuses iNES bytes after a non-extractable %s target is replaced",
async (format) => {
const root = await createTestTempDirectory("rea-extract-current-");
const path = join(root, "source.bin");
await writeFile(
path,
format === "plist" ? buildBinary({ value: "old" }) : nativeAotPeFixture(),
);
const target = await parseBinaryTarget(path);
if (!target.ok) throw target.error;
expect(target.value.format).toBe(format);
const client = createOwnedArtifactClient(target.value);
await writeFile(path, inesFixture());
const output = join(root, "output");
const result = await client.execute(
"extract_artifact",
artifactExtractionExecutionSchema.parse({ output_root: output }),
);
expect(result).toMatchObject({
ok: false,
error: {
_tag: "AnalysisUnsupportedTargetError",
operation: "extract_artifact",
reason: expect.stringContaining(
"Artifact format has no extraction reader:",
),
},
});
await expect(access(output)).rejects.toThrow();
},
);
describe("artifact extraction format support", () => {
it("refuses unsupported formats before inventory or output", async () => {
it.each(["zip", "asar", "hap", "hsp", "app"] as const)(
"refuses a replaced iNES target that scans as %s",
async (format) => {
const root = await createTestTempDirectory("rea-extract-replaced-");
const path = join(root, `source.${format}`);
await writeFile(path, inesFixture());
const target = await parseBinaryTarget(path);
if (!target.ok) throw target.error;
expect(target.value.format).toBe("ines");
const client = createOwnedArtifactClient(target.value);
if (format !== "asar") {
await writeOrderedZip(path, ["payload.txt"]);
} else {
const source = join(root, "source");
await mkdir(source);
await writeFile(join(source, "payload.txt"), "evidence\n");
await createPackage(source, path);
}
const output = join(root, "output");
const result = await client.execute(
"extract_artifact",
artifactExtractionExecutionSchema.parse({ output_root: output }),
);
expect(result).toMatchObject({
ok: false,
error: {
_tag: "ArtifactOperationError",
operation: "extract_artifact",
reason: "integrity",
},
});
await expect(access(output)).rejects.toThrow();
},
);
it("refuses a scanned non-container instead of selecting a stale reader", async () => {
const root = await createTestTempDirectory("rea-extract-replaced-file-");
const path = join(root, "source.bin");
await writeFile(path, inesFixture());
const target = await parseBinaryTarget(path);
if (!target.ok) throw target.error;
const client = createOwnedArtifactClient(target.value);
await writeFile(path, "ordinary replacement bytes");
const output = join(root, "output");
expect(
await client.execute(
"extract_artifact",
artifactExtractionExecutionSchema.parse({ output_root: output }),
),
).toMatchObject({
ok: false,
error: {
_tag: "ArtifactOperationError",
reason: "integrity",
},
});
await expect(access(output)).rejects.toThrow();
});
it("refuses a currently unsupported format before child inventory or output", async () => {
const root = await createTestTempDirectory("rea-extract-format-");
const image = join(root, "Image.dmg");
const udif = Buffer.alloc(512);
@@ -57,7 +153,9 @@ describe("artifact extraction format support", () => {
});
await expect(access(output)).rejects.toThrow();
});
});
describe("artifact extraction executable formats", () => {
it("refuses a thin Mach-O that inventory lists as root-only", async () => {
const root = await createTestTempDirectory("rea-extract-thin-macho-");
const binary = join(root, "thin");
tests/boundary/providers/native/artifactProvider.universalExtraction.test.ts
+6 -1
View File
@@ -7,9 +7,14 @@ import { expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { CPU_ARM64, universalMacho } from "../../../fixtures/universalMacho.js";
import { thinMach } from "../../../../src/domain/binaryTarget.fixture.js";
import { parseBinaryTarget } from "../../../../src/application/BinaryTargetResolver.js";
import { ArtifactReaderFailure } from "../../../../src/artifacts/ArtifactReader.js";
import {
CPU,
fatImage,
machoImage,
} from "../../../../src/artifacts/apple/MachoImage.fixture.js";
import { parseBinaryTarget } from "../../../../src/application/BinaryTargetResolver.js";
import { artifactExtractionExecutionSchema } from "../../../../src/contracts/artifactToolContracts.js";
const sliceReader = vi.hoisted(() => ({
tests/fixtures/ines.ts
Vendored
+25
View File
@@ -0,0 +1,25 @@
/** Source-owned iNES bytes; the payload is test data, not a game ROM. */
export const inesFixture = (
options: {
readonly trainer?: boolean;
readonly chrUnits?: number;
readonly trailing?: Buffer;
} = {},
): Buffer => {
const header = Buffer.alloc(16);
header.write("NES\x1a", "latin1");
header[4] = 2;
header[5] = options.chrUnits ?? 1;
header[6] = 0x31 | (options.trainer === true ? 4 : 0);
header[7] = 0x10;
return Buffer.concat([
header,
...(options.trainer === true ? [Buffer.alloc(512, 0x54)] : []),
Buffer.alloc(16_384, 0x41),
Buffer.alloc(16_384, 0x42),
...Array.from({ length: header.readUInt8(5) }, () =>
Buffer.alloc(8_192, 0x43),
),
options.trailing ?? Buffer.alloc(0),
]);
};