Hiloop reports a new state of the art on Karpathy’s autoresearch benchmark, reaching 0.9016 validation bits-per-byte on B200 hardware with unmodified stock coding agents. The result is compared with a published score of 0.9109. The article describes the 0.9016 result as a new state of the art.
Hiloop ships user-defined annotations as a primitive for telemetry data. Users register a JSON Schema, attach it to a point, range, or run, and query annotations with the same SQL as other telemetry events. Promoted fields become typed columns, filtering a 20-million-row benchmark in under 100 milliseconds.
Hiloop builds autoresearch infrastructure that separates research goals from harnesses, compute resources, and data models. Its interceptor wraps a user’s harness as a child process and captures telemetry, network calls, and output without assuming a particular framework. The system records raw activity for later reconstruction into views such as span trees, trajectories, or training datasets, while linking experiment lineage to both observability and compute state.
