MIB Memory Intelligence Benchmark · Reference Architecture

MIB Memory Intelligence Benchmark · Reference Architecture A architecture diagram generated by Archify. Hidden Eval Store · hidden.py · MIB_OFFICIAL_PACK · Architecture component · 30 hidden + 6 holdout Hidden Eval Store hidden.py · MIB_OFFICIAL_PACK 30 hidden + 6 holdout Evaluation Service · evaluation_service.py · Architecture component · Ed25519 signed jobs Evaluation Service evaluation_service.py Ed25519 signed jobs Leaderboard · leaderboard.py · Architecture component · paired comparison Leaderboard leaderboard.py paired comparison Scenario Registry · scenarios/ · pack manifest · Architecture component · 24 public templates Scenario Registry scenarios/ · pack manifest 24 public templates Benchmark Driver · runner.py · benchmark.py · Architecture component · timeline replay Benchmark Driver runner.py · benchmark.py timeline replay Agent Adapter · transports.py · stdio / HTTP · Architecture component · MIBAgent protocol Agent Adapter transports.py · stdio / HTTP MIBAgent protocol Agent + Memory System · participant-supplied · Submission sandbox sandbox.py · resource caps · network off · architecture-neutral Agent + Memory System participant-supplied architecture-neutral Report · Capability Card · report.py · capability.py · Architecture component · mib-report.schema Report · Capability Card report.py · capability.py mib-report.schema Ablation Controller · relevant / irrelevant / harmful · Architecture component · replay_excluding_events Ablation Controller relevant / irrelevant / harmful replay_excluding_events World Simulator · world.py · WorldState · Architecture component · virtual clock · hidden truth World Simulator world.py · WorldState virtual clock · hidden truth Evaluators · evaluator.py · Architecture component · answer · world state · trajectory Evaluators evaluator.py answer · world state · trajectory Scoring Engine · scoring.py · benchmark.py · Architecture component · hierarchical bootstrap Scoring Engine scoring.py · benchmark.py hierarchical bootstrap hidden templates signed job · sandboxed run load scenario pack observation · future probe observe / respond / act paired condition replay tool call · advance virtual time final world state answer · action trajectory dimension scores MIB Score · capability card signed result Submission sandbox sandbox.py · resource caps · network off Legend Frontend Backend Database Cloud External

Execution path

  • • Driver replays the timeline; future probes never leak into formation
  • • World Simulator owns hidden truth, the virtual clock and the seed
  • • The agent never writes world state — it observes tool-call outcomes

Causal evidence

  • • One scenario, paired conditions: relevant / irrelevant / harmful
  • • Paired deltas yield Memory Benefit, Stability, Harm Resistance
  • • Evaluators judge answer, final world state and action trajectory

Integrity and publication

  • • 60 templates: 24 public / 30 hidden / 6 holdout; the last two stay private
  • • Job manifests and result attestations are Ed25519 signed
  • • Track A and Track B are ranked separately, never one list