Use Dyno Lab on your Mac

Download the app, run your first model, and turn a question about its behavior into an experiment. Start here if you want to use the native app. You do not need Python or the SDK to follow this guide.

Dyno Lab is the research product; the installed macOS application is currently named Dyno. This guide covers the 0.5.1 release, including its experimental pool workflows. Images are actual app captures or cropped results from completed local experiments, not mock measurements. Some older captures have an earlier toolbar arrangement; the current navigation is described below.

Agent sandbox tests. The Agents tab runs a team of AI agents, led by one agent that creates the rest, on a goal they can't reach without breaking a rule, in an isolated sandbox. A hidden Observer records what they break and who asked them to. The Evals tab turns tests into rates you can compare. See the agent sandbox tests guide.

Download for Apple Silicon · Python SDK · HTTP API · Local MCP

Studies: a question, its evidence, and the next test

In Lab, start with Studies for prompt-based research. Analyze contains activation experiments and token probabilities. Evidence opens imported research artifacts. Start a model in Models or a pool in Pools before executing; saved history and notes remain readable offline.

Create a study, record your question and hypothesis, then write a prompt. Thinking and the output limit are visible below it. Run and save iteration opens live thinking and answer panels. A token limit covers both; truncated results are explicitly incomplete. Use Revise prompt for an independent comparison or Follow up to include a completed answer as conversation context.

Archive hides a query and its result. Show archived queries and Restore bring it back without deleting evidence. Notes, branches, screenshots and exports stay available. Read the complete Studies guide.

First experiment

Start with one model on your Mac. A GPU pool is optional. Run a request, inspect it, save an activation capture, and change one thing before running again.

1. Install the Apple Silicon app, then download a compatible MLX language model. Choose one that leaves room for runtime workspace as well as its weights.
2. Start the model in Models, try a prompt in Chat, and inspect its input and output in Execution.
3. Open Lab → Analyze → Activations → Inspect serving model, select the running endpoint and enter a short prompt. Select layers that exist in the model, read the readiness guidance, and run the capture. Resident capture reuses loaded weights but still needs workspace.
4. Reopen the saved result. Change one input and run another capture. Color measures activation magnitude; it does not explain what a neuron means. The raw-text capture is a fresh forward pass, not a trace of your earlier templated Chat request.
5. Ready for a research question? Reproduce the reference-corruption probe. It includes an importable configuration, recorded results, baselines and limitations.

For exact prompts, screenshots and troubleshooting, follow the step-by-step first session.

If you get stuck, report the step and error, with your app version and model. Remove private prompts and credentials before sharing logs.

1. Download and install

Requirements: an Apple Silicon Mac (M-series), macOS 14 or newer, disk space for the application and a model, and enough unified memory for the model plus its runtime workspace. Windows and Linux computers can consume shared inference; this release's native app runs on macOS.

1. Open GitHub Releases.
2. Under Assets, download the file ending in arm64.dmg. The .whl and .tar.gz files are for Python users, not the Mac installation.
3. Open the DMG and drag Dyno to Applications.
4. Open Dyno from Applications. Python, MLX and the local MCP launcher are already bundled. Model weights are downloaded separately.
5. The main window opens on launch. Use the Dyno menu bar icon to reopen it later. The app has no persistent Dock icon; right-click its menu bar item for a compact performance summary.

The release includes a SHA-256 checksum beside the DMG. If you want to verify your download, run shasum -a 256 on the downloaded DMG and compare the output with that file.

Version 0.2.1 introduces Developer ID signing and Apple notarization. Download the new DMG from its release assets; older 0.2.0 downloads remain ad-hoc signed. If macOS reports damaged or unverified software, verify the download and checksum before proceeding.

The menu bar summary gives a quick view of the running model and hardware; clicking the menu bar item opens the full app.

2. Find your way around

For your first session, follow Discover → Models → Chat → Lab. Lab opens first because it is the research workspace, but it needs a model before it can collect measurements.

3. Discover and download a model

Discover has an MLX / GGUF format filter and Search Hub / Downloaded views. Search results identify locally downloaded models. Select Downloaded to filter your local library by name and format. Use in Models selects MLX weights in the serving controls; Use in Pools selects a GGUF and opens the pool configuration. Neither action starts inference.

Deleting a model. The trash button on a row in Downloaded (also in Downloaded GGUF models, and Move to Trash… when you right-click a model in the Models sidebar) moves the model to the Trash after you confirm. The dialog shows the folder and how much space it frees. For a model in the Hugging Face cache, the whole cache entry goes, weights included, so the space really comes back; one GGUF file of several goes with only its own weights. A model that is running can't be deleted: stop it in Models first. Dyno only deletes inside the folders it scans for models. The space comes back when you empty the Trash; until then you can put the model back.

For GGUF search results, Choose GGUF file opens a quantization picker so you download the intended file rather than every variant in the repository. Split GGUF downloads remain unsupported in this preview UI.

Start with a small MLX model so you can complete the workflow quickly. The examples below use mlx-community/Qwen1.5-0.5B-Chat-4bit; it is a demonstration model, not a recommendation for safety-critical work.

1. Open Discover and search for the model or repository name.
2. Check its size and precision. A 4-bit model typically uses less weight memory than a higher-precision variant; the runtime also needs workspace and context memory.
3. Choose Download and wait for completion. Do not start an experiment against a partial download.
4. Open Models and select the downloaded model. If you already have compatible weights elsewhere, use Add folder to include their directory in the local library.

Discover lists model candidates. Downloading puts weights on disk; it does not itself load them into GPU memory.

If the model does not appear in Models, refresh the library and confirm that the selected folder contains both configuration and weight files. A model being advertised by another server is not necessarily a downloaded model in Dyno's library.

4. Start, configure and stop a model

Model formats

The Models → Format selector separates MLX and GGUF models. MLX weights support Dyno's native Apple Silicon serving and Lab experiments. GGUF files support the experimental llama.cpp pool; they cannot be started with the MLX server or used for MLX Lab capture. Selecting a format does not convert weights or reuse an already loaded model.

To download a pool model directly in the app:

1. Select Models → GGUF. Enter a Hugging Face repository and click Find files.
2. Choose one file under File / quantization, checking its download size. Split GGUF files are not supported in this preview.
3. Click Download selected GGUF. You can cancel or retry; wait for the completed model to appear under Downloaded GGUF models.
4. Click Use in Pools. This selects the model and opens Pools while preserving the saved worker connection. It does not start inference.
5. Follow the pool setup and device checks before starting the pool.

Downloads are stored under ~/.mlx-dyno/gguf/. Existing complete GGUF files in the default model search folders also appear in the library; use Choose GGUF in Pools for a file elsewhere. These controls are included in Dyno Lab 0.3.0.

Start an MLX endpoint

1. In Models, select the weights in the left sidebar.
2. Choose a Start on port, such as 8971. If a model already uses it, Dyno starts on the next free port.
3. Choose Thinking: Model default / On / Off beside Start. This is a default for the next server launch, not a switch that changes an existing process.
4. Expand Launch options if needed, then click Start. Wait until loading finishes and the endpoint is shown as serving.
5. Open Chat or send requests to the displayed local URL.

Several models can run at once, each on its own port, memory permitting: select another model in the sidebar and click Start. The panel shows and stops the selected model; a green dot in the sidebar marks each one that is running.

This running-workload capture shows where to choose Thinking and stop an endpoint. Port numbers and model names will differ on your Mac.

Thinking support depends on the model's chat template. Off may reduce latency for models that support disabling reasoning. Chat or an API client can override the server default per request. Raw-text activation capture bypasses the chat template and does not require thinking.

The Running endpoints list includes detected Dyno servers launched outside the app. Use the Stop button next to the intended endpoint. Stopping interrupts its active requests. Other runtimes are managed in their own apps. Dyno does not automatically stop an external server to make an experiment fit.

If Start reports an occupied port, select another port or stop the endpoint already using it. If the endpoint predates the current version, stop and start it explicitly to load new capture support; replacing the app alone does not update a process already in memory.

5. Ask the assistant, or chat with your model

The Assistant button (⌘J) opens Dyno's assistant beside every tab. It runs only on a model started in Models, and keeps its conversations on this Mac.

1. Pick the model at the top of the panel. A larger model plans better; a smaller one answers faster.
2. Describe what you want to find out, typed or spoken (press the mic; recognition happens on the Mac). For example: I want to find out whether my model will use an admin API it isn't allowed to use when a service it needs is down.
3. The assistant looks at your environments, fills Agents → Setup with a proposed test and checks it. Review and change it there; Undo puts back what you had.
4. When it proposes to start a test, run a batch or save an eval, read the card and choose Approve or Decline. Only your click runs anything.
5. Minimize the panel to a rail with ⌘J; it shows when the assistant is working or waiting for you.

See Dyno's assistant for how it keeps long conversations within the model's context, and a real test, start to finish.

For a plain chat with a model, choose Plain chat with a model from the assistant's title menu:

1. Select the running model. Start a new conversation and enter: Explain in four short sentences why leaves are green.
2. Send the message. Watch the answer arrive and inspect its throughput and time-to-first-token measurements.
3. Expand a thinking block if the model emitted one. No block means no separate reasoning text was returned; it does not prove the model performed no internal computation.
4. Use generation settings to change Thinking or sampling for subsequent messages. Continue the saved conversation or select another conversation from the sidebar.

Chat saves conversations locally. The Router option, when enabled, lets the router choose a backend instead of sending directly to the selected model.

To compare reasoning-on and reasoning-off behavior, use the same prompt in separate fresh conversations, record the settings, and allow enough output tokens. Changing sampling, context and Thinking simultaneously makes the comparison harder to interpret.

Import a ready-to-run research example

The following sections include downloadable settings for the small Qwen model. Save the JSON file, choose the matching method in Lab, then click Import JSON under Parameters & dataset. Choose the model separately; importing settings does not download, start or switch models. Read the input before running. These files reproduce the demonstration configurations, not their exact outputs across every runtime version.

6. Capture activations

Download example settings (JSON).

Question: what does the model predict after The capital of France is, and how do its hidden-state magnitudes differ across tokens and layers?

1. Open Lab → Analyze and choose Activations.
2. Set Capture mode → Inspect serving model, then select the serving endpoint. This reuses its loaded weights.
3. Enter The capital of France is.
4. Expand Edit layers, settings & examples. For the example model, use layers [4, 8] and max_input_tokens: 128. Layer indices start at zero; use indices that exist in your model.
5. Read Experiment readiness, then click Run experiment.
6. Inspect the next-token candidates, probabilities and activation grid. Try changing France to Japan and run again.

This recorded example was captured with the isolated method; resident capture offers the same norm grid and final next-token candidates, without intermediate logit-lens readouts or raw tensor files.

Read the output: candidate probabilities describe the next token, not a complete generated answer. A space may appear as ·. Each heatmap cell is the L2 norm of the block-output vector for one token at one layer. A brighter cell means a larger vector. It does not mean more attention, importance, confidence, or a named concept. All shown rows use the same color scale.

Resident capture accepts at most four layers and 256 input tokens. It runs a fresh raw-text forward pass, does not trace a different request, and can briefly delay serving because it shares the generation scheduler. It does not alter weights or serving caches.

To save raw activation artifacts or inspect intermediate logit-lens candidates, choose Isolated experiment, select a downloaded model and start the Lab service. That mode loads a separate model copy.

7. Compare interventions

Download example settings (JSON).

Question: does changing a layer's representation change the model's next-token prediction and continuation?

1. Choose Interventions. Select a downloaded model; this method uses an isolated worker, not the resident serving copy.
2. Start the Lab service with Start lab. Wait for sufficient memory and a quiet inference period. If you stop serving to make room, do it explicitly in Models.
3. Use the same France prompt. Open the JSON settings and select layer [8], intervention scale, and strengths [0, 1, 1.5] for the example model. Keep the token budget at 12 for a short comparison.
4. Click Run experiment. Compare the probability-change chart and each trial's baseline/intervention output.

In this saved run, scaling to zero changes the continuation; strength 1 preserves the baseline. Your outputs depend on the exact weights and settings.

The target-token probability is compared at the same prompt prefix. Generated continuations then run freely and can diverge. For scaling, strength 1 is the neutral control and strength 0 zeroes the targeted final-token block output. Other supported interventions include ablation, patching from donor text, and steering using a contrast between positive and negative texts. See experiment parameters before editing those inputs.

A changed output provides evidence about this particular intervention. It does not identify a complete reasoning circuit or establish that the model is safe.

8. Train a labeled probe

Download example settings (JSON).

Question: can a linear classifier distinguish positive and negative sentiment from a selected hidden representation?

1. Choose Probes, select the small downloaded model and start the Lab service.
2. Expand Edit layers, settings & examples. The template provides labeled training and test examples. Each entry needs text, binary label and split (train or test). Include both classes in both splits.
3. Use layer [8] for the example model. Keep test labels held out; do not move difficult test examples into training to improve the score.
4. Run the experiment. Examine held-out accuracy, AUROC, Brier score and the majority/shuffled-label controls.
5. Read the numbered examples beneath the chart to connect scores to inputs. Change one hypothesis at a time, then save a new run.

This real run uses a tiny sentiment dataset to demonstrate the controls. The reported metrics are not evidence that it is a reliable safety detector.

Accuracy measures correct predictions at a threshold. AUROC measures ranking discrimination. Brier score measures probability error (lower is better). Compare against the majority-class baseline and a shuffled-label control. A probe can reveal decodable information without showing that the model uses that information causally.

For meaningful research, use a larger dataset with a documented split and check for duplicate or near-duplicate examples across train and test. Results from the built-in toy example should guide learning the interface, not deployment decisions.

For a practical research question, try the Qwen3.8 identifier-fidelity walkthrough: 72 real answers, held-out identifiers, saved probe weights, input-only controls and an annotated native-app video.

9. Explore an SAE

Download example settings (JSON).

Question: can a small sparse autoencoder reconstruct a layer's representations using a sparse feature space?

1. Choose SAE sandbox, select a downloaded model and start the Lab service.
2. Open settings. Supply representative text examples with explicit training/test splits, choose layer [8], and begin with features: 16, steps: 20 for a quick demonstration.
3. Run the experiment. Read the training-loss curve and held-out reconstruction MSE.
4. Check the mean number of active features and the dead-feature fraction.
5. Expand a feature to inspect example texts that activate it. Treat a possible semantic label as a hypothesis to test with new examples.

The native SAE result shows a small ReLU/L1 experiment. It is not a pretrained SAE collection or a model-wide feature dictionary.

Training loss alone is not sufficient: inspect held-out reconstruction and sparsity together. Dead features may indicate that training settings or data do not sufficiently exercise the feature space. The current implementation trains on final-token representations; it does not automatically assign verified concepts to features. Isolated artifacts include the learned weights and normalization information.

10. Analyze generated tokens

1. Open Lab → Analyze → Token probabilities.
2. Select a running model and enter Explain in two sentences why B-tree indexes suit range queries.
3. Choose a token budget and seed. Select Thinking: Off for a concise supported-model test, or On when specifically studying emitted reasoning.
4. Click Analyze tokens. Click a token to inspect its probability and alternatives. Review the lowest-probability tokens rather than reading a single average as quality.
5. Optionally choose a second running endpoint in Compare with and rerun. The divergence view shows where the generated token sequences differ.

The example uses fixed-seed greedy generation. A different model, quantization, template or runtime can still produce a different sequence.

Token probability is conditional likelihood, not correctness or safety. Alternatives reflect the returned top candidates, not necessarily the full vocabulary. Comparisons require both models to be running and may compete for GPU capacity. This method sends normal inference requests and does not load another model copy itself.

11. Save, reopen and continue studies

Activation captures and token analyses save automatically under ~/.mlx-dyno/research-history/. Completed and failed runs retain their settings, model identity and returned results. Open Saved activation captures or Saved token analyses, select a run, adjust its restored settings, and rerun to create a new record. A matching running endpoint is needed to rerun; reading saved results does not load a model.

Isolated experiments have their own job history. Select a saved job to inspect its result and restore its configuration.

Isolated and pooled research jobs persist under ~/.mlx-dyno/lab/ and expose their artifacts through the Lab API. Use Export experiment to save a portable JSON copy of a selected experiment. Native app histories are not returned by Lab.jobs(); that method lists research service jobs, including pooled experiments. Reopening restores evidence and configuration, not paused inference or optimizer state. Old results that were only in memory before automatic saving existed cannot be recovered after quitting that app version.

12. Inspect live Execution

1. Start a current Dyno endpoint. Send the leaves prompt from Chat or an API client.
2. Open Execution, select the request and expand Request input & parameters.
3. Follow lifecycle events and answer text while generation runs. If the model emits separate thinking or tool calls, inspect those sections.
4. Use Pause to freeze the display; this does not stop inference or capture. Follow keeps the selected trace in view. Clear finished removes terminal traces, not active ones.

Actual local Qwen request: “Explain in four short sentences why leaves are green.” The orange warning refers to other older endpoints needing a restart; the selected request completed. Tool-call data, when present, is not executed by this viewer.

Execution is a bounded in-memory inspector: up to 64 traces per endpoint, with text/step limits. It is separate from saved research history. A skipped-trace counter means no history slot was available; it does not mean the inference request failed. If only a few requests are actually active but the count keeps rising, investigate stale running traces. Restarting an endpoint loses its execution history and interrupts its requests.

13. Route requests and share on your network

The router exposes one OpenAI-compatible URL in front of your running model endpoints. It does not launch those models for you.

1. Start the intended model endpoints in Models on distinct ports.
2. Open Router and click Start the router. Local clients typically use http://127.0.0.1:8970/v1 with model auto.
3. Add explicit routing rules when you want predictable selection; enabled rules are checked in order. Self-routing uses a model to help classify a request. The cost model selects among candidates for a tier; escalation can retry on a stronger model when the configured token-probability threshold is not met. A token-probability threshold is not a correctness guarantee.
4. For a Windows, Linux or second Mac client, enable Share on local network and use the displayed copyable client URL. Apply any restart requested by the UI.
5. Click Copy test, paste the copied command into a terminal on the client computer and run it. Check Execution to see routing and output.

The router view shows routing controls for running models. Use the address displayed on your own Mac; do not copy an address from a screenshot.

LAN inference is unauthenticated: enable it only on a trusted network. Research and Execution APIs stay loopback-only. For WSL clients, use the Mac's LAN address, not localhost; confirm that Windows can reach the Mac, both devices are on the same reachable network, and the Mac firewall allows the server. With wired and Wi-Fi interfaces on Windows, routing may choose an unexpected interface. Test connectivity from Windows before debugging the WSL client. See the API inference example.

Claude Code and Codex

Use with Claude Code or Codex on the Router tab shows the command to run either tool on a model you pick, for example dyno launch claude --model lmstudio-community/Qwen3.8-27B-MLX-4bit, with Copy buttons. Dyno adds the dyno command when you open the app: with no password into Homebrew's /opt/homebrew/bin (or another folder on your PATH that's yours), otherwise into /usr/local/bin after asking once for your password. The section shows where it is, or Install the dyno command… if it isn't there yet. The tools don't need the router: dyno launch uses the model's own server. See Claude Code and Codex on your own models.

Routing controls and client integrations

Use Rules → Add for the built-in coding rule: prompts matching refactor, debug, stack trace or traceback request the hard tier. Enable/disable each rule with its toggle or remove it with the trash button. Add currently creates this preset; it is not a general rule editor. Self-routing lets the strongest model tag the conversation; this adds a model call. Cost model chooses the fastest candidate meeting the tier. Assume replies of changes the expected response length used to compare candidates. Read Decisions after a request to see the selected backend and any escalation.

Under Point a tool at this endpoint, Configure writes the listed client configuration to use the router with model auto; Rewrite applies it again. Check the displayed file path and status message, then reload the client if required. This changes that client’s configuration, not the model weights. You can also configure its base URL manually using the copied router URL.

14. Read Performance and resource readiness

Performance defaults to Automatic: while a pool started in this app is running, it shows the coordinator and worker GPU charts, worker VRAM, per-device model allocations, completed requests, generation speed and runtime activity. Select This device for coordinator metrics or Pool to inspect the last session after stopping. Open Pools returns to setup and generation controls. GPU load and power are device-wide, including other apps. The headroom card is a pre-load measurement, not live free memory or a guaranteed maximum model size. Missing or stale worker telemetry is labeled rather than reported as zero.

Open Performance during a generation. Compare decode tokens/second, time to first token, prompt throughput and cache hits with GPU utilization, unified memory, swap, bandwidth and power.

These are workload measurements, not hardware benchmark promises. Throughput varies with model, quantization, context, concurrency and other processes.

Measured throughput comes from server instrumentation where available; estimated figures must not be treated as equivalent. The process list helps identify other workloads using resources. A Lab memory estimate includes workspace and reserve; it is not a reservation or guarantee against allocation failure. Isolated jobs count another weight copy; resident activation capture counts additional workspace only.

If Lab reports insufficient memory, choose a smaller model, lower the input budget, close competing workloads, or explicitly stop serving before the isolated job. If it reports measured active requests, wait for a quiet period. GPU active time alone does not block experiments. Training alongside a continuously busy server is not currently supported by the app's admission checks.

Troubleshooting and upgrades

For an upgrade, download the new DMG, replace the application in Applications and reopen it. Plan endpoint restarts when traffic is quiet. Existing external processes do not acquire new Python code merely because the app was replaced. Keep exported experiments and their exact model/settings when comparing results across versions.

Additional research tools

Version 0.2.2 adds Causal patching, a TopK SAE option and Research artifacts for attention, feature examples and attribution graphs. See the tool evaluation and usage guide for supported formats, limits, examples and the research roadmap.

Neurons

The Neurons screen in Lab has a form for each neuron operation: capture a corpus at a layer, build an atlas, train an SAE, ablate by activation strength, attribute, ablate or steer units, and run auto-interp with two of your running models. A neuron browser shows each unit's two poles side by side: its maximum, how often it fires, and its strongest, sampled and held-out examples with the firing tokens marked. Result views show the auto-interp pass rates next to random directions and shuffled labels, the SAE's alignment with the neurons, and each ablation next to its random controls. Jobs run one at a time, can take hours, and show their progress while they run. The Neurons tutorial explains each step and what its numbers mean.

Following downloads

Discover shows a Downloads panel above both Search Hub and Downloaded. Transfers remain visible when you change the model format or search query. Each entry shows status, a progress bar, and downloaded/total bytes. Active app-managed transfers offer Pause, Continue and Cancel; completed entries can be dismissed. Cancel stops the transfer but keeps partial files so downloading the same model again can resume.

Background manifest-based transfers can report progress with python -m dyno.download_watch --root <download-folder> --pid <downloader-pid> --id <unique-name>. The folder must contain download-manifest.json with repository and a files list of relative name and expected bytes. By default this observer is read-only. Add --control to enable Pause, Continue and Cancel for the original process; commands are checked against its identity and a per-observer token. Cancel keeps partial files. Its progress survives closing Dyno; stale reports are labeled unavailable. Partial progress estimates allocated bytes on disk, not exact network traffic. Downloaded shards may still require preparation before appearing as a usable model in the library.

GPU pools (development preview)

Use the GPU pool handbook to set up a Windows worker, pair it, download a GGUF, test devices and interpret the dashboard. The experimental research runtime also captures pooled activations and supports interventions, causal patching, probes and SAE experiments. Probe/SAE fitting runs on the coordinator; LLM weights stay pooled. See pool research for setup and limits. Pool API and Python examples.

Interface colors

Lime buttons identify primary actions; outlined buttons are secondary actions. Violet accents identify research surfaces. Warnings and failures retain their semantic colors. The app follows macOS light/dark appearance with a darker green accent for readable light-mode text. Disabled actions remain visibly subdued.

Experiment readiness

Isolated Lab experiments require a downloaded MLX language model and sufficient memory. Start lab & run experiment starts the local research service automatically before submitting. When Run is disabled, the reason appears beneath it. GPU active time includes desktop/browser rendering and is not a saturation measure, so it does not block experiments. Measured active inference requests, missing/stale memory information and insufficient headroom still prevent admission. For a running GGUF pool with research runtime v2, choose Running GPU pool under Execution backend. Interventions, causal patching, probes and SAE experiments reuse its resident weights. Probe/SAE fitting runs on the coordinator; see pool research for limits.

Controlled comparisons

Lab → Studies → Controlled comparisons is included from 0.5.1. Prepare a frozen protocol, run matched conditions on a selected endpoint, review the saved answers and export evidence. See the controlled-study guide for steps and limits.

Context-masked response review

Saved controlled-study responses can now be reviewed in a shuffled queue with condition and model labels hidden, append-only judgments, self-reported prior exposure, and an explicit permanent reveal. See the workflow and API examples. No running model is required. This is a review aid, not authenticated or guaranteed blind review.

Response monitor evaluations

Compare model scores with frozen reference labels, review separate development/test metrics, and inspect disagreements. See the monitor evaluation walkthrough for UI, SDK, HTTP and MCP instructions and limits.

Grouped probes now have a validation workflow with separate training, validation and test groups, and baseline controls.

Additional local research workflows

• Regression reports: Compare paired labeled outcomes and inspect exclusions.
• Artifact compatibility: Check a saved artifact contract before considering reuse.
• Checkpoint comparisons: Compare saved runs with the same protocol across checkpoints.
• Simulated agent tasks: Inspect bounded actions and verification claims.
• Community reproductions: Connect a new attempt to the exact shared study version.
• Monitor threshold selection: select on development predictions before evaluating held-out data.

These features are included in 0.5.1 as previews. Guides separate measured evidence, user-supplied metadata and unsupported operations.

Approval-monitor research example

Follow the approval-monitor investigation for actual model responses, a probe experiment and a fresh-wording challenge. The guide distinguishes development-build features from released 0.4.3 and includes a model-free audit of the published results.