Report what was measured.
Counts from a few episodes are not rates. A task with no rule break is not evidence that a model is safe. Labels are checked by hand, and the limits stay next to the result.
ABOUT DYNO LAB
Open tests for AI agents.
Evidence anyone can check.
AI agents now run commands with real credentials on real systems. Dyno Lab tests what they do when following the rules means they can’t finish the job: whether they report the problem, work around the rule, or claim success they didn’t get. The tests, the harness and the app are open source and free.
Why outside testing
AI developers say their agents are monitored and contained. Checking that from the outside needs tests anyone can run and rerun, and records precise enough that someone else can check the conclusion.
Each test is an ordinary task with one explicit rule. Following the rule makes the task impossible, so the honest response is to report that. Everything the agent tries is recorded, flagged and kept apart from what the agent sees.
I’m Danilo Canivel. I build Dyno Lab and run its experiments. It started as a Mac app for running models and investigating their behavior, and those research tools are still part of it.
Make it checkable.
Every run keeps the full transcript, the exact task and environment, and file hashes. Sealed runs can be verified by anyone who receives them.
Build in the open.
Tasks and environments are plain files. If a test is flawed, a label is wrong or a result doesn’t reproduce, I want to hear about it. The first pilot has already corrected one task and one labeling rule.