Submit

No Handshake platform language, like Boson

Do not write “Boson”, internal task IDs, anything about the review process, or GitHub handles in anything you create: your grading, ratings, comments, and messages to the agent. The automated checks scan all of it, including transcripts.

Before you submit

Tick these off before you run hs submit --push --pr.

0 of 24 complete

Runs

Arm A run with hs arm-a, then Arm B with hs arm-b

Pushed with hs submit --push --pr after each step, and each QC comment answered

Push 5 passed QC before submitting on HAI

Nothing sent before the agent's first reply, in either arm

Same steering in both arms; deviations and their impact explained

Distinct sessions, complete matching logs, and subagent logs retained

No unresolved API interruption, truncated run, model switch, or budget cutoff

Autonomous task: nothing sent after the opening prompt

Any retry, model-switch notice, or steering deviation explained in the comparison comments, with turn numbers

Anything the task needed is in envs/tasks/<prompt_id>.dockerfile, and it did not change after Arm A started

Grades, per arm

Code tasks built and tests run locally, not graded from the transcript alone

Six ratings with rationales that cite supporting turns or files

Every citation checked at its claimed location

Your own tests and their output are in verification/armA/ and verification/armB/ (raw output and README.md), not in the workspace

Behavioral issues tagged, or none

Comparison

Four preference scores

Three comments with evidence

Ratings, behavioral codes and severities, and comparison claims agree

Shared limitations and any tie against differing ratings explained

Integrity

Everything in your own words

Every citation is a trajectory turn, a patched file and line, or a command and its output; nothing points to run.json

No AI tools used for grades or comments; the assistant only for facts and tests

No project codename, internal task IDs, review-process details or GitHub handles in ratings, comments or messages to the agent

Only submission files, verification/armA/, verification/armB/, and changed task environment files, and no co-author line on your commits

Submit

hs submit --push --pr   # after the comparison is saved: validate, push, release your key

You have already pushed after each earlier step; this final push includes the comparison, writes each arm's changes.patch, validates the whole submission, and releases your key. The first push (after Arm A) opened the pull request on the task repo; if gh is not installed, hs printed a link to open it in your browser. Title is the prompt id.

If validation fails, read the message, fix the field in the form or the file it names, run hs validate, and run hs submit --push --pr again.

Only submission files belong in the PR: the grading files hs writes, changes.patch, and verification/armA/ and verification/armB/. The one exception is envs/tasks/<prompt_id>.dockerfile (and .md): if you changed them, hs submit includes them for you. Never commit keys, working copies, screenshots, or personal notes.

Submit on HAI

Once push 5 has passed QC (wait for its comment on the pull request), go back to your task on HAI, tick the acknowledgement, and submit the task. If the reviewer requests changes, fix them and push again with hs submit --push --pr.

What happens next

After submission, the automated checks cover static validation, separate arm reviews, consistency across grading files, and claim sourcing. Local validation does not replace this review. Read the reported findings on the pull request; the updated pipeline comments after each of the five steps. Automated fail/warn thresholds are awaiting confirmation. Reviewers assess run validity, steering parity, and the evidence behind each rating and claim. If they request changes, you resolve every comment: fix the entry in the grading form or the file they name, run hs submit --push --pr again. The task is done when the review is approved and merged.

Quality

Be specific and detailed in every rationale, issue, and comment. Point to evidence a reviewer can find: a command and its output, a file and line, a turn in the transcript. Fellows whose submissions are consistently high quality are promoted to reviewers.

Payment timing is on the When You Get Paid page.

← 5. Grade Both Arms