Final Results — PROCEDURE Track¶
This page is the official final ranking of the PROCEDURE track, computed on the test phase.
The leaderboards of the pre-evaluation and test phases on this platform sort entries by mean accuracy. They are not the final ranking. Rank by mean accuracy is shown in the last column below for comparison.
PROCEDURE has two leaderboards: one over all questions and one over the clinically relevant questions only. Each is ranked independently with the same method.
How to read the leaderboard¶
- Rank comes from pairwise comparisons, not from mean accuracy. For every pair of entries and every bucket, a paired bootstrap over source videos (10,000 resamples) tests whether one entry is significantly more accurate (α = 0.05). Entry A beats entry B if A wins in more buckets than B does.
- Copeland score = number of entries beaten − number of entries lost to. Entries with the same score share a rank.
- Tie-break win rate is used only for ties within the top three places: the average share of video-bootstrap resamples in which an entry has the highest mean accuracy within its tie group.
- The two official baselines (in italics) are ranked like every other entry. A team is eligible for a prize if it ranks strictly above both baselines. Teams that do not are listed as "Anonymous".
- The seed of all bootstraps, 67, comes from random.org (integer generator, min 1, max 100, drawn 2026-09-23 10:45:31 UTC) and was fixed before this ranking was computed.
PROCEDURE (all questions)¶
15 teams were ranked together with the two official baselines; 13 of them rank above both baselines.
| Rank | Team | Copeland score | Opponents beaten / lost to | Tie-break win rate | Mean accuracy (%) | Rank by mean accuracy | Links |
|---|---|---|---|---|---|---|---|
| 1 | mlo-lab | +16 | 16 / 0 | 66.2 | 1 | Team page · GitHub | |
| 2 | CBH | +14 | 15 / 1 | 64.9 | 2 | Team page · GitHub | |
| 3 | FOKillerTeam | +12 | 14 / 2 | 64.9 | 3 | Team page · GitHub · Hugging Face | |
| 4 | Jmees | +9 | 12 / 3 | 57.7 | 4 | Team page · GitHub | |
| 4 | UTN-FunAI | +9 | 12 / 3 | 57.1 | 5 | Team page · GitHub | |
| 6 | Incision Impossible | +6 | 11 / 5 | 53.5 | 7 | Team page · GitHub | |
| 7 | HeyDonto Labs | +4 | 10 / 6 | 55.6 | 6 | Team page · Hugging Face | |
| 8 | SurgLedger | +2 | 9 / 7 | 52.1 | 8 | Team page · GitHub · Hugging Face | |
| 9 | BBMLL | +0 | 8 / 8 | 51.1 | 9 | Team page · GitHub · Hugging Face | |
| 10 | GenMI-Star | -3 | 6 / 9 | 48.9 | 10 | Team page · Hugging Face | |
| 11 | TUHH-mtec | -4 | 5 / 9 | 48.4 | 11 | Team page · GitHub | |
| 12 | Camma | -6 | 4 / 10 | 46.6 | 13 | Team page · GitHub | |
| 13 | ICVS-2Ai | -7 | 4 / 11 | 47.2 | 12 | Team page · Hugging Face | |
| — above: ranked higher than both baselines — | |||||||
| 14 | Official fine-tuned baseline | -10 | 3 / 13 | 37.7 | 14 | ||
| 15 | Anonymous | -13 | 1 / 14 | 37.3 | 15 | ||
| 16 | Anonymous | -14 | 1 / 15 | 36.9 | 16 | ||
| 17 | Official proprietary baseline | -15 | 0 / 15 | 36.4 | 17 |
Accuracy per bucket (%)¶
| Rank | Team | Object recognition (ID) | Object recognition (OOD) | Temporal grounding (ID) | Temporal grounding (OOD) | Aggregation (ID) | Aggregation (OOD) | Event understanding (ID) | Event understanding (OOD) | Complex reasoning (ID) | Complex reasoning (OOD) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | mlo-lab | 84.5 | 67.0 | 40.1 | 36.7 | 68.5 | 59.5 | 89.3 | 77.5 | 76.3 | 62.5 |
| 2 | CBH | 81.8 | 69.5 | 41.2 | 39.1 | 65.6 | 58.0 | 86.8 | 76.9 | 72.5 | 57.4 |
| 3 | FOKillerTeam | 86.3 | 66.6 | 41.2 | 32.7 | 67.4 | 55.2 | 87.6 | 74.0 | 77.3 | 60.6 |
| 4 | Jmees | 80.1 | 60.2 | 34.0 | 27.7 | 55.5 | 49.7 | 85.1 | 69.0 | 66.9 | 48.4 |
| 4 | UTN-FunAI | 77.5 | 55.7 | 33.1 | 27.1 | 62.7 | 52.9 | 82.8 | 66.5 | 62.2 | 50.3 |
| 6 | Incision Impossible | 72.5 | 59.5 | 37.1 | 31.8 | 57.3 | 53.2 | 59.3 | 62.7 | 54.4 | 47.3 |
| 7 | HeyDonto Labs | 78.2 | 57.3 | 29.8 | 20.7 | 47.5 | 50.5 | 83.9 | 65.4 | 71.7 | 50.8 |
| 8 | SurgLedger | 75.9 | 59.2 | 28.0 | 22.3 | 49.0 | 43.5 | 65.0 | 65.2 | 69.7 | 43.4 |
| 9 | BBMLL | 71.8 | 56.4 | 16.5 | 16.0 | 58.8 | 50.8 | 76.3 | 67.2 | 56.4 | 40.9 |
| 10 | GenMI-Star | 69.0 | 53.3 | 22.2 | 15.7 | 50.2 | 51.2 | 66.5 | 60.5 | 58.5 | 41.8 |
| 11 | TUHH-mtec | 76.0 | 57.8 | 34.3 | 29.9 | 48.8 | 46.2 | 45.9 | 58.5 | 58.5 | 28.4 |
| 12 | Camma | 58.2 | 52.5 | 12.5 | 8.0 | 47.9 | 51.7 | 64.6 | 59.7 | 66.1 | 44.7 |
| 13 | ICVS-2Ai | 72.4 | 60.0 | 32.9 | 27.6 | 42.6 | 36.4 | 59.5 | 62.9 | 50.0 | 27.6 |
| 14 | Official fine-tuned baseline | 59.7 | 43.1 | 22.0 | 19.0 | 38.8 | 38.9 | 40.5 | 61.7 | 39.2 | 14.3 |
| 15 | Anonymous | 48.3 | 45.1 | 11.0 | 12.9 | 47.6 | 50.6 | 28.9 | 35.6 | 53.1 | 39.7 |
| 16 | Anonymous | 67.9 | 49.2 | 14.4 | 9.2 | 35.1 | 41.2 | 48.8 | 62.6 | 19.7 | 20.6 |
| 17 | Official proprietary baseline | 52.5 | 47.2 | 16.5 | 18.2 | 36.3 | 34.9 | 47.6 | 63.1 | 24.4 | 23.4 |
Stability¶
Re-running the whole ranking with 5 other seeds (1, 2, 3, 4, 5) moves these entries; every other rank is unchanged. No team's prize eligibility changes under any of them. These re-runs are reported for transparency only and do not change the official result.
| Official rank | Team | Rank under another seed | Seeds that differ |
|---|---|---|---|
| 7 | HeyDonto Labs | 6 | 1 of 5 |
| 12 | Camma | 11 | 1 of 5 |
| 15 | Anonymous | 16 | 1 of 5 |
| 16 | Anonymous | 15 | 1 of 5 |
PROCEDURE (clinically relevant questions)¶
15 teams were ranked together with the two official baselines; 12 of them rank above both baselines.
| Rank | Team | Copeland score | Opponents beaten / lost to | Tie-break win rate | Mean accuracy (%) | Rank by mean accuracy | Links |
|---|---|---|---|---|---|---|---|
| 1 | mlo-lab | +15 | 15 / 0 | 0.526 | 68.1 | 2 | Team page · GitHub |
| 2 | CBH | +15 | 15 / 0 | 0.474 | 69.2 | 1 | Team page · GitHub |
| 3 | FOKillerTeam | +12 | 14 / 2 | 67.2 | 3 | Team page · GitHub · Hugging Face | |
| 4 | UTN-FunAI | +10 | 13 / 3 | 55.7 | 6 | Team page · GitHub | |
| 5 | Incision Impossible | +8 | 12 / 4 | 59.3 | 4 | Team page · GitHub | |
| 6 | Jmees | +6 | 11 / 5 | 58.4 | 5 | Team page · GitHub | |
| 7 | HeyDonto Labs | +4 | 10 / 6 | 49.1 | 11 | Team page · Hugging Face | |
| 8 | TUHH-mtec | +2 | 9 / 7 | 53.0 | 9 | Team page · GitHub | |
| 9 | ICVS-2Ai | -2 | 7 / 9 | 53.7 | 8 | Team page · Hugging Face | |
| 9 | BBMLL | -2 | 6 / 8 | 47.3 | 14 | Team page · GitHub · Hugging Face | |
| 11 | GenMI-Star | -4 | 5 / 9 | 54.2 | 7 | Team page · Hugging Face | |
| 12 | SurgLedger | -5 | 5 / 10 | 49.6 | 10 | Team page · GitHub · Hugging Face | |
| — above: ranked higher than both baselines — | |||||||
| 13 | Official proprietary baseline | -8 | 3 / 11 | 48.4 | 12 | ||
| 14 | Anonymous | -10 | 2 / 12 | 48.0 | 13 | ||
| 15 | Anonymous | -11 | 2 / 13 | 45.9 | 15 | ||
| 16 | Official fine-tuned baseline | -14 | 1 / 15 | 40.7 | 16 | ||
| 17 | Anonymous | -16 | 0 / 16 | 35.7 | 17 |
Accuracy per bucket (%)¶
| Rank | Team | Object recognition (ID) | Object recognition (OOD) | Temporal grounding (ID) | Temporal grounding (OOD) | Aggregation (ID) | Aggregation (OOD) | Event understanding (ID) | Event understanding (OOD) | Complex reasoning (ID) | Complex reasoning (OOD) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | mlo-lab | 82.2 | 69.8 | 53.0 | 39.6 | 75.5 | 69.6 | 89.5 | 85.3 | 50.0 | 66.7 |
| 2 | CBH | 82.8 | 81.1 | 49.0 | 49.7 | 74.8 | 64.2 | 81.1 | 84.6 | 75.0 | 50.0 |
| 3 | FOKillerTeam | 85.1 | 68.6 | 39.6 | 26.8 | 75.3 | 67.8 | 84.6 | 82.6 | 75.0 | 66.7 |
| 4 | UTN-FunAI | 67.8 | 61.5 | 32.2 | 20.1 | 71.2 | 65.0 | 79.0 | 76.4 | 50.0 | 33.3 |
| 5 | Incision Impossible | 73.0 | 65.1 | 50.3 | 32.9 | 63.8 | 65.5 | 66.4 | 72.0 | 37.5 | 66.7 |
| 6 | Jmees | 74.1 | 62.7 | 38.3 | 30.2 | 57.7 | 59.9 | 80.4 | 80.4 | 50.0 | 50.0 |
| 7 | HeyDonto Labs | 74.1 | 60.9 | 30.2 | 18.8 | 46.7 | 61.6 | 80.4 | 76.2 | 25.0 | 16.7 |
| 8 | TUHH-mtec | 70.7 | 59.8 | 39.6 | 32.2 | 62.0 | 59.3 | 60.1 | 63.4 | 50.0 | 33.3 |
| 9 | ICVS-2Ai | 74.7 | 68.0 | 40.9 | 28.2 | 42.4 | 36.9 | 45.5 | 71.1 | 62.5 | 66.7 |
| 9 | BBMLL | 62.1 | 60.9 | 6.7 | 10.7 | 64.8 | 58.1 | 65.0 | 78.2 | 50.0 | 16.7 |
| 11 | GenMI-Star | 51.7 | 52.1 | 22.1 | 16.1 | 51.8 | 57.5 | 72.7 | 67.8 | 50.0 | 100.0 |
| 12 | SurgLedger | 64.4 | 58.6 | 23.5 | 14.8 | 50.2 | 50.4 | 46.9 | 74.9 | 62.5 | 50.0 |
| 13 | Official proprietary baseline | 76.4 | 58.0 | 23.5 | 14.1 | 36.5 | 35.8 | 53.1 | 74.2 | 62.5 | 50.0 |
| 14 | Anonymous | 71.3 | 59.2 | 25.5 | 15.4 | 29.9 | 44.5 | 47.6 | 70.1 | 50.0 | 66.7 |
| 15 | Anonymous | 50.6 | 52.7 | 3.4 | 1.3 | 49.2 | 58.6 | 72.0 | 67.2 | 37.5 | 66.7 |
| 16 | Official fine-tuned baseline | 52.3 | 55.0 | 16.1 | 13.4 | 41.8 | 41.1 | 40.6 | 67.7 | 62.5 | 16.7 |
| 17 | Anonymous | 43.7 | 39.1 | 6.0 | 2.7 | 55.8 | 63.7 | 34.3 | 40.7 | 37.5 | 33.3 |
Stability¶
Re-running the whole ranking with 5 other seeds (1, 2, 3, 4, 5) moves these entries; every other rank is unchanged. No team's prize eligibility changes under any of them. These re-runs are reported for transparency only and do not change the official result.
| Official rank | Team | Rank under another seed | Seeds that differ |
|---|---|---|---|
| 5 | Incision Impossible | 4 | 5 of 5 |
| 15 | Anonymous | 14 | 3 of 5 |
Data and decisions¶
- 6 test questions were removed for every entry because their video clip in the test archive was damaged.
- Withdrawn submissions and submissions without a method description are not ranked.
- Links point to each team's page on this platform and to the repository the team gave for this track (with its test-phase submission, by email or in its method description), where it gave one.