Final Results — PROCEDURE Track

This page is the official final ranking of the PROCEDURE track, computed on the test phase.

The leaderboards of the pre-evaluation and test phases on this platform sort entries by mean accuracy. They are not the final ranking. Rank by mean accuracy is shown in the last column below for comparison.

PROCEDURE has two leaderboards: one over all questions and one over the clinically relevant questions only. Each is ranked independently with the same method.

How to read the leaderboard

  • Rank comes from pairwise comparisons, not from mean accuracy. For every pair of entries and every bucket, a paired bootstrap over source videos (10,000 resamples) tests whether one entry is significantly more accurate (α = 0.05). Entry A beats entry B if A wins in more buckets than B does.
  • Copeland score = number of entries beaten − number of entries lost to. Entries with the same score share a rank.
  • Tie-break win rate is used only for ties within the top three places: the average share of video-bootstrap resamples in which an entry has the highest mean accuracy within its tie group.
  • The two official baselines (in italics) are ranked like every other entry. A team is eligible for a prize if it ranks strictly above both baselines. Teams that do not are listed as "Anonymous".
  • The seed of all bootstraps, 67, comes from random.org (integer generator, min 1, max 100, drawn 2026-09-23 10:45:31 UTC) and was fixed before this ranking was computed.

PROCEDURE (all questions)

15 teams were ranked together with the two official baselines; 13 of them rank above both baselines.

Rank Team Copeland score Opponents beaten / lost to Tie-break win rate Mean accuracy (%) Rank by mean accuracy Links
1 mlo-lab +16 16 / 0 66.2 1 Team page · GitHub
2 CBH +14 15 / 1 64.9 2 Team page · GitHub
3 FOKillerTeam +12 14 / 2 64.9 3 Team page · GitHub · Hugging Face
4 Jmees +9 12 / 3 57.7 4 Team page · GitHub
4 UTN-FunAI +9 12 / 3 57.1 5 Team page · GitHub
6 Incision Impossible +6 11 / 5 53.5 7 Team page · GitHub
7 HeyDonto Labs +4 10 / 6 55.6 6 Team page · Hugging Face
8 SurgLedger +2 9 / 7 52.1 8 Team page · GitHub · Hugging Face
9 BBMLL +0 8 / 8 51.1 9 Team page · GitHub · Hugging Face
10 GenMI-Star -3 6 / 9 48.9 10 Team page · Hugging Face
11 TUHH-mtec -4 5 / 9 48.4 11 Team page · GitHub
12 Camma -6 4 / 10 46.6 13 Team page · GitHub
13 ICVS-2Ai -7 4 / 11 47.2 12 Team page · Hugging Face
— above: ranked higher than both baselines —
14 Official fine-tuned baseline -10 3 / 13 37.7 14
15 Anonymous -13 1 / 14 37.3 15
16 Anonymous -14 1 / 15 36.9 16
17 Official proprietary baseline -15 0 / 15 36.4 17

Accuracy per bucket (%)

Rank Team Object recognition (ID) Object recognition (OOD) Temporal grounding (ID) Temporal grounding (OOD) Aggregation (ID) Aggregation (OOD) Event understanding (ID) Event understanding (OOD) Complex reasoning (ID) Complex reasoning (OOD)
1 mlo-lab 84.5 67.0 40.1 36.7 68.5 59.5 89.3 77.5 76.3 62.5
2 CBH 81.8 69.5 41.2 39.1 65.6 58.0 86.8 76.9 72.5 57.4
3 FOKillerTeam 86.3 66.6 41.2 32.7 67.4 55.2 87.6 74.0 77.3 60.6
4 Jmees 80.1 60.2 34.0 27.7 55.5 49.7 85.1 69.0 66.9 48.4
4 UTN-FunAI 77.5 55.7 33.1 27.1 62.7 52.9 82.8 66.5 62.2 50.3
6 Incision Impossible 72.5 59.5 37.1 31.8 57.3 53.2 59.3 62.7 54.4 47.3
7 HeyDonto Labs 78.2 57.3 29.8 20.7 47.5 50.5 83.9 65.4 71.7 50.8
8 SurgLedger 75.9 59.2 28.0 22.3 49.0 43.5 65.0 65.2 69.7 43.4
9 BBMLL 71.8 56.4 16.5 16.0 58.8 50.8 76.3 67.2 56.4 40.9
10 GenMI-Star 69.0 53.3 22.2 15.7 50.2 51.2 66.5 60.5 58.5 41.8
11 TUHH-mtec 76.0 57.8 34.3 29.9 48.8 46.2 45.9 58.5 58.5 28.4
12 Camma 58.2 52.5 12.5 8.0 47.9 51.7 64.6 59.7 66.1 44.7
13 ICVS-2Ai 72.4 60.0 32.9 27.6 42.6 36.4 59.5 62.9 50.0 27.6
14 Official fine-tuned baseline 59.7 43.1 22.0 19.0 38.8 38.9 40.5 61.7 39.2 14.3
15 Anonymous 48.3 45.1 11.0 12.9 47.6 50.6 28.9 35.6 53.1 39.7
16 Anonymous 67.9 49.2 14.4 9.2 35.1 41.2 48.8 62.6 19.7 20.6
17 Official proprietary baseline 52.5 47.2 16.5 18.2 36.3 34.9 47.6 63.1 24.4 23.4

Stability

Re-running the whole ranking with 5 other seeds (1, 2, 3, 4, 5) moves these entries; every other rank is unchanged. No team's prize eligibility changes under any of them. These re-runs are reported for transparency only and do not change the official result.

Official rank Team Rank under another seed Seeds that differ
7 HeyDonto Labs 6 1 of 5
12 Camma 11 1 of 5
15 Anonymous 16 1 of 5
16 Anonymous 15 1 of 5

PROCEDURE (clinically relevant questions)

15 teams were ranked together with the two official baselines; 12 of them rank above both baselines.

Rank Team Copeland score Opponents beaten / lost to Tie-break win rate Mean accuracy (%) Rank by mean accuracy Links
1 mlo-lab +15 15 / 0 0.526 68.1 2 Team page · GitHub
2 CBH +15 15 / 0 0.474 69.2 1 Team page · GitHub
3 FOKillerTeam +12 14 / 2 67.2 3 Team page · GitHub · Hugging Face
4 UTN-FunAI +10 13 / 3 55.7 6 Team page · GitHub
5 Incision Impossible +8 12 / 4 59.3 4 Team page · GitHub
6 Jmees +6 11 / 5 58.4 5 Team page · GitHub
7 HeyDonto Labs +4 10 / 6 49.1 11 Team page · Hugging Face
8 TUHH-mtec +2 9 / 7 53.0 9 Team page · GitHub
9 ICVS-2Ai -2 7 / 9 53.7 8 Team page · Hugging Face
9 BBMLL -2 6 / 8 47.3 14 Team page · GitHub · Hugging Face
11 GenMI-Star -4 5 / 9 54.2 7 Team page · Hugging Face
12 SurgLedger -5 5 / 10 49.6 10 Team page · GitHub · Hugging Face
— above: ranked higher than both baselines —
13 Official proprietary baseline -8 3 / 11 48.4 12
14 Anonymous -10 2 / 12 48.0 13
15 Anonymous -11 2 / 13 45.9 15
16 Official fine-tuned baseline -14 1 / 15 40.7 16
17 Anonymous -16 0 / 16 35.7 17

Accuracy per bucket (%)

Rank Team Object recognition (ID) Object recognition (OOD) Temporal grounding (ID) Temporal grounding (OOD) Aggregation (ID) Aggregation (OOD) Event understanding (ID) Event understanding (OOD) Complex reasoning (ID) Complex reasoning (OOD)
1 mlo-lab 82.2 69.8 53.0 39.6 75.5 69.6 89.5 85.3 50.0 66.7
2 CBH 82.8 81.1 49.0 49.7 74.8 64.2 81.1 84.6 75.0 50.0
3 FOKillerTeam 85.1 68.6 39.6 26.8 75.3 67.8 84.6 82.6 75.0 66.7
4 UTN-FunAI 67.8 61.5 32.2 20.1 71.2 65.0 79.0 76.4 50.0 33.3
5 Incision Impossible 73.0 65.1 50.3 32.9 63.8 65.5 66.4 72.0 37.5 66.7
6 Jmees 74.1 62.7 38.3 30.2 57.7 59.9 80.4 80.4 50.0 50.0
7 HeyDonto Labs 74.1 60.9 30.2 18.8 46.7 61.6 80.4 76.2 25.0 16.7
8 TUHH-mtec 70.7 59.8 39.6 32.2 62.0 59.3 60.1 63.4 50.0 33.3
9 ICVS-2Ai 74.7 68.0 40.9 28.2 42.4 36.9 45.5 71.1 62.5 66.7
9 BBMLL 62.1 60.9 6.7 10.7 64.8 58.1 65.0 78.2 50.0 16.7
11 GenMI-Star 51.7 52.1 22.1 16.1 51.8 57.5 72.7 67.8 50.0 100.0
12 SurgLedger 64.4 58.6 23.5 14.8 50.2 50.4 46.9 74.9 62.5 50.0
13 Official proprietary baseline 76.4 58.0 23.5 14.1 36.5 35.8 53.1 74.2 62.5 50.0
14 Anonymous 71.3 59.2 25.5 15.4 29.9 44.5 47.6 70.1 50.0 66.7
15 Anonymous 50.6 52.7 3.4 1.3 49.2 58.6 72.0 67.2 37.5 66.7
16 Official fine-tuned baseline 52.3 55.0 16.1 13.4 41.8 41.1 40.6 67.7 62.5 16.7
17 Anonymous 43.7 39.1 6.0 2.7 55.8 63.7 34.3 40.7 37.5 33.3

Stability

Re-running the whole ranking with 5 other seeds (1, 2, 3, 4, 5) moves these entries; every other rank is unchanged. No team's prize eligibility changes under any of them. These re-runs are reported for transparency only and do not change the official result.

Official rank Team Rank under another seed Seeds that differ
5 Incision Impossible 4 5 of 5
15 Anonymous 14 3 of 5

Data and decisions

  • 6 test questions were removed for every entry because their video clip in the test archive was damaged.
  • Withdrawn submissions and submissions without a method description are not ranked.
  • Links point to each team's page on this platform and to the repository the team gave for this track (with its test-phase submission, by email or in its method description), where it gave one.