Skip to content

Leaderboard: RoRa + Electric Beatle resubmits, +UTDA, blocker notes #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
willpartcl wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Leaderboard: RoRa + Electric Beatle resubmits, +UTDA, blocker notes #82

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 17 additions & 16 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,37 +230,38 @@ Submissions are ranked by **average proxy cost** across all 17 IBM benchmarks (l

| Rank | Team | Avg Proxy Cost | Best | Worst | Overlaps | Runtime | Verified | Notes |
|------|------|---------------|------|-------|----------|---------|----------|-------|
| 1 | "Cezar" (ReFine) | **1.2224** | 0.8843 | 1.5115 | 0 | 5min/bench | :white_check_mark: | Verified 1.2224 vs self-reported 1.0666; beats RePlAce by 16.2%, SA by 42.5%; resubmitted 4/25, contesting result — re-verification pending |
| 2 | "MTK" (DreamPlace++) | **1.2818** | 0.9073 | 1.6529 | 0 | 37s/bench (GPU) | :white_check_mark: | Verified better than self-reported 1.317; beats RePlAce on all 17 benchmarks |
| 3 | "RoRa" (RipPlace) | **1.3241** | — | — | 0 | 694s/bench | | |
| 4 | "UToronto Analytical" (MOSAIC) | **1.3323** | 0.9371 | 1.6545 | 0 | 24min total | :white_check_mark: | Verified 1.3323 (self-reported 1.3325 — exact match); gradient-based with smooth surrogates, hard+soft |
| 5 | "Shoom" (MultiDREAMPlace) | **1.3381** | — | — | 0 | 350s/bench | | New 4/27; multi-start DREAMPlace + min-displacement legalization + SA |
| 6 | "V5" (TierPlace) | **1.3382** | — | — | 0 | 850s/bench | | New 4/23; GPU-based, multi-density-formulation pilot + phased optimization |
| 7 | "Archgen" (AutoDMP++) | **1.3479** | — | — | 0 | 2404s total | | New 4/24; multi-start + fast proxy screening + bounded refinement |
| 8 | "Electric Beatel" (ePlace-Lite) | **1.3913** | 0.9773 | 1.7253 | 0 | 155s/bench (GPU) | :white_check_mark: | |
| 1 | "Cezar" (ReFine) | **1.2224** | 0.8843 | 1.5115 | 0 | 5min/bench | :white_check_mark: | Verified 1.2224 vs self-reported 1.0666; resubmitted 4/25, contesting result — re-verification pending |
| 2 | "RoRa" (RoomPlace) | **1.2723** | — | — | 0 | 700s/bench | | Resubmitted 5/1 (was RipPlace 1.3241; previous version blocked on `place(plc=...)` signature) — re-verification pending |
| 3 | "MTK" (DreamPlace++) | **1.2818** | 0.9073 | 1.6529 | 0 | 37s/bench (GPU) | :white_check_mark: | Verified better than self-reported 1.317; beats RePlAce on all 17 benchmarks |
| 4 | "Electric Beatle" (ePlace-Lite) | **1.3253** | — | — | 0 | 2000s/bench (GPU) | | Resubmitted 4/30 (previously verified 1.3913); Adam-based stochastic descent with multi-start hyperparameter sweep — re-verification pending |
| 5 | "UToronto Analytical" (MOSAIC) | **1.3323** | 0.9371 | 1.6545 | 0 | 24min total | :white_check_mark: | Verified 1.3323 (self-reported 1.3325 — exact match); gradient-based with smooth surrogates, hard+soft |
| 6 | "Shoom" (MultiDREAMPlace) | **1.3381** | — | — | 0 | 350s/bench | | New 4/27; multi-start DREAMPlace + min-displacement legalization + SA |
| 7 | "V5" (TierPlace) | **1.3382** | — | — | 0 | 850s/bench | | New 4/23; GPU-based, multi-density-formulation pilot + phased optimization |
| 8 | "Archgen" (AutoDMP++) | **1.3479** | — | — | 0 | 2404s total | | New 4/24; multi-start + fast proxy screening + bounded refinement |
| 9 | "Varun's Parallel Worlds" (GRPlace) | **1.4017** | 1.0362 | 1.7298 | 0 | 27s/bench | :white_check_mark: | |
| 10 | "UT Austin" - AS (DREAMPlace Analytical) | **1.4076** | — | — | 0 | 17s/bench | | |
| 11 | "ByteDancer" (Incremental CD) | **1.4151** | 1.0236 | 1.7792 | 0 | 38min/bench | :white_check_mark: | |
| 12 | "vmallela" (Incremental CD+LNS) | **1.4152** | 1.0236 | 1.7817 | 0 | 12h total | :white_check_mark: | Verified 1.4152 (self-reported 1.1172 — 27% worse on our hardware); pure Python+numpy, single-threaded |
| 13 | "TAISPlAce" (ALNS + Thompson Sampling) | **1.4321** | — | — | 0 | 28min/bench | | |
| 14 | "ArzunPD" (HyperPlace SA+LNS) | **1.4421** | 1.0323 | 1.7851 | 0 | 6h total | :white_check_mark: | Verified 1.4421 (self-reported 1.4174); Stage 5 LNS disabled — missing `networkit` dep, fell back to Stage 4 across all benchmarks |
| 15 | "Pragnay" (SweepingBellPlacement) | **1.4427** | — | — | 0 | 632s/bench | | |
| 15 | "Pragnay" (SweepingBellPlacement) | **1.4427** | — | — | 0 | 632s/bench | | Blocked on `compute_proxy_cost(..., plc=None)` in fallback path |
| 16 | "Convex Optimization" (UWaterloo Student) | **1.4556** | 1.0432 | 1.7867 | 0 | 11s/bench | :white_check_mark: | Resubmitted 4/13; fixed from DQ (was 846 overlaps) |
| 17 | "another Waterloo kid" (Batched Nesterov GP) | **1.4568** | — | — | 0 | 118s/bench | | |
| 17 | "another Waterloo kid" (Batched Nesterov GP) | **1.4568** | — | — | 0 | 118s/bench | | Blocked on Modal cloud dispatch — can't run air-gapped |
| — | RePlAce (baseline) | **1.4578** | 0.9976 | 1.8370 | 0 | — | :white_check_mark: | |
| 18 | "W3 Solutions" (GRACE) | **1.4824** | — | — | 0 | 90s/bench | | |
| 18 | "W3 Solutions" (GRACE) | **1.4824** | — | — | 0 | 90s/bench | | HybridPlacer runtime exceeds 1h/bench cap on our hardware |
| 19 | "Jiangban Ya" (Spectral-Seed + Adaptive Legalizer) | **1.4943** | 1.0891 | 1.8099 | 0 | 49s/bench | :white_check_mark: | |
| 20 | "UTAUSTIN-CT" (PLC-Exact Congestion-Aware SA) | **1.5062** | 1.1363 | 1.7941 | 0 | 6s/bench | :white_check_mark: | |
| 21 | "oracleX" (Oracle) | **1.5130** | 1.1340 | 1.7937 | 0 | 11s/bench | :white_check_mark: | |
| 22 | "SEVmakers" (Hybrid Legalization + SA) | **1.5200** | — | — | 0 | 200s/bench | | Private repo — pending judge access |
| 23 | "CA" (congestion_aware) | **1.5247** | 1.2226 | 1.7945 | 0 | 2s/bench | :white_check_mark: | Verified 1.5247 vs self-reported 1.5238 |
| 24 | "#5 ubc cpen student" (Gene Pool Shuffle) | **1.5337** | 1.1411 | 1.8084 | 0 | 13s/bench | :white_check_mark: | |
| 25 | Will Seed (Partcl) | **1.5338** | 1.1625 | 1.7965 | 0 | 35s total | :white_check_mark: | |
| 26 | "UT Austin" - RH (DREAMPlace) | **1.6037** | — | — | 0 | 4.5s/bench | | |
| 27 | "UT Austin" - CT (PROXYCost) | **1.8706** | — | — | 0 | 187s/bench | | |
| 28 | "AS" (Shelf Stacker) | **1.9121** | 1.4614 | 2.3508 | 0 | 0.16s total | :white_check_mark: | |
| 29 | "Adi's Team" (GNN-ePlace Hybrid) | **2.0025** | — | — | 0 | 3726s/bench | | |
| 30 | "Sharc #1" (Auction Placer) | **2.0433** | 1.5143 | 2.4336 | 0 | 96s/bench | :white_check_mark: | |
| 26 | "UTDA" (Phase 12 Multi-start CD+LNS) | **1.5767** | — | — | 0 | 57s/bench | | New 5/1; multi-start CD+LNS with numba-accelerated proxy |
| 27 | "UT Austin" - RH (DREAMPlace) | **1.6037** | — | — | 0 | 4.5s/bench | | |
| 28 | "UT Austin" - CT (PROXYCost) | **1.8706** | — | — | 0 | 187s/bench | | |
| 29 | "AS" (Shelf Stacker) | **1.9121** | 1.4614 | 2.3508 | 0 | 0.16s total | :white_check_mark: | |
| 30 | "Adi's Team" (GNN-ePlace Hybrid) | **2.0025** | — | — | 0 | 3726s/bench | | Blocked on `compute_proxy_cost(skip_congestion=True)` kwarg |
| 31 | "Sharc #1" (Auction Placer) | **2.0433** | 1.5143 | 2.4336 | 0 | 96s/bench | :white_check_mark: | |
| — | SA (baseline) | 2.1251 | 1.3166 | 3.6726 | 0 | — | :white_check_mark: | |
| — | Greedy Row (demo) | 2.2109 | 1.6728 | 2.7696 | 0 | 0.3s total | :white_check_mark: | |
| — | "Binghamton" (feng shui) | pending | — | — | — | — | | |
Expand Down