slurmlet
an all-or-nothing scheduler that places big AI training jobs across a fleet of GPUs, built in Go and Python
About
Big AI training runs need many GPUs at the same moment: if even one is missing, the whole job stalls and the rest sit idle. slurmlet is the traffic controller that holds a job back until every GPU it asks for is free, then starts them all together, so expensive hardware never waits half-used while a run gets the machines it needs.
It is built twice, once in Go and once in Python, against one shared design, so the two implementations can be compared head to head on the same problem (and the trade-offs each language makes show up first-hand). Everything runs against a simulated fleet of stand-in GPUs, so the full lifecycle, scheduling a job, draining a machine, and rescheduling, is exercised without renting expensive hardware. The Go version is active; the Python version lands next.
Liner notes
- All-or-nothing start: a job only runs once every GPU it needs is reserved, so none sit idle waiting.
- Built twice, in Go and Python, against one shared design, to compare the two approaches directly.
- Runs on a simulated GPU fleet, so the whole workflow can be tested without real GPU costs.