Ready-set tracker

Hands out the tasks of a graph without cycles as soon as every task they depend on is done, so a slow task holds up only the tasks that wait for it.

Time complexity O(V+E), gograph function dag.NewTracker.

How it works

A Tracker runs a dependency graph the way a build system does. Ready returns the tasks that can start, and Done marks tasks as finished, which makes ready every task that was waiting only on them. An edge from A to B means B depends on A, so Ready returns B once A is done.

Levels split the same graph into rounds, and each round waits for its slowest task. The tracker starts a task the moment its own dependencies finish, so a long task in one round doesn't hold up the tasks of the next round that never waited for it.

The tracker leaves running the tasks to the caller, and with them how many run at once. This page uses two workers that take the ready tasks first in, first out, and each task takes its weight in time. NewTracker copies the graph in O(V+E), and all the Ready and Done calls of a run add up to O(V+E) too. It returns ErrDAGHasCycle for a graph with a cycle, and Done returns ErrNotReady for a task Ready hasn't returned.

Course prerequisites

Fourteen courses of a computer science degree, each one semester long. Each edge points from a course to a course that requires it.

A directed graph with 14 vertices and 18 edges.

Taking two courses a semester, each as soon as its prerequisites are passed, how many semesters does the degree take?

With 2 workers, the last of the 14 tasks finishes at time 7. Running the levels one after another on the same workers takes 9, since each level waits for its slowest task. One task at a time would take 14.

Step by step

The tracker hands out a task once every task it depends on is done, and 2 workers take the ready tasks in the order it returns them. Each task takes its weight in time, and the time it ends shows next to its name once it starts. The first call to Ready returns the 3 tasks that depend on nothing: Programming 1, Discrete math and Calculus 1.
At time 0, Programming 1 and Discrete math start, to end at 1 and 1. Every worker is busy, so 1 ready task waits for one to come free.
At time 1, Programming 1 and Discrete math finish. Done makes Programming 2 ready. Calculus 1 and Programming 2 start, to end at 2 and 2.
At time 2, Calculus 1 and Programming 2 finish. Done makes Calculus 2, Linear algebra, Computer systems and Data structures ready. Calculus 2 and Linear algebra start, to end at 3 and 3. Every worker is busy, so 2 ready tasks wait for one to come free.
At time 3, Calculus 2 and Linear algebra finish. Done makes Probability ready. Computer systems and Data structures start, to end at 4 and 4. Every worker is busy, so 1 ready task waits for one to come free.
At time 4, Computer systems and Data structures finish. Done makes Operating systems, Databases and Algorithms ready. Probability and Operating systems start, to end at 5 and 5. Every worker is busy, so 2 ready tasks wait for one to come free.

2 more steps lead to the last one:

Done. At time 7, Compilers and Machine learning finish, the last of the 14 tasks. Running the levels one after another on 2 workers would take 9, and one task at a time 14.

Use it in Go

t, err := dag.NewTracker(g)
if err != nil {
	return err
}
workers := make(chan struct{}, 2)
finished := make(chan string, g.Order())
for t.Remaining() > 0 {
	for _, task := range t.Ready() {
		go func() {
			workers <- struct{}{}
			run(task) // task.Weight() is how long it takes
			<-workers
			finished <- task.Label()
		}()
	}
	if err := t.Done(<-finished); err != nil {
		return err
	}
}

Example graphs

Build pipeline, 7 vertices and 9 edges
Course prerequisites, 14 vertices and 18 edges
Spreadsheet formulas, 25 vertices and 41 edges
Monorepo build, 100 vertices and 266 edges