Dynamic topological sort

Keeps a topological order while edges are added one at a time, and moves only the vertices between the two ends of an edge that points backward.

Time complexity O(V+E) per edge, gograph function Graph.AddEdge.

gograph does this inside AddEdge on a graph created with gograph.Acyclic. It doesn't expose the order it keeps, so the order on this page comes from the step-by-step copy.

How it works

A build tool that adds dependencies one at a time has to know right away when a new one closes a loop. Sorting the whole graph again after every edge costs O(V+E) each time, which adds up as the graph grows. Pearce and Kelly's dynamic topological sort keeps an order instead, and when a new edge points backward in it, reorders only the vertices between the edge's two ends.

An edge that already points forward needs nothing. Neither does an edge from a vertex with no incoming edges, which can move to the front, or to a vertex with no outgoing edges, which can move to the end. Any other backward edge first searches forward from its destination among the vertices placed before its source. Reaching the source means the edge would close a loop, and AddEdge returns ErrDAGCycle without adding it.

Otherwise gograph runs two walks side by side, each allowed a number of edges that doubles every round, and uses the first to finish. The walk back from the source finds the vertices placed after the destination that reach the source. They swap places with the ones the forward search found, so each set keeps its order and every edge still points forward. The walk forward finds everything the destination reaches, and if it finishes first, all of it moves to the end of the order. Either way an edge costs little more than one search over the vertices the destination reaches.

All of this happens inside g.AddEdge on a graph created with gograph.Acyclic, and gograph.TopologySort still gives an order on demand. The order AddEdge keeps isn't exposed, so the steps here come from a copy of it, checked against the edges gograph turns away, and in the site's tests against gographtest.IsTopologicalOrder.

Monorepo build

One hundred build targets, T1 to T100, in the repository of an online shop. T1 to T32 are shared libraries, T33 to T41 the code generator and API definitions, T42 to T58 services, T59 to T63 batch jobs, T64 to T71 the API client and UI packages, T72 to T76 apps, T77 to T95 container images, and T96 to T100 the deploys and the contract and end-to-end tests. A target takes 1 to 20 minutes to build. Each edge points from a target to a target that builds on it.

A directed graph with 100 vertices and 266 edges.

Add the dependencies of the monorepo's targets one by one and keep a valid build order the whole time.

AddEdge accepts all 266 edges, and the order it keeps is a valid build order after each one.

Step by step

  1. Start with the 100 vertices and no edges, from T100 back to T1. Each new vertex goes after the others, so the order starts as the order they were added in, and every edge below is added in a fixed mixed order.
  2. Add the edge from T20 to T46. T46 is at 55, before T20 at 81, but nothing points to T20 yet, so T20 moves to the front without a search.
  3. Add the edge from T35 to T64. T64 is at 38, before T35 at 67, but nothing points to T35 yet, so T35 moves to the front without a search.
  4. Add the edge from T31 to T54. T54 is at 49, before T31 at 71, but nothing points to T31 yet, so T31 moves to the front without a search.
  5. Add the edge from T77 to T80. T80 is at 24, before T77 at 27, but nothing points to T77 yet, so T77 moves to the front without a search.
  6. Add the edge from T77 to T84. T77 is at 1 and T84 at 21, so the edge already points forward and nothing moves.

292 more steps lead to the last one:

  1. Done. AddEdge added 266 edges, 31 of them needed a reorder and 159 vertices moved in all. The searches visited 112 vertices in all, where sorting the whole graph after every edge would visit all 100 each time, 26600 in all.

Use it in Go

g := gograph.New[string](gograph.Directed(), gograph.Acyclic())
for _, label := range labels {
	g.AddVertexByLabel(label)
}
for _, d := range deps {
	// AddEdge keeps a topological order and reorders only what it must.
	_, err := g.AddEdge(g.GetVertexByID(d[0]), g.GetVertexByID(d[1]))
	if errors.Is(err, gograph.ErrDAGCycle) {
		fmt.Println(d[0], "->", d[1]) // would close a loop, turned away
	}
}

Example graphs

More in Ordering

  • Topological sort: Puts the vertices of a graph without cycles in an order where every edge points forward, taking vertices in the order they become ready.
  • Stable topological sort: Puts the vertices of a graph without cycles in an order where every edge points forward, taking the ready vertex that comes first alphabetically.
  • Topological levels: Splits a graph without cycles into levels, so that every edge goes to a later level and each level can run at the same time.
  • Ready-set tracker: Hands out the tasks of a graph without cycles as soon as every task they depend on is done, so a slow task holds up only the tasks that wait for it.

All algorithms