MOOR security

Dieser Eintrag ist auf Englisch verfasst.

Work graphs on a Mac: a GPU that schedules its own work, one level at a time

Entry 049818 of 918 tests pass

D3D12 work graphs, where shaders launch other shaders by writing records, now run on an M1 Mac through a level-by-level scheduler ported from vkd3d-proton; all ten vkd3d-proton work graph tests pass.

What changed

Work graphs are D3D12's newest way to feed a GPU (shader model 6.8). Instead of the CPU recording every dispatch, a graph of nodes (compute shaders) runs on the GPU. A node writes records (small structs) to other nodes, or to itself, and the GPU launches those nodes on the records. Metal has nothing like this.

The layer now implements them with the design vkd3d-proton (the Vulkan translation layer used by Proton) uses on Vulkan, which has no work graphs either. At state object creation the layer sorts the nodes and places each one on every level it can run at: level 0 holds the entry nodes, level 1 what they can write to, and so on, with a recursive node repeated up to its declared depth. Each node is translated once (DXIL to SPIR-V with dxil-spirv, then to Metal Shading Language) into up to four Metal pipelines that differ only in function constants.

DispatchGraph then runs level after level. After a level, two small scheduling kernels turn the record counts that the nodes wrote into threadgroup counts and give each record its place in memory. Then comes one indirect dispatch per node of the next level (see the diagram). Both kernels are ports of vkd3d-proton's GLSL to Metal, using 32-wide SIMD groups.

How one DispatchGraph becomes Metal dispatches.CreateStateObject (EXECUTABLE, WORK_GRAPH) → Program build on the CPU: sort nodes by ID and array index, place each on every level it can run at, unroll recursion; Program build on the CPU: sort nodes by ID and array index, place each on every level it can run at, unroll recursion → Each node translated once DXIL → SPIR-V → MSL; up to 4 Metal pipelines via function constants; DispatchGraph (CPU or GPU records) → Level 0: dispatch the entry node on the uploaded records; Each node translated once DXIL → SPIR-V → MSL; up to 4 Metal pipelines via function constants → Level 0: dispatch the entry node on the uploaded records; Level 0: dispatch the entry node on the uploaded records → Nodes write records to a payload ring and count them per node (atomics in the backing memory); Nodes write records to a payload ring and count them per node (atomics in the backing memory) → wg_distribute: counts → threadgroups per node; wg_distribute: counts → threadgroups per node → wg_offsets: each record's payload offset, grouped by node; wg_offsets: each record's payload offset, grouped by node → Level n: one indirect dispatch per node, reading the other ring; Level n: one indirect dispatch per node, reading the other ring → more levels?; more levels? → Nodes write records to a payload ring and count them per node (atomics in the backing memory) (yes: swap the rings); more levels? → Done: 3 dispatches and barriers per level (no)CreateStateObject(EXECUTABLE, WORK_GRAPH)Program build on the CPU:sort nodes by ID and array index,place each on every level it can run at,unroll recursionEach node translated onceDXIL → SPIR-V → MSL;up to 4 Metal pipelines viafunction constantsDispatchGraph(CPU or GPU records)Level 0: dispatch the entrynode on the uploaded recordsNodes write records to a payloadring and count them per node(atomics in the backing memory)wg_distribute: counts →threadgroups per nodewg_offsets: each record'spayload offset, grouped by nodeLevel n: one indirectdispatch per node,reading the other ringmore levels?Done: 3 dispatchesand barriers per levelyes: swap the ringsno
How one DispatchGraph becomes Metal dispatches. (Mermaid source)
Gallery scene workgraph: a quadtree built by a work graph: one recursive broadcasting node gets fifteen 128-pixel squares from DispatchGraph and, where its 8x8 samples disagree on inside or outside the Mandelbrot set, sends records for its four quarters back to itself, down to 4 pixels; leaves are coloured by depth, from dark blue (0) to orange (5); animated, the view zooms in 6 times and back. All scenes in the gallery

The interesting part

The last three failing tests had two unrelated causes, and neither logged an error.

An entry node that takes no input is started with one record but no record data: NULL and stride 0. My DispatchGraph treated the NULL pointer as an application error and dropped the input, so the whole graph silently did nothing. Two tests read back zeroes.

The other was a C++ use-after-move. When padding node arrays (a node ID with indices 0, 1, 3 needs an empty slot at 2), the loop compared each node with the one before it. But the previous node had already been moved into the result, so its name was an empty string. Every array element after the first got a spurious hole in front of it, and records addressed to C[1] went to a slot that no node runs. Both are now covered by tests.

Numbers

  • vkd3d-proton test_workgraph_*: 10 of 10 pass (they were all skipped before, because the device reported no work graph support).
  • Full run: 818 pass, 24 fail, 76 skip of 918, up from 807 pass. Nothing newly failing. The 11 newly passing tests are the ten work graph tests and test_rasterizer_ordered_views_dxil, the intermittent failure from last session.
  • A later review found no layout bugs. It did find three gaps, now fixed. Oversized CPU record uploads were truncated to 32 bits. Backing memory below the reported minimum was accepted, although the payload rings are not bounds-checked. A GPU input naming no entry point matched every non-entry node.
Tests by suite at the end of this entry: 818 of 918 pass (data as CSV)
  • pass
  • fail
  • skip

suite · share of each row's total

  1. vkd3d-proton (+11)490 pass, 15 fail, 72 skip of 577
  2. vkd3d177 pass, 9 fail, 4 skip of 190
  3. apps77 pass, 0 fail, 0 skip of 77
  4. api70 pass, 0 fail, 0 skip of 70
  5. samples4 pass, 0 fail, 0 skip of 4