A zero the Metal compiler must not see: keeping wave operations inside their loop

What changed

Wave operations are instructions that combine a value across the 32 threads ("lanes") that a GPU runs together, for example WaveActiveSum. D3D runs them with only the lanes that reach the instruction. Shaders use that in a "scalarization loop": each iteration takes the lanes whose key matches the first active lane's, sums over just those lanes, and lets them leave the loop.

On our layer such a loop returned the sum of the whole wave on every lane. vkd3d-proton's test_shader_waveop_maximal_convergence had failed on this for many sessions. It now passes, and so does a new test, api/wave_loop_break, with 225 checks (full run).

The interesting part

DXC, Microsoft's HLSL compiler, already knows this pattern is fragile. It makes each break out of such a loop conditional on loading a constant zero, @dx.break.cond, so that later optimisation passes can't prove the break is always taken and move the sum below the loop. Our translation chain turned that global into a constant array (dxil-spirv to SPIR-V, then SPIRV-Cross to the Metal Shading Language). From the results, the Metal compiler then folded the test and ran the sum after the loop with every lane. Its output can't be inspected, so that part is an inference.

A spike measured which conditions the Metal compiler folds (M1 Max, macOS 26). An unconditional break, a constant array and even a mutable one-element thread array all give the wrong result on every lane. A condition read from device memory or through a volatile load gives D3D's per-iteration sums. The fix marks DXC's global Volatile in dxil-spirv (patch 0006). SPIRV-Cross (patch 0043) then reads it through *(volatile thread uint*)&array[0], because its array wrapper type can't be declared volatile.

This leans on unspecified compiler behaviour; I found no reconvergence control in Metal. The new test is there to notice if a later compiler starts folding volatile loads.

One more finding: DXC 1.9 adds dx.break.cond only in pixel and compute shaders. In vertex, hull and domain shaders it keeps the same loop with a plain break. The spike shows the Metal compiler moves wave operations out of such a loop, and I have no fix for that case.

Numbers

• Spike: 6 break forms give 0 of 16 lanes the D3D result, 4 give 16 of 16.
• Full run: 853 pass, 28 fail, 74 skip of 955, against 851/29/74 of 954. Newly passing: api/wave_loop_break and test_shader_waveop_maximal_convergence. Tutorial08_Tessellation failed with its known intermittent 270-pixel frame and passed three solo reruns.
• The design notes say 127 of the new test's 225 checks fail without the patches. That was measured when the work was written and not re-measured for this entry.