Derivatives in compute shaders, and a compiler crash that a texture write avoids

What changed

mtl12 translates Direct3D 12 to Apple's Metal, so Windows games can run under Wine on Apple Silicon. Until this session the device reported Shader Model 6.0. Last session, reporting 6.6 for a single run showed seven tests failing for separate reasons. This session went through them in turn, writing a failing test first for each. The device now reports 6.6.

• Wave size. A wave is a group of threads that run in lockstep; on Metal that is a SIMD-group, and it always has 32 threads. The device now reports 32-lane waves. A compute shader that demands another size fails to compile, as it would on D3D12 hardware with fixed 32-lane waves.
• Helper lanes. Pixel shaders run in 2x2 quads. A pixel that calls discard stays in its quad as a "helper", so its neighbours can still compute derivatives. IsHelperLane() came out as an undeclared variable in the translated shader. The fix was a small change in SPIRV-Cross, the shader translator (patch 0016).
• Bitwise multi-prefix scans. Metal's SIMD-group scans only add and multiply. Patch 0017 builds AND, OR and XOR scans out of shuffles.
• Derivatives in compute shaders. This is the main piece, described below.
• Raw buffers read as typed buffers. A shader may read a raw buffer view through a typed buffer declaration. These views now get a texture-buffer view as well.
• Per-sample shading. See below.

An architecture decision records the mappings. A review found one bug: when the extra texture-buffer view for a raw buffer couldn't be created, the whole view was lost. It now keeps the raw view.

The interesting part

Derivatives. SM 6.6 lets compute shaders call ddx() and sample textures with automatic mip selection. Both need derivatives. In compute, derivatives come from 2x2 groups of threads. Metal rejects every derivative function in a kernel. dxil-spirv (the DXIL-to-SPIR-V compiler) already arranges threads so that each quad is four consecutive SIMD lanes. That matches Metal's quadgroups, so the derivative becomes the difference of two quad_shuffle reads. Sampling becomes an explicit-gradient sample built from those differences (patch 0018). The vkd3d-proton test passes 823 checks.

The crash. A per-sample pixel shader runs once per MSAA sample and reads the sample's position. With indirect-command-buffer support turned on, Apple's shader compiler crashes if such a shader writes depth; I found that in entry 018. It turns out a plain buffer store or an atomic crashes it too. A texture write fails with a clean error instead. The compiler never returns in the crashing cases, so there is nothing to catch. Any shader that reads a sample position now gets a pipeline without ICB support.

The one that stays. -denorm preserve asks the GPU to keep very small floats (denormals) instead of flushing them to zero. On an M1 Max, every Metal math mode flushed 32-bit denormals, including precise::fma. That test keeps failing unless floats are emulated in software.

Numbers

• Full run at the end, with 6.6 reported: 558 pass, 79 fail, 162 skip of 799 tests. At the end of entry 021: 544, 78 and 170 of 792.
• Of the 14 new passes, seven are new api tests. Six are vkd3d-proton tests that skipped at 6.0. test_update_tile_mappings_remap_smem is flaky and passed this time.
• Failures went up by one. Two tests that skipped before now fail one check each: denormals, and a helper lane on the seam of a clipped triangle. The flaky tile test took one failure off.
• Failed checks in the vkd3d and vkd3d-proton suites: 3,322 to 3,297.
• A full run with 6.6 reported, just before switching it on for good: 557 pass, 79 fail, 162 skip of 798 tests.
• Checks passed by the SM 6.x tests with 6.6 reported: wave_intrinsics 582, wave_size 279, compute_derivatives 823, heap_mismatch_types 523.