Mesh shaders on Metal, and a grid limit D3D12 doesn't have
The D3D12-on-Metal layer now runs mesh and amplification shaders as Metal mesh and object functions; all six vkd3d-proton mesh tests pass and nothing regressed.
What changed
Mesh shaders replace the classic front of the pipeline. There is no vertex buffer and no vertex shader. Instead, a mesh shader (MS) threadgroup writes a small batch of vertices and triangles directly. An optional amplification shader (AS) runs first, decides how many MS threadgroups to launch, and hands them a payload of up to 16 KB. Games use it to cull whole chunks of a model before any triangle exists.
Metal has the same shape: an object function, a mesh function, and a payload. So unlike geometry shaders or tessellation (entries 026–030), no emulation was needed:
- stream-based pipeline creation accepts the AS and MS subobjects;
DispatchMeshbecomesdrawMeshThreadgroups;- ExecuteIndirect's mesh arguments turn into indirect mesh draws, because D3D12_DISPATCH_MESH_ARGUMENTS has the same three-uint layout as Metal's indirect arguments.
Two gaps needed fixes:
- SPIRV-Cross declared a mesh shader's clip distances with one array dimension too many, which Metal rejects. Patch 0027 fixes that.
- Pixel shader inputs were not checked against the mesh shader's outputs, so a mismatched pipeline reached the Metal compiler instead of failing with E_INVALIDARG. The check follows vkd3d-proton's rules. It matches by semantic across the per-vertex and per-primitive outputs, and SV_PrimitiveID must come from the MS.

The interesting part
D3D12 lets a draw launch up to 2^22 threadgroups, and lets each AS threadgroup launch up to 2^22 more. A spike (a small throw-away test program) measured what an M1 Max actually runs:
- a draw's own grid, of mesh or object threadgroups, runs up to 2^21 threadgroups;
- a grid set from inside an object function runs up to 1024 threadgroups, the
maxTotalThreadgroupsPerMeshGridthe pipeline reports.
One threadgroup over either limit, and nothing runs. There is no error and no hang. One odd case: 2^21 + 1 from an object function ran exactly one threadgroup, which looks like the count is kept modulo 2^21 (unverified beyond that one point). The layer drops draws over 2^21 with a FIXME. The 1024 limit inside the AS is open: no test hits it yet.
Numbers
- Full run: 659 pass, 27 fail, 138 skip of 824 tests, against 653 / 27 / 144 at the end of entry 030. Six tests went from skip to pass, and none newly fail.
- The six: test_mesh_shader_create_pipeline, test_mesh_shader_rendering (260 checks), test_mesh_shader_execute_indirect, test_mesh_shader_execute_indirect_state, test_amplification_shader (57 checks) and test_amplification_shader_execute_indirect_state.
- test_shader_io_mismatch: 116 checks, all passing after the fix.