Integer border colours on a GPU with three fixed ones

What changed

mtl12 translates Direct3D 12 to Apple's Metal, so Windows games can run under Wine on Apple Silicon. This entry covers one short session; its gallery scene, diagram and note were committed the session after.

Shader Model 6.7 bundles four texture features under one cap, AdvancedTextureOps: sampling integer textures, raw gathers (reading four texels' bits unconverted), offsets computed at run time, and a depth comparison at an explicit mip level. With the cap simply switched on, three of the four already worked: the shader translators (dxil-spirv and SPIRV-Cross) produce Metal code the compiler accepts. One piece did not: integer border colours, the value a sampler returns when the coordinate falls outside the texture and the address mode is "border".

The interesting part

Metal has exactly three border colours and no custom one. For float textures the project already emulated arbitrary colours: sample once with a black border and once with a white one, and blend between them with the colour as weight.

Integer sampling is point-filtered (it never blends neighbouring texels), so that trick becomes a select. Inside the texture both samples return the same texel. Outside, one returns 0 and the other 1, so any channel where the two differ is border, and it takes the colour. SPIRV-Cross patch 0021 emits that select for integer results.

The trap was in storage. The sampler record used to hold the border colour as floats, and a 32-bit float holds integers exactly only up to 2^24. The gallery scene's third border, 0x01000007, would round to 0x01000008 and its low bits, which pick the colour, would turn it grey. The record now keeps the bits.

The new test also hit a Metal assertion on a view with an empty mip range, which hangs under Wine; such views now become null views.

Numbers

• The new api test advanced_texture_ops: 45 checks. Before patch 0021 only the 4 custom-colour checks failed; with it all pass.
• Full run at the end: 580 pass, 82 fail, 145 skip of 807. Against the previous full run: vkd3d-proton test_sm67_sample_cmp_level and test_sm67_dynamic_texture_offset go from skip to pass, and the new api test passes. Nothing that passed before fails.