MOOR security

Dieser Eintrag ist auf Englisch verfasst.

Inverted viewports for free, and the depth clamp that wasn't there

Entry 074870 of 964 tests pass

mtl12 now reports inverted viewport depth, which Metal does natively, and clamps the depth a pixel shader writes to the viewport's range, which Metal doesn't do.

What changed

A D3D12 viewport has a depth range, MinDepth to MaxDepth, that the 0-to-1 depth of each triangle is mapped into. A newer D3D12 cap (a feature flag the device reports), InvertedViewportDepthFlipsZSupported, says that MinDepth may be larger than MaxDepth: depth then runs backwards, and the triangle is still drawn. Metal's MTLViewport already behaves this way. Depth flips, nothing is culled, and clipping uses the range. I wrote an api test for it, api/inverted_viewport_depth, with 13 cases on a 32-bit float depth buffer, and reported the cap.

Three of those cases failed, and not because of inversion. In D3D12, depth that a pixel shader writes itself (SV_Depth) is clamped to the viewport's range, smaller end to larger end. Metal clamps it to [0, 1] and no further. A shader writing 0.9 with a viewport from 0.2 to 0.8 stored 0.9, where D3D12 stores 0.8. Ranges that aren't inverted were just as wrong, so this was an old gap that the new test happened to find.

Which draws take the clamp variant, and what that costs.draw → pixel shader writes depth, a depth buffer is bound, a viewport range isn't [0, 1]?; pixel shader writes depth, a depth buffer is bound, a viewport range isn't [0, 1]? → plain pipeline Metal clamps to [0, 1], which is right here (no (most draws)); pixel shader writes depth, a depth buffer is bound, a viewport range isn't [0, 1]? → clamp variant made on first use (yes); clamp variant made on first use → per-draw buffer 30 words 24+: (min, max) for each viewport; per-draw buffer 30 words 24+: (min, max) for each viewport → fragment function ends with depth = clamp(depth, min, max) of its viewport index; plain pipeline Metal clamps to [0, 1], which is right here → indirect command buffers allowed; fragment function ends with depth = clamp(depth, min, max) of its viewport index → no indirect command buffers: ExecuteIndirect runs per recorddrawpixel shader writes depth,a depth buffer is bound,a viewport range isn't [0, 1]?plain pipelineMetal clamps to [0, 1],which is right hereclamp variantmade on first useper-draw buffer 30words 24+: (min, max)for each viewportfragment function ends withdepth = clamp(depth, min, max)of its viewport indexindirect commandbuffers allowedno indirect command buffers:ExecuteIndirect runs per recordno (most draws)yes
Which draws take the clamp variant, and what that costs. (Mermaid source)
Gallery scene depthclamp: three viewports, one per panel, get the same sine wave as pixel-shader depth (SV_Depth); a second pass plots the depth each column kept. Left, range [0, 1]: the whole wave. Middle, [0.25, 0.75], and right, inverted 0.9 to 0.4: the wave cut flat at the white lines, the viewport's range; animated, the wave scrolls one period. All scenes in the gallery

The interesting part

The fix is a clamp at the end of the pixel shader. The cost is where it lives. The shader needs each viewport's range, which comes from a small per-draw buffer, and draws that read such a buffer can't go into Metal's indirect command buffers (pre-recorded draw lists that the GPU replays). So the clamp is in a pipeline variant, a second copy of the pipeline that is made on first use, like the one for variable-rate shading. Only a draw whose pixel shader writes depth, with a depth buffer bound and a viewport range other than [0, 1], uses it. Every other draw keeps the plain pipeline, where Metal's own [0, 1] clamp is already right.

The clamp itself is a patch to SPIRV-Cross, the library that turns shaders into Metal's language (patch 0046). It adds the line depth = clamp(depth, min, max) with the bounds for the viewport the triangle went to, adding a viewport index input when the shader has none. When a draw also uses a depth bounds test, both features share one buffer slot, so the clamp reads its ranges through the other feature's argument. A review after the first commit asked for a test of that path, and the test now has three depth bounds cases.

Switching Metal's depth clip mode to clamp would have been shorter. It did clamp written depth to the range, but it also stops clipping, which D3D12 keeps while DepthClipEnable is on.

Numbers

  • api/inverted_viewport_depth before the clamp: 215 checks, 48 failed, the three pixel-shader-depth cases with 16 pixels each. After it: 215 ok. With a [0, 1] case and two viewport-index cases: 263 ok. With the depth bounds cases: 311 ok.
  • Full run: 870 pass, 26 fail and 68 skip of 964. 1 test newly passes, the new api test, and none newly fail.
api/inverted_viewport_depth checks per run (data as CSV)
  • ok
  • failed

run · checks

  1. before the clamp167 ok, 48 failed of 215
  2. clamp variant215 ok, 0 failed of 215
  3. + 3 cases263 ok, 0 failed of 263
  4. + 3 bounds cases311 ok, 0 failed of 311
Tests by suite at the end of this entry: 870 of 964 pass (data as CSV)
  • pass
  • fail
  • skip

suite · share of each row's total

  1. vkd3d-proton499 pass, 13 fail, 65 skip of 577
  2. vkd3d178 pass, 9 fail, 3 skip of 190
  3. api (+1)93 pass, 0 fail, 0 skip of 93
  4. apps80 pass, 0 fail, 0 skip of 80
  5. wine-dxgi16 pass, 4 fail, 0 skip of 20
  6. samples4 pass, 0 fail, 0 skip of 4
Zur Eintragsliste