Wide and antialiased lines on an M1: every line becomes a rectangle

What changed

D3D12 has four ways to draw a line. ALIASED is the thin line every GPU draws. QUADRILATERAL_WIDE and QUADRILATERAL_NARROW cover a rectangle 1.4 or 1.0 pixels wide between the endpoints. ALPHA_ANTIALIASED fades the line's edges: the pixel shader's alpha is multiplied by how much of the pixel the line covers. Older pipelines select these modes with two bits, MultisampleEnable and AntialiasedLineEnable. Metal only has aliased lines. Its multisampled lines didn't match any of D3D12's models in a spike that drew 300 random lines (see the chart).

The layer now draws lines in the other three modes itself. A generated object function (a Metal compute stage that runs before the rasterizer) takes each line and emits two triangles that cover the rectangle. For ALPHA lines, the rectangle is 2 pixels wide with half-pixel caps, and each corner carries its distance across and along the line. A SPIRV-Cross patch multiplies the shader's output alpha by the box-filtered coverage computed from those distances.

The session's first commit made D3D12 depth bias work on 24-bit depth buffers. D3D12 counts the constant bias in units of 1/(2^24 − 1) for D24 formats. Metal uses the float format's units, so the layer adds the missing part as a small clip-space offset per viewport.

The interesting part

The new line tests all passed. Then the full run failed two bgfx examples, one of them with a frustum edge that was about 70 pixels short next to its visible end (the second picture).

The frustum's far corner is just behind the camera, at w = −0.49. The object function cut each line at w = 10^-6 to drop the part behind the eye. That was correct, but the cut point projects to x/w ≈ 10^9 pixels. At that size a 32-bit float can only step in 64 or 128 pixel increments. The ALPHA mode measures the distance along the line from that far end, so its coverage near the visible end was off by tens of pixels.

Random lines and a scaled-down copy of the frustum edge didn't reproduce it. It only happens with a vertex barely behind the eye and a large x. The fix clips the line in clip space, before any division, to the viewport plus 8 pixels on each side, so nothing reaching the divide is far off screen. A new test case draws exactly this line. Without the fix it has 17 wrong pixels.

The rest of the two examples' difference was correct. bgfx asks for antialiased lines, and D3D12 draws them alpha-antialiased. The references came from bgfx's own Metal renderer, whose lines are aliased. After a review showed that every differing pixel is on a line, both references became reviewed mtl12 frames.

Numbers

• Full run: 807 pass, 25 fail, 86 skip of 918; 1 newly failing, 4 newly passing. The newly failing test, vkd3d-proton test_rasterizer_ordered_views_dxil, is intermittent. It failed once in 43 runs this session, and passed in the full run just before at the same code (808 pass, 0 newly failing). It is still an open question.
• Previous entry's run: 804 pass, 26 fail, 86 skip of 916.
• api/line_rasterization: 303 checks over 23 cases. vkd3d-proton test_line_rasterization and test_depth_bias_formats pass now.
• The spike: of 300 random lines, how many Metal's multisampled lines matched each D3D12 model at 1 and 4 samples.
• The frustum edge: 17 wrong pixels in the new test case with the w cut alone, 0 with the clip. The cases in front of the eye fail 8 checks if noperspective members are mixed at the clip-space fraction instead of the screen's.