MOOR security

Dieser Eintrag ist auf Englisch verfasst.

A third of the draw path was reference counting

Entry 058839 of 943 tests pass

Under Rosetta, the automatic retain and release calls Objective-C inserts around every draw cost about a third of the CPU time mtl12 spent replaying a draw-heavy scene; removing them made that scene's frames 23% faster.

What changed

mtl12's Unix side (the part that turns recorded D3D12 commands into Metal calls) is written in Objective-C with ARC (automatic reference counting: the compiler inserts retain and release calls to keep objects alive). On a Mac, Wine runs as x86_64 code under Rosetta, Apple's translator, and every one of those calls is a call into the Objective-C runtime.

Per draw, the code loaded the Metal encoder and the argument table (the list of buffers a draw binds) into local variables and passed them to three helper functions. At the -O1 optimisation level, ARC retained and released each of them every time. The objects belong to the encoder for the whole draw, so those references are now marked __unsafe_unretained, which tells ARC to leave them alone. Two subtler cases followed. A function that returned the encoder made ARC retain it for the caller. A check run on every draw lived in a function whose object locals ARC released on exit, even when they were still nil. After both were fixed, a counter that intercepts the runtime's retain and release functions found no call per draw left. A review of every unretained reference found none that can outlive its object. It also found one comment that needed tightening.

What each draw did to reference counts before and after.before: each draw: cmd_draw table = e->render_table RETAIN, render_encoder() returns e->render RETAIN for the caller, draw_state(enc) draw_bindings(table) draw_primitives(enc, table) RETAIN each parameter on entry, RELEASE on exit, Metal draw call, cmd_draw returns RELEASE its object locals; after: each draw: cmd_draw __unsafe_unretained table, render_pass_matches() plain C, no objects, draw_state, draw_bindings, draw_primitives unretained parameters, Metal draw call; cmd_draw table = e->render_table RETAIN → render_encoder() returns e->render RETAIN for the caller; render_encoder() returns e->render RETAIN for the caller → draw_state(enc) draw_bindings(table) draw_primitives(enc, table) RETAIN each parameter on entry, RELEASE on exit; draw_state(enc) draw_bindings(table) draw_primitives(enc, table) RETAIN each parameter on entry, RELEASE on exit → Metal draw call; Metal draw call → cmd_draw returns RELEASE its object locals; cmd_draw __unsafe_unretained table → render_pass_matches() plain C, no objects; render_pass_matches() plain C, no objects → draw_state, draw_bindings, draw_primitives unretained parameters; draw_state, draw_bindings, draw_primitives unretained parameters → Metal draw callbefore: each drawafter: each drawcmd_drawtable = e->render_tableRETAINrender_encoder()returns e->renderRETAIN for the callerdraw_state(enc)draw_bindings(table)draw_primitives(enc, table)RETAIN each parameter on entry,RELEASE on exitMetal draw callcmd_draw returnsRELEASE its object localscmd_draw__unsafe_unretained tablerender_pass_matches()plain C, no objectsdraw_state, draw_bindings,draw_primitivesunretained parametersMetal draw call
What each draw did to reference counts before and after. (Mermaid source)
Why the profile put cmd_draw's time under its caller.what ran: replay, cmd_draw, objc_msgSend (no stack frame of its own), Metal: drawPrimitives; what the sampled stack showed: replay (return address into replay), leaf PC inside objc_msgSend (translated code, no symbol); replay → cmd_draw; cmd_draw → objc_msgSend (no stack frame of its own); objc_msgSend (no stack frame of its own) → Metal: drawPrimitives; replay (return address into replay) → leaf PC inside objc_msgSend (translated code, no symbol); what the sampled stack showed → cmd_draw's own frame is missing: objc_msgSend did not push one, so its time looked like replay's. Read the return address in the disassembly first.what ranwhat the sampled stack showedreplaycmd_drawobjc_msgSend(no stack frame of its own)Metal: drawPrimitivesreplay(return address into replay)leaf PC inside objc_msgSend(translated code, no symbol)cmd_draw's own frame is missing: objc_msgSend did not push one,so its time looked like replay's. Read the return address in the disassembly first.
Why the profile put cmd_draw's time under its caller. (Mermaid source)

The interesting part

An earlier session split the Unix side's roughly 0.5 µs per draw by area of code by skipping parts of it, because timing marks disturbed the measurement. They could not see inside the functions, because sample shows no symbols for translated code. Xcode's xctrace does work on a Wine process, but its stacks mislead in one specific way. objc_msgSend (the function behind every Objective-C method call) sets up no stack frame of its own. A sample taken inside it skips the function that made the call, so time spent in cmd_draw's Metal calls showed up under its caller. Reading the disassembly at each return address, not the line numbers, is what turned up the retain calls in the helpers' prologues.

A dead end with a cost: attaching lldb to the Wine process left Rosetta in a state where every newly built x86_64 program hangs at launch and cannot be killed.

Numbers

Three alternating rounds against the previous commit:

  • bench-draws (10,000 draws a frame), submit: 4.99 to 5.08 ms before, 3.51 to 3.56 ms after.
  • bgfx drawstress, Unix CPU per frame: 31.4 to 32.3 ms before, 21.7 to 22.3 ms after; frame 43.3 to 44.2 ms before, 33.6 to 34.3 ms after.
  • ModelViewer: no change outside the run-to-run spread (ab4_modelviewer.txt). It waits for vsync and makes few draws.
  • Compiling the Unix side at -O2 instead of -O1 would give another 2 to 3% (ab5_O2.txt). Not adopted yet.
  • A single run of the perf history agrees: drawstress frame -25%.
  • Tests: 839 of 943 pass, as before; none newly passing or failing.
Draw path without ARC calls, % of the old build. Step 1: locals and parameters unretained; step 2: no ARC call per draw (medians of 3 rounds) (data as CSV)
  • old
  • step 1
  • step 2

measurement · % of old build

  1. bench-draws submit100 old, 79 step 1, 70 step 2
  2. drawstress frame100 old, 85 step 1, 77 step 2
  3. drawstress unix CPU100 old, 79 step 1, 69 step 2
Tests by suite at the end of this entry: 839 of 943 pass (data as CSV)
  • pass
  • fail
  • skip

suite · share of each row's total

  1. vkd3d-proton490 pass, 15 fail, 72 skip of 577
  2. vkd3d177 pass, 9 fail, 4 skip of 190
  3. apps77 pass, 0 fail, 0 skip of 77
  4. api75 pass, 0 fail, 0 skip of 75
  5. wine-dxgi16 pass, 4 fail, 0 skip of 20
  6. samples4 pass, 0 fail, 0 skip of 4