Translating each shader once: a disk cache in front of SPIR-V and MSL

What changed

When a D3D12 program creates a pipeline (a bundle of shaders plus fixed-function state), this layer translates each shader from DXIL (Direct3D's shader bytecode) to SPIR-V with dxil-spirv, and from SPIR-V to MSL (Metal Shading Language) with SPIRV-Cross. Then Metal compiles the MSL. Metal already caches its own compile on disk. The translation in front of it was not cached anywhere, so every run of a game translated every shader again.

Now each translation's results are written to ~/Library/Caches/mtl12/<build>/<key>. The results are the MSL of each stage and everything else the translator works out, such as threadgroup sizes, tessellation layout and vertex attribute locations. The key is a SHA-256 hash of everything the translator reads: the shader bytecode, the root signature's binding ranges, static samplers, the input layout, the stream-output declaration and the fixed-function fields. The hash also covers the UUID of the translator library itself, so a rebuilt translator never reads an older one's output.

The interesting part

A cache like this has only one dangerous failure: a wrong hit, where two different pipelines share a key and one gets the other's shaders. A miss only costs time. So the checks this session aimed at wrong hits:

• api/translation_cache creates pairs of pipelines that differ only in what the key must tell apart: a sampler's address mode, a root constant against a root constant buffer, the order of root parameters, the order of input elements. With the static samplers taken out of the key, the test fails, as it should.
• The whole conformance suite ran twice over one empty cache directory. The cold run left 1,805 entries. The warm run, where nearly every pipeline was a hit, gave the same 837 passing tests and added no entries. A pointer hashed by its address instead of its contents would have shown up as new entries, because addresses change between processes.
• To check that the warm side really loads entries, the api suite ran twice over a fresh cache. The second run rewrote none of its 161 entries, so every translation was a hit.

One timing test, api/timestamp_after_work, failed in both runs. Its precondition is that a round of draws lasts at least 0.5 ms, and some rounds measured 0.05 to 0.3 ms. It fails just as often with the cache turned off (8 of 10 runs passed), so the cache isn't the cause. I don't yet know whether the draws or the timestamps are at fault.

Numbers

From the bench-pso scene (48 pipelines that need 16 distinct translations):

• A pipeline whose translation misses takes 1.9 to 5.5 ms. With the cache filled, no pipeline after the first takes more than 0.82 ms.
• Pipelines 2 to 48 together: 52.5 ms with an empty cache, 20.6 ms with a filled one.
• The first pipeline, which also carries one-time setup: 15.3 ms to 10.7 ms.