Dunfey · Hotel WWDC as data, est. 1983
Front desk everything
Years
Topics

WWDC26 · 22 min · AI & Machine Learning / Graphics & Games

Build real-time neural rendering pipelines with Metal

Discover how to integrate machine learning into your real-time rendering pipeline using Metal 4. We’ll explore practical adoption patterns and best practices for achieving production-quality results with MetalFX neural denoising, featuring real-world insights from Maxon’s Redshift Live. Learn how to train and deploy a neural tone mapper using the ML command encoder inline with your graphics work. Finally, dive into the new tensor API to build and evaluate small, specialized neural networks directly within your shaders.

Watch at developer.apple.com ↗

Transcript all transcripts

Chapters

  • 0:00 — Introduction
  • 2:16 — MetalFX Denoising
  • 9:57 — Deploy custom ML networks with Metal 4
  • 13:40 — Inline neural networks with tensorOps
  • 20:55 — Next steps

Code shown on screen · 1 snippet

Compute camera-only motion vectors cpp · at 8:46 ↗ 
#include <metal_stdlib>
using namespace metal;

// Compute camera-only motion vectors
float4 clipCurrent = viewProjCurrent * float4(worldPos, 1.0);
float2 ndcCurrent = clipCurrent.xy / clipCurrent.w;

float4 clipPrevious = viewProjPrevious * float4(worldPos, 1.0);
float2 ndcPrevious = clipPrevious.xy / clipPrevious.w;

float2 motion = ndcPrevious - ndcCurrent;

// Get subpixel offset for current and previous frames
float2 jitterCurrent = getJitter(frameIndex);
float2 jitterPrevious = getJitter(frameIndexPrevious);
motion -= jitterPrevious - jitterCurrent;

Resources