Dunfey · Hotel WWDC as data, est. 1983
Front desk everything
Years
Topics

WWDC25 · 15 min · Audio & Video / Spatial Computing

Learn about Apple Immersive Video technologies

Explore the capabilities of Apple Immersive Video and Apple Spatial Audio Format technologies to create truly immersive experiences. Meet the new ImmersiveMediaSupport framework, which offers functionality to read and write the necessary metadata for enabling Apple Immersive Video. Learn guidelines for encoding and publishing Apple Immersive Video content in standalone files for playback or streaming via HLS. To get the most out of this session, we recommend first watching “Explore video experiences for visionOS.”

Watch at developer.apple.com ↗

Transcript all transcripts

Chapters

  • 0:00 — Introduction
  • 0:48 — Apple Immersive Video overview
  • 2:36 — Apple Immersive Video metadata
  • 5:13 — Read AIVU files
  • 7:16 — Write AIVU files
  • 8:43 — Publish Apple Immersive Video content
  • 10:29 — Preview Apple Immersive Video content
  • 11:21 — Apple Spatial Audio Format
  • 12:39 — Apple Positional Audio Codec

Code shown on screen · 6 snippets

Read VenueDescriptor from AIVU file swift · at 6:23 ↗ 
func readAIMEData(from aivuFile: URL) async throws -> VenueDescriptor? {
    let avAsset = AVURLAsset(url: aivuFile)
    let metadata = try await avAsset.load(.metadata)
    let aimeData = metadata.filter({ $0.identifier == .quickTimeMetadataAIMEData }).first
    if let dataValue = try await aimeData.load(.value) as? NSData {
        return try await VenueDescriptor(aimeData: dataValue as Data)
    }
    return nil
}
Read PresentationDescriptor from AIVU playback swift · at 6:50 ↗ 
func presentation(timedMetadata: [AVTimedMetadataGroup]) async throws ->   
[PresentationDescriptor] {
    var presentations: [PresentationDescriptor] = [] 
    for group in timedMetadata {
        for metadata in group.items {
            if metadata.identifier == .quickTimeMetadataPresentationImmersiveMedia {
                let data = try await metadata.load(.dataValue) {
                    presentations.append(
                        try JSONDecoder().decode(PresentationDescriptor.self, from: data)
                    )
                }
            }
        }
    }
    return presentations
}
Create AVMetadataItem from VenueDescriptor swift · at 7:52 ↗ 
func getMetadataItem(from metadata: VenueDescriptor) async throws -> AVMetadataItem {
    let aimeData = try await metadata.aimeData
    let aimeMetadataItem = AVMutableMetadataItem()
    aimeMetadataItem.identifier = .quickTimeMetadataAIMEData
    aimeMetadataItem.dataType = String(kCMMetadataBaseDataType_RawData)
    aimeMetadataItem.value = aimeData as NSData
        
    return aimeMetadataItem
}
Create timed AVMetadataItem from PresentationDescriptorReader swift · at 8:02 ↗ 
func getMetadataItem(reader: PresentationDescriptorReader, 
                     time: CMTime, frameDuration: CMTime) -> AVMetadataItem? {
    let commands = reader.outputPresentationCommands(for: time) ?? []
    if commands.isEmpty { return nil }

    let descriptor = PresentationDescriptor(commands: commands)
    let encodedData = try JSONEncoder().encode(descriptor)
    let presentationMetadata = AVMutableMetadataItem()
    presentationMetadata.identifier = .quickTimeMetadataPresentationImmersiveMedia
    presentationMetadata.dataType = String(kCMMetadataBaseDataType_RawData)
    presentationMetadata.value = encodedData as NSData
    presentationMetadata.time = time
    presentationMetadata.duration = frameDuration
    
    return presentationMetadata
}
Validate AIVU file swift · at 8:20 ↗ 
func validAIVU(file aivuFile: URL) async throws -> Bool { 
    return try await AIVUValidator.validate(url: aivuFile)
}
Save AIME file swift · at 9:31 ↗ 
let aimeFile = FileManager.default.temporaryDirectory.appendingPathComponent("primary.aime")
try? await venueDescriptor.save(to: aimeFile)

Resources