Dunfey · Hotel WWDC as data, est. 1983
Front desk everything
Years
Topics

WWDC21 · 19 min · AI & Machine Learning / SwiftUI & UI Frameworks / Audio & Video

Discover built-in sound classification in SoundAnalysis

Explore how you can use the Sound Analysis framework in your app to detect and classify discrete sounds from any audio source — including live sounds from a microphone or from a video or audio file — and identify precisely in a moment where that sound occurs. Learn how the built-in sound classifier makes it easy for you to identify over 300 different types of sounds without the need for a custom trained model. This includes a variety of noises, ranging from human sounds, musical instruments, animals, and various items. For custom models, see how you can leverage the Audio Feature Print feature extractor to create smaller models with variable sound window control to better serve your app’s purposes. For more about Sound Classification and the Sound Analysis framework, watch “Training Sound Classification Models in Create ML” from WWDC19.

Watch at developer.apple.com ↗

Transcript all transcripts

Code shown on screen · 3 snippets

Get list of recognized sounds swift · at 8:12 ↗ 
func getListOfRecognizedSounds() throws -> [String] {
    let request = try SNClassifySoundRequest(classifierIdentifier: .version1)
    return request.knownClassifications
}
Create sound classification request swift · at 9:19 ↗ 
let request = try SNClassifySoundRequest(classifierIdentifier: .version1)

let analyzer = try SNAudioFileAnalyzer(url: url)

var observer: SNResultsObserving // TODO

try analyzer.add(request, withObserver: observer)
analyzer.analyze()
Implement sound classification observer swift · at 9:52 ↗ 
class FirstDetectionObserver: NSObject, SNResultsObserving {
    var firstDetectionTime = CMTime.invalid
    var label: String
    
    init(label: String) {
        self.label = label
    }
    
    func request(_ request: SNRequest, didProduce result: SNResult) {
        if let result = result as? SNClassificationResult,
           let classification = result.classification(forIdentifier: label),
           classification.confidence > 0.5,
           firstDetectionTime == CMTime.invalid {
                firstDetectionTime = result.timeRange.start
        }
    }
}

Resources