OpenAL on iOS - The 12 Step Program
So you have a problem. It's ok, you're not alone. I am here to help you! This 12 Step Program for… programming, will help you get up and running with OpenAL for your iOS projects.
However, as with all self-help programs, this will help get you started. You will need to continue to work after completing all 12 steps. There are many amazing resources available to you. Here are a few I have used while developing these 12 steps, and I strongly recommend you review them as well.
Background
These are the first tutorials I have ever written, and OpenAL is a fairly complex subject, so why have I started with OpenAL? In my own projects, I need low latency sounds that can be mixed together. Thats it! OpenAL is a very powerful tool and it is used in many games to generate some pretty cool audio effects for moving objects. For me though, effects were not important. I just needed to know that my apps would play a sound at the exact time they needed to, and continue to play when a new sound was initiated. I am the lead developer of the
Musicopoulos series of music education apps. I have developed a number of music apps that needed to play concurrent audio samples at the exact time they are needed. OpenAL gives me the power to do this. If you are looking for a tutorial on how to generate 3D sounds in your game, then look elsewhere. If you too need to play low latency, mixing sounds in your apps, then read ahead.
Review
In the last tutorial, I showed you how to play a sound in OpenAL using the following steps:
- Open a device
- Create and activate a context
- Generate sound sources
- Generate data buffers
- Open your audio data files
- Transfer your audio data to a buffer
- Attach a buffer to a sound source
- Play the audio sample
However, there were some serious problems with the code. Our goal with OpenAL is to play low latency sounds that could be played simultaneously. The sample project did not let us do this. We were loading a single sound into a buffer each time we wanted to play that sound, and each time we play a new sound the original audio was cut off.
Sounds bad, but it's even worse than that! There are a few other housekeeping chores that we didn't tend to that could lead to some serious problems. Do you remember when Steve Jobs announced the original iPhone? It was a phone, an iPod, and an internet device. What this meant from an audio perspective is that all apps, be they native Apple apps or third party apps will be competing for audio. If you don't manage your audio properly, your apps may loose the ability to produce sound.
So lets get into it. Here are the 12 steps you need to not only play audio samples, but get the most out of OpenAL:
- Set up an Audio Session
- Open a device
- Create and activate a context
- Generate sound sources
- Manage a collection of sources
- Open your audio data files
- Transfer your audio data to a buffer
- Generate data buffers
- Manage a collection of data buffers
- Attach a buffer to a sound source
- Play the audio sample
- Clean up and shutdown OpenAL
We will be modifying the code from the previous tutorial. This will leave us with a project that is usable, but still a little limited. I will go into the reasons why at the end of the tutorial. You will be able to download a complete project called SimpleMetronome that you should be able to adapt to your own needs.
So where did we leave off in the last tutorial? We have a class called AudioSamplePlayer that had a single method called playSound.
AudioSamplePlayer.h
#import <Foundation/Foundation.h>
#import <OpenAl/al.h>
#import <OpenAl/alc.h>
#include <AudioToolbox/AudioToolbox.h>
@interface AudioSamplePlayer : NSObject
- (void) playSound;
@end
AudioSamplePlayer.m
#import "AudioSamplePlayer.h"
@implementation AudioSamplePlayer
static ALCdevice *openALDevice;
static ALCcontext *openALContext;
- (id)init
{
self = [super init];
if (self)
{
openALDevice = alcOpenDevice(NULL);
openALContext = alcCreateContext(openALDevice, NULL);
alcMakeContextCurrent(openALContext);
}
return self;
}
- (void) playSound
{
NSUInteger sourceID;
alGenSources(1, &sourceID);
NSString *audioFilePath = [[NSBundle mainBundle] pathForResource:@"ting" ofType:@"caf"];
NSURL *audioFileURL = [NSURL fileURLWithPath:audioFilePath];
AudioFileID afid;
OSStatus openAudioFileResult = AudioFileOpenURL((__bridge CFURLRef)audioFileURL, kAudioFileReadPermission, 0, &afid);
if (0 != openAudioFileResult)
{
NSLog(@"An error occurred when attempting to open the audio file %@: %ld", audioFilePath, openAudioFileResult);
return;
}
UInt64 audioDataByteCount = 0;
UInt32 propertySize = sizeof(audioDataByteCount);
OSStatus getSizeResult = AudioFileGetProperty(afid, kAudioFilePropertyAudioDataByteCount, &propertySize, &audioDataByteCount);
if (0 != getSizeResult)
{
NSLog(@"An error occurred when attempting to determine the size of audio file %@: %ld", audioFilePath, getSizeResult);
}
UInt32 bytesRead = (UInt32)audioDataByteCount;
void *audioData = malloc(bytesRead);
OSStatus readBytesResult = AudioFileReadBytes(afid, false, 0, &bytesRead, audioData);
if (0 != readBytesResult)
{
NSLog(@"An error occurred when attempting to read data from audio file %@: %ld", audioFilePath, readBytesResult);
}
AudioFileClose(afid);
ALuint outputBuffer;
alGenBuffers(1, &outputBuffer);
alBufferData(outputBuffer, AL_FORMAT_STEREO16, audioData, bytesRead, 44100);
if (audioData)
{
free(audioData);
audioData = NULL;
}
alSourcef(sourceID, AL_PITCH, 1.0f);
alSourcef(sourceID, AL_GAIN, 1.0f);
alSourcei(sourceID, AL_BUFFER, outputBuffer);
alSourcePlay(sourceID);
}
@end
1. Set up the Audio Session
While it is not needed to play audio using OpenAL, you need to manage your Audio Session to ensure the user experience is predictable. We are competing for audio on iOS devices. We need to clearly state what we need and when we need it. For example, you have an app that includes some background music as well as sound effects. If a user is listening to their own music through the iPod app when they launch your app what should happen? Should the iPod stop playing? Should your app forgo playing the background music and continue to play the users selection? Let's modify our init method so that we handle our Audio Session appropriately:
- (id)init
{
self = [super init];
if (self)
{
AudioSessionInitialize(NULL, NULL, AudioInterruptionListenerCallback, NULL);
UInt32 session_category = kAudioSessionCategory_MediaPlayback;
AudioSessionSetProperty(kAudioSessionProperty_AudioCategory, sizeof(session_category), &session_category);
AudioSessionSetActive(true);
openALDevice = alcOpenDevice(NULL);
openALContext = alcCreateContext(openALDevice, NULL);
alcMakeContextCurrent(openALContext);
}
return self;
}
We are using AudioSessionInitialize() to initialise our Audio Session. This function takes 4 parameters:
- The run loop that the interruption listener callback should be run on. We are passing NULL to indicate that we want to use the main run loop.
- The mode for the run loop that the interruption listener function will run on. We are passing NULL to indicate that we want to default mode.
- The function we want to call when our Audio Session is interrupted. Our function is called AudioInterruptionListenerCallback.
- Data that you would like to be passed to the interruption callback function.
Do you notice a theme here? When you initialise an Audio Session, you need to specify how your app will behave when it is interrupted. In our case, when the app is interrupted, we are going to call the function AudioInterruptionListenerCallback(). Lets have a look at this methods and see what we need to take care of:
void AudioInterruptionListenerCallback(void* user_data, UInt32 interruption_state)
{
if (kAudioSessionBeginInterruption == interruption_state)
{
alcMakeContextCurrent(NULL);
}
else if (kAudioSessionEndInterruption == interruption_state)
{
AudioSessionSetActive(true);
alcMakeContextCurrent(openALContext);
}
}
Now, when our app is interrupted we need to handle 2 cases:
- kAudioSessionBeginInterruption - What we do when our app is interrupted
- kAudioSessionEndInterruption - What we do when our app is relaunched
Let's discuss the Audio Session first. When your app is interrupted, your Audio Session will automatically deactivated. There is no need to call AudioSessionSetActive(false). However, when the interruption has ended, we need to explicitly state that we want to restore our Audio Session by calling AudioSessionSetActive(true).
We are also managing the OpenAL context within this function. If we think back to the previous tutorial, we described the context as a combination of the person hearing the audio, and the space or air in which the audio travels. Just like our Audio Session, we share the context with other apps. When we are interrupted, we need to give up that context. We do this by calling alcMakeContextCurrent(NULL). When our app is restored, we need to take back the context using the same function, but this time passing in our openALContext variable, alcMakeContextCurrent(openALContext).
2. Open a device and 3. Create and activate a context
In the previous tutorial, we created our device, created our context and made it active. Now that we are managing our context with the Audio Session, we can move on to dealing with sources.
4. Generate sound sources and 5. Manage a collection of sources
In the previous tutorial, we generated a single sound source. This meant that we could only play a single audio sample at any given time. We could not mix our audio samples. With OpenAL on the iPhone, we can generate up to 32 unique sound sources. In order to take advantage of this, we need to manage a collection of sound sources. First, lets define a constant for the maximum number of concurrent sources. Place this at the top of the AudioSamplePlayer.m file, underneath the #import "AudioSamplePlayer.h" statement. Also, declare a static variable that we can use to store our collection of sound sources:
#import "AudioSamplePlayer.h"
#define kMaxConcurrentSources 32
@implementation AudioSamplePlayer
static ALCdevice *openALDevice;
static ALCcontext *openALContext;
static NSMutableArray *audioSampleSources;
In our init method, initialise the audioSampleSources array and generate our sound sources:
- (id)init
{
self = [super init];
if (self)
{
openALDevice = alcOpenDevice(NULL);
openALContext = alcCreateContext(openALDevice, NULL);
alcMakeContextCurrent(openALContext);
audioSampleSources = [[NSMutableArray alloc] init];
NSUInteger sourceID;
for (int i = 0; i < kMaxConcurrentSources; i++) {
/* Create a single OpenAL source */
alGenSources(1, &sourceID);
/* Add the source to the audioSampleSources array */
[audioSampleSources addObject:[NSNumber numberWithUnsignedInt:sourceID]];
}
}
return self;
}
After we initialise the audioSampleSources array, we declare a single sourceID of type NSUInteger. We then go through a loop and use alGenSources() to generate our sound sources, and store the sound source ID in our array. Note, in order to store this number in an Objective-C array we must convert it to an NSNumber object. We now have an array of 32 sound sources that can be used to play 32 concurrent sounds.
6. Open your audio data files, 7. Transfer your audio data to a buffer, 8. Generate data buffers and 9. Manage a collection of data buffers
In the last tutorial, we were loading our file into a buffer each time we wanted to play an audio sample. This created a lag in our sound, something we really don't want. To remedy this, we need to preload all of our audio samples into buffers, and there they will wait until we are ready to play them. Preloading our audio samples into buffers is how we achieve low latency sounds. Now, instead of doing this in our playSound() method, we need to create an new method that takes the name of a sound and loads it into a buffer. We then need to maintain a collection of buffers that we can select when we want to play an specific sound. You can preload up to 256 buffers, so lets start by defining another constant and declaring a collection for storing our data buffers:
#import "AudioSamplePlayer.h"
#define kMaxConcurrentSources 32
#define kMaxBuffers 256
@implementation AudioSamplePlayer
static ALCdevice *openALDevice;
static ALCcontext *openALContext;
static NSMutableArray *audioSampleSources;
static NSMutableDictionary *audioSampleBuffers;
We are using an NSMutableDictionary to store our buffers so that we can provide a key and get the related buffer ID. Our key will simply be the name of our audio sample. Now, lets create a method that takes the name of an audio sample and loads it into a buffer:
- (void) preloadAudioSample:(NSString *)sampleName
{
if ([audioSampleBuffers objectForKey:sampleName])
{
return;
}
if ([audioSampleBuffers count] > kMaxBuffers) {
NSLog(@"Warning: You are trying to create more than 256 buffers! This is not allowed.");
return;
}
NSString *audioFilePath = [[NSBundle mainBundle] pathForResource:sampleName ofType:@"caf"];
AudioFileID afid = [self openAudioFile:audioFilePath];
UInt32 audioFileSizeInBytes = [self getSizeOfAudioComponent:afid];
void *audioData = malloc(audioFileSizeInBytes);
OSStatus readBytesResult = AudioFileReadBytes(afid, false, 0, &audioFileSizeInBytes, audioData);
if (0 != readBytesResult)
{
NSLog(@"An error occurred when attempting to read data from audio file %@: %ld", audioFilePath, readBytesResult);
}
AudioFileClose(afid);
ALuint outputBuffer;
alGenBuffers(1, &outputBuffer);
alBufferData(outputBuffer, AL_FORMAT_STEREO16, audioData, audioFileSizeInBytes, kSampleRate);
[audioSampleBuffers setObject:[NSNumber numberWithInt:outputBuffer] forKey:sampleName];
if (audioData)
{
free(audioData);
audioData = NULL;
}
}
We discussed how to create buffers in the last tutorial, so here we will concentrate on how to manage a collection of buffers. Our preLoadAudioSample method takes a single parameter, the name of an audio sample. Now, the name of the audio sample should be stripped on any file type information. For example, if you want to play an audio sample called laser.caf, we will only be passing 'laser' into this method.
The first thing we do is check and make sure this sample hasn't already been added to the buffer. We need to make sure we are not using more memory than we need to! Then we check and make sure we are not trying to create more than 256 buffers. If all goes well, we open the file, read in its audio data and copy it to the buffer, just like we did in the last tutorial.
Now, when we generate our buffer ID, outputBuffer, we are storing this in our AudioSampleBuffers dictionary using the name of the audio sample as the key. This will make the task of playing our sample much easier.
In addition to adding our buffer to the audioSampleBuffers dictionary, there is also 2 new helper methods, openAudioFile and getSizeOfAudioComponent. While not necessary, this helps make a code a little more presentable and easier to read. Here is the new helper method:
- (AudioFileID) openAudioFile:(NSString *)audioFilePathAsString
{
NSURL *audioFileURL = [NSURL fileURLWithPath:audioFilePathAsString];
AudioFileID afid;
OSStatus openAudioFileResult = AudioFileOpenURL((__bridge CFURLRef)audioFileURL, kAudioFileReadPermission, 0, &afid);
if (0 != openAudioFileResult)
{
NSLog(@"An error occurred when attempting to open the audio file %@: %ld", audioFilePathAsString, openAudioFileResult);
}
return afid;
}
This method takes a file path as a string, opens the audio sample and returns the AudioFileID.
- (UInt32) getSizeOfAudioComponent:(AudioFileID)afid
{
UInt64 audioDataSize = 0;
UInt32 propertySize = sizeof(UInt64);
OSStatus getSizeResult = AudioFileGetProperty(afid, kAudioFilePropertyAudioDataByteCount, &propertySize, &audioDataSize);
if (0 != getSizeResult)
{
NSLog(@"An error occurred when attempting to determine the size of audio file.");
}
return (UInt32)audioDataSize;
}
This method takes an AudioFileID and returns the size of the audio data as a UInt32.
10. Attach a buffer to a sound source and 11. Play the audio sample
So we have now generated our sources and preloaded our sounds into buffers. Its time to attach a buffer to a source and play an audio sample! The old playSample method is not longer sufficient. We need to provide the name of the audio sample we would like to play. Here is the replacement method you use to do this:
- (void) playAudioSample:(NSString *)sampleName
{
ALuint source = [self getNextAvailableSource];
alSourcef(source, AL_PITCH, 1.0f);
alSourcef(source, AL_GAIN, 1.0f);
ALuint outputBuffer = (ALuint)[[audioSampleBuffers objectForKey:sampleName] intValue];
alSourcei(source, AL_BUFFER, outputBuffer);
alSourcePlay(source);
}
The first thing we need to do is get one of the sources from our audioSampleSources array. We are using the helper method, getNextAvailableSource, to do this. Here is the method:
- (ALuint) getNextAvailableSource
{
ALint sourceState;
for (NSNumber *sourceID in audioSampleSources) {
alGetSourcei([sourceID unsignedIntValue], AL_SOURCE_STATE, &sourceState);
if (sourceState != AL_PLAYING)
{
return [sourceID unsignedIntValue];
}
}
ALuint sourceID = [[audioSampleSources objectAtIndex:0] unsignedIntegerValue];
alSourceStop(sourceID);
return sourceID;
}
The source ID's are store in an array. What we need to do is move through that array and locate the first source that isn't currently playing an audio sample. We do this by querying the source for its state, using the function alGetSourcei() which takes the following parameters:
- A source ID (remember, we stored our source ID's as NSNumbers, so we now need to called unsighedIntValue to retrieve the source ID)
- What information we are after, in this case we want the AL_SOURCE_STATE.
- A pointer to an ALint in which the source state will be stored
If we find a source that is not currently playing an audio sample, we return the source ID. Note, if we find that all the sources are currently being used, we will return the first source ID in the array. This means that any audio sample being played through this source will be cut off.
Back to our playAudioSample method, once we have our source, we then set some parameters for that source. In our case, we are setting the pitch and gain to 1.0. This method can easily be modified to take float values for these parameters, which you can see in the sample project.
Next, we retrieve the buffer for our audio sample. The buffer was stored in a dictionary using the audio sample name as the key. We retrieve the buffer ID using this key, convert it to and int value and finally casting it to be of type ALuint.
Now we have our source and our buffer, we can attach the buffer to the source using the function alSourceI(). Finally, we play our audio sample using the function alSourcePlay().
12. Clean up and shutdown OpenAL
This is our final step. When we have finished using OpenAL, we need to clean up and shut it down.
- (void) shutdownAudioSamplePlayer
{
ALint source;
for (NSNumber *sourceValue in audioSampleSources)
{
NSUInteger sourceID = [sourceValue unsignedIntValue];
alGetSourcei(sourceID, AL_SOURCE_STATE, &source);
alSourceStop(sourceID);
alDeleteSources(1, &sourceID);
}
[audioSampleSources removeAllObjects];
NSArray *bufferIDs = [audioSampleBuffers allValues];
for (NSNumber *bufferValue in bufferIDs)
{
NSUInteger bufferID = [bufferValue unsignedIntValue];
alDeleteBuffers(1, &bufferID);
}
[audioSampleBuffers removeAllObjects];
alcDestroyContext(openALContext);
alcCloseDevice(openALDevice);
}
First, we locate our source, stop it if it is playing an audio sample then delete the source. We then remove all the objects from the audioSampleSources array. Next, we locate and delete all the buffers. To do this, we are extracting and array off all the values in the audioSampleBuffers dictionary (we are not interested in the keys here as the buffer ID's are in the values), then we move through that array and delete all the buffers. We then remove all the objects from the audioSampleBuffers dictionary. Finally, we destroy the context and close the device.
And our 12 Step Program is complete. Hopefully you now have a better understanding of OpenAL and how to use it in your iOS projects. Remember, these 12 steps are just the beginning. You will need to continue to search for new information and try new techniques. It is the only way to stay on top of it.
The Sample Project
The code we have gone through today will work just fine in most cases, but it still has a some problems. Our biggest problem is that our AudioSamplePlayer class can be initialised from anywhere in your application. This means you could have more than 1 instance of AudioSamplePlayer. Well, actually, thats not true. If you try to create more than 1 of these objects you app will crash! Why? Remember the beginning of the first tutorial? We can only have a single context and a single device. If you try to create more than one of these your app is doomed!
The solution is to make AudioSamplePlayer a singleton class. That way you can call it from anywhere you choose and there will only every be a single instance of AudioSamplePlayer. The sample project includes AudioSamplePlayer as a singleton.
The sample project is a very simple metronome. I choose this genre because metronomes are all about keeping time. We need to make sure our sounds are being played with low latency. Lag is not acceptable! The project is full of comments, so it should be clear and easy to understand. Feel free to ask any questions if you are not sure about something.
