5. White Papers | Studio API 3D Events

Studio 3D Events

This section will introduce you to using 3D sound with FMOD Studio events.

Coordinate Systems and Handedness

FMOD Studio shares the same coordinate system as the core API. See the 3D Sounds white paper for details.

Updating orientations

The programmer needs to call Studio::System::setListenerAttributes once per frame for the listener, and to update 3D events with Studio::EventInstance::set3DAttributes. It is important to update all orientations before calling Studio::System::update. If some orientations are set before Studio::System::update and some are set afterwards, then some frames may end up having old positions relative to others. This is particularly important when both the listener and the events are moving fast and together - if there are frames where the listener moves but the event does not it becomes very noticeable.

FMOD Spatializer

FMOD Studio supports spatializing events by placing an FMOD spatializer effect on the event master track. It is possible to use other sorts of spatializers by replacing the FMOD spatializer with a different type of effect, for example a third party spatializer.

FMOD Object Spatializer

The object spatializer is a special type of spatializer that interfaces with object-based output modes such as Dolby Atmos. These output modes accept mono signals with a 3D position and do their own spatialization and mixing to the final speaker configuration. To use object spatializers, the programmer has to specify an output mode that supports object based spatialization, otherwise the signal will be mixed down at the final stage by FMOD.

The benefit of the object spatializer is that it allows the sound designer to leverage object-based technologies. It does come at a cost, however, as the signal leaves the mix at the object spatializer and does not receive the benefit of DSP effects on parent buses like signals run though normal spatializers do. The object spatializer automatically bases its volume on the combined volumes of parent buses for basic mixing, but no complex effects can be used. For this reason, the mix has to be set up very carefully with knowledge of the limitations of the object spatialization.

It is possible for the sound designer to use a mixture of normal 3D spatialized events and object-spatialized 3D events. Normal events will have signal going through the mixer hierarchy, and object-based events will have signal that leaves the mix at the master track. As far as the programming API goes, both kinds of events are treated exactly the same.

Automatic Parameters

FMOD Studio supports setting automations based on parameters that automatically update based on position. For example, the sound designer could add a volume automation based on Distance, with a 2D panning that is automated on the Direction parameter. The event is still considered 3D in that case, even if it has no spatializer on the master track.

An event may have both a spatializer on the master track, as well as an automation based on a Distance parameter. As the event and listener moves, both the spatializer and the automation will be updated.

Multiple Listeners

FMOD Studio supports multiple listeners. Call Studio::System::setNumListeners to set the number of listeners, and use Studio::System::setListenerAttributes to set the orientations for listeners, with an index for the listener.

Studio Spatialization for Multiple Listeners

Consider the case of an event with three nearby listeners. In this case, listener A is slightly closer to the event than B, and C is the furthest away, outside the max distance of the event.

The Studio spatializer will take listener A and B into account. The gain will be based off the closest listener distance (in this case, the distance to listener A). Listener B will have an effect on the spatialization. However, both A and B agree that the event is to the front, so the final pan matrix will be towards the front speakers. Listener C has no effect on the calculation since it is out of range.

Consider this case where listener A and B have moved and now the event is to the right of A and to the left of B. In this case, the gain will be based of the closest listener distance (which is B), but the pan matrix will have signal out of both the left and the right since both listeners have an effect on the mix. If A moved further away then the contribution of A would diminish and to the signal would start to come more out of the left speakers. If A moved further enough away, the signal would smoothly interpolate to just B having an influence on the spatialization.

Studio Spatialization for Listener Weights

Listener weights can be set using Studio::System::setListenerWeight. This allows listeners to fade in and out of existence, as well as to allow cross-fading of listeners to a new position. In the following picture, we have 4 listeners. Listener C is out of range so it has no influence, and listener D has 0% weighting so it has no influence either. The remaining two listeners have a weight of 40% and 60%. In this example, perhaps the camera is teleporting to a new position and the game is smoothly interpolating to a new orientation.

The gain is a weighted average between A and B, so it is equivalent to having a distance somewhere between the two listeners. The spatialization of the signal is a mixture of A and B. A is further away and has a lower weight, so the biggest contribution is due to B, meaning the signal sounds mostly in the front speakers. If you imagine blending from A to B, the signal will smoothly interpolate from the back speakers to the front and get louder when the weights scale from A to B.

Listener Mask

Events can have a mask that specifies which listeners are active for that event. By default all listeners apply to all events. By calling Studio::EventInstance::setListenerMask, some listeners can be disabled for that event so that they have no influence on the panning. This could be used to group some events and listeners together and have that set only affected by that one listener. When performing the calculation above, any listener not included in the mask is ignored and is as if it does not exist for that event. It is an error to set a combination of mask and weight such that no listener is active for an event.

Doppler

FMOD events support doppler. The sound designer specifies doppler on a per event basis with a scale, so some events may be affected less than others. Doppler is calculated using listener and event velocities; it is up to the programmer to correctly specify velocity values using Studio::EventInstance::set3DAttributes and Studio::System::setListenerAttributes. The scale of doppler can be specified at initialization time using System::set3DSettings.

For the case of multiple listeners, the doppler is based on the closest listener. If listener has a weight then it is a combination of the closest listeners up to 100%. For example if there were three listeners at increasing distance with weight of 60%, 60% and 60%, then the doppler would be calculated from 60% of the first listener, 40% of the second, and 0% of the third.

Automatic Parameters and Multiple Listeners

If there are multiple listeners, an FMOD Studio event's automatic parameters are based on the closest listener. If the closest listener has a weight, then the event's automatic parameters instead use a combination of the closest listeners, up to a total weight of 100%. For example, if there are three listeners at increasing distance with weights of 60%, 60% and 60%, then the automatic parameters would be calculated from 60% of the first listener, 40% of the second, and 0% of the third.

Interface with Core API

When calling Studio::System::setNumListeners and Studio::System::setListenerAttributes, there is no need to call the equivalent Core API functions System::set3DNumListeners and System::set3DListenerAttributes, as FMOD Studio passes the information into the Core API automatically. This means it is possible to have a mixture of FMOD Studio 3D events and Core API 3D Channels playing at the same time.