FMOD Engine User Manual 2.03
Running FMOD as HTML5 on the web is made possible by having a web server host and deliver the JavaScript content to the client web browser. In a production environment this would be hosted using a professional web server such as Apache, but during development we recommend an easy to set up local server such as Web Server for Chrome
FMOD is compiled using the following tools.
FMOD supports the following minimum browser versions.
The web presents a unique set of challenges that make this implementation less than perfect when compared to other platforms supported by the FMOD Engine.
One significant disadvantage is the lack of multi-threading, or more specifically, limited browser support of features such as SharedArrayBuffer that prevent our toolchain from implementing pthread support. This limitation means we must run mixing, streaming and loading all in the game thread requiring added latency to hide the additional CPU cost.
Another disadvantage of the web platform is the lack of networking sockets. There are alternatives provided by our toolchain, however they require additional servers running as a proxy. This limitation means we cannot implement Live Update and Profiling features between the running code and FMOD Studio. Additionally this means we cannot support internet streaming of audio (netstreams).
Many platforms benefit from targeted instruction level optimization known as SIMD (single instruction multiple data), these optimizations allow FMOD to perform several times faster than with standard C code instructions alone. Unfortunately development of SIMD for the web is an ongoing process and not ready for us to consume. This coupled with the fact the native FMOD C code has been converted to run in a browser means you will experience worse performance than the equivalent application running natively (outside the browser).
There are a large number of binaries for HTML5 covering different runtime formats and compatibility.
The folder structure we use represents the choices you must make following this pattern: /api/{fmod-api}/lib/{binary-format}/*
{fmod-api} Select either core or studio depending on which API you wish to use. For HTML5 (unlike other platforms) studio already contains the core library so including the core library separately is not necessary.{binary-format} Select wasm for the most performant feature-rich version targeting modern browsers or js for compatibility with older browsers.
To include FMOD using JavaScript in a html file, refer to the following example in your HTML. Filename will vary based on what HTML technology or logging/reduced variant you might be using.
Core API Only:
<script type="text/javascript" src="fmod.js"></script>
Studio API:
<script type="text/javascript" src="fmodstudio.js"></script>
Select from below for your technology choice, and refer to the table for the folder the FMOD libraries are located in, and any accompanying files that must be copied in with them to your project folder.
Faster (approximately 30%) and uses half the memory of Javascript, but requires the web browser to support the WASM technology, and also a web server that understands the .wasm 'application/wasm' mime type.
Core API libraries - Located in /api/core/lib/wasm/
| Core API file | Companion file (put in same folder) | Description |
|---|---|---|
| fmod.js | fmod.wasm | Release library for production code |
| fmodL.js | fmodL.wasm | Release library with logging output, use for debugging |
| fmod_reduced.js | fmod_reduced.wasm | Smaller release library for production code, with reduced features |
Studio API libraries - Located in /api/studio/lib/wasm/. The Studio API files already contain the Core API, so you do not need to include the Core API files as well.
| Studio API file | Companion file (put in same folder) | Description |
|---|---|---|
| fmodstudio.js | fmodstudio.wasm | Release library for production code. Includes fmod_reduced.js |
| fmodstudioL.js | fmodstudioL.wasm | Release library with logging output, use for debugging |
Older Javascript compiled. Slower and uses more memory, but best for compatibility.
Core API libraries - Located in /api/core/lib/js/
| Core API file | Companion file (put in same folder) | Description |
|---|---|---|
| fmod.js | fmod.js.mem | Release library for production code |
| fmodL.js | fmodL.js.mem | Release library with logging output, use for debugging |
| fmod_reduced.js | fmod_reduced.js.mem | Smaller release library for production code, with reduced features |
Studio API libraries - Located in /api/studio/lib/js/. The Studio API files already contain the Core API, so you do not need to include the Core API files as well.
| Studio API file | Companion file (put in same folder) | Description |
|---|---|---|
| fmodstudio.js | fmodstudio.js.mem | Release library for production code. Contains fmod_reduced.js |
| fmodstudioL.js | fmodstudioL.js.mem | Release library with logging output, use for debugging. Contains fmodL.js |
For using Emscripten with your own C/C++ code to convert to JavaScript. W32 is an intermediate format.
Core API libraries - Located in /api/core/lib/w32/
| Core API file | Description |
|---|---|
| fmod.a | Release binary for production code |
| fmodL.a | Release binary with logging enabled for development |
| fmod_reduced.a | Smaller release binary with reduced features, for production code. |
Studio API libraries - Located in /api/studio/lib/w32/. The Studio API files already contain the Core API, so you do not need to include the Core API files as well.
| Studio API file | Description |
|---|---|
| fmodstudio.a | Release binary for production code. Contains fmod_reduced.a |
| fmodstudioL.a | Release binary with logging enabled for development. Contains fmodL.a |
Note: The fmodstudio versions of the library include the fmod_reduced version of the core, so accessing things like .ogg/.mp3 (see reduced ) through Studio::System::getCoreSystem are not supported. Use the fmodstudioL version to get full access to all features.
If you have a C/C++ program that uses FMOD and are converting it to JavaScript using Emscripten, you will need to link the relevant FMOD binaries as per the Emscripten Documentation, and ensure the following flag is added in your Emscripten compilation arguments:
-s EXPORTED_RUNTIME_METHODS=ccall,cwrap,setValue,getValue
The FMOD engine supports two FMOD_OUTPUTTYPE modes for Web Audio.
To start with FMOD in a JavaScript environment you must initialize a global FMOD object, which you can then call FMOD functions with.
This involves declaring a variable (ie 'FMOD') then calling a constructor function on it ( 'ie FMODModule(FMOD); )
Before this constructor is called, some optional user callbacks can be set for pre / post emscripten initialization, as well as the FMOD memory block size for ASM.JS users.
var FMOD = {}; // FMOD global object which must be declared to enable 'main' and 'preRun' and then call the constructor function.
FMOD['preRun'] = prerun; // Will be called before FMOD runs, but after the Emscripten runtime has initialized
FMOD['onRuntimeInitialized'] = main; // Called when the Emscripten runtime has initialized
FMOD['INITIAL_MEMORY'] = 64*1024*1024; // (ASM.JS ONLY) FMOD Heap defaults to 16mb which is enough for this demo, but set it differently here for demonstration (64mb)
FMODModule(FMOD); // Calling the constructor function with our object
Most browsers have a user interaction requirement, or audio will not be audible.
This was implemented to stop unscrupulous websites from auto playing audio in things like advertisements.
FMOD output will become audible upon detecting a user interaction.
fmod_reduced.js and fmodstudio.js feature removal
The fmod_reduced library is a smaller subset of features. The following is a list of what is removed.
fmodstudio.js includes fmod_reduced.js by default. To get the full feature set use fmodstudioL.js instead.
The following are removed in favour of .FSB only. .FSB support includes vorbis, FADPCM and pcm compression formats.
The following are removed due to relative expensiveness of the effect and usage rates being below other types of effects.
The following features have been removed
This is an important section if coming from knowledge of a C/C++ background of the FMOD API.
Javascript parameters are passed by value, therefore when you pass one to a function, it makes the concept of a 'getter' function difficult.
The variable's value cannot be changed by the function from the caller's perspective, but it can add a new member to the variable, which is the mechanism FMOD always uses when 'getting' data from a function.
In C the variable can be altered as it would be passed by reference (using pointers).
In FMOD for JavaScript, the variable you pass in gets a new member called val which contains the new data.
i.e.
var outval; // generic variable to reuse and be passed to FMOD functions.
var name; // to store name of sound.
sound.getName(outval);
name = outval.val; // 'val' contains the data. Pass it to the variable we want to keep.
console.log(name);
All FMOD functions that produce data in a variable after calling a function return data this way.
In FMOD for JavaScript the enumerated values are members of the object declared at the top of the file.
As this would normally be a variable called 'FMOD' it would be redundant to keep the FMOD_ as part of the constant name, so it is removed to avoid duplication.
For example, instead of
FMOD.FMOD_OK
in FMOD for HTML5 it becomes:
FMOD.OK
similarly
FMOD_INIT_NORMAL
FMOD_DEFAULT
FMOD_ERR_FILE_NOTFOUND
FMOD_GUID
FMOD_3D_ATTRIBUTES
etc
becomes
FMOD.INIT_NORMAL
FMOD.DEFAULT
FMOD.ERR_FILE_NOTFOUND
FMOD.GUID()
etc
Not that for attributes that start with a number after the FMOD namespace, it has to start with an underscore.
For example.
FMOD._3D
FMOD._2D
FMOD._3D_ATTRIBUTES()
In the above example you may notice that () is used to 'construct' a javascript object which represents a C structure in the FMOD API.
When using a structure to pass information to FMOD, a helper/constuctor function must be called to create the structure before using it/filling in its members, so that FMOD can understand what is being passed to it.
If these constructor functions are not used, the function it is being passed to will probably result in a 'binding' error (in the browser debug/log console/window).
var guid = FMOD.GUID();
var info = FMOD.STUDIO_BANK_INFO();
Some API constructs are redundant for JavaScript, so are removed from the API.
Examples are 'cbsize' inside a struct. The JavaScript version of a struct already knows its own size, so it is removed/redundant here.
Structures like FMOD_CREATESOUNDEXINFO, FMOD_ADVANCEDSETTINGS and FMOD_STUDIO_ADVANCEDSETTINGS have these members, it can just be left out.
FMOD lets you load data from the host in a few different ways, depending on your setup.
FMOD exposes an emscripten mechanism to mount/pre-load files in the 'prerun()' function, that is described above in the "Essential Startup Information" section of this document.
Call FMOD.FS_createPreloadedFile to register your files so that FMOD can use filenames in file related functions. See Emscripten docs for more on this function.
For example the playsound example
// Will be called before FMOD runs, but after the Emscripten runtime has initialized
// Call FMOD file preloading functions here to mount local files. Otherwise load custom data from memory or use own file system.
function prerun()
{
var fileUrl = "/public/js/";
var fileName;
var folderName = "/";
var canRead = true;
var canWrite = false;
fileName = [
"dog.wav",
"lion.wav",
"wave.mp3"
];
for (var count = 0; count < fileName.length; count++)
{
FMOD.FS_createPreloadedFile(folderName, fileName[count], fileUrl + fileName[count], canRead, canWrite);
}
}
Then later in your app you can simply reference a file by path/filename.
result = gSystem.createSound("/lion.wav", FMOD.LOOP_OFF, null, outval);
CHECK_RESULT(result);
If you have raw data in memory that you want to pass to FMOD , you can. Warning though, javascript passes data by value, so the memory usage will temporarily double for the sound data when it is passed to fmod.
The load_from_memory has an example of this
var chars = new Uint8Array(e.target.result);
var outval = {};
var result;
var exinfo = FMOD.CREATESOUNDEXINFO();
exinfo.length = chars.length;
result = gSystem.createStream(chars.buffer, FMOD.LOOP_OFF | FMOD.OPENMEMORY, exinfo, outval);
CHECK_RESULT(result);
If you have a file system you can use file callbacks to load data into FMOD. See System::setFileSystem, or Studio::System::loadBankCustom
In load_bank example, there is a demonstration of this
var info = new FMOD.STUDIO_BANK_INFO();
info.opencallback = customFileOpen;
info.closecallback = customFileClose;
info.readcallback = customFileRead;
info.seekcallback = customFileSeek;
info.userdata = filename;
result = gSystem.loadBankCustom(info, FMOD.STUDIO_LOAD_BANK_NONBLOCKING, outval);
CHECK_RESULT(result);
In this case the filename is passed as userdata to let the callback get information for what file to load.
Here is the customFileOpen callback getting the filename and opening a file handle.
The file is opened in this case with a special FMOD provided file API. Replace this with your own API.
function customFileOpen(name, filesize, handle, userdata)
{
var filesize_outval = {};
var handle_outval = {}
// We pass the filename into our callbacks via userdata in the custom info struct
var filename = userdata;
var result = FMOD.file_open(gSystemLowLevel, filename, filesize_outval, handle_outval)
if (result == FMOD.OK)
{
filesize.val = filesize_outval.val;
handle.val = handle_outval.val;
}
return result;
}
To read data in the callback, into the buffer that FMOD has provided, FMOD has used a special technique where it passes the memory address, instead of a javascript buffer.
This means to write data to FMOD's buffer you must use FMOD.setValue using the buffer address, writing the data in a loop, a byte at a time. This could be optimized somewhat by using i32 to write 4 bytes at a time.
function customFileRead(handle, buffer, sizebytes, bytesread, userdata)
{
var bytesread_outval = {};
var buffer_outval = {};
// Read from the file into a new buffer. This part can be swapped for your own file system.
var result = FMOD.file_read(handle, buffer_outval, sizebytes, bytesread_outval) // read produces a new array with data.
if (result == FMOD.OK)
{
bytesread.val = bytesread_outval.val;
}
// Copy the new buffer contents into the buffer that is passed into the callback. 'buffer' is a memory address, so we can only write to it with FMOD.setValue
for (count = 0; count < bytesread.val; count++)
{
FMOD.setValue(buffer + count, buffer_outval.val[count], 'i8'); // See https://kripken.github.io/emscripten-site/docs/api_reference/preamble.js.html#accessing-memory for docs on setValue.
}
return result;
}
FMOD comes with some functions to aid with reading file data and writing to memory buffers.
Here is the list of functions that are provided.
FMOD['preRun'] = callback // Variable to assign a function callback to. Will be called before FMOD runs, but after the Emscripten runtime has initialized
FMOD['onRuntimeInitialized'] = callback // Assign a function to this member of FMOD, pre Called when the Emscripten runtime has initialized **
FMODModule(fmodobject) // Constructor function to set up FMOD object before use. fmodobject = the main FMOD object you will use throughout the application lifetime.
FMOD.FS_createPreloadedFile // Mounts a local file so that FMOD can recognize it when calling a function that uses a filename (ie loadBank/createSound). See https://emscripten.org/docs/api_reference/Filesystem-API.html#FS.createPreloadedFile for more on this function.
FMOD.System_Create // Only exported in FMOD.JS or FMODL.JS (Only use this function if using pure Core API). See docs for System_Create
FMOD.Studio_System_Create // Only exported in FMODSTUDIO.JS or FMODSTUDIOL.JS (Only use this function if using Studio API). See docs for Studio::System_Create
FMOD.ReadFile // Read the entire contents of a file into a memory variable, as preloaded by FMOD.FS_createPreloadedFile. Call FMOD.Memory_Free on the variable after using it. NOTE: output variable has 'val' and 'length' as the JS members to use as parameters to System::loadBankMemory.
FMOD.Memory_Free // Free memory allocated by FMOD internally in FMOD.ReadFile
FMOD.file_open
FMOD.file_close
FMOD.file_read
FMOD.file_seek
FMOD.setValue // See https://kripken.github.io/emscripten-site/docs/api_reference/preamble.js.html#accessing-memory for docs on setValue.
You should be able to port existing C code with no special HTML5 related functionality. Link the fmod.bc file or the fmodstudio.bc file (not both!) into your application to get it to link.
As mentioned previously FMOD_NONBLOCKING will not be needed, so your program might hang if System::update is not processed during any logic that waits on a getState function.
By default FMOD mixes at 48000hz. If the browser is not running at that rate, it will introduce a resampler to convert the rate, which consumes CPU time and adds a DSP block worth of latency.
This can be solved by querying the hardware's rate before System::init, and calling System::setSoftwareFormat to the same rate as the output. Here is an example (from the PlaySound example):
var outval = {};
result = gSystem.getDriverInfo(0, null, null, outval, null, null);
CHECK_RESULT(result);
result = gSystem.setSoftwareFormat(outval.val, FMOD.SPEAKERMODE_DEFAULT, 0)
CHECK_RESULT(result);
Some devices cannot handle the default buffer size of 4 blocks of 1024 samples (4096 samples) without stuttering, so to avoid this set the buffer size in your application to 2 blocks of 2048.
Here is an example
result = gSystem.setDSPBufferSize(2048, 2);
CHECK_RESULT(result);
As stated in the introduction, FMOD for HTML5 does not have access to any threads so any loading/mixing/decoding/streaming/dsp has to be run in a blocking mode, from the main application loop.
Keep this in mind when loading sounds, or implementing DSP that may take a large time slice. If the application pre-loads sounds, and has a fast enough framerate that the FMOD mixing can execute in the same frame (with time left over) then there should be no audible glitching or frame rate glitching.
FMOD defaults to 16mb of memory to load sounds with and create FMOD objects with. Use the following global before calling the main FMOD constructor object, to control how much memory to use.
FMOD['INITIAL_MEMORY'] = 64*1024*1024; // (ASM.JS ONLY) FMOD Heap defaults to 16mb which is enough for this demo, but set it differently here for demonstration (64mb)
For WASM support, FMOD uses ALLOW_MEMORY_GROWTH, which will allow memory to grow dynamically from INITIAL_MEMORY up to a maximum of 2GB.
FMOD_NONBLOCKING and FMOD_STUDIO_LOAD_BANK_NONBLOCKING do not allow sound to load in the background, but it will process loading from the update loop instead.
Streaming is executed from the main thread. This may impact performance, so pre-loading sounds is preferred.
Streaming infers that the source is being loaded from 'disk' which is not usually done with HTML5, because data is usually pre-packaged with the javascript file, or streamed over http (from an external URL) so loading sounds into memory might make more sense here.
The following functions are not supported.
Some have no place in JavaScript (ie loadPlugin) but others are incomplete at this point and can be added in a later point release.
The following callbacks remain unimplemented at this point, so they will not work.