Rubber Band Library

Integration notes and examples

Conceptual notes
Which options to use
Common problems and queries
Examples

Conceptual notes

A time-stretcher such as Rubber Band Library has to be handled a little differently from a typical audio effect.

When speeding up or slowing down audio, there is by definition a difference between the input and output "rates" of the processor. If you supply N samples as input to the processor, then you will get more than N (when slowing down) or fewer (when speeding up) as output.

This must be the case for any audio time stretcher library. But on top of this, with Rubber Band the effective rate can also differ locally depending on the locations of detected features such as percussive transients.

This means that the input and output rates are not always in a stable ratio: the ratio you set behaves a little bit like a long-term average, rather than a fixed ratio for every processing block.

A consequence of this is that you cannot just provide a certain number of samples at the input and expect to see a fixed number of samples at the output. This is true even if you are using the processor only for pitch-shifting, and in fact can be the case even if you have the time and pitch ratios both set to 1.

After supplying samples to the input with process, you must always query how many samples have now been made available at output using available. See this discussion for more details of how that might work when pitch-shifting, as well as the examples below.

For real-time use especially, it may be simplest to use Rubber Band if you are able to integrate it using a "pull" configuration rather than a "push" one, so that the number of samples you supply to it (for a given number of output samples) is allowed to vary.

That is:

you have some sort of time-processing object, which holds a Rubber Band stretcher object;
your audio output driver or graph requests a certain number of samples from it;
your time-processing object checks how many samples it can get from Rubber Band already (available), as there may be some in its internal buffers;
if that is not enough:
- it queries how many the Rubber Band stretcher needs at input before it can produce more at output (getSamplesRequired);
- it calls back into its audio source to obtain that number of samples;
- it runs the stretcher (process), receives the available output (retrieve), and repeats if necessary
and the time-processing object then returns the accumulated output samples to the audio output.

This structure can be seen at work in the Mini Example with Qt and Sonic Visualiser code examples below.

Which options to use

Rubber Band has many configurable options, which you select by passing a bitwise "or" of RubberBandStretcher::Option constants to the constructor of the stretcher.

The number of possibilities can be daunting, but the good news is that — once you have decided between offline and real-time processing modes, a choice normally dictated by the application architecture — the stretcher will work tolerably well with almost any combination of options.

So you can integrate it without messing about too much, then tweak it afterwards.

That said, here are some rules of thumb for ideal option selection using Rubber Band 3.2 or newer:

Start by choosing offline or real-time mode, with either OptionProcessOffline or OptionProcessRealTime . Read about the difference here.
This is the only essential decision.
If CPU power is not a major limitation, add OptionEngineFiner | OptionChannelsTogether ;
- or if CPU power is limited and you are dealing with percussive material, instead add OptionEngineFaster ;
- or if CPU power is limited and you are dealing with vocals, acoustic recordings, speech, or situations where you know nothing about the audio material, instead add the combination OptionEngineFiner | OptionChannelsTogether | OptionWindowShort .
If you are time-stretching without pitch-shifting, stop there.
- or if you are pitch-shifting with dynamically-changing ratios, add OptionPitchHighConsistency ;
- or if you are pitch-shifting with fixed ratios and CPU power is not a major limitation, instead add OptionPitchHighQuality .
- If you are pitch-shifting vocals, or mixes including vocals, and wish to avoid the chipmunk effect, also add OptionFormantPreserved .
Skip the rest of the options, at least to begin with.

Some examples of applying the above to various scenarios:

Real-time dynamically-changing pitch-shifting of vocals, with relatively limited CPU capacity:
OptionProcessRealTime | OptionEngineFiner | OptionWindowShort | OptionFormantPreserved | OptionPitchHighConsistency
Real-time time-stretching of drum loops without pitch shifting, with relatively limited CPU capacity:
OptionProcessRealTime | OptionEngineFaster
Offline pitch-shifting of maximum quality: start with
OptionProcessOffline | OptionEngineFiner | OptionChannelsTogether | OptionPitchHighQuality
and be prepared to do two passes and read the documentation for study.
Offline time-stretching of maximum quality: start with
OptionProcessOffline | OptionEngineFiner | OptionChannelsTogether
and be prepared to do two passes and read the documentation for study.

Common problems and queries

Playback is dropping out because the stretcher doesn't return enough samples in my play callback.
This is usually the problem described in the conceptual notes above. Review those.

Output in real-time mode is not aligned with the timing I expected.
Check the documentation for getPreferredStartPad and getStartDelay. You need to pad the start of the input with the number of samples returned by getPreferredStartPad, and drop from the start of the output the number of samples returned by getStartDelay.
Offline mode does this for you, but real-time mode doesn't. (You may wish to ensure the initial pad and delay handling occurs outside the real-time callback cycle. Although real-time safe, these first calls may take longer to complete than a regular process cycle because the stretcher has to prime its internal buffers. The same applies at the end of processing, to any process call in which the final flag is set.)

Output in real-time mode is aligned from the start, but misaligned after a scheduled change of ratio.
This is related to the previous issue. The stretcher applies ratio changes immediately to the next input it processes, but there is a lag between input being received and output being produced. Therefore the change unavoidably takes place slightly later than the moment at which you request it. To get the timing right after a scheduled ratio change, schedule the change a little early — by the same getStartDelay as used when managing the initial delay as above.

Output in real-time mode has a number of extra zero-valued samples at the end.
Real-time mode does not strictly guarantee to produce a precise number of samples at output, for the reasons given earlier. The "final" flag provided to the last process call just tells the stretcher to drain its internal buffers; the output is effectively rounded up to the next internal processing block. The application should keep track of how many samples are expected (taking into account the start delay as described above) and truncate the output where necessary.

What is the typical delay returned by getStartDelay?
This depends on the engine and some other parameters as well as the sample rate. With a typical configuration at 44100/48000 Hz the values are:

Engine	Window setting	Delay (samples)
R2	Short	512
	Default	1024
	Long	2048
R3	Short	1280
	Default	2048

Can I get better results by using a higher sample rate?
Generally no. Rubber Band is tuned for audio sample rates of 44100 and 48000 Hz, and both "faster" and "finer" engines are designed to perform at least as well at those rates as any. Although you can run a Rubber Band stretcher at any sample rate from 8000 to 192000 Hz, it's not expected that you would get detectably better output with higher rates than 48000 Hz.

Examples

Here are some code examples. We don't recommend trying to use any of these as a cut-and-paste source, but they may help as documentation to address specific points of concern.

Illustrative examples published by Breakfast Quay:

Pitch-shifter plugin example — a simple LADSPA and LV2 format pitch-shifter included in the Rubber Band Library source distribution. Here is the core of the process loop.
Mini Example with Qt — A simplistic Qt-based UI that can load an audio file and adjust its playback speed dynamically.
iOS test project — This project just contains a build example for an old-school Objective-C iOS project. It exists only to verify that it is possible to build to an iOS target using the Rubber Band Library. There is no real iOS development here (we aren't iOS developers).
Android test project — This project just contains a build example for an old-school Java/Ant/NDK Android project. It similarly exists only to verify that it is possible to build to an Android target using the Rubber Band Library.

Third-party examples from open-source code using Rubber Band Library:

Time-stretcher wrapper from Sonic Visualiser (GPL).
Time-stretcher effect from the Ardour DAW (GPL).
Two different Android integrations into ExoPlayer, neither of which we have tested ourselves: pblandford/ExoRubber (Apache license), devingryu/RubberBandAndroidExample (GPL).
Stretching looper from SooperLooper (GPL).