Rubber Band Library

Integration notes and examples

Conceptual notes
Common problems
Examples

Conceptual notes

A time-stretcher such as Rubber Band Library has to be handled a little differently from a typical audio effect.

When speeding up or slowing down audio, there is by definition a difference between the input and output "rates" of the processor. If you supply N samples as input to the processor, then you will get more than N (when slowing down) or fewer (when speeding up) as output.

This must be the case for any audio time stretcher library, but on top of this, with Rubber Band the effective rate can also differ locally depending on the locations of detected features such as percussive transients, which it usually tries to pass through unmodified.

This means that the input and output rates are not always in a stable ratio: the ratio you set behaves like a long-term average, rather than a fixed ratio for every processing block.

A consequence of this is that you cannot just provide a certain number of samples at the input and expect to see a fixed number of samples at the output. This is true even if you are using the processor only for pitch-shifting, and in fact it is true even if you have the time and pitch ratios both set to 1! After supplying samples to the input with process, you must always query how many samples have now been made available at output using available. See this discussion for more details of how that might work when pitch-shifting, as well as the examples below.

For real-time use especially, it may be simplest to use Rubber Band if you are able to integrate it using a "pull" configuration rather than a "push" one, so that the number of samples you supply to it for a given number of output samples is allowed to vary freely. That is:

you have some sort of time-processing object, which holds a Rubber Band stretcher object;
your audio output driver or graph requests a certain number of samples from it;
your time-processing object checks how many samples it can get from Rubber Band already (available), as there may be some in its internal buffers;
if that is not enough:
- it queries how many the Rubber Band stretcher needs at input before it can produce more at output (getSamplesRequired);
- it calls back into its audio source to obtain that number of samples;
- it runs the stretcher (process), receives the available output (retrieve), and repeats if necessary
and the time-processing object then returns the accumulated output samples to the audio output.

This structure can be seen at work in the Mini Example with Qt and Sonic Visualiser code examples below.

Common problems

Playback is dropping out because the stretcher doesn't return enough samples in my play callback. This is usually the problem described in the conceptual notes above. Review those.

Output from real-time mode is not aligned with the timing I expected. Check the documentation for getPreferredStartPad and getStartDelay. You need to pad the start of the input with the number of samples returned by getPreferredStartPad, and drop from the start of the output the number of samples returned by getStartDelay. Offline mode does this for you, but real-time mode doesn't.

I don't know which combination of RubberBandStretcher::Options to use. Some suggestions:

Real-time dynamically-changing pitch-shifting, where CPU capacity is limited: start with
OptionProcessRealTime | OptionEngineFaster | OptionPitchHighConsistency
and consider also OptionFormantPreserved when dealing primarily with vocals or solo instruments
Real-time time-stretching without pitch shifting, where CPU capacity is limited: start with
OptionProcessRealTime | OptionEngineFaster
Either of the above, but with a more liberal CPU budget: replace OptionEngineFaster with OptionEngineFiner
Offline pitch-shifting of maximum quality: start with
OptionProcessOffline | OptionEngineFiner | OptionPitchHighQuality
and be prepared to do two passes and read the documentation for study
Offline time-stretching of maximum quality: start with
OptionProcessOffline | OptionEngineFiner
and be prepared to do two passes and read the documentation for study

Finally, if you're doing large-scale data-augmentation for machine learning, please default to not using OptionEngineFiner or the -3 option to the command-line tool. The Finer engine uses a lot more energy and is probably unnecessary for this task.

Examples

Here are some code examples. We don't recommend trying to use any of these as a cut-and-paste source, but they may help as documentation to address specific points of concern.

Illustrative examples published by Breakfast Quay:

Mini Example with Qt — A simplistic Qt-based UI that can load an audio file and adjust its playback speed dynamically.
Pitch-shifter plugin example — a simple LADSPA and LV2 format pitch-shifter included in the Rubber Band Library source distribution.
iOS test project — This project just contains a build example for an old-school Objective-C iOS project. It exists only to verify that it is possible to build to an iOS target using the Rubber Band Library. There is no real iOS development here (we aren't iOS developers).
Android test project — This project just contains a build example for an old-school Java/Ant/NDK Android project. It similarly exists only to verify that it is possible to build to an Android target using the Rubber Band Library.

Third-party examples from open-source code using Rubber Band Library:

Time-stretcher wrapper from Sonic Visualiser (GPL).
Time-stretcher effect from the Ardour DAW (GPL).
Stretching looper from SooperLooper (GPL).