A time-stretcher such as Rubber Band Library has to be handled a little differently from a typical audio effect.
When speeding up or slowing down audio, there is by definition a difference between the input and output "rates" of the processor. If you supply N samples as input to the processor, then you will get more than N (when slowing down) or fewer (when speeding up) as output.
This must be the case for any audio time stretcher library. But on top of this, with Rubber Band the effective rate can also differ locally depending on the locations of detected features such as percussive transients.
This means that the input and output rates are not always in a stable ratio: the ratio you set behaves a little bit like a long-term average, rather than a fixed ratio for every processing block.
A consequence of this is that you cannot just provide a certain number of samples at the input and expect to see a fixed number of samples at the output. This is true even if you are using the processor only for pitch-shifting, and in fact can be the case even if you have the time and pitch ratios both set to 1.
After supplying samples to the input
with process, you must always query how many samples have
now been made available at output
using available. See this
discussion for more details of how that might work when
pitch-shifting, as well as the examples below.
For real-time use especially, it may be simplest to use Rubber Band if you are able to integrate it using a "pull" configuration rather than a "push" one, so that the number of samples you supply to it (for a given number of output samples) is allowed to vary.
That is:
available), as there may be some in its internal buffers;getSamplesRequired);process), receives the available output (retrieve), and repeats if necessaryThis structure can be seen at work in the Mini Example with Qt and Sonic Visualiser code examples below.
Rubber Band has many configurable options, which you
select by passing a bitwise "or" of RubberBandStretcher::Option constants to
the constructor of the stretcher.
The number of possibilities can be daunting, but the good news is that — once you have decided between offline and real-time processing modes, a choice normally dictated by the application architecture — the stretcher will work tolerably well with almost any combination of options.
So you can integrate it without messing about too much, then tweak it afterwards.
That said, here are some rules of thumb for ideal option selection using Rubber Band 3.1:
OptionProcessOffline
or OptionProcessRealTime.OptionEngineFiner;OptionEngineFaster;
OptionEngineFiner | OptionWindowShort.OptionPitchHighConsistency;OptionPitchHighQuality.OptionFormantPreserved.Some examples of applying the above to various scenarios:
OptionProcessRealTime |
OptionEngineFiner | OptionWindowShort | OptionFormantPreserved |
OptionPitchHighConsistencyOptionProcessRealTime |
OptionEngineFasterOptionProcessOffline | OptionEngineFiner |
OptionPitchHighQualitystudy.OptionProcessOffline |
OptionEngineFinerstudy.Finally, if you're doing data augmentation on a large
scale for machine learning, please use one of the less CPU-intensive
combinations. This means either using the R2/Faster engine (with the
command-line tool this is the -2 flag) or, from v3.1
onwards, using the short-window option with the R3 engine (-3
--window-short). The full R3 engine uses a lot more energy and
is probably unnecessary for this task.
Playback is dropping out because the
stretcher doesn't return enough samples in my play callback.
This is usually the problem described in
the conceptual notes above. Review
those.
Output in real-time mode is not aligned with the
timing I expected.
Check the documentation
for getPreferredStartPad
and getStartDelay. You
need to pad the start of the input with the number of samples returned
by getPreferredStartPad,
and drop from the start of the output the number of samples returned
by getStartDelay.
Offline mode does this for you, but real-time mode doesn't. (You
may wish to ensure the initial pad and delay handling occurs outside
the real-time callback cycle. Although real-time safe, these first
calls may take longer to complete than a regular process cycle because
the stretcher has to prime its internal buffers. The same applies at
the end of processing, to any process call in which
the final flag is set.)
Output in real-time mode is aligned
from the start, but misaligned after a scheduled change of
ratio.
This is related to the previous issue. The stretcher applies
ratio changes immediately to the next input it processes, but there is
a lag between input being received and output being
produced. Therefore the change unavoidably takes place slightly later
than the moment at which you request it. To get the timing right after
a scheduled ratio change, schedule the change a little early —
by the
same getStartDelay
as used when managing the initial delay as above.
What is the typical delay returned
by getStartDelay?
This depends on the engine and some other parameters as well as
the sample rate. With a typical configuration at 44100/48000 Hz the
values are:
| Engine | Window setting | Delay (samples) |
|---|---|---|
| R2 | Short | 512 |
| Default | 1024 | |
| Long | 2048 | |
| R3 | Short | 1280 |
| Default | 2048 |
Can I get better results by using a
higher sample rate?
Generally no. Rubber Band is tuned for audio sample rates of
44100 and 48000 Hz, and both "faster" and "finer" engines are designed
to perform at least as well at those rates as any. Although you can
run a Rubber Band stretcher at any sample rate from 8000 to 192000 Hz,
it's not expected that you would get detectably better output with
higher rates than 48000 Hz.
Here are some code examples. We don't recommend trying to use any of these as a cut-and-paste source, but they may help as documentation to address specific points of concern.
Illustrative examples published by Breakfast Quay:
Third-party examples from open-source code using Rubber Band Library: