How does alphaTab handle drum synthesis?

[sandrojologua]

Hi there. I'm interested in how does alphaTab play drums. I've seen that there are around hundred drums in guitar pro in general. Are there any samples for all of those drums somewhere in the project? Or is it generated? I'm really curious. Thanks a lot.

[Danielku15]

alphaTab is using a midi+soundfont2¹² based audio synthesis engine. This is quite a common combination which you can find in many popular libraries and applications like:

1. FluidSynth
2. Java (javax.sound.midi.Synthesizer)
3. MuseScore

We are using an adapted and customized TypeScript port of TinySoundFont an amazing work of Bernhard Schelling and Steve Folta.

So what does this mean in detail:

• alphaTab is organizing the drums in a way compatible to General Midi.
  • There is one special midi channel reserved for percussion: channel 10 (9 on zero-based indexing)
  • The individual percussion sounds are depending on the "note height" ³
  • This is similar to when you have a real (midi) keyboard and you enable the percussion mode.
  • In the data model internall things are aligned with Guitar Pro. There is a list of "percussion articulations" ⁴ describing the display and playback.
• When it comes to actual audio generation the percussion is not different from normal instruments. Simplified this means:
  • The playback is controlled via a midi sequencer which provides the right events at the right time.
  • When midi events are processed internally "channels" are used to hold the state on what note is played and what program should be used.
  • The SoundFont file contains audio samples and descriptions how to use those samples.
  • SoundFont in general describes a combination of filters, envelopes, modulators,... (9.1 SoundFont Spec)
  • The raw samples are sent through this synthesis model to produce the right output for the note height and length.
  • The audio samples from all channels are mixed together.
  • The generated audio samples are buffered inside alphaTab and sent to the audio output (e.g. Audio Worklets or ScriptProcessorNode)

Footnotes

1. https://en.wikipedia.org/wiki/SoundFont ↩

2. https://www.synthfont.com/sfspec24.pdf ↩

3. https://en.wikipedia.org/wiki/General_MIDI#Percussion ↩

4. https://github.com/CoderLine/alphaTab/blob/main/src/model/PercussionMapper.ts ↩

[sandrojologua]

Hi @Danielku15. Is there any way I can extract wav (or similar) files from sonivox.sf2 file and get all drum sounds pre-recorded that alphaTab produces? I've tried to extract aforementioned sf file with an online tool but couldn't find all the drum samples there, only few ones. Thanks.

[Danielku15]

I personally use Polyphone for editing soundfonts. It has an option to export samples as wav.

https://www.polyphone.io/en/documentation/manual/soundfont-editor/tools/sample-tools#doc_cat_files

[sandrojologua]

I've checked sonivox.sf2 file with Polyphone and I got samples numbered from 000 to 150, however couldn't find the drum samples here. Looks like in alphaTab drum sounds are produced from another soundfont file? I'm little bit confused. Thanks.

[Danielku15]

You might want to do some reading how the SoundFont file is organized and how Midi handles drums. alphaTab is not doing anything magical to generate drums. You have to drill in to the structure to find out what sample is used where. The samples are not mapping 1:1 to midi programs.

1. You need to inspect the preset 128 which defines what is used for drums (SoundFont2 spec defines this). There you will see what "instruments" should be used defined for this preset. In sonviox there is just one basic definition without special overrides:
   
2. When you inspect the "Instrument" you will see the definitions what the synthesizer needs to use when generating the audio. In Sonivox:
   

SoundFonts are designed in a way that allows heavy reusing of same definitions. e.g. have 1 sample, for 3 instruments but using different parameters.

When actually synthesizing the audio for playback, all details within the soundfont have to be respected and the raw samples are just the foundation.

[sandrojologua]

Thanks a lot @Danielku15. Could you give me an advice on how can I somehow convert each drum keys in sonivox.sf2 file to wav file sample? Like for drums, each midi number to have its sample generated from sonivox file. I want to play drums using Tone.js which doesn't have a soundfont player. Thanks again.

[Danielku15]

You can export simply all samples using Polyphone. Then you need to translate the "Key-Range" to the respective Midi key number and then simply pick the sample from the exported list. If you want to develop something which does this automatically, I fear you'll have to learn about the repective formats and program the relevant bits.

You could look into these areas to learn more:
https://github.com/CoderLine/alphaTab/tree/develop/src/synth/soundfont
https://www.npmjs.com/package/soundfont2

Generally this goes beyond what should be discussed in this repository as here things are about alphaTab and alphaTab doesn't expose features to do that low level soundfont handling. You might be better helped on Stackoverflow if you require general programming assistence.