trackswitch

Timeline Units

Timelines are an abstract concept in trackswitch. Per definition, a timeline is an ordered sequence of numerical positions that allows for linear interpolation between consecutive elements.

Trackswitch supports the following timeline units, depending on the type of media:

Media type Allowed Timeline Units Default Unit
Audio seconds, samples seconds
MIDI seconds, ticks seconds
Sheet Music measures measures
Image percent (of the image width), pixels percent (of the image width)

Note: For MIDI ticks, the tick conversion runs through the MIDI header rather than a plain division by PPQ, so a file with tempo changes converts correctly across the whole piece.

Timelines are usually bound to a medium, except for the reference timeline. The reference timeline will always be printed as seconds.

We show an example of this in the player below with our running example of Gefrorne Tränen from Schubert’s Winterreise, D. 911. We show the sheet music, a piano roll (MIDI) with MIDI ticks as a timeline unit, and the recorded performance HU33 with audio samples as a timeline unit.

Audio data: Schubert Winterreise Dataset, Gefrorne Tränen, D. 911, No. 3, performances HU33 and SC06.

Declaring a unit

All timelines (except for the reference timeline) need to be tied to a medium, so the unit is declared on the media entry, with timelineUnit:

"media": {
  "takeA": { "type": "audio", "src": "take-a.wav", "timelineUnit": "samples" },
  "takeB": { "type": "audio", "src": "take-b.wav" }
}

That single declaration does both jobs: the medium’s alignment column is read in that unit, and every surface the medium owns reads out in it. A medium without a timelineUnit uses the native unit of its type.

In the example above

The alignment above was computed on a 20 ms grid. Its HU33 column was written out as sample indices and its MIDI column as ticks, while the score column holds measure positions:

measure,hu33_samples,notes_ticks
0.000000,0,0
0.020000,441,19
0.040000,882,38
0.060000,1323,58

Two of the three media declare a unit; the score needs none, because measures is already what a score plays back in:

"media": {
  "score": { "type": "musicxml", "src": "Schubert_D911-03.xml" },
  "notes": { "type": "midi", "src": "Schubert_D911-03.mid", "timelineUnit": "ticks" },
  "hu33": { "type": "audio", "src": "HU33.wav", "timelineUnit": "samples" }
},
"alignment": {
  "src": "alignment.csv",
  "referenceTimeline": "score",
  "timelines": {
    "score": "measure",
    "hu33": "hu33_samples",
    "notes": "notes_ticks"
  },
  "outsideCoverage": "hold",
  "duplicatePlacements": "average"
}

A score makes a good reference timeline for exactly the reason its unit is unusual: measure positions are the coordinate every performance of the piece has in common, so the main timer names a place in the music rather than a place in one recording.

Further Notes

See Aligned Timelines for the alignment model itself, and the alignment reference for the complete list of configuration properties.