kkkkl
"Litmus Test - def: any kind of indicator used to classify something either favorably or unfavorably"
Over many years of testing and checking the timing performance characteristics of different electronic event sequencing devices I rely on a very simple test that gives a very accurate indication of just how well a musical sequencing device keeps time.
Take any sequencing device - in a sampler use a tight edited fast transient sound like a rim shot - look at it closely on an editor software application to make sure the start time is tight. On a ROM Player or sequencer with built-in sounds - select a similarly tight percussive sound or patch.
Make certain any sample output or patch VCA/EG settings are set to absolute zero/fastest possible attack time.
Sequence a simple two bar pattern at exactly 120 BPM with hard quantised quarter notes playing only your test sample.
Now play the pattern and at the same time record the audio output into any reasonable audio recording software application at 44.1 kHz for 30 seconds or so - Soundforge, Wavelab etc.
[It is not necessary to use a lab oscilloscope for this type of testing. We are interested primarily in analysing timing discrepancies that you can hear - my initial interest in testing came about because of sloppy hats that I need to fix in a sequence - measuring rhythmic precision down to the individual sample at 44.1kHz (22 microseconds) is more than accurate enough.]
Stop the pattern and open up the waveform of your recording.
Make sure your editor is set to view and edit your waveform in samples rather than absolute time.
Zoom right in on the very front of a waveform (any of them will do) and place a marker - do it at the absolute maximum magnification on screen so you are certain the marker is where the waveform begins.
Now zoom back out and locate the very next waveform in the recording and again, zoom all the way back in and place another marker at the very start of this one too.
Repeat this for at least 8 of your recorded 'clicks' in the recording - 16 is better if you have the patience.
Now 'select' the block between the first two markers - usually clicking in the area with the mouse does the job.
Somewhere on screen the number of actual samples between your two markers will be indicated.
Write this number down.
Repeat this for each 'block' between all markers you have placed.
At 120 BPM - a perfect quarter note interval recorded at 44.1 kHz is exactly 22050 samples.
In a perfect world all your measured 'blocks' should be 22050 samples in length.
They won't be.
The amount the quarter note intervals vary will indicate how tight your sequencer or drum machine really is.
The bigger the variation, the sloppier the rhythm.
Two things obviously affect this measurement - the real tempo (as opposed to what it tells you on the screen) and jitter (tempo fluctuations). What is critical is the amount of variation you find in your measurements. If your blocks vary by more than 50 samples then the device's ability to keep tight rhythmic time is questionable at best. Remember that our test is very easy on the hardware and software resources - a single track of well spaced events. There are no polyphony or multitimbral compromises to hide behind.
Sequencer Tempo Jitter/Event Slop has two primary causes - Clock Jitter (where the clock driving the internal sequencer or slave device is a bit rough) and Internal Event Trigger Lag (where a devices ability to correctly trigger an event referenced to the tempo grid/clock position is compromised in someway leading to push-pull within the device itself).
The end timing performance stability in any sequencing system is a combination of both these factors.
Keep in mind also that these test results are the very best your box or software application can produce. Under normal usage you will usually find things get much worse.
Published hardware and software specifications in owner's manuals, on-line and in the glossy brochures and print media focus almost exclusively on the myriad of bells and whistles on offer. The aim being of course to tempt and tease in a bigger, faster, wider, louder, and 'newer is better' display of spec-superiority - polyphony, memory, number of patches, high sample rates, track count, plug-in support, data I/O speed, CD Burning etc.
What they never publish or even mention is timing performance.
I'm still uncertain if this is a deliberate strategy to hide poor design, that R&D teams dismiss the notion that humans have a very high degree of sensitivity to rhythmic variance or that the marketing departments find this area of analysis just too dry and uninteresting for the general public.
These days it would be my first consideration before buying anything at all.
You can spend as much as you like on furnishings and fancy wallpaper but a house is only as good as the foundation it is built on and in sequencing the foundation is the clock.
After your own tests are completed and the numbers don't look so crash hot - tell the manufacturer.
Email them the numbers and tell them you expect better.
Let them know you would prefer a tight sequencing engine with a few less frills not an expensive rubber band with everything and the kitchen sink included.
[On a recent forum regarding sequencer timing stability - the audio/DAW method of measurement as outlined here was strongly critisised as being too inaccurate to be considered meaningful when measuring tempo jitter in sequencers. For those of you reading this with similar views I thought it important to quote a direct reply from one of the design/R&D team involved in the product in question: ' There is nothing wrong with your method of measuring the jitter, any fully working sound card will do.']
All In Good Time
It seems logical to my mind that time is the core foundation on which all music is created because what makes sound into music is how the sounds are played or replayed in time.
There is a common belief that certain sequencers and drum machines have a magic ‘groove’ or feel built right into them. I personally don't think there is any 'magic' beat box feel - vintage or contemporary. What gives any rhythmic pattern 'feel' is how we anticipate where sounds fall in time and because every individual hears subjectively it makes practical analysis and criticism of timing performance in sequencers very difficult. This I well understand.
I do believe strongly, however, that adding any random element to step/event placement in any sequencing device does not create feel. All it serves to do is blur the edges.
The exact opposite applies when deliberate push-pull placement of steps and events against a strict, quantised tempo-grid is used to customise rhythmic feel - pushed hats, late snares and of course added amounts of shuffle/swing.
Discussions about timing precision in music always invoke passionate debate and there are many who see any quest for tighter performance as an obsessive desire for extreme rigidity and control. The assertion being that this is a very negative characteristic and that ‘going with the flow’ and accepting some element of ‘human-feel’ is a far more musically appealing way to be.
Rigidity has negative connotations for most musicians but I must stress again that a desire for precision and consistency in sequencing is not about rigidity or stiffness at all. Quite the reverse in fact.
Feel is rhythmic anticipation – and that very human anticipation relies on the strict principle that if a snare is deliberately placed 5 ticks late it must always sound 5 ticks late to faithfully maintain the groove. The potential feel in any rhythm becomes less focused when the snares fall 3 ticks late sometimes and 7 ticks late other times in a pattern or loop when the timing variation is of a random nature.
It is important not to confuse random timing errors with any concept of human feel.
It is not feel in any sense because the timing variation is random.
It is simply software and hardware not keeping time.
The Good, The Bad and The Lumpy.
As I get around to it I will list here my measurements on various boxes and software sequencing applications.
All tests as specified above - 120 BPM recorded at 44.1 kHz into Soundforge 8 or Soundforge 9.
The group of measured consecutive quarter note intervals will be listed in samples along with a maximum variance between any two.
On first inspection it may seem that measuring the error margin away from the theoretical 'perfect' grid value of 22050 is a more accurate way to look at these numbers, however, our perception of feel is all about how our brain interprets the length of time BETWEEN consecutive rhythmic events. This is why the push-pull timing of consecutive events is the most significant set of numbers to consider when looking at sequencer timing stability.
Date of test and current Operating System is listed.
Obviously a change of OS may mean a change in test results.
Equipment listed is in no order of personal preference.
Feel free to email me if your own testing provides a different set of numbers.
Remember that this is a highly biased rating system based on one feature only. Important as timing performance is, there are many other aspects to software and hardware sequencers that contribute their overall worth as creative tools.
My genuine wish is that the designers and manufacturers of our current and future music machines start placing timing performance at the very top of the priority list when it comes to features and OS updates they offer.
At the end of the day it comes down to sales. If we continue to buy our toys and tools as they are then there is little incentive for the manufacturers to change anything. If enough customers began refusing new drum machine, sequencer, groovebox and workstation purchases based on this simple test and let the company know the reasons why then I think accurate timing performance might soon become a much higher design priority than it is now.
Email: info@innerclocksystems.com
Originally, this page included devices still current and in production at the time of testing. The motivation for the Litmus testing was not to cause any individuals or organisations grief but simply to provide reliable timing information to those who wished to know. There have been some that view this page as detrimental to companies that manufacture products that rank low on the timing stability scale. There are also some that have actively suggested on a number of forums that this page is simply a way for me to generate Sync-Shift sales. For those that understand the design of the Sync-Shift this notion is ridiculous.
The Litmus Ranking scale indicates Event/Tempo Jitter within a device itself. If a sequencer or drum-machine has internal coding or hardware issues that cause timing irregularities, then no amount of Sync-Shifting will cure that. The Sync-Shift is a Start-Offset device. Nothing more and nothing less.
Nevertheless I have decided to make the Litmus Page restricted to devices no longer in current production. If you wish to test current products yourself, the method listed above holds true.
One exception to this is the Mansell-Labs Vailixi OS Upgrade for the Akai MPC-3000 which deserves a special mention. Even though Akai's production of this fine instrument ceased some years back, Rohan Mansell now offers a custom OS Eprom System set that refines Roger Linn's original vision further by adding a number of very valuable features without sacrificing any aspect of the core design and performance that make the unit so special to this day. If you own an MPC-3000 - visit the website and get the OS Upgrade. If you don't own one and you have an ear for timing - find one, buy it and then get the Vailixi Upgrade. You will not regret either purchase I assure you. A truly remarkable achievement in every respect.
UREI 964 Digital Metronome
Test Date:18.06.07
Note: Tempo on the 964 is set in Frames Per Beat rather than more common BPM and therefore precise decimal Tempo setting is not possible for some values. Again, what is significant is the interval variance rather than the numerical average value.
[22033/22033/22033/22033/22033/22033/22033/22033/22033/22033/22033/22033/22033/22033/22033/22033]
Maximum variation between any two consecutive quarter note intervals: Zero Samples (0.00ms)
Litmus Ranking: 
Garfield Electronics Masterbeat
Test Date: 09.08.07
[22041/22041/22042/22041/22041/22041/22042/22041/22041/22041/22041/22042/22041/22041/22041/22042]
Maximum variation between any two consecutive quarter note intervals: 1 Sample (0.02ms)
Litmus Ranking:
Garfield Electronics MiniDoc
Test Date: 03.11.07
This Test - Urei 964 Audio Click generating output trigger pulse in real-time.
Maximum variation between input audio and output pulse trigger: Zero Samples (0.00ms)
Litmus Ranking:
Garfield Electronics Digital Click
Test Date: 06.11.07
[22040/22040/22040/22043/22040/22039/22041/22040/22040/22041/22041/22040/22039/22040/22040/22040]
Maximum variation between any two consecutive quarter note intervals: 3 Samples (0.07ms)
Litmus Ranking:
Akai MPC-3000/OS Version:Vailixi 3.50
Mansell-Labs Web - http://www.mansell-labs.com/
Test Date:18.03.07
This Test - Pattern Mode/Cycle
[22049/22051/22048/22050/22050/22048/22052/22050/22049/22048/22050/22052/22048/22050/22052/22051]
Maximum variation between any two consecutive quarter note intervals: 4 Samples (0.09ms)
Litmus Ranking:
Akai MPC-3000/OS Version:Vailixi 3.50
Mansell-Labs Web - http://www.mansell-labs.com/
Test Date:01.06.07
This Test - Song Mode with Tempo Slaved to Ultra Stable External Midi Clock to Midi Input B.
[22051/22052/22052/22052/22053/22052/22052/22052/22051/22052/22052/22052/22051/22053/22052/22052]
Maximum variation between any two consecutive quarter note intervals: 2 Samples (0.045ms)
Litmus Ranking:
Roland SH-101
Test Date:15.11.07
This Test - Internal Step Sequencer clocked via Ext.Clock Input from Garfield MiniDoc. VCA set to [Gate], Sawtooth VCO at full level, Filter wide open with zero resonance. The numbers represent the number of samples between source trigger pulse and the generated audio from the SH-101. The offset measurments taken by recording an analogue (no latency) split from the Garfield reference Click Pulse and the SH-101 triggered notes to a stereo audio capture in Sound Forge 9 at 16/44.1 hard panned Left/Right respectively.
[203/214/230/243/253/280/167/177/189/203/219/227/250/251/265/274/174/188/199/218/225/239]
Minimum external trigger lag to open VCA via internal Sequencer: 167 Samples (3.79 ms)
Maximum external trigger lag to open VCA via internal Sequencer: 274 Samples (6.21 ms)
Litmus Ranking: 
This Test - Identical reference Clock Pulse Input from Garfield MiniDoc but patched directly to the SH-101 External Gate Input jack and bypassing the on-board sequencer. VCA set to [Gate], Sawtooth VCO at full level, Filter wide open with zero resonance. The numbers represent the number of samples between source trigger pulse and the generated audio from the SH-101. The offset measurements taken by recording an analogue (no latency) split from the Garfield reference Click Pulse and the SH-101 triggered notes to a stereo audio capture in Sound Forge 9 at 16/44.1 hard panned Left/Right respectively.
[12/14/12/12/13/12/14/12/12/12/11/14/12/13/14/12/12/14/13/14]
Minimum external trigger lag to open VCA via Gate Input: 11 Samples (0.25 ms)
Maximum external trigger lag to open VCA via Gate Input: 14 Samples (0.32 ms)
Litmus Ranking:
What these two tests show very clearly is just how much CPU processing means in terms of sequencer design and performance. The SH-101 is well regarded as having a very snappy VCA/EG - the 0.32 ms external gate trigger time is solid proof of that reputation. However, a quick look at the schematics shows that the internal Step-Sequencer is built right in to the central CPU of the SH-101. It does not have its own dedicated hardware but instead shares resources with everything else going on under processor control. This accounts for the very late trigger response times when driven by the internal sequencer. Instead of the almost instant trigger response when the VCA is opened via the gate input or a key press, triggers created by the internal sequencer and then fed to the VCA take much longer to respond to external clocks and are far more variable - a product of CPU/Serial task management. If you look at the first set of numbers again you will see also that they form a cyclic pattern - this shows the Internal Sequencer trigger performance varies depending on what other tasks the CPU is looking after at various times.
Roland CR-8000
Test Date:22.11.07
This Test - Programmed quarter note Cow Bell hits. Internal Sync at 120 BPM as reported by the Tempo LED display.
[22087/22070/22161/22099/22086/22077/22107/22188/22071/22078/22085/22174/22056/22170/22076/22121]
Maximum variation between any two consecutive quarter note intervals: 118 Samples (2.68ms)
Litmus Ranking:
This Test - Identical programmed quarter note Cow Bell hits. External Din Sync at 120 BPM as set by Friend-Chip Mark I SRC [Maximum clock drift +/- 1 sample].
[22082/21983/22073/22079/22082/21984/22066/22087/21991/22074/22065/22094/21986/22065/22074/22085]
Maximum variation between any two consecutive quarter note intervals: 108 Samples (2.45 ms)
Litmus Ranking:
Two tests with very similar results - close maximum variance but also note the same cyclic pattern in the numbers. Three close together followed by a big variance then another three close and so on. This is again repeated across both tests. What does this indicate? The event timing engine driving the CR-8000 is under CPU control which, like the SH-101 are at the mercy of the code program timing and any data interrupts that occur while running. Even supplying the External Din-Sync input with a very tight clock stream makes almost no difference to the timing performance of this machine. It can only ever be as tight as its CPU program will allow it to be.
Roland TR-808
Test Date:26.3.07
This Test: Rim Shot Individual Output and Tempo Gen. External Din Sync from Friend-Chip SRC Mark 1 at 120 BPM
[22000/22086/22083/22004/22088/22004/22084/22081/22003/22082/22005/22077/22090/22003/22082/22081]
Maximum variation between any two consecutive quarter note intervals: 87 Samples (1.97ms)
Litmus Ranking:
Roland MC-4B
Test Date:26.3.07
Test (A) - Channel 1 Gate Pulse Output - Internal Tempo Generator (note:the MC-4B uses an analogue oscillator for internal tempo control and 120 BPM is a guide only - the important data is the variance of the values around the mean/average)
[22381/22380/22382/22387/22382/22380/22372/22379/22381/22380/22382/22378/22381/22371/22387/22382]
Maximum variation between any two consecutive quarter note intervals: 8 Samples (0.18ms)
Litmus Ranking:
Test (B) - Channel 1 Gate Pulse Output - Tempo Gen. External Din Sync from Friend-Chip SRC Mark 1 at 120 BPM
[22055/22046/22055/22050/22053/22051/22052/22067/22040/22060/22044/22066/22039/22052/22049/22056]
Maximum variance between consecutive quarter note intervals: 27 Samples (0.61ms)
Litmus Ranking:
Roland MC-4B
Test Date:26.3.07
This test: Channel 1 Gate Pulse Output driving CS-30M EG1 at fastest possible attack setting and the MC-4B running off internal Tempo Generator.
[22320/22324/22325/22325/22325/22329/22325/22328/22320/22320/22324/22327/22324/22326/22327/22328]
Maximum variation between any two consecutive quarter note intervals: 8 Samples (0.18ms)
Litmus Ranking:
LinnDrum/Midified J.L.Cooper Mod
Test Date:21.02.08
This Test - Internal Sync/Pattern Mode/Cycle/Rim Shot
[21095/21146/21105/21135/21113/21129/21119/21101/21121/21128/21107/21135/21114/21133/21120/21114]
Maximum variation between any two consecutive quarter note intervals: 51 Samples (1.16 ms)
Litmus Ranking:
This Test - SRC Mk 1:48 ppq External Pulse Sync/Pattern Mode/Cycle/Rim Shot
[22046/22058/22052/22060/22044/22053/22051/22059/22040/22058/22053/22051/22059/22044/22053/22052]
Maximum variation between any two consecutive quarter note intervals: 19 Samples (0.43 ms)
Litmus Ranking:
Akai MPC-60/OS Version: 2.12
Test Date:20.02.08
This Test - Pattern Mode/Cycle
[22041/22030/22040/22040/22048/22041/22038/22050/22030/22050/22039/22041/22037/22040/22050/22039]
Maximum variation between any two consecutive quarter note intervals: 20 Samples (0.45 ms)
Litmus Ranking:
Akai MPC-60/OS Version: 2.02
Test Date:10.07.09
This Test - Pattern Mode/Cycle
[23996/24003/24002/24002/24003/24022/23982/24007/23999/24003/24002/24002/23996/24002/24002/24003]
Maximum variation between any two consecutive quarter note intervals: 40 Samples (0.90ms)
Litmus Ranking:
This Test - Song Mode
[24004/24002/24003/24004/24004/24005/24002/24002/24002/24002/24002/24003/24002/24003/24002/24002]
Maximum variation between any two consecutive quarter note intervals: 3 Samples (0.067ms)
Litmus Ranking:
