HOME | BENCHMARKS | FORUM| CONTACT |
 
DAWbench - Reference Benchmarks :.

   

Audio Interface - Low Latency Performance : Part II :.

   

Following on from the initial testing and report investigating Audio Interface Low Latency Performance , over the last 12-18 months I have placed over 2 dozen interfaces on the bench and ran a suite of benchmarks across each individually to get a good overall grounding of the comparative Low Latency Performance of the respective interfaces .

In the process I also developed a Low Latency Performance Rating system which allows me to place the interfaces into an easily understandable scale, taking into account performance at respective latencies , weighed up against the Round Trip Latency of the units.

I also further investigated the OEM controllers/drivers and their relationship in regards to performance, and to top it off helped develop a utility to measure actual Round Trip Latency.

Its definitely been an interesting ride , so lets begin.

 

audio-int

llp-logo-blk

Hardware Buffer Setting v Actual Latency - Redux :.    

This is an area that I touched on in the previous article, and it is a minefield to navigate at the best of times.

The further I investigated the more complex and convoluted it became.

As noted in the previous article , the listed values and settings in the respective hardware control panels in many cases have little to no correlation what so ever to the actual latency achieved. Further complicated by various factors including added safety and double buffers as well as non reporting of the AD/DA.

 

Some interfaces even resorting to simply reporting nominal figures , which would compromise the ability of the host DAW's to be able to keep everything in sync due to wide variances between the reported and actual latency.

The variances at respective latency settings was also quite substantial , for example the RTL at the listed 064 setting varied anywhere from under 3.5ms all the way to over 12ms , so obviously there is no real consistency in regards to what the actual panel setting represents.

 

Honestly some of the listed panel settings were little more than window dressing.

To help navigate the minefield I collaborated with Andrew Jerrim of Oblique Audio and helped develop a Round Trip Latency Utility which allowed me to measure the actual RTL of each respective interface .

The utility is similar in principle to the CENtrance Latency Tool that has been around for a few years, but is far more advanced and has proved accurate to within a few samples.

         
Firewire / USB Controllers and Driver Performance :.    

This is another area that I touched on briefly in the previous article, and if the area of actual latency was a minefield, then this made the previous feel like a few minor dips in the road.

As the testing pool increased it was becoming quite evident that there was a consistency amongst various interfaces from different manufacturers in regards to not only control panel settings and corresponding latency values, but also the comparative performance of the units . It was pretty obvious we were dealing with not only an identical OEM FW controller , but also a baseline driver.

Trying to get official confirmation of the OEM controllers used was proving difficult so I needed to dig a little deeper with some trusted industry contacts who gave me some clues where to dig and managed to narrow down the OEM FW controllers that were most widely used.

 

The mostly used OEM controller is the Dice range from TC Applied Technologies - Dice Jr and Dice Mini , the 2 vary by the number of available I/O. Some manufacturers using this controller - TC ( obviously ) , AVID ( Mbox 3 Pro) , M-Audio ( Profire Range ) , Presonus ( Current ), Focusrite , Mackie ( Current Onyx ) , Midas , Allen and Heath, the list goes on .

It is worth noting that AVID and M-Audio differ from the other units listed in that they do not use the base OEM driver, instead developing and using their own.

The second most widely used range of controllers is by ArchWave ( Formally BridgeCo) , list of some of the manufacturers using the controllers include - Apogee, Lynx, M-Audio ( Older FW units), MOTU, Presonus ( Older units), Prism and Roland.

 

The 3rd controller is a joint venture developed by Echo and Mackie which was used in the Echo Audiofire line of interfaces , the earlier Mackie Onyx rack/desktop models and is still used in the Mackie Onyx Mixer range.

The OEM USB2 controller most widely used I have recently discovered is the XMOS , which I detail later in the report.

3rd party OEM controllers cover a large % of FW/USB audio interfaces available , the obvious exception is RME , who develop not only their own proprietary controllers in house, but also couple that with development of the drivers.

RME are unique not only in the level of development that they apply at both controller and driver level , but also in the level of performance that they have achieved across all of the available protocols.

     
Reference Test Sessions / System :.    

The Reference test sessions are the DAWbench DSP Universal - RXC and DAWbench VI - Kontakt4

These sessions have a large data base of collated results over the years that we can directly correlate the results to, is freely available for numerous DAW applications, and even tho the results listed will be for Cubase 6.x the individual results for the respective interfaces will be relative to any application that utilizes the ASIO driver efficiently.

 

All interfaces have been tested on my tried and tested / never to be sold i7 reference / report system which has been specifically configured for low latency audio application.

Base reference hardware is the RME HDSP line of audio interfaces that have proven to be the top performers on both platforms , all other hardware will be graded comparatively against the base hardware results

 

Reference System Detail:
Intel i7 920 Quadcore/ 2.66 GHZ/
Intel X58 / 6 GB DDR3-PC12800.

O.S Detail:
Windows 7 SP1 x64 Pro

         

llp-2

LLP ( Low Latency Performance ) Rating :.

I decided to develop a rating system that takes into account numerous variables relative to overall low latency performance , an explanation of how the LLP- Low Latency Performance Rating is derived is as follows.

3 reference benchmarks are used - DAWbench DSP - RXC , and the 2 variations of the DAWbench VI benchmark CV ( utilizing convolution verbs) and NCV ( no convolution verbs)

The results for the DAWbench DSP RXC across the latencies of 032 thru to 256 ( which has been the M.O for the last 5 years ) are added and the total is then % wise gauged against the result for the RME HDSPe baseline card. The same is then calculated for the DAWbench VI CV/ NCV tests for 032-512.

Those 3 % results are then added and divided by 3 to give an average % .

 

I thought it important for the I/O and RTL figures to be an influencing factor on the rating as some cards have a lot lower overall latency than others, so the average % results is then multiplied by the last % result for the RTL.

How the RTL % is calculated is I combine the total of the RTL values across the specific available buffer settings for the cards. There are 2 default values listed for RTL for the RME HDSPe baseline , first being for 032-512, second being 064-512 , the appropriate value being used depending on the respective test cards range of available latencies.

In the instance that the test interface has a different range of latency values available- i.e. 128-512 , then calculations are adjusted accordingly from the appropriate values of the base reference interface.

 

I then calculate the % variable against the baseline.

To summarize, the average % value across the 3 benchmarks is multiplied by the RTL% value to give the final rating.

I think that is a fair appraisal using the collated data, and it gives deserved credit and advantage to those cards that do have lower individual In/Out and Round Trip Latencies.

The charts below give a clearer indication of what I have outlined

         

llp-results

Getting into the Details and Navigating the Curves :.    

The results clearly show that there are huge variables in not only the dedicated I/O Latency and RTL but also overall performance at the respective buffer settings. Its not as simple as comparing performance results at any given dedicated control panel buffer setting when there are such large variables in play - i.e. RTL for buffer settings of 064 range from 3.5 ms to over 12ms as I noted earlier.

Its also not as clear cut as saying that for example FW interfaces offer better performance than USB 2.0 or vise versa , as there are instances of respective interfaces clearly outperforming others using the various protocols.

The tested PCI/PCIe interfaces do however lead in performance , clearly indicated by the tables above , but the variance is not as large as it once was. The RME FW and USB interfaces for example are prime examples by not being too far off the base reference in both performance , I/O and RTL.

It is also evident that numerous manufacturers are utilizing not only the identical FW controllers ( TC Applied Dice ) but also the bundled OEM driver , which for the most part is convenient for them to get products to market, but the products using the above cover a large end user demographic and the drivers are not necessarily the best catch all in some working environments.

Whereas they will be fine for live tracking and in session environments focussing on audio where low latencies are not a priority , once the latencies are dialled down for live Virtual Instrument playing / Guitar Amp Simulators for example, the performance variable at those lower /moderate latencies can be in the vicinity of 40+%. That is a huge loss of potential overhead on any system, let alone current multi core systems.

Trying to get more detailed info of the controllers and drivers from the appropriate sources was near impossible , either the requests and concerns were totally ignored or dismissed as irrelevant to Real World application.

Through my own investigation I did manage to conclude that one manufacturer despite using the TC DICE FW controller did take the time to write their own drivers and perhaps even firmware, which greatly improved the performance .

 

That manufacturer , M-Audio/AVID on their Profire and MBox Pro ranges , which are only bettered by the RME units in I/O, RTL and overall LLP.

What that indicates is that for those that have the resources available, better LLP is indeed available for the 3rd party OEM FW controllers , the reality is that sadly most of the manufacturers either do not have the resources available, or worse, couldn't care less , IMO.

With the current shift for many manufacturers away from FW, most of the current crop of USB 2 interfaces apart from RME also share an identical OEM controller from XMOS , base drivers are provided by numerous 3rd parties, CENtrance, Ploytec, Thesycon, and in short, its an absolute crap shoot.

There have been recent instances of USB2 interfaces that have come to market with their drivers being in a state on Windows which can only be described as unworkable in any sense of the word when it comes to DAW usage. High I/O and RTL , performance at the respective latencies which collapsed under any type of loading.

One of those interfaces was the Steinberg UR28M from their new USB2 range , which thankfully has had a driver rewrite since the initial release which addressed the issue. The results above are for the 2nd revision of the driver , the first driver had a LLP Rating of 1.44

There is another instance brought to my attention by a colleague ( which I have not tested as yet so will remain nameless) that has a 40ms RTL at a supposed 256 buffer setting, yes you did read that correctly. That driver has not been addressed and we managed to get some insight from the developers that indicated that they did not see a problem ??? !!!

Despite the interfaces using the same OEM controller , performance varies greatly depending on the choice of OEM 3rd party base driver and the added optimizations ( if any ) being deployed by the manufacturers , so its not as blatant as the FW interfaces all using the DICE and bundled OEM driver , which all have identical performance.

 

 

The last option for device protocol is the one that many are holding out great hope for , which is of course Thunderbolt - i.e. copper version of Lightpeak , which is essential a dual protocol external interconnect with dedicated PCIe x4 and PCIe x16 for DisplayPort.

This in theory will allow manufacturers to achieve PCIe LLP from an external interconnect , as well as being backward compatible with all current interconnects running on the PCIe bus- FW and USB2/USB3.

Despite the hype and near hysteria surrounding TB in regards to potentially advantages it can offer digital audio interface manufacturers , its been a slow road in regards to adoption and we are yet to see at the time of my typing, any interface with a TB interconnect, native or bridged.

There have been numerous theories as to the reason for the slow adoption and its evident that we are not really getting the whole story. There are uploaded videos from an Intel Development Forum in September 2010 where there was a proof of concept demo of Lightpeak being used for Professional Digital Audio application by none other than AVID on Protools HD, using a BETA HD I/O on a Compal white box PC Laptop running Windows 7. Videos can be seen Here and Here

There are more than a few questions that get thrown up watching the videos that were from a conference a full 5 months before Apples initial release and exclusivity. First off of course is that the demo was on Windows 7 running on an ODM PC notebook , so the whole area of Apples exclusivity comes into serious questioning. Secondly the interconnect cable being used is not the current Apple licenced proprietary cabling which is another area of concern with the added electronic component at each end of the cable ( looks like a standard USB style interconnect in the videos). Thirdly , if PC ODM manufacturers were already heavily invested in Lightpeak ( its not a simple bolt on with the DisplayPort aspect needing direct dedicated plumbing to the PCIe x16 bus) , why almost 2 years later are we still yet to see any significant roll out on the PC Platform.

I do have some insight and opinions, which I'll leave for the next round.

         
Conclusion :        

So what have I learned from the ongoing adventure in trying to bring some added clarity to the area of Low Latency Performance of the available interfaces under Windows 7 ?

Well one thing is this is an area that some manufacturers and their representatives are not overly happy about having an added focus on , but I'll get to that later. What I have learned over the course of the last 12-18 months is that Low Latency Performance is something that is not very well understood by a large section of the end users, and the developers of the 3rd party controllers and the manufacturers are all too happy to continue to market and blur the lines when talking about interface latencies.

One typical method is in regards to marketing the interface with good Low Latency , but they are specifically referring to Low Latency Monitoring thru a DSP Mixer Applet / ASIO Direct Monitoring - which is essentially the latency of the AD/DA + a few added samples for arbitration to/from the DSP/FPGA.

Are they specifically marketing the interface in a false manner, of course not , as pretty much all interfaces now have a good DSP powered onboard mixing facility, so direct monitoring sans FX is essentially real time. This however has been available since the late 90's on Windows and isn't anything particularly new or anything to be hyping about IMO.

 

Where it gets murkier is when we are dealing with I/O and RTL latencies monitoring thru FX and/or real time playing Virtual instruments, as that is when the efficiency of the controllers/ drivers come into play. Further to that, the actual scaling performance of the above at the respective working latencies is an area that has huge variables as I noted earlier.

Now to be clear , I am focussing purely on the area of Low Latency Performance for those needing that facility when using Guitar Amp Simulators and Virtual Instruments. For those needing basic low latency hardware monitoring for tracking audio and have minimal focus on larger FX/MIDI/Virtual Instrument environments , then its not as much as a priority.

If the drivers are stable at any given working latency that allows the end user to track and mix with minimal interruption, then all well and good.

The question then is - how many manufacturers using the lesser performing controller/driver options specify the preferred working environments that their interfaces will be best suited ?

We all already know the answer to that, and its an area that is very sensitive to try and approach with them without it getting volatile I have found.

 

I had the unfortunate experience of discussing some concerns with a representative of one of the larger audio interface manufacturers regarding their driver performance, which ended in the rep saying- and I need to paraphrase and tread lightly here -

" The Low Latency Performance aspect that you are focusing on is irrelevant to the majority of end users, and your benchmarks are simply designed to baffle people with bullshit ! "

My reaction after I picked my jaw up off the floor was wow, O.K , so obviously my clients who have these specific requirements are either idiots or deluding themselves as to the advantages they are experiencing using the interfaces displaying the over all better LLP in their respective working environments- i.e. heavy use of monitoring with FX / Virtual Instruments at low latencies.

Needless to say it was obvious that my work has ruffled a few feathers by shining a light on this specific area of audio interface performance , but it also made me even more determined to continue to update the data base and hold the manufacturers to account if they release sub standard drivers.

Till next time .

Vin Curigliano
AAVIM Technology
August 18 2012

Part I | II |III

© AAVIMT 2005-2012