Roon vs Mosaic SQ - Redux

[struts001]

I have always been intrigued by the claims that people hear differences between the same tracks replayed via Roon and Mosaic, and although I have never noticed any in casual listening (most of my listening is via Roon, I only use Mosaic for occasional testing) I have never actually tested this in a systematic way myself. So when some free time opened up I decided to give it a try.

Just to declare my bias, I fully understand that there should be no difference and according to everything we know there can’t be a difference. However I am also prepared to leave a little opening for what we don’t yet know, and that there may be mechanisms at work here that we haven’t identified yet and aren’t explained by our current models. The frontiers of science and engineering are constantly being pushed back and simply put, we don’t know what we don’t know. However I fully accept that going into this I had a clear bias that there shouldn’t be a difference. And I realize that completely undermines the validity of my results, but I couldn’t think of a way to eliminate expectation bias of no difference. So I present this for what it is worth, my findings based on the best test I could devise.

Methodology

Now I don’t claim to be an expert in test design by any means but this is what I came up with. I couldn’t come up with a simple way to implement a double blind test so I settled on single blind. If anyone can see any obvious flaws in my methodology or offer any suggestions (particularly how I can eliminate expectation bias of no difference) please let me know.

1. I selected 10 test tracks (list below), 6 from the library on my NAS (Qnap TS-i410X powered by a HDPlex LPS), a mixture of rips and downloads, and 4 from Qobuz. I tried to include tracks where others here had reported hearing differences between Roon and Mosaic specifcally or that have been suggested for illuminating other system or config differences. The 6 tracks from my own library are very familiar to me, the other 4 are more recent acquaintances.
2. I had my son play these tracks 4 times each, randomly selecting to do so via Roon or Mosaic controlled from the respective apps running on a Macbook Air each time. The randomness was generated by asking Siri to pick a number between 0 and 1. I asked him to note which app he had used each time.
3. I listened blindfolded and noted any differences I heard. Where I heard differences I tried to assign them to one or other system by denoting them A and B.

System

• Roon Server and MinimServer are running on a SGC sonicTransporter i5 (powered by an HDPlex LPS) which is connected via wired ethernet to the Vivaldi Upsampler. I have an Uptone EtherREGEN (powered by an Uptone UltraCap LPS 1.2) and a Baaske medical ethernet isolator between the switch and the Upsampler and the cable to the Upsampler is custom made with the shield lifted at the Upsampler end.
• For this test I decided to use my headphone system, a Kevin Gilmore DIY Stax SRM-T2 I built myself as a participant in the initial group as described in this thread. The output tubes are Mullard XF1 EL34s. The T2 is connected directly to my Vivaldi APEX DAC with 1m runs of balanced Audioquest Firebird. The headphones were a pair of late Stax SR-007 Mk I. This setup is one of the most revealing I have ever had the pleasure to listen to.

Tracks:

1. Waltz for Debby from Waltz for Debby (live at the Village Vanguard) by Bill Evans. 24/192 HDTracks download
2. Walking on the Moon from Walking on the Moon by The Police. 16/44.1 CD Rip
3. Horsin’ Around from Steve McQueen by Prefab Sprout. 16/44.1 CD Rip
4. Fratres from Pärt: Spiegel im Spiegel/Fratres/Fur Alina; by Vadim Gluzman. 16/44.1 CD Rip
5. Afro Left from Leftism by Leftfield. 16/44.1 CD Rip
6. Mendelsson: Piano Concerto No.1, Murray Perahia. 16/44.1 CD Rip
7. Purple from Freedom Flight by Shuggie Otis, Qobuz 16/44.1
8. Beethoven: Op.126 Bagatelles, Alfred Brendel. Philips 1990. Qobuz 16/44.1
9. Bologna 1666, Kammerorchester Basel, Julia Schroder. Qobuz 24/96.
10. Debussy: Prelude a l’apres midi d’un faune, Pierre Monteux/LSO Australian Decca Eloquence, Qobuz 16/44.1

Take 1

So, first track, “Waltz for Debby”. Sounded good. Very good. I usually listen through my speakers and I was immediately struck by how much deeper into the recording I could hear through the cans. Second playing. Sounded good too. I was straining to hear any slight difference while being mindful not to just make something up. Not easy! Focusing on this, focusing on that. Nope, if there was a difference it was extremely subtle. Third playing. Wow! Huge difference. Both the double bass and the piano sounded very different (like someone had moved the mics) and the musicians had sort of moved around in the room. I took the headphones off within a few seconds in shock. DAMN! I didn’t want to hear a difference but this was indisputable!

I was explaining my surprise to my son when a thought suddenly struck me. Damn it. I had forgotten to check that the signal path in Roon was clean and sure enough my Digital Room Correction Filter and its compensating 9dB of procedural gain were both still enabled. Doh! Turns out 1 and 2 were Mosaic and 3 was Roon. Back to the drawing board.

This time I went through Roon with a toothcomb:

1. In Device Setup press Load defaults (just to be on the safe side, cross-fade (not to be confused with cross-feed) is not active during tracks but it does enable SRC).

2. In MUSE ensure:
   1. No presets are set
   2. Headroom management and Sample rate conversion are Disabled
   3. All Filters are removed (not just disabled - not taking any chances)

3. Ask son to verify when playing via Roon that “Signal Path Lossless” is lit.

Take 2

Well, I spent most of the day yesterday on this and I have to say that once the Roon config was sorted, try as I might I couldn’t identify any significant differences at all. Even where I thought I heard something and asked my son to replay the track with the prior app to check I concluded I was risking imagineering a difference. I could never be certain enough of any difference to be able to make a guess of A or B.

So based on my empirical testing I am unable to detect any difference. Either there isn’t one, my system is somehow not revealing it, I am simply not hearing it or my expectation bias is blocking it.

I am guessing that people who are hearing a difference have either inadvertently enabled something in Roon as I had (I think if this test proved anything it was how easy that is to do!), there is some other explanation that applies to their system and not to mine, their listening skills are somehow way more discerning than mine or that they are imagineering it. However I am not going to discount it out of hand just because it contradicts our current model of how things should be working or jibe with my own findings.

My purpose with this post is not to reigntite the theoretical debate, we have done that to death many times. Nor is it to bait you @Anupc, honest! I am genuinely curious to hear the difference if one exists, even if we can’t currently explain why that might be. I haven’t managed to find it but if anyone here who does hear a difference has any specific tracks they can suggest (many thanks to @Miguelito who has already posted one, unfortunately it was the only one I could find) please could you let me know which track, which version (resolution etc) and what sort of differences you hear and I will be more than happy to re-test in my own system.

[miguelito (Miguel Barrio)]

Interesting. I have casually tried this over the years. For some reason I forget, I once tried a specific track. It was Shuggie Otis’s “Purple”, a redbook download from Qobuz - I buy music I really like as things disappear from streaming all the time. Playing this file over Roon and compared to Mosaic was shocking. Even the soundstage was different. I don’t use any DSP at all so that was not it.

I should definitely try this again.

Miguel

[ARDDCS (Anthony)]

Shocking…but which was better Roon?

[miguelito (Miguel Barrio)]

Mosaic on UPnP via MinimServer had a deeper more coherent soundstage. It surprised me so much that I did the same comparison the next day and found the same to be true.

[Anupc (Anup)]

In Roon’s early days, I would have sworn it sounded inferior to Mosaic/UPnP (I may have even posted as much here on dcs.community).

During the Covid19 lock-downs in 2020, partly driven by a debate with Martin Colloms (HiFiCritic), I found the time to set-up an objective testing methodology that completely removes any subjective opinion or biases in evaluating whether or not there’s a difference between Roon and Mosaic (though originally intended to debunk Audiophile Ethernet Switches/Cables, which it also does quite well).

On the hardware side, the setup requires a Vivaldi Upsampler & Clock, and a PCM Recorder; one that can receive a PCM stream and record it, like the TASCAM DA-3000, which has the added benefit of being able to be externally clocked to the same clock source. So, all comparisons are done in the digital domain where there’s no chance of analog noise causing capture sample variations.

On the software side at the time I used Audio DiffMaker which compares two PCM .wav files for any differences. These days there’s also the DeltaWave Audio Null Comparator which does an even better job.

Using a single source file played back through various ways, in this case Roon vs. Mosaic, and the digital outputs from the Upsampler captured, one can then compare the captured .wav files and generate a difference, which you can then listen to for any audible variances, or view with something like Adobe Audition right down to an individual PCM sample.

When the tests are done correctly, you’ll find that the difference between the captured streams is null

[struts001]

Yes, a wise man (was it Confucius?) once said that searching for something that doesn’t exist will take a long time.

I like your methodology because it is compelling, I can’t see any obvious flaws. The only thing I don’t like is that it isn’t subjective so still doesn’t rule out some effect that is “outside the model” (I.e. not coded into the PCM samples). But since I can’t suggest any candidate for what that might be given that the job of the digital delivery pipe is solely to deliver PCM samples I concede that that reservation falls a little bit flat.

Anyway, thanks for sharing. I presume you didn’t fork out for the Tascam just for this test!?

[hifimckinney]

I do not think this method is right. Bring in the speakers too and then record the output from speakers and then compare. I do not think anything changes in digital domain but the final experiences (what we hear) are different.

[Anupc (Anup)]

@struts001

I like your methodology because it is compelling, I can’t see any obvious flaws.

Glad to hear it. It’s really quite a straightforward approach.

The one feature on the MSB DAC worth copying by dCS is MSB’s bit-perfect testing; using their specific test files, the DAC will indicate whether it’s receiving a bit-perfect stream. Designed to eliminate all subjective arguments about the quality of bit-perfect transport software/platforms upstream.

@struts001

I presume you didn’t fork out for the Tascam just for this test!?

Yeah, I’ve had the unit since my Scarlatti stack days actually, so, well over a decade old. It was mainly used with my Reel-to-Reel deck for realtime A-D conversion into my dCS stack ever since I eliminated my analog Pre-amp (but has since been replaced in that job when I acquired a mint-condition dCS 905 ).

TASCAM discontinued the model just prior to Covid19. Pity really as it’s a very versatile product; it does any-to-any A-D/D-A/D-D/Recording including DSD128. And it records directly to SD-Flash, so there’s no need to fiddle with a Computer next to it (unlike similar machines from RME etc). The only thing it doesn’t do is DXD (unlike the 905).

@hifimckinney

I do not think this method is right. Bring in the speakers too and then record the output from speakers and then compare. I do not think anything changes in digital domain but the final experiences (what we hear) are different.

You might want to take your time and rethink that carefully

(When the AES digital stream flowing out of the Upsampler is identical between playing the same source file via Roon vs. Mosaic, there can be no difference in the DAC output, and hence at the speaker output).

[sourav (Sourav Mazumder)]

@Anupc What Reel-to-Reel deck do you use ? Just curious.

Regards,
Sourav

[Anupc (Anup)]

I have a run-of-the-mill Revox PR99 MkII. Wish I had the space for a Studer A80, or the Metaxas Tourbillon!

[sourav (Sourav Mazumder)]

@Anupc wow. Saw Revox releasing a limited edition of 10 B77 MkII at 13.5K Euro.

I never got a chance to listen to an R2R deck. I have been watching a few on eBay for the last few years but have not yet got the confidence to buy a used one given that too many things can go wrong.

My follow-up question to you - do u think analog reproduction from a R2R system can be beaten (or paralleled) by digital reproduction by the world’s top digital reproduction system like dCS (given all other components are the same)?

(I know I’m deviating from the original topic. But could not resist asking this qs - ha ha)

Regards,
Sourav

[barryr1]

I have many years of r2r experience and none now. Given that you are purchasing expensive source material and playing on well maintained heads i think r2r is still superior but it takes commitment. Inho

[jacobacci (Rudi)]

Re. Revox equipment: Pascal Vogel of der Reparateur in Winterthur / Switzerland has quite a stock of rebuilt Revox equipment.

[Anupc (Anup)]

@sourav

do u think analog reproduction from a R2R system can be beaten (or paralleled) by digital reproduction by the world’s top digital reproduction system like dCS (given all other components are the same)?

The answer is, it depends.

Barry’s spot on, the main issue with R2R is maintaining the machine and problems getting good quality source tapes, which are generally very expensive and limited in selection. Good quality tapes are usually north of $400 per tape. For example, from;

https://tapeproject.com/catalog/

Reel To Reel|Acoustic Sounds

If you’re planning to get stock old tapes from all over the place (like Ebay), they’ll sound just so so and can’t really compare to the thousands of good quality digital files/discs.

[sourav (Sourav Mazumder)]

@Anupc @barryr1 @jacobacci thanks for the inputs.

Yes, I’m aware of the cost of the new tapes. That’s also one of the reasons for staying away from R2R. Hope the tapes will be more cost-effective in the next few years.

Regards,
Sourav

[Urbanluthier]

@Anupc

TASCAM discontinued the model just prior to Covid19. Pity really as it’s a very versatile product; it does any-to-any A-D/D-A/D-D/Recording including DSD128. And it records directly to SD-Flash, so there’s no need to fiddle with a Computer next to it (unlike similar machines from RME etc). The only thing it doesn’t do is DXD (unlike the 905).

Could you remind us which Tascam unit you had? I currently use an RME babyface pro FS to do A to D conversion from my phono stage into my Bartok. On the plus side one can adjust both the input and output levels to ensure an unclipped signal is sent to the Bartok. it is a little awkward to use however

[Anupc (Anup)]

It’s the TASCAM DA-3000. While the model has long been discontinued, you can still find plenty of them, some BNIB, on Ebay and such. Sound quality wise your RME might be a better performer considering it’s much newer, but the TASCAM is very flexible, including level adjustments etc. and being able to Word-Clock sync to your dCS system.

If you’re seriously considering it, pull down the manual and have a browse through for suitability before you do anything else.

[struts001]

Just saw that Hervé Delatraz just had a good old vent about Roon on the DarTZeel FB page:

Does ROON suck?

Long time ago I wrote about MQA insanity, and audio God heard us.

Today I have serious doubts about ROON, which brings nothing but bugs, complaints, lack of support, and dictatorial policy.

It is one year we are waiting for our certification.

I know we are not the only ones in this situation.

Nevertheless, this is just unbearable and completely irrespectuous.

Especially when DNLA/UPnP allows for untouched playback quality.

Sorry, I had to write it.

[Anupc (Anup)]

Good thing he didn’t take a swipe at Roon’s sonic quality, that would have been like shooting himself in the foot

[glevethan (Gregg)]

@struts001

Especially when DNLA/UPnP allows for untouched playback quality.

I interpreted this as a backhand reference to Roon’s sonic quality