Tuesday, 28 February 2017

More Ripping Yarns

Background

In this post, I wrote about ripping CDs and the apparently large number of errors you get when you actually bother to check for them. I actually asked the question "Why don't programs provided on CD fail because they have so many bit errors that the code is defective?".  I discussed this with some ex-work colleagues over lunch yesterday, and we reckoned something else must be going on... A bit of research reveals that that is in fact the case!

The underlying storage mechanism is the same for CD-DA and CD-ROM (and its derivatives), but essentially, the "file system" is different.

Firstly, all CDs use the same encoding scheme - 

The Red Book specifies the physical parameters and properties of the CD, the optical "stylus" parameters, deviations and error rate, modulation system (eight-to-fourteen modulation, EFM) and error correction facility (cross-interleaved Reed–Solomon coding, CIRC), and the eight subcode channels. These parameters are common to all compact discs and used by all logical formats, such as CD-ROM(wikipedia). 

The CIRC encoding system basically wraps lots of extra bits around the data as part of the encoding. It also scatters the logical audio frames around a number of physical frames, so that disk faults can be more easily recovered from, since they will generally degrade recoverably a number of logical frames and not obliterate the one with the error.

However, CD-ROM, as opposed to CD Digital Audio, adds in more error checksums and another layer of error correction... because it is assumed that computer data cannot be interpolated like audio data, but must be bit-perfect!!

The structures used to group data on a CD-ROM (the Yellow Book) are also derived from the Red Book. Like audio CDs (CD-DA), a CD-ROM sector contains 2,352 bytes of user data... To structure, address and protect this data, the CD-ROM standard further defines two sector modes, Mode 1 and Mode 2, which describe two different layouts for the data inside a sector.

Both Mode 1 and 2 sectors use the first 16 bytes for header information, but differ in the remaining 2,336 bytes due to the use of error correction bytes. Unlike an audio CD, a CD-ROM cannot rely on error concealment by interpolation; a higher reliability of the retrieved data is required. 
  • Mode 1, used mostly for digital data, adds a 32-bit cyclic redundancy check (CRC) code for error detection, and a third layer of Reed–Solomon error correction leaving 2,048 bytes per sector available for data. 
  • Mode 2, which is more appropriate for image or video data (where perfect reliability may be a little bit less important), contains no additional error detection or correction bytes, having therefore 2,336 available data bytes per sector. 
Note that both modes, like audio CDs, still benefit from the lower layers of error correction at the frame level.
Format← 2,352 byte sector structure →
CD digital audio:2,352 (Digital audio)
CD-ROM Mode 1:12 (Sync pattern)3 (Address)1 (Mode, 0x01)2,048 (Data)4 (Error detection)8 (Reserved, zero)276 (Error correction)
CD-ROM Mode 2:12 (Sync pattern)3 (Address)1 (Mode, 0x02)2,336 (Data)
So I guess CD-DA is more liable to uncorrected errors, because nobody gives a toss about audio errors... 

Ripping programs can
  • Read a sector multiple times and check they get the same result
  • Use the "C2" feature implemented in many drives that allows the drive to report potential errors as guidance to the reading program
but of course, nothing can actually fix the errors if they can't be recovered, even if detected. Boo. Oh well. Off to eBay to source another CD-ROM drive...

New Ripping Software - Perfect Sound, Forever!! 

I've spent a lot of time trying to use XLD to rip my copies of Wagner's Ring to disk, and it's been very hard work. I don't know why these are so hard, but there are lots of errors, an they almost never match the AccurateRip database. So I thought I'd try dbpowerasp, the ripper produced by Spoon/Illuminate, the people who own the AccurateRip database. 

The software seems to work well in itself, it needs some adjusting in terms of tagging and filenames, but it it is taking an extraordinarily long time to rip the firstdisk. I started ripping last night, it read the first track, then announced it would retry 2000+ errors sectors. Great. I left it running when I went to bed... and when I came down this morning it was onto the 3rd track! It's 15.19, and it's still ripping the 4th, with 2676 frames to retry! The first track took 6.30 hours to rip, with its 2000 duff sectors. What is going on?! Are all my CDs going to be this hard? I reckon I should forget ripping CDs, and just play stuff off Tidal, because I can and life's too short!

Saturday, 25 February 2017

Network Problems

Debugging the dodgy network... On the current running system (cheated on the systemd entry :-) ):

[root@Frontroom ~]# journalctl --output=short-iso | grep "eth0: carrier lost"
2017-02-22T18:35:43+0000 Frontroom systemd[1]: Time has been changed 
2017-02-24T04:35:42+0000 Frontroom dhcpcd[677]: eth0: carrier lost
2017-02-25T20:28:51+0000 Frontroom dhcpcd[677]: eth0: carrier lost

giving about 143,589 and 122,399 seconds between events. The lease is for 86400 seconds.

Annoyingly the Rune image clock starts on 22 Feb 2016, which is annoyingly close to the current date (25 Feb!). The --output=short-iso is required to display the year on the clock...

More data required... so I need to make the journal files non-volatile...

By default, the journal stores log data in /run/log/journal/. Since /run/ is volatile, log data is lost at reboot. To make the data persistent, it is sufficient to create /var/log/journal/ wheresystemd-journald will then store the data:
mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal

Tried that. Let's see if there are any more entries and if they survive a reboot!
I note that no entries have appeared in that directory yet... I wonder if I have to do something more?

Update: I edited /etc/systemd/journald.conf to change the Storage=volatile to Storage=persistent, then did systemctl restart systemd-journald.service to restart the logging with non-volatile storage. Hurrah - works!

Update:
The router log has the following entries around the time of the last outage (relevant bold):

[DHCP IP: (192.168.1.16)] to MAC address 00:BB:3A:33:62:DA, Saturday, Feb 25,2017 20:35:46
[DHCP IP: (192.168.1.3)] to MAC address B8:27:EB:25:C4:1A, Saturday, Feb 25,2017 20:28:53
[UPnP set event: Public_UPNP_C3] from source 192.168.1.11, Saturday, Feb 25,2017 20:25:28
[UPnP set event: Public_UPNP_C3] from source 192.168.1.11, Saturday, Feb 25,2017 20:10:27
[Time synchronized with NTP server] Saturday, Feb 25,2017 20:09:33
[Internet connected] IP address: 192.168.0.3, Saturday, Feb 25,2017 20:08:45
[UPnP set event: Public_UPNP_C3] from source 192.168.1.11, Saturday, Feb 25,2017 19:55:28
[UPnP set event: Public_UPNP_C3] from source 192.168.1.11, Saturday, Feb 25,2017 19:40:27
[Time synchronized with NTP server] Saturday, Feb 25,2017 19:38:43

That doesn't tell me a lot, but at least they agree! And there is no apparent fault with the router. I do note that my phone gets a lot of IP address assignments, at sub-10 minute intervals, which I notice. Maybe the signal is bad for that?

As an experiment, I might try plugging in the network extender I have and using that to provide a wired interface, to see if that makes a difference. The office RPi is using one of those and has no wired problem, although the wireless connection does fail a lot, despite using the wireless extender in the same room!!!

Update 2017-02-28:

Well, the link has been up now for 64 hours, without failing, using the Powerline network extender and a new, short Cat5 cable. This is positive. Maybe it's like my N years spent complaining about the laser printer's inadequacy and the uselessness of the drivers, when it was the cable all along...

Rune vs. Roon?

Up until now I've used Rune as the audio player on my Raspberry Pis. It's a cut-down ArchLinux using mpd as the player and a PHP-based web interface and control server to manage and configure the mpd and o/s.

It works ok most of the time. However, and maybe this is not a problem specific to Rune, it is fairly flakey in respect of network connectedness. Yeah, on startup it usually connects and exposes itself as host.local, connects to the server for ripped data, is ready to receive DLNA/uPnP streams and reports to interested parties like Linn Kazoo and Lumin on iOS.

It's hard to work out what's going on, because the problem is network connectivity, so beaming in on SSH is not possible, and a restart appears to kill the log files - I need to research this a bit more. The log shows outage of the wired connection, which is weird... These entries show it's down for around 1 second! What's going on?

Feb 25 20:28:51 Frontroom kernel: smsc95xx 1-1.1:1.0 eth0: link down
Feb 25 20:28:51 Frontroom dhcpcd[677]: eth0: carrier lost
Feb 25 20:28:51 Frontroom dhcpcd[677]: eth0: deleting route to 192.168.1.0/24
Feb 25 20:28:51 Frontroom dhcpcd[677]: eth0: deleting default route via 192.168.1.1
Feb 25 20:28:51 Frontroom avahi-daemon[275]: Withdrawing address record for 192.168.1.3 on eth0.
Feb 25 20:28:51 Frontroom avahi-daemon[275]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.1.3.
Feb 25 20:28:51 Frontroom avahi-daemon[275]: Interface eth0.IPv4 no longer relevant for mDNS.
Feb 25 20:28:52 Frontroom ifplugd[280]: Link beat lost.
Feb 25 20:28:52 Frontroom dhcpcd[677]: eth0: carrier acquired
Feb 25 20:28:52 Frontroom kernel: smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Feb 25 20:28:52 Frontroom dhcpcd[677]: eth0: IAID eb:25:c4:1a
Feb 25 20:28:53 Frontroom dhcpcd[677]: eth0: rebinding lease of 192.168.1.3
Feb 25 20:28:53 Frontroom dhcpcd[677]: eth0: probing address 192.168.1.3/24
Feb 25 20:28:53 Frontroom ifplugd[280]: Link beat detected.
Feb 25 20:28:54 Frontroom ntpd[348]: Deleting interface #5 eth0, 192.168.1.3#123, interface stats: received=73, sent=75, dropped
Feb 25 20:28:54 Frontroom ntpd[348]: 178.79.160.57 local addr 192.168.1.3 -> <null>
Feb 25 20:28:57 Frontroom dhcpcd[677]: eth0: leased 192.168.1.3 for 86400 seconds
Feb 25 20:28:57 Frontroom avahi-daemon[275]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.1.3.
Feb 25 20:28:57 Frontroom avahi-daemon[275]: New relevant interface eth0.IPv4 for mDNS.
Feb 25 20:28:57 Frontroom avahi-daemon[275]: Registering new address record for 192.168.1.3 on eth0.IPv4.
Feb 25 20:28:57 Frontroom dhcpcd[677]: eth0: adding route to 192.168.1.0/24
Feb 25 20:28:57 Frontroom dhcpcd[677]: eth0: adding default route via 192.168.1.1
Feb 25 20:28:57 Frontroom dhcpcd[677]: eth0: removing route to 192.168.1.0/24
Feb 25 20:28:59 Frontroom ntpd[348]: Listen normally on 6 eth0 192.168.1.3:123
Feb 25 20:28:59 Frontroom ntpd[348]: 178.79.160.57 local addr 192.168.1.12 -> 192.168.1.3

Feb 25 20:28:59 Frontroom ntpd[348]: new interface(s) found: waking up resolver

Anyway, I've been investigating Roon, which I first encountered at Keith's house when I heard the Kii Audio Threes. He's running a MacMini as the Roon Core, the Rune Control on an iPad and using the Mac Mini as a Roon Output. it all seems very well thought through, and pretty robust. The one drawback is that it costs money, continuing money, at $120/year or $500 lifetime! It does have a fantastic interface, some amazing metadata capabilities and it also interfaces utterly seamlessly with Tidal. I'd have to provide a sensibly powerful box for the Core, I suspect an RPi isn't going to cut it, in fact, the Roon doco explicitly says it won't. 

But all this is moot if my hub is letting me down - how do I debug the network interface??