Still Livin' La Vida Linux

It's been over a year since I wrote about my conversion to a Linux based digital media environment, and since it's the holiday season (or just after) I thought it was time to update the story, and describe some new Linux based devices I'm using that others might find useful.

In the original essay I spoke about converting all my physical CD's to digital files into the patent-free FLAC (Free Lossless Audio Codec) format. At the time I was looking at the Sonos multi-room music system to play the files. I took the plunge and ended up buying a four room system last year. They aren't cheap, but they're the most robust devices I own. They never crash (and for a device as sophisticated as this that's a real pleasure). I've owned televisions that fail more often than the Sonos boxes.

The Sonos platform is based on an embedded Linux kernel, but the user interface is completely custom, using separate Linux based controller devices to select play lists (although you can also use an iPhone). This is embedded Linux done right. It is incredibly easy to use. My wife, who is an avowed technophobe, finds it simple to program the music she wants. I'm trying to find excuses to buy more of these devices as it's such fun to have all your music available anywhere you want, or even listen to Chinese traffic radio broadcasts (available on the Sonos over the Internet) – although I'm running out of rooms to put them in :-).

I was so impressed with the ease of use of the Sonos that I ended up buying one of their all in one (including amplifier and speakers) S5 systems and giving it to my brother in the UK for Christmas. Bear in mind this is someone who would call me if his desktop background on Windows got accidentally changed and scream down the phone "my computer is broken!", so I really didn't want to give him a system requiring any kind of technical support. I had converted him from his old Windows XP system to a desktop Linux system running Ubuntu earlier this year, and had only one support call since then (involving OpenOffice) and I really wanted to keep that score. So far it's been a complete success, although I did have one tech support call caused by him resting his chessboard on top of the "volume minus (-)" button on the unit, and wondering why he couldn't hear any sound or turn up the volume using the controller.

As I explained to his wife later, just when you think something is foolproof, the universe gives birth to a better fool.

In order for the Sonos to play the music, the files need to be available on a Network Attached Storage (NAS) device. Now this is where my Samba experience becomes useful, as Samba is a core component of most home NAS devices. As the total amount of music I have is under 200Gb, I was able to re-use an old HP Media Vault NAS box I had spare at home. The HP Media Vaults are Linux and Samba based NAS units. This is one of the original units, not the modern versions they're selling now. It's so old it still uses ReiserFS as its primary file system, which gave me some concern before using it, but it's been fine as a mirrored 200Gb storage unit. It's really easy to set up with any client as it has a web-based administration interface, and as a basic music store it's given me no trouble. Point the Sonos devices at a Samba share on the box, give them a user name and password to use and they'll go off and index all your music with no troubles at all. A few minutes later you're listening to anything from your complete collection.

I had to set up a NAS server for my brother as well, in order for them to encode their CD's for their Sonos unit. This is where I got lucky. I was given a modern Netgear ReadyNAS NVX NAS server with two 500GB mirrored drives. It's a Linux based NAS, using Samba to provide Windows network storage and I'm very familiar with the developers.

It's built like a tank (which I didn't appreciate whilst having to carry it in my luggage from San Jose, California to Sheffield in the UK) and is incredibly robust. It's really easy to set up and use. It has a modern web interface that is very intuitive to use, has two gigabit ethernet interfaces and hot swappable slots for four more drives. I was sorely tempted to keep it for myself but my brother also needed something to back up his work so his needs ended up being greater than mine. The fan was a little too noisy for them to tolerate it sitting in their kitchen so I ended up hiding it in the attic room connected via a D-Link Powerline Ethernet-over-power setup. It made me weep to see the gigabit network port throttled by that bottleneck (the ethernet over power speed doesn't get close to gigabit) but it works well enough in their environment. I expect after a while they'll forget the server exists, as it just sits there silently doing its job in the best traditions of an appliance.

Last year I spoke about tackling the problem of my ever growing DvD movie collection. I managed to solve that problem this year with the help of a really clever new device. After looking at all the options available I settled on the SageTV HD Theater system. At under two hundred dollars they're cheap enough to get one for each television and they're completely silent. They play anything. They can play disk based DvD images, DvD images stored as an .ISO file, AVI files, music in any non DRM (Digital Restrictions Management) format, files in the the Open Source Matroska Media format, and any variant of MPEG format I've ever seen. They also play video directly off the Internet and allow you to watch YouTube videos directly on your TV. Makes you realize how bad flash video encoding is when you see it that large :-).

They have HDMI and digital audio outputs (both coaxial and fiber optic), and are small enough that they're pretty unobtrusive next to a big screen TV (except for a really annoying blinking ethernet light that everyone ends up covering with black tape). When not in use they default to a really nice slideshow application displaying all your photographs that they have detected on the NAS storage.

Again these devices are Linux and Samba based, and you can even telnet into the box as the root (super) user and mess around. The user interface is really nice (written in Java), with an easy set up, and they also connect to a NAS server via the Linux kernel CIFS/SMB (Common Internet File System protocol – the protocol that Samba implements) to play movies and music. I only use them for movies as I have the Sonos boxes for music. My only problem with them is they're not as robust as the Sonos boxes, and occasionally I find I have to reboot one of them that seems to have gone to sleep all by itself. It's not often enough to be irritating, but I wish they were as reliable as the Sonos boxes.

One thing I love about them is that the developers at Sage are really fast at fixing any problems you report. It's a matter of a few minutes to update to a new version over the Internet (and just like a Linux distribution you are in control of when you update, not them) and new versions are posted very regularly. So far they've fixed a problem I found navigating a certain DvD menu, an issue with aspect ratio display, and rather embarrassingly for me they found a bug in the latest version of the Samba server by the way they were driving the Linux client. I always test the latest Samba versions on my home server ("eating our own dogfood" as the saying goes) so hopefully I catch issues like these before regular customers and vendors get to see them.

Back in the original story I rather quaintly estimated I'd need "one or two terabytes of storage" to cope with my DvD collection. Ha. Ha. Ha. I severely underestimated my squirrel like tendencies with movies. So far I have about twenty terabytes worth of movies on those annoying physical disks, and that's just with regular DvD's. I haven't moved to BluRay yet as the DRM on this format is so offensive that I know of no reliable way (yet) to copy the data from these disks onto network available storage.

I have a perfectly serviceable Ubuntu desktop at home (with 8GB of memory) running Linux so I decided to turn that into my movie media server. I bought an eight bay hotswappable eSATA TowerRaid box from the local Frys Electronics in San Jose, and eight 1.5 Tb disks. That gave me 12 Tb of storage total, but given the failure rates of large drives I ended up using the TowerRaid box as a JBOD (Just a Bunch Of Disks) tower and using the Ubuntu Linux kernel software RAID5 code instead of the custom Linux RAID driver code that came with the TowerRaid controller card. I formatted it with ext4 and set up the latest Samba code (from our git tree) to provide NAS service for the 9.5 Tb available after the RAID formatting. Other than the problem I reported earlier that the SageTV boxes found it's been a completely reliable setup, and all for under fifteen hundred dollars.

I still don't have enough space for all my movies, so I've been migrating them onto the NAS server as I want to watch them. It seems to work, and hopefully by the time I'm running out of space on the array I'll be looking at upgrading to 4TB disks (or maybe even Solid State Disks by then). I've been using a combination of the Linux "vobcopy" command or the excellent KDE created "k9copy" command to copy the DvD's onto my home server, both of which seem to deal nicely with the evil CSS (content scrambling system) DRM that DvD's are encoded with. This time I guess it's the Motion Picture Association of America (MPAA) who would be calling me a criminal, rather than the Recording Industry Association of America (RIAA) but other than that it's the same thing I've been doing with my music for my own enjoyment, so I don't feel bad about it. Let me know how you have been solving your music and movie storage problems as you too move to "La Vida Linux", I'd love to hear from you!

Jeremy Allison
Samba Team.
San Jose, California.
30th December 2009.



Comments

my own setup

Qnap nas device (with a power consumption of only 14W)
Live XBMC installation on an ASROCK mobo based mini pc (with a power consumption of 35W)
full gigabit switched network (Netgear ProSafe 16 port) using Cat6 wiring
Twonky and UPNP services enabled for media
Apache and MYSQL as our intranet server using Joomla for managing it all
plans to install an all in one touch device in the kitchen (where else?)
looking around for a not so expensive built in touch panel for in the bathroom (must be waterproof)

UPNP

Yeah I also thought about UPNP but I didn't want to have another service running, plus that protocol has no authentication. Scary IMHO.

Jeremy.

Mediacenter

I recently completed building my own debian-based media center PC and am very happy with it (after many hours of fiddling). It's housed in a Lian Li box with 4 removable SATA drives, in addition to one non-removable system drive and DVD. This gives me 8 TB of storage, 4 TB for the original ISOs and 4 TB for backup. I've copied all my DVDs to the system as ISO files and play them using the Kaffeine media player. The system works flawlessly, except for a few DVDs/ISOs which give me the dreaded "nav packet not found" error.

I also use the system as a Squeezecenter server so I can stream my FLAC audio collection wirelessly to my Logitech Transporter, which has a decoder that is actually superior to my old NAD CD player. The Transporter also uses my wireless network for accessing Internet radio streams. Sweet.

With the exception of the problem DVDs/ISOs, I'm very pleased with this single-box solution to all my media needs.

BTW--It's stock Lenny OS/KDE interface with a newer kernel for the wireless NIC and a few newer media packages that I hoped would solve the "nav packet" problem but which failed to do so.

My Setup

I have a HTPC/server running with 6TB storage (6x1TB), playing music through MPD (awesome music player, you can control it through telnet and clients created for it, virtually from any device) and MythTV for my television needs.

I also have a Popcorn Hour connected at the living room for others to enjoy the movies I store on my computer, it's also linux based box that plays pretty much everything over ethernet (samba/upnp/nfs) and hardly uses any power even when playing 1080p stuff.

traveler_AL

i have a dreambox 800 HD DVR .. it does the primary job to view all pay tv packets for free via cardsharing :D .. and secondary jobs as streaming HD movies from pc (any os ) or playing music ..
and internet tv too as some images have internet browser ..
linux ftw

Multiple Devices + samba server

Well, for me, $99 per TV device is my limit, now. I reuse an old PC, old laptop, old TiVo and old MediaGate device to get media to the rooms desired.

Samba Media Server
# Athlon XP2000
## JBOD 4x320GB disks (just under 1TB usable); Addonics 4M
## 2GB RAM - most becomes disk cache anyway
## Ubuntu Srv LTS
## GigE NIC and switch
## RAID5 - using Linux software RAID, mdadm, even though I have a Promise RAID card, the drivers for it required a kernel downgrade. Software RAID rocks.
# TiVo Series2 - there's nothing like a TiVo.
# MediaGate MG35 - no disk, just network playback $99 (5 yrs old)
# WD TV Live - supports newer codecs and HDMI $99
# Laptop running Win7 MC
## Hauppauge 950Q HD Capture card

GigE wired access upstairs and WiFi access down.

The main thing that I do, which you don't is convert all DVDs into xvid. This lets me skip the commercials. I hate commercials. Also, with proper encoding settings, file sizes are reduced by 50% - 75%. 4 xvid-ac3 encoded movies fit in the same space that 1 MPEG2 encoded movie fits. Cartoons compress the most with ZERO visible impact.

The MediaGate is a linux device, but doesn't have the power to playback some of the newer codecs like h.264. Also, it can't display any content with more than 720 lines.

The WD-TV-Live is a new addition (thanks Mom!) since Xmas. It plays HD content, but they have a way to go to a user friendly, yet workable, GUI. All the reviews love this device, but it lacks some simple features and caches content lists WAY TOO LONG. I think this runs Linux.

Win7 Media Center ... first, if the software wasn't given to me, I wouldn't have it. The Hauppauge ClearQAM recording is why. 7MC supports the device well even if the GUI is slow to respond on a 2GHz C2D w/4GB, but the HiDef recordings are beautiful. I use it with ClearQAM - about 100 channels on my cable system. Convert recordings into xvid/ac3 at whatever the recorded resolution was.

The samba server performs many other tasks. It is the central backup server for about 10 machines receiving rdiff-backups of Xen images and incremental Windows7 backups nightly.

TiVo - I use kmttg (old perl version) to archive TV and premium movies from cable into xvid/mp3 files. The TiVo accesses audio files from my collection using an old Galleon Java server running on the Athlon.

Eventually, the recorded content migrates to the Samba server and batch xvid conversion occurs. I prefer xvid still for backwards compatibility with the MediaGate.

As the disk array fills, I burn DVDs of the media with par2 files to help correct media errors. Eventually, most of the content is removed to make room for newer content. We all know that RAID is not a backup, so I'll continue to burn DVDs. Reconverting all that media is too time-effort-expensive NOT to backup.

While my setup isn't perfect, the real world often isn't. Without Samba and Linux, it would be much more costly.

I was wondering about transcoding..

the DvD's into something smaller. But I use subtitles a lot and keeping the DvD menu format makes this easy. Maybe conversion to .MKV might work as you can select subtitles from there.

I don't worry about the commercials as the SageTV box lets you skip all the non-skippable bits anyway :-).

Jeremy.

Questions about SageTv

Reading a few reviews I found

http://arstechnica.com/gadgets/news/2009/05/budget-upgrades-the-sagetv-h...

"""As you navigate around the interface, there's a noticeable lag as you move from icon to icon. This lag gets much worse if you have to load a whole new screen—for instance, if you clicked on an icon or if you're entering or exiting a folder in the browser view. This screen lag makes browsing folders fairly painful, and it's the number one strike against the SageTV."""

This was May 2009. Did you experience this lag? Is it now gone?

It's still there...

But not as bad as described. Also you learn to compensate :-). Yeah, the UI on the SageTV box isn't the fastest thing in the world, but I'm ok with it mostly because of the features and number of formats it will play.

Jeremy.

mkv/x264 ?

Have you thought about converting your DVD's to MKV and x264 format ?

I've recently started using Handbrake, which makes it dead simple. And the quality is simply stunning. The average DVD comes down from 5.0 GB to between 800 MB and 1.5 GB - a good saving.

I started of with a Netgear

I started of with a Netgear EAV8000 which I was very disappointed with mainly because whatever the forum beta tested suggested netgear coders would not allow direct browsing of media folders, you have to scan your network drives. Yes this device doesn't have its own hard drive though you can plug in a USB device. When my friend saw it in my bedroom he said "Its not a brick, its a paving slab!". Also it will not play any HD stuff above 720p, the lie like cheap japanese watch (Quote from UK 70's)

All my media was stored on my Debian Linux box (at the time a 400MHz mini box) and on my XP windows BOX a 3GHz Mini box. The netgear could never get access to the XP shared drives but always got access to the Linux samba shares, the XP also could access the samba shares. This has been going on for years and I never know what is wrong, I just have to change settings in XP to get it to access the samba shares, its at that point the the XP box stops sharing its drives with the Don't have permission, contact myself alert.

Then last year I got the DIVCO TViX 6500A, fantastic HD player, plays everything so far. Got a Green 1Tb WD drive for it. It has ftp/samba servers, can be a USB drive, can have USB devices like blueray players into it etc etc, anyway it was at this point 3GHz XP box died :-( So bought new quad core cheapy installed XP finally (Was trying VMware @ hardware level) so could install XP and Debian, in end installed dual boot. The old XP machine went in the MAGIC box (A network managers box used to keep completely broken but cant through away hardware cos ones suspects its fixable when have time), ebryone should have one.

Six weeks later during a panic with the 400Mhz Linux box dying too, the old XP 3GHz was taken out the MAGIC repair box and the old linux drives plugged into her, switched on and Blimey she started up!! Even better and I love Linux for this, it just booted and carried on working. Try getting a windows box to do that unless its a virualised windows.

The Linux box is called Jill, she does all my home automation x10 with heyu software and IRtrans for remote controls, she speaks using festival and use motion to say hello to me when I pass by her room ;-) She also controls the TViX and TV's and XP machine.

Sonos

But they don't support Linux...

I also live la vida (GNU/)linux

Nearly all my devices run GNU/Linux:
*my freerunner does
*my htcdream is capable of it
*my 2 bug devices do
*my computers and router(pentium III 500)do(I've also an unused wrt54gsv4)
Today I thought....what about installing mpd(music player daemon) on my bug device.
I compiled it(bitbake mpd),synced with my repos,and installed it.
In no time I got my phone(freerunner with pythm) and my desktop computer controlling mpd that was running on my bug device,over wifi.
Nice.
I've still some wifi stability issues on the bug device part but maybe I could fix them installing wireless-compat.
The music reside on a 16GB microsd that is in an usb->microsd adapter that is connected to the usb port on the von hippel module on the bug device.
There is not enough place for all my music(maybe I should use samba or NFS) but in another hand I use the same microsd for all my devices(the htcdream is principally used as an ogg creative common music player)

Denis.

Back to top