Tag: h.264

Categories
Other

Cutting the Cord: Free Whole-house DVR without Cable TV

Buh Bye, Time Warner Cable!  Hello, cleaner, crisper OTA HDTV!

IMG_6831

After years of marginal support for CableCard-based DVRs (a Moxi and a TivoHD), incessant rate increases, and interminable waits on hold for support, I finally decided to do away with my $100/month cable bill.

Of course, it’s not as simple as just canceling the service.  Thanks to the aforementioned Moxi and Tivo, the DW and I are now very accustomed to DVRing our favorite shows (watching live TV is so 48 seconds ago,) so we needed the same capability if we were to get our programming over the air.  The solution would also need to have a high WAF (wife acceptance factor,) meaning it is easy to navigate and use.

Benefits

  1. Free!!! After NRE (non-recurring engineering, or setup) costs
  2. Depending on number of TVs supported, NRE is recouped within months, saving $100/mo
  3. Whole-house DVR (or at least to every TV with an AppleTV connected)
  4. Consistency–all TVs now provide same look and feel with respect to DVR usage
  5. No more monthly support calls when the tuning adapter(s) goes out!!!

The Setup

My solution is a combination of the following hardware:

(Street price of the each piece of hardware, with the exception of the iMac, is less than $100. So for me setup costs were less than $400.)

Along with the following software:

  • Elgato EyeTV v3.6 (DVR Software)
  • mc2xml v1.2 (for guide data)
  • iTunes (for sharing DVR recordings)
  • A couple of custom AppleScripts (for EyeTV and iTunes cleanup and organization)

(The EyeTV software I’d purchased long ago, but even now it is less than $80 at Elgato’s website.)

Wiring Up TVs and HDHomeRun

Nothing too complicated here–not even worth taking pictures.

  1. Run a coax from the OTA antenna into the drop amp input.
  2. From the drop amp outputs, run a coax into each of the HDHomeRun inputs.
  3. From the drop amp outputs, run a coax to each TV that will tune OTA signals (for live TV.)
  4. Connect a network patch cable from each HDHomeRun to your router/switch.
  5. Apply power to the drop amp and the HDHomeRun units.
  6. Connect an AppleTV to each TV, connect network patch cable (or use wifi) to each AppleTV

Mac Setup

Follow installation instructions for the individual software packages.

  1. The iMac must be on the same network as the HDHomeRun units.
  2. Install EyeTV software
  3. Run EyeTV,
    1. Choose EyeTV | EyeTV Setup Assistant
    2. Follow wizard to detect HDHomeRun units and scan for channels
    3. Skip ‘TV Guide’ or TitanTV account setup steps
  4. Plug Turbo.264 HD into a USB port
  5. Install mc2xml
  6. Configure mc2xml for use with EyeTV (a good reference can be found here.)
  7. Go back to EyeTV and set the EPG setting for all channels to XMLTV
  8. In iTunes, turn on Home Sharing (File | Home Sharing | Turn On Home Sharing)

AppleTV Setup

The AppleTVs must be on the same network as the iMac. Turn on Home Sharing (Settings | Computers | Turn On Home Sharing) using the same account used in iTunes above.

Customizations

Automate Guide Data Download

First a script to simplify things, mc2xml.sh:

#!/bin/sh
cd /Users/girls/guide
./mc2xml

# open EyeTV with file
open -a EyeTV /Users/girls/guide/xmltv.xml

(I used my girls’ account, ‘/Users/girls’, on the iMac since this is the account which is setup for autologin on the iMac and the account most used.)

And a simple cron job, as mentioned in the reference above, to automate things:

00 20 7,14,21,28 * * /Users/girls/guide/mc2xml.sh

This job will download new guide data on the 7th, 14th, 21st, and 28th of each month. A daily download is not necessary since the guide data is generally available two weeks in advance. But to play it safe, a once a week download is used to allow for a 50% failure rate on the update.

/Library/Application Support/EyeTV/Scripts/TriggeredScripts/ExportDone.scpt

EyeTV allows users to extend its functionality through the use of event handlers.

These event handlers are implemented in AppleScript and are invoked by name, e.g. ExportDone, if a script with the event name is found in the folder /Library/Application Support/EyeTV/Scripts/TriggeredScripts.

To make the recorded shows easier to navigate on AppleTV, the following script adds the date of recording to the name of the show in iTunes, e.g. ‘Elementary (1/4)’ in the image above.

It also resets the genre of the show in iTunes to ‘EyeTV’. This genre will be used later for maintaining the iTunes library.

-- ExportDone.scpt
on ExportDone(recordingID)
  set myid to recordingID as integer

  tell application "EyeTV"
    set theRec to recording id myid

    -- gather some info from EyeTV about the recording that just finished
    set origdur to get the actual duration of theRec
    set myshortname to get the title of theRec
    set episodeID to get the episode of theRec
    set thisdate to get the start time of theRec
    set mm to (month of thisdate) as integer
    set dd to day of thisdate

    -- add date to name for iTunes
    set itunesname to myshortname & " (" & mm & "/" & dd & ")"
  end tell

  -- wait a while to make sure iTunes has imported the exported recording
  delay 30

  tell application "iTunes"
    -- all EyeTV exports go to playlist 'EyeTV'
    tell playlist "EyeTV"
      set theShows to tracks whose name is myshortname
      if (count of theShows) = 0 then
        set theShows to tracks whose episode ID is episodeID
      end if
      set a_show to (the first item of theShows)
    end tell
    if a_show is not {} then
      -- change genre so we can find it later
      set genre of a_show to "EyeTV"

      set video kind of a_show to TV show
      set show of a_show to myshortname
      set name of a_show to itunesname
    end if
  end tell
  log myshortname & " exported to iTunes"
  display dialog (myshortname & " exported to iTunes") giving up after 10
end ExportDone
-- ExportDone.scpt

Automator Script — iTunesCleanup

Without some maintenance the iTunes library would grow continually.

I decided to automate the maintenance by deleting nightly news show daily, and other shows after thirty days.

While this could be done with a cron job, I chose to use an Automator event in iCal (this will be easier for those who are Terminal averse.)

In addition to cleaning up iTunes this script also deletes the recording from EyeTV since, once they are exported, I no longer need them in the EyeTV archive.

Before deleting from EyeTV a comparison is made between the exported duration in iTunes and the recorded duration in EyeTV–if the difference is too great then the export was probably not successful so the recording is not deleted in case we want to re-export by hand.

I’ve configured the following script to run daily at 6am in iCal:

on run
  cleanup_iTunes()
end run

to cleanup_iTunes()
  log "cleanup_iTunes()"

  set tracksToDelete to {}
  set now to current date

  tell application "iTunes"
    -- EyeTV exported videos automatically get added to playlist "EyeTV"
    set shows to (every track of playlist named "EyeTV")

    repeat with a_show in shows
      set showName to (name of a_show)

      -- look at recordings (we know them because we set the genre)
      if genre of a_show is "EyeTV" then
        -- delete older recordings, news daily, otherwise 30 days old

        -- we record NBC news, local and national, so regex match to get news shows
        set regexscript to "echo \"" & showName & "\" | awk /^NBC.*News.*$/"
        tell current application to set news to (do shell script regexscript)

        if length of news is greater than 0 then
          -- delete news daily
          set theDate to (now - 18 * hours)
        else
          -- delete everything else older than 1 month
          set theDate to (now - 30 * days)
        end if

        -- add to delete list if old enough
        set toDel to ((date added of a_show) is less than theDate)

        if toDel then
          -- cache files to delete so we don't alter library while iterating
          set tracksToDelete to tracksToDelete & (get database ID of a_show)
        end if
      end if
    end repeat
  end tell

  if tracksToDelete is not {} then
    log "tracksToDelete -- " & tracksToDelete as string
    set filesToDelete to {}

    tell application "iTunes"
      set myLib to playlist 1
      repeat with theID in tracksToDelete
        set toDel to (first track of myLib whose database ID is theID)
        if (class of toDel) is file track then
          set filesToDelete to filesToDelete & (location of toDel)
        end if
        delete toDel
      end repeat
    end tell

    tell application "Finder"
      repeat with theFile in filesToDelete
        my delete_the_file(theFile)
      end repeat
    end tell
  end if

  --delete EyeTV recordings if exported
  my delete_exported_recordings()

  try
    -- quit iTunes to force library sync on a regular basis
    tell application "iTunes" to quit

    delay 60

    tell application "iTunes"
      -- just do something here so we know iTunes is running 
      -- and ready for Home Sharing connections from the AppleTV
      set shows to (every track of playlist named "EyeTV")
    end tell
  end try
end cleanup_iTunes

to delete_exported_recordings()
  log "delete_exported_recordings"
  set recs to {}
  tell application "EyeTV"
    repeat with a_rec in recordings
      set thisdate to get the start time of a_rec
      set mm to (month of thisdate) as integer
      set dd to day of thisdate
      set itunesname to (title of a_rec) & " (" & mm & "/" & dd & ")"
      set recs to recs & [[(title of a_rec), itunesname, actual duration of a_rec]]
    end repeat
  end tell
  
  repeat with a_title in recs
    set myshortname to get item 1 of a_title
    log "  checking export " & myshortname
    set exportdur to get_duration_of_show(get item 2 of a_title)
    set origdur to get item 3 of a_title
    
    if origdur > exportdur then
      set thediff to (origdur - exportdur)
    else
      set thediff to (exportdur - origdur)
    end if
    
    if thediff < origdur * (0.15) then
      my delete_recording(myshortname)
    end if
  end repeat
end delete_exported_recordings

on delete_recording(shortname)
  log "delete eyetv recording -- " & shortname
  tell application "EyeTV"
    repeat with a_rec in recordings
      if title of a_rec is equal to shortname then
        delete recording id (unique ID of a_rec)
        log "-Recording " & shortname & " deleted"
        exit repeat
      end if
    end repeat
  end tell
end delete_recording

on get_duration_of_show(show_name)
  set exportdur to 0
  
  tell application "iTunes"
    tell playlist "EyeTV"
      try
        set theShows to tracks whose name is show_name
        if (count of theShows) > 0 then
          set exportdur to (duration of first item of theShows)
        end if
      end try
    end tell
  end tell
  
  log "Exported duration " & exportdur
  return exportdur
end get_duration_of_show

to delete_the_file(floc)
  log "Attempt to delete file" & POSIX path of (floc as string)
  try
    do shell script "rm -f " & quoted form of POSIX path of (floc as string)
  on error
    log "Done. However, the file could not be deleted."
  end try
end delete_the_file

Everyday Use

Using ‘Smart Guides’ in EyeTV makes it simple to set up a ‘Season Pass’ for any show. In the smart guide options you can also configure the recordings to export automatically to iTunes. When they do, the ExportDone script is invoked on completion of the export which puts the date on the show in iTunes.

Using AppleTV to connect to the computer hosting iTunes, it is simple to traverse the TV shows in iTunes. Selecting the ‘TV Shows’ category from the shared computer and navigating back to the top menu puts the TV shows in the top row on the AppleTV (see image above,) with the most recent recordings on the left, and the date of the recording in the title (again, see ‘Elementary (1/4)’ above.) It is then a simple matter of selecting shows from this top row–pressing ‘play’ plays the most recent recording of a show, selecting the show will present the list of episodes if multiple episodes exist.

IMG_0316

As I complete this post, our new whole-house DVR system has been up and running for two months. There were a few hiccups along the way–the scripts above have been revised a few times–but at this point all the rough edges appear to have been eliminated and the WAF is high.

Mission Accomplished!

Six Month Followup

After having this solution up for a little more than six months, I thought it might be useful to share some updates.

Some of the processes and scripts have been updated. Some of the updates are bug fixes, others are for reducing the load on my aging iMac when transcoding the recordings.

MC2XML Updates

After rereading the xc2xml install instructions, I noticed an EyeTV “clear EPG database” command that I’d missed the first time around. Instead of performing this manually, I decided to add it to the mc2xml.sh script:

#!/bin/bash
cd /Users/girls/guide
./mc2xml

# clear epg db
osascript<

Serialization of Transcoding


I’ve moved away from the auto-export (ExdportDone.scpt) and now serialize the transcoding through the use of a couple of other scripts.  I had issues with transcoding taking days when there were two or three shows being exported at once.  I needed a solution to serialize this process so as to not overload the iMac.  After a bit of trial and error I came up with this multi-stage solution.


First, I use the EyeTV RecordingDone hook to invoke a shell script, RecordingDone.scpt:


property SHELL_SCRIPT_SUFFIX : " >> /Users/girls/Documents/TriggeredScripts/eyetv_script.log 2>&1 "

on RecordingDone(recordingID)
  my logger("RecordingDone id: " & recordingID)
  do shell script "/Library/Application\\ Support/EyeTV/Scripts/TriggeredScripts/invoke_script.bash " & recordingID 
end RecordingDone

on logger(logThis)
  set dtg to do shell script "date \"+%I:%M:%S %p -- \""
  do shell script "echo \"" & dtg & logThis & "\"" & SHELL_SCRIPT_SUFFIX
end logger



In the second step I create a bash script-based Turbo.264 job queue from within the bash script invoked by RecodingDone.scpt.  Here’s invoke_script.bash:


#!/bin/bash
logFile="/Users/girls/Documents/TriggeredScripts/eyetv_script.log"
jobFile="/Users/girls/Documents/TriggeredScripts/__${1}"
lockFile="/Users/girls/Documents/TriggeredScripts/__turbo_lock"

touch $logFile
chmod 777 $logFile
dtg=`date "+%I:%M:%S %p -- "`
echo "${dtg}=================================" >> $logFile
echo "${dtg}recordingID = $1" >> $logFile

cat > ${jobFile} < /dev/null 2>&1 || [ -f ${lockFile} ]; do
  sleep $(( 10 + $RANDOM % 30 ))
done
touch ${lockFile}
# start new job
nohup osascript /Library/Application\ Support/EyeTV/Scripts/TriggeredScripts/RecordingDone-called.scpt $1 >> $logFile 2>&1 &
# remove thyself
rm -rf $jobFile
DELIM

chmod +x ${jobFile}
${jobFile} &
echo "${dtg}${jobFile} queued" >> $logFile



This script creates job scripts which are invoked in the background.  The job scripts all wait until the previous invocation of Turbo.264 has exited, as indicated by the absence of an active Turbo.264 task and removal of the lock file created when transcoding starts.  The use of a RANDOM delay reduces the chances that jobs for two shows which ended about the same time will not wakeup simultaneously.The transcoding itself, as well as removal of the lock file is done by another AppleScript, RecordingDone-called.scpt:


property TARGET_PATH : "/Users/girls/Documents/EyeTV Archive/Transcoded/"
property TARGET_TYPE : ".mp4"
property SOURCE_TYPE : ".mpg"
property SHELL_SCRIPT_SUFFIX : " >> /Users/girls/Documents/TriggeredScripts/eyetv_script.log 2>&1 "

property CLEAN_FILENAME_DISALLOWED_CHARS : ";|!@#$%^&*+()/"
property CLEAN_FILENAME_DISALLOWED_CHARS2 : ":,'"
property CLEAN_FILENAME_REPLACEMENT : "_"

on run argv
  set recordingID to item 1 of argv

  -- Obtain some show information from EyeTV
  -- Transcode recorded video to conform to desired format
  -- Delete original EyeTV recording

  with timeout of (480 * 60) seconds
    tell application "EyeTV"
      set myid to recordingID as integer
      set show_title to title of recording id myid as text
      set show_episode to episode of recording id myid as text
      set thisdate to start time of recording id myid
      set mm to (month of thisdate) as integer
      set dd to day of thisdate
      set timestamp to " (" & mm & "/" & dd & ")"
      if show_episode = "" then
        set show_episode to thisdate as text
        set suffix to timestamp 
      else
        set suffix to " - " & show_episode & timestamp 
      end if
      set show_description to description of recording id myid as text
      set recording_location to location of recording id myid as text
    end tell

    set AppleScript's text item delimiters to "."
    set recording_path to text items 1 through -2 of recording_location as string
    set AppleScript's text item delimiters to ""
    set recording_path to POSIX path of recording_path
    set input_file to (recording_path & SOURCE_TYPE) as string
    set show_filename to (my clean_filename(show_title & " - " & show_episode) & TARGET_TYPE)
    set transcoded_file to (TARGET_PATH & show_filename) as string

    my logger("Turbo.264 HD (" & recordingID & ") - " & input_file & " to " & transcoded_file)
    tell application "Turbo.264 HD"
    	add file input_file with destination transcoded_file exporting as HD720p
    	encode
    	set busyEncoding to true
    end tell
    
    -- Loop until this export is finished
    repeat while busyEncoding
    	do shell script "sleep 60"
    	tell application "Turbo.264 HD"
    		set busyEncoding to isEncoding
    	end tell
    end repeat
    
    -- quit Turbo.264 HD
    tell application "Turbo.264 HD" to quit
        
    -- Remove lock file 
    do shell script "rm -f /Users/girls/Documents/TriggeredScripts/__turbo_lock"

    -- prep target for iTunes
    set cmd to "chmod 666 " & (quoted form of TARGET_PATH) & "*" & TARGET_TYPE
    my logger(cmd)
    do shell script cmd 

    -- delete recording from EyeTV
    my logger("Delete recording " & quoted form of input_file)
    tell application "EyeTV"
      delete recording id myid
    end tell
  end timeout

  -- Add the video file as it resides on the NAS server to the 
  -- iTunes library as a TV show.

  my logger("Add '" & show_title & suffix & "' to iTunes")
  tell application "iTunes"
    set transcoded_folder to ("Macintosh HD:Users:girls:Documents:EyeTV Archive:Transcoded:") as string
    set newShow to (add (transcoded_folder & show_filename))
    set genre of newShow to "EyeTV"
    my logger("  genre '" & (genre of newShow) & "'")
    set video kind of newShow to TV show
    my logger("  kind '" & (video kind of newShow) & "'")
    set name of newShow to (show_title & suffix)
    my logger("  name '" & (name of newShow) & "'")
    set show of newShow to show_title
    my logger("  show '" & (show of newShow) & "'")
    set episode ID of newShow to show_episode
    my logger("  episode ID '" & (episode ID of newShow) & "'")
    set description of newShow to show_description
    my logger("  description '" & (description of newShow) & "'")
  end tell
  my logger("Finished")
end run

on logger(logThis)
  set dtg to do shell script "date \"+%I:%M:%S %p -- \""
  do shell script "echo \"" & dtg & logThis & "\"" & SHELL_SCRIPT_SUFFIX
end logger

on clean_filename(theName)
  set newName to ""
  repeat with i from 1 to length of theName
    -- check if the character is in CLEAN_FILENAME_DISALLOWED_CHARS
    -- replace it with the CLEAN_FILENAME_REPLACEMENT if it is
    if ((character i of theName) is in CLEAN_FILENAME_DISALLOWED_CHARS) then
      set newName to newName & CLEAN_FILENAME_REPLACEMENT
    -- check if the character is in CLEAN_FILENAME_DISALLOWED_CHARS2
    -- remove it completely if it is
    else if ((character i of theName) is in CLEAN_FILENAME_DISALLOWED_CHARS2) then
      set newName to newName & ""
    -- if the character is not in either CLEAN_FILENAME_DISALLOWED_CHARS or
    -- CLEAN_FILENAME_DISALLOWED_CHARS2, keep it in the file name
    else
      set newName to newName & character i of theName
    end if
  end repeat
  return newName
end clean_filename



This AppleScript invokes the Turbo.264 HD application to transcode the recording indicated on the command line.  It waits until transcoding completes, then quits the Turbo.264 HD app, and removes the lock file created by the job queue script.  When one of the waiting job queue scripts wakes from its sleep and sees Turbo.264 HD app is not running and the lock file from the previous show has been removed, then is invokes RecodingDone-called.scpt for its own recording.  It also deletes the recording from EyeTV so we don’t have the original lying around after the transcoded version is available after export to iTunes.


Again I use /Users/girls as the base account for my solution.  if you use these scripts you will need to update the paths according to your own installation.


Good Luck!