HOWTO: Auto transcode MYTHTV recorded shows to DVD, XVID or DIVX cutting out the co

[GrammatonCleric]

PLEASE READ POSTS 1 thru 3 to fully!


How to auto transcode MYTHTV recorded shows to DVD, XVID or DIVX cutting out the commercials.

This how to is assuming that MYTHTV and MYTHWEB are already installed and running correctly.

Step 1: Install dependencies for nuvexport.

sudo apt-get install mjpegtools libdbix-profile-perl libdate-manip-perl

Step 2: Build ffmpeg with the xvid and other formats.

 Follow these instructions 

Step 3: Create a directory for nuvexport.

mkdir nuvexport

Step 4: Change into nuvexport.

cd nuvexport

Step 5: Grab nuvexport.

wget http://www.forevermore.net/files/nuv...latest.tar.bz2

NOTE: When MythTV 0.21 is released nuvexport will be integrated into MYTHTV.

Step 6: Extract.

 tar jxvf nuvexport-latest.tar.bz2

Step 7: Change into extracted directory.

cd nuvexport-*

Step 8: Install nuvexport.

sudo make install

Step 9: Test the output quality manually.

Before setting up the automated transcoding of your recorded shows run nuvexport manually a few times and play with the quality settings to suit your needs.

• Login as the user that you have mythtv running.
• From the command line run nuvexport.

nuvexport

• step through the prompts.
• remember the quality settings that you like, better write them down. I've included the settings that I use below in my nuvexportrc.

Step 10: Edit nuvexportrc

sudo gedit /etc/nuvexportrc

Step 11: Change Settings in the nuvexportrc.

NOTE: Below is my nuvexportrc. Change your nuvexportrc to match mine or tweak the quality settings in your nuvexportrc to suit your needs (see step9).
Also you might want to change the output path from "path = /media/videos/converted_tv" to something that you prefer.

<nuvexport>

    
export_prog=ffmpeg
# mode=divx
underscores=yes
# require_cutlist=yes

#
#  By default, nuvexport picks what it thinks is a good name for your file
#    (doing its best to avoid printing "Untitled" into the filename).  Setting
#    name will let you change the output format of the filename generated by
#    nuvexport.  Even after this formatting, nuvexport will still do some basic
#    replacements to make sure that illegal filename characters (eg. /\:*?<>|)
#    are replaced with a dash (or " with a ').  The following format variables
#    are supported:
#
#    %f -> full path to the filename
#    %c -> the chanid of the show
#    %a -> start time in YYYYMMDDHHMMSS format
#    %b -> end time in YYYYMMDDHHMMSS format
#    %t -> title (show name)
#    %s -> subtitle (episode name)
#    %h -> hostname where the file resides
#    %m -> showtime in human-readable format (see --date below)
#    %d -> description
#    %% -> a % character
#
#    

filename=%t_-_%s_(%m) 

#
#  By default, nuvexport uses an American-style date to represent showtimes in
#    lists and filenames.  Use --date to override that with the format of your
#    choosing.  See the UnixDate section `perldoc Date::Manip` for formatting
#    options.
#
#    date=%m/%d, %i:%M %p

date=%m.%d.%y


#
#  Nuvexport has the option to crop a percentage of the border of each recording
#  in order to get rid of the unsightly edges of the tv signal.  The default 2%
#  approximates the overscan of an average TV, but you can alter this from 0 to
#  5% to fit your preferences.  Please keep in mind that this amount is removed
#  prior to making any aspect conversions like removing black bars from 4:3
#  recordings to make a 16:9 export.
#
    
crop_pct = 2

#
#  Alternatively, you can override the general crop_pct to crop a different
#  amount from specific sides of the recording.
#
#   crop_top    = 2
#   crop_right  = 2
#   crop_bottom = 2
#   crop_left   = 2

</nuvexport>

#
#  The sections below work as above, with each more specific section overriding
#    the more generic.
#

<generic>

#
# Default to export to the current directory
#
    
path = /media/videos/converted_tv

#
# Use the cutlist (not to be confused with the commercial flag list) when
#    exporting.
#
    
use_cutlist = yes

#
# Tell mythcommflag to generate a cutlist from the commercial flags before
#    exporting.  Don't forget to enable use_cutlist above, too.
#
#  

 
gencutlist = yes

#
# Contrary to popular belief, enabling multipass will not make your recordings
#    look better.  What it will do, however is guarantee that the bitrate you
#    choose will be the average bitrate of your entire encode (meaning that your
#    exports will end up being about the same size per-minute), and that you
#    will receive the best overall quality for a files of the same size.
#
    
multipass = no

#
# Disabling noise reduction can speed up your exports dramatically, but at the
#    expense of some quality.  You can also access this on the commandline via
#    the --denoise (or --nodenoise) flag.
#
    
noise_reduction = no

#
# Deinterlace the video so that it looks better on software players.
#
    
deinterlace     = yes

#
# Crop about 2% from the border of the recording before encoding.  This is done
#    to get rid of part of the broadcast signal that is usually obscured by the
#    tv's overscan.
#
    
crop = yes

#
#  You can create settings for each export module type.  These are the
#    second-most generic sections, and will only be reached if there are no
#    matches in the full or generic module names.
#
#  If you have a particularly dirty signal, you might want to try to disable
#    fast_denoise (it's actually part of yuvdenoise, which both the ffmpeg
#    and transcode exporters call).  It can be almost twice as slow as the
#    default "fast" normal noise reduction, but it considerably more effective.
#    The latest version of yuvdenoise (which is called directly by the ffmpeg
#    exporters) does not support this option, so it is ignored in that case.
#
    
fast_denoise = no

#
#  If nuvexport is having trouble detecting the *input* aspect ratio of your
#    recordings (MythTV used to hard-code all software-encoded files as 1:1
#    regardless of the true aspect), set this option to one of the following:
#
#   force_aspect = [ 1:1 4:3 16:9 2.21:1 ]

</generic>

<ffmpeg>
#
#  ffmpeg is almost twice as fast if you disable noise reduction
#
#    noise_reduction = no
#
#  By default, nuvexport's ffmpeg module lets ffmpeg handle deinterlacing.
#    I've found that this provides the best results, but if you wish to let
#    yuvdenoise do it instead, set deint_in_yuvdenoise to a true value.
#
#    deint_in_yuvdenoise = no
#
</ffmpeg>

<transcode>

#
#  Mythtranscode will always be used for nupplevideo recordings because
#    transcode can't read them, but setting force_mythtranscode to yes will
#    force nuvexport to call mythtranscode when using the transcode exporter for
#    mpeg recordings, too.  This may help problems that some people have been
#    having with transcode not recognizing certain dvb recordings, as well as
#    transcode not working properly on certain ivtv recordings.
#
    
force_mythtranscode = yes

#
#  Setting both force_mythtranscode and mythtranscode_cutlist to yes will tell
#    nuvexport to use mythtranscode's built-in cutlist functions, rather than
#    having transcode use its own.  I've found that the cutlists for a handful
#    of ivtv recordings that do not work properly with transcode's internal
#    cutlist handler.
#
    
mythtranscode_cutlist = yes

</transcode>

<mencoder>
</mencoder>

#
#  You can also create settings for generic export module names.  These will
#    only be overridden by full module names.
#

<XviD>

    vbr          = yes   # Enable vbr to get the multipass/quantisation options
                         # (enabling multipass or quantisation automatically enables vbr)
    multipass    = no   # You get either multipass or quantisation; multipass will override
    quantisation = 3   # 4 through 6 is probably right...  1..31 are allowed (lower is better quality)

    a_bitrate    = 256   # Audio bitrate of 256 kbps
    v_bitrate    = 4500   # Remember, quantisation overrides video bitrate

    width        = 624   # Height adjusts automatically to width, according to aspect ratio
    height       = auto

</XviD>
<DIVX>

    vbr          = yes   # Enable vbr to get the multipass/quantisation options
                         # (enabling multipass or quantisation automatically enables vbr)
    multipass    = no   # You get either multipass or quantisation; multipass will override
    quantisation = 2  # 4 through 6 is probably right...  1..31 are allowed (lower is better quality)

    a_bitrate    = 256   # Audio bitrate of 256 kbps
    v_bitrate    = 4500   # Remember, quantisation overrides video bitrate

    width        = 624   # Height adjusts automatically to width, according to aspect ratio
    height       = auto

</DIVX>
<DVD>

    vbr          = yes   # Enable vbr to get the multipass/quantisation options
                         # (enabling multipass or quantisation automatically enables vbr)
    multipass    = no   # You get either multipass or quantisation; multipass will override
    quantisation = 2  # 4 through 6 is probably right...  1..31 are allowed (lower is better quality)

    a_bitrate    = 256   # Audio bitrate of 256 kbps
    v_bitrate    = 4500   # Remember, quantisation overrides video bitrate

    width        = 624   # Height adjusts automatically to width, according to aspect ratio
    height       = auto

</DVD>

#
#  Default mp3 bitrate in MythTV is 128
#
<MP3>
    bitrate = 256
</MP3>

#
#  If you want to provide settings for a very specific export module, you can
#    use its full name, and it will override any more generic settings.
#

#
# The MP4 encoder for ffmpeg has a couple of options unique to itself
#

<ffmpeg::MP4>

# Codec to use (mpeg4 or h264).  Please note that h264 support requires the
# SVN version of ffmpeg (not CVS!).  In fact, even the mpeg4 codec works
# better with the SVN version.
    mp4_codec = mpeg4

# Framerate to use:  auto, 25, 23.97, 29.97.  PAL will always be 25 fps, and
# auto will set 29.97 for everything over 320x288 and 23.97 for the rest.
    mp4_fps = auto

</ffmpeg::MP4>

#
# As does the PSP exporter
#
<ffmpeg::PSP>

# PSP framerate (high=29.97, low=14.985)
    psp_fps = low

# PSP resolution (320x240, 368x208 or 400x192)
    psp_resolution = 320x240

# PSP video bitrate (high=768, low=384)
    psp_bitrate = high

# Create a thumbnail to go with the PSP video export?
    psp_thumbnail = yes

</ffmpeg::PSP>

#
# You can also add flags to the one and only mencoder option
#
<mencoder::XviD>

    multipass = no

</mencoder::XviD>

#
# You can also make specific profiles called with the --profile parameter that
# will override other config options (but not commandline arguments).
#
# For example, you could make a profile that would encode your favorite show
# with your favorite settings.
#
<profile::sample>

    title       = test

    export_prog = transcode
    mode        = xvid
    confirm     = true

</profile::sample>

Step 12: Open a browser to your MythWeb.

The following is assuming that MythWeb is running on your MythTV box. The same process can be run from the MythTV box while logged in as your mythtv user. Run mythtv-setup and navigate to the last page of the General section until you see the "User Jobs."

• Click on the "Edit MythWeb and MythTV Settings" button (i.e. the Key and the Wrench).
• Click on the link "MythTV settings table."
• Scroll down to the UserJob1.
• Enter the following.

nuvexport-xvid --nice 19 --input="%FILE%"

• Now enter a Description for UserJob1 under the setting UserJobDesc1 (i.e. Convert to XVID).
• Scroll to the UserJob2.

nuvexport-divx --nice 19 --input="%FILE%"

• Now enter a Description for UserJob2 under the setting UserJobDesc2 (i.e. Convert to DIVX).
• Scroll to the UserJob3.

nuvexport-dvd --nice 19 --input="%FILE%"

• Now enter a Description for UserJob3 under the setting UserJobDesc3(i.e. Convert to DVD).

Step 13: Set your scheduled shows to auto transcode.

• From MythWeb click on "Recording Schedules (Manual, Custom)"
• Click on one of your scheduled shows.
• On the right side under "Schedule Options:" make sure that "Auto-flag commercials:" has a check next to it.
• Just under the "Auto-transcode:" option you should now see the three options that you specified in step 12 (i.e. Convert to XVID, Convert to DIVX, and Convert to DVD). Select one to auto trancode your show.
• Repeat Step 13 for other shows that you wish to auto transcode.

Enjoy!
-GC


Update: I no longer use ffmeg to trancode to divx or xvid but instead I use mencoder solely now, as seen in post #6. This change is due to quality issues that I've encountered with ffmpeg.

[GrammatonCleric]

Started having quality issues with the ffmpeg encoder. So I switched to using mencoder for xvid transcodes. Mencoder is slower than ffmpeg but the quality is much better.

In the nuvexportrc file I've remarked out the following lines:

# export_prog=ffmpeg    (call the encoder from MythTV userjobs)
# mode=xvid    (request mode from MythTV userjobs)

I've also changed my userjobs to look like:

UserJob1 - Convert to DVD

nuvexport-dvd --ffmpeg --nice 19 --input="%FILE%"

UserJob2 - Convert to XVID

nuvexport-xvid --mencoder --nice 8 --input="%FILE%"

UserJob3 - Convert to DIVX

nuvexport-divx --ffmpeg --nice 10 --input="%FILE%"

[GrammatonCleric]

I've recieved an email or two asking that I post a sample output of the trancoded video. The following was transcoded from my one of PVR-250's MPEG2 to XVID using the settings listed in the above posts namely...

<XviD>

    vbr          = yes   
    multipass    = no 
    quantisation = 3 
    a_bitrate    = 256  
    v_bitrate    = 4500 
    width        = 624
    height       = auto

</XviD>

...with the mencoder.

Here's the sample video output.....

Battlestar Intro

Next video sample is from the show "Heroes" this is an example of the auto commercial removal.  
The edit occurs at 1min and 2secs.

Heroes 

[mirr0r]

hi, I need help with mythtv...

Everything is ok but I m not able to set correct user job to transcoding my recording in xvid: in a shell I tested this command:

First Pass:

mencoder -oac mp3lame -lameopts cbr:br=128 -ovc xvid -xvidencopts pass=1:interlacing:chroma_opt:turbo:me_quality=6:vhq=4:chroma_me:hq_ac:max_key_interval=250:bf_threshold=3:noqpel:nogmc:quant_type=h263:min_iquant=2:max_iquant=8:min_pquant=3:max_pquant=8:min_bquant=2:max_bquant=8:trellis -vf spp,scale /path/to/file/recorded -o /path/to/tmp/folder

Second Pass:

mencoder -oac mp3lame -lameopts cbr:br=128 -ovc xvid -xvidencopts bitrate=-180000:pass=2:keyframe_boost=10:kfthreshold=1:kfreduction=20: /path/to/tmp/folder  -o /path/to/xvid/file_name

this work great in a shell but if I try to set this command in mythtv user job I get nothing.. this is my job as write in mythtv-setup:
job1(name "first pass"):

mencoder -oac mp3lame -lameopts cbr:br=128 -ovc xvid -xvidencopts pass=1:interlacing:chroma_opt:turbo:me_quality=6:vhq=4:chroma_me:hq_ac:max_key_interval=250:bf_threshold=3:noqpel:nogmc:quant_type=h263:min_iquant=2:max_iquant=8:min_pquant=3:max_pquant=8:min_bquant=2:max_bquant=8:trellis -vf spp,scale %DIR%%FILE% -o /media/registrazioni/job/1

job2(name "second pass"):

mencoder -oac mp3lame -lameopts cbr:br=128 -ovc xvid -xvidencopts bitrate=-180000:pass=2:keyframe_boost=10:kfthreshold=1:kfreduction=20: /media/registrazioni/job/1  -o /media/archivio/filesharing/myth/xvid/[xvid-ita]-%TITLE%-%STARTTIMEISO%

I think I m wrong with variable %DIR% and %FILE%...

Thanks and sorry for my bad english

[GrammatonCleric]

I believe if you are running these from a "UserJob" the variable %file% includes the path where MythTV recorded file has already output to. So try removing the %dir% variable.

Here's some additional variables that can be used with MythTV UserJobs.

   1. * %DIR% - the directory component of the recording’s filename
   2. * %FILE% - the filename component of the recording’s filename
   3. * %TITLE% - the title of the recording (e.g., name of the series)
   4. * %SUBTITLE% - the subtitle of the recording (e.g., name of the episode)
   5. * %DESCRIPTION% - description text for the recording (from guide data)
   6. * %HOSTNAME% - the backend making the recording
   7. * %CATEGORY% - the category of the recording (from guide data)
   8.  * %RECGROUP% - the recording group
   9. * %CHANID% - the MythTV channel ID making the recording
  10. * %STARTTIME% - the recording start time (YYYYMMDDhhmmss)
  11. * %ENDTIME% - the recording end time (YYYYMMDDhhmmss)
  12. * %STARTTIMEISO% - the recording start time in ISO 8601 format (YYYY-MM-DDTHH:MM:SS)
  13. * %ENDTIMEISO% - the recording end time in ISO 8601 format
  14. * %PROGSTART% - the recording’s start time (from guide data; YYYYMMDDhhmmss)
  15. * %PROGEND% - the recording’s end time (from guide data)
  16. * %PROGSTARTISO%, %PROGENDISO% - the recording’s start and end time in ISO 8601 format.

-GC

[GrammatonCleric]

I was asked if it is possible to setup UserJobs to transcode for differnet XVID or DIVX levels of quality (i.e. Best, High, Medium, etc.). Yes, most of the parameters that are in the nuvexportrc file can be pipped via command line or MythTV UserJobs.

Some of the nuvexport options are...

nuvexport-xvid
nuvexport-divx
nuvexport-dvd
 --ffmpeg
 --transcode
 --mencoder
--require_cutlist
--crop_pct
--gencutlist
--use_cutlist
--path
--noise_reduction
--deinterlace
--crop
--fast_denoise
--force_mythtranscode
--mythtranscode_cutlist
--vbr
--multipass
--quantisation
--a_bitrate
--v_bitrate
--width

When using the above options do not use the "=" as you see in the nuvexportrc file. Just put a space and the option response you want.

nuvexport-xvid --mencoder --fast_denoise yes --quantisation 4 --nice 10 --a_bitrate 128  --v_bitrate 2500 --path /path/to/some/directory --input="%FILE%

So back to the question. If you wanted to setup different quality levels for either XVID or DIVX change your MythTV UserJobs to something like.


Best Quality:

nuvexport-xvid --mencoder --quantisation 1  --a_bitrate 192 --v_bitrate 5000 --nice 10 --input="%FILE%"

High Quality:

nuvexport-xvid --mencoder --quantisation 2  --a_bitrate 160 --v_bitrate 4500 --nice 8 --input="%FILE%"

Medium Quality:

nuvexport-xvid --mencoder --quantisation 4  --a_bitrate 128 --v_bitrate 4000 --nice 10 --input="%FILE%"

-GC

[georgefairbanks]

Thanks, this has been very helpful. I haven't been able to find a way to delete the original recording after it's been transcoded. Is there a way to do this? I'm capturing HD streams so they pile up quickly.

[GrammatonCleric]

Thanks, this has been very helpful. I haven't been able to find a way to delete the original recording after it's been transcoded. Is there a way to do this? I'm capturing HD streams so they pile up quickly.

Your welcome. I'm glad that my how to helped. As to the auto-deletion this functionality is not available...yet. Kyle Hill appears to have written a patch and has summited it to the nuvexport dev team for consideration. With luck it will make it in.

You could adjust your settings for "No. of recordings to keep." So that Mythtv will auto-delete them. The down side of doing this is that if you want to record some sort of all day/all weekend show fest that your nuvexport process may miss a show if the file is deleted before nuvexport can finish. I go in daily via MythWeb and delete the show that are finished.

fyi... Kyle's site appears to be down.

-GC

[deoncarr]

Thanks for the howto. I am having problems getting mythtv to automatically run the nuvexport-xvid script after recording a program or through the Watch recordings -> job options -> convert xivd menu.

I have tested that the script works by typing nuvexport-xvid --nice 19 --input=/mnt/storage/mythtv/recordings/some_mpg.mpg which completed successfully and dumped the output in the expected location.

However when I try to do the conversion with some recordings I already have via Watch recordings->job options->start xvid convert, nothing happens. Then in job options the start xvid convert has changed to stop xvid convert but when I click on this nothing happens. When I go back to job options it still says stop xvid convert so it doesnt look like it can stop whatever has been started.

Does anyone have any ideas about what could be going on or how I can find out what is happening.

btw I am using the following line as job 1 in mythtv:
nuvexport-xvid --nice 19 --input="%FILE%"

Thanks
d

[GrammatonCleric]

Hi D,
From MythWeb click on the Backend Status and scroll down to the Job Queue section. Are there any errors for the failed jobs? I have seen that in some cases that you may need to add the full path to nuvexport-xvid. Try changing your UserJob to something like...

/usr/local/bin/nuvexport-xvid --nice 19 --input="%FILE%"

...or wherever you put your nuvexport-xvid. Also make sure that the user that you are running mythtv as has permission to write the the output directory you have specified in the nuvexportrc.


-GC