I spent several years maintaining my own forked mythbuntu (HD-PVR was the point of no return) and I'd rather not do that again right away.
I'm probably going to fork again eventually (is anybody ever happy with UPnP?) and at that point rejoining -fixes can get really heavyweight. But that's my problem.
was build a few days before Final Freeze (which is where we can't put any more new builds in).Code:2:0.25.0+fixes.20120410.1f5962a-0ubuntu1
As for 0.25.1 (if there is a 0.25.1), we won't be making a special build of the released version of that. We'll make a build of the 0.25 branch for that day, just like we do every day. We'll increment the number to 0.25.1+fixes.... and keep building.April 12th
The thread title is "Mythbuntu 0.25 experience", not 12.04-release. Users still seem to be using the terms interchangeably.
Continuous build/ship scares me; sid is a scary kid. Somebody who doesn't have eight years of data can install before me. You don't have to do anything special for me to pin or hold or whatever my packages. I just think you'll get better feedback and less confused users if you name significant releases in a way they can see and speak compactly.
Regardless of my silly attempt to impose my naming and release system on your project, there is still a problem: a bunch of people are seeing giant performance regressions with some kinds of DataDirect feeds. This seems to be a Mythbuntu 12.04 problem.
Unfortunately, people don't usually post their full logs when opening a thread, so it does require some digging. Some digging is going to be required if we release 1 build per Ubuntu release or 1000 builds.
Anyway, you had originally stated that you didn't want to use the Mythbuntu repos because you wanted stability over cutting edge. I responded to correct that statement. I'm not entirely sure why you feel the need to keep arguing about the version numbers.
Back on topic, to troubleshoot previous mythfilldatabase issues, upstream has wanted
The pcap probably isn't necessary for this issue, but I'd say lets gather it in case this needs to go upstream.mythfilldatabase run with -v file,network --loglevel
debug, AND a pcap file of it failing.
Hi guys. After reading the discussion about which version is in place, etc, I will clarify my info as much as I can:
My fresh install is from the iso downloaded from the mythbuntu web site 5/1/12. (so my understanding is that this is Ubuntu 12.04 + MythTV 0.25), 64 bit.
Additionally, the very first line of my mythfilldatabase.log file says:
May 4 21:13:30 media mythfilldatabase: C thread_unknown mythcommandlineparser.cpp:2534 (ConfigureLogging) mythfilldatabase version: fixes/0.25 [v0.25] www.mythtv.org
I was looking through the log just now, and my last run of mythfilldatabase occured a little bit before 2:30 am last night and finished around 7:30 am. It appears that it took roughly 20-30 minutes to finish each day's worth of data.
If you like, I can post that section of my log. I am not super advanced, but if there is any other information that you would like me to get, please let me know and I would be happy to try to accomodate.
I decided to log database as well, since the interesting thing is how much faster MyISAM is than InnoDB--all that's changed between the two runs is the default storage engine. The InnoDB run will be going for about another hour I think; here's a snapshot of MyISAM vs InnoDB:
You will note the first run took six minutes, and the second is chugging away 40 minutes later, with the disk light on. I'll upload the complete logs when they finish.Code:nop@mythtv:/tmp/mfdblog$ head -2 mythfilldatabase.20120507114153.27610.log 2012-05-07 11:41:53.551208 C [27610/27610] thread_unknown mythcommandlineparser.cpp:2534 (ConfigureLogging) - mythfilldatabase version: fixes/0.25 [v0.25] www.mythtv.org 2012-05-07 11:41:53.551226 N [27610/27610] thread_unknown mythcommandlineparser.cpp:2536 (ConfigureLogging) - Enabled verbose msgs: general file network database nop@mythtv:/tmp/mfdblog$ tail -2 mythfilldatabase.20120507114153.27610.log 2012-05-07 11:47:45.781819 I [27610/27610] CoreContext datadirect.cpp:573 (~DataDirectProcessor) - DataDirect: Deleting temporary files 2012-05-07 11:47:45.785840 D [27610/27610] CoreContext mythhttppool.cpp:94 (RemoveListener) - MythHttpPool: RemoveListener(0x7fffe01ed9d0) nop@mythtv:/tmp/mfdblog$ head -2 mythfilldatabase.20120507115634.28267.log 2012-05-07 11:56:34.991773 C [28267/28267] thread_unknown mythcommandlineparser.cpp:2534 (ConfigureLogging) - mythfilldatabase version: fixes/0.25 [v0.25] www.mythtv.org 2012-05-07 11:56:34.991790 N [28267/28267] thread_unknown mythcommandlineparser.cpp:2536 (ConfigureLogging) - Enabled verbose msgs: general file network database nop@mythtv:/tmp/mfdblog$ tail -2 mythfilldatabase.20120507115634.28267.log 2012-05-07 12:34:32.533679 D [28267/28267] CoreContext mythdbcon.cpp:675 (exec) - MSqlQuery::exec(DataDirectCon) INSERT INTO dd_program ( programid, title, subtitle, description, showtype, category_type, mpaarating, starrating, stars, runtime, year, seriesid, colorcode, syndicatedepisodenumber, originalairdate) VALUES ('EP015048820003', 'Hotel Impossible', 'The Penguin Hotel, South Beach, Fl.', 'Anthony travels to South Beach to save a family-run boutique hotel.', 'Series', 'series', '', '', '0', '', '', 'EP01504882', '', '102', '2012-04-16') 2012-05-07 12:34:32.569703 D [28267/28267] CoreContext mythdbcon.cpp:675 (exec) - MSqlQuery::exec(DataDirectCon) INSERT INTO dd_program ( programid, title, subtitle, description, showtype, category_type, mpaarating, starrating, stars, runtime, year, seriesid, colorcode, syndicatedepisodenumber, originalairdate) VALUES ('EP015048820005', 'Hotel Impossible', 'The Ocean Manor Resort: Fort Lauderdale Fl', 'The Ocean Manor is full of potential with its beachfront bar and sushi restaurant, but dirty rooms are ruining the hotel's chances.', 'Series', 'series', '', '', '0', '', '', 'EP01504882', '', '104', '2012-05-07')
I'm wondering if another approach is to have DataDirectProcessor::GrabData() begin a transaction() before the XML parse into tables, and commit() after. It's not that we care about the consistency of the temp tables, but autocommit defaults to on, turning each request into a transaction. That sounds like a forced disk barrier for each. The MySQL docs suggest the InnoDB log only needs a barrier at the end of a transaction.
I was wrong; it only took 70 minutes for the InnoDB run. Full logs and summarized captures at http://place.org/~nop/m/mfdblog-2012-05-07.tar.xz. Tar file is 6M, uncompressed 150M.
I'd like to blow away the tarball in the near future; it shouldn't be of much use to anybody who doesn't like to debug database performance issues.
I did a fresh 64-bit install Friday 20120504 with an ISO downloaded from mythbuntu.org also from Friday 20120504. I too ran into a problem with mythfilldatabase where the fill took 6 hours of HDD churn to complete. I made a change, and again waited out 6 additional hours of what sounded like pain and discomfort for the HDD.
A brief search for knowledge seemed to center around ext4 defaults set to barriers on.
Since this current release of Mythbuntu is straight LTS, and after years of wading through MythTV/Mythbuntu upgrades, I decided to wait it out until more knowledge and available time presents itself or one of the kind distribution/package maintainers sweats the dirty work for us all.
I dusted off my backend, from its dark and hidden place, and retooled it only very recently. I have only been on 11.10 for a couple of weeks. All of that previous hardware was at least 10 years old, minus storage, and the last time I touched the box seems to be 3 years ago over the schedules direct migration. The amount of dust in that forgotten and unappreciated box was horrendous.
It is funny what we take for granted.
If the issue is ext4 barriers, I am interested in the final chosen direction. I am thinking 3 paths:
2) separate partition for the database
3) rewrite the extraction translation to have all of the needed data physically on the platter before the final load.
Anyway, thanks to everyone for the years of function and enjoyment the community and dedicated individuals have provided. And thanks for making me track down what on earth my old UbuntuForums username and password was...
Version information is easy to get from dpkg.
dpkg truncates if the window is too narrow, make your xterm full screen before you run it. There is a commandline option to make sure it doesn't truncate anything but I can't recall the details. All your key mythtv packages will be the same version number.Code:nick@media:~$ dpkg -l mythtv-backend Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-=================================-=================================-================================================================================== ii mythtv-backend 2:0.24.2+fixes.20120316.322de47-0 A personal video recorder application (s[erver)
Alternatively each of the main binaries has a --version switch, so:
Code:nick@media:~$ mythbackend --version Please attach all output as a file in bug reports. MythTV Version : v0.24.2-27-g322de47 MythTV Branch : fixes/0.24 Network Protocol : 63 Library API : 0.24.20110505-1 QT Version : 4.6.2 Options compiled in: linux profile using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_crystalhd using_directfb using_dvb using_firewire using_frontend using_hdhomerun using_hdpvr using_iptv using_ivtv using_joystick_menu using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtdbus using_qtwebkit using_v4l using_x11 using_xrandr using_xv using_bindings_perl using_bindings_python using_mythtranscode using_opengl using_vdpau using_ffmpeg_threads using_live using_mheg
That can be run while the backend is running without interfering with its operation.
Note the git version 322de47 is included in the dpkg output and the --version output.
Last edited by nickrout; May 8th, 2012 at 06:49 AM.