Let's boot with Upstart (GDM ready in 15s)

I got bored and rewrite my entire system to boot up with Upstart. Yes, I rewrote all jobs, essentially wrapping them to call init scripts individually and tweaking the dependencies correctly.


Short Statistics:
Time spent hacking: 5+ hours. Ran into a lot of snags getting udev and filesystem mount ordered correctly, and each time I failed at that it meant needing a livecd!


The Victim: Me. Err, oh you meant the computer: Macbook Core 2 Duo 2.16, 2GB RAM, running Hardy.

Enjoyment: Infinite. This kind of nerdy thing keeps me going.

Stopwatch/Visual statistics:
I'll get to bootchart in a moment, but bootchart doesn't tell the whole story -- what bootchart calls a bootup time and what the user sees as the bootup time are two entirely different things. These boot times are from a stopwatch from GRUB to login chime:

Native Upstart: 18s

SysV Upstart: 30s

Bootchart Stats:

Native Upstart: (22s) http://jdong.mit.edu/~jdong/macbook/...rt-upstart.png

SysV Upstart: (27s) http://jdong.mit.edu/~jdong/macbook/bootchart-sysv.png


Observations:

(1) Note how slow my laptop hard drive is. Around 12MB/s is its peak read speeds. If I tried this with a powerful desktop hard drive, the 10 second long "readahead-list" could be shaved to 5 seconds or less.

(2) Ignore the first 17s of both bootup charts for the time being: Time spent probing modules and slurping readahead cannot be optimized by parallelized startup and are more or less hardware and kernel limitations on bootup speed. Focus on what's after the big blue trapezoid @ 15s: The actual time it took to set up dbus, hal, avahi, GDM, anacron, sysklog, and other system services. It took 5 seconds to do all of that on Upstart while SysV took over 10s. That's effectively doubling the speed at which services are coming up. In addition, notice how Upstart was more consistently able to break the one-core barrier of CPU usage.


The event.d:
For educational purposes, here's how I wrote my upstart events: http://jdong.mit.edu/~jdong/event.d.tar.gz

You're welcome to play with it all you want, but DO IT ON AN EXPERIMENTAL BOX. It's extremely easy for one typo or logic error to result in an unbootable system.


EDIT

As a proof-of-concept, I "defragged" (rewrote rapidly) all the files readahead identified as being used at bootup, and got peak IO to around 15MB/s and upstart boot time to 19s by bootchart

EDIT 2: Youtube video of bootup: (note how rough around the edges it is :D)
http://www.youtube.com/watch?v=cY8MWwbESy4

Bootchart for this bootup: http://jdong.mit.edu/~jdong/macbook/...-defragged.png

EDIT 3: After some requests I am posting some dependency diagrams of my boot:
http://jdong.mit.edu/~jdong/macbook/...pstart-dep.png

For comparison, SysV: http://jdong.mit.edu/~jdong/macbook/...t/sysv-dep.png

For the most part it fans out massively parallel quite well, which is encouraging news. Now, to optimize the longest chain length....

You can generate this diagram on your own system by grabbing http://jdong.mit.edu/~jdong/macbook/...rt/depchart.py and http://jdong.mit.edu/~jdong/macbook/...pchart-sysv.py

They don't come with manuals; figure it out :D

Wow, if these speed boosts are correct, that is friggen amazing. This needs to be added to Intrepid so badly. I mean like it should be one of the major focuses to get a stable event based bootup up.

John, you need a hobby. One that involves interaction with other humans.

Honestly, I never was interested in this sort of thing. For Windows, maybe, where reboots are common and frequent, shaving time off of booting is important, but I almost never reboot Linux, and if I did, I really don't mind a few extra seconds.

I guess it's one of those "why not" things, because if it can be faster, you might as well make it faster, but I've just never been too interested.

:) Nice results, though.

I added a youtube of the upstart bootup process so one can get a feel for what kind of bootup time it is. It definitely feels noticeably faster than stock Hardy boot.

Note that all services that are a part of the regular bootup sequence are started up (and a few more for this laptop to set up wireless and power saving and webcam), I did not perform any trickery involving trimming/stripping down services!

Quote:

---

Originally Posted by Penguin Power

John, you need a hobby. One that involves interaction with other humans.

---

I'm screwed then.

Quote:

---

Originally Posted by DoktorSeven

Honestly, I never was interested in this sort of thing. For Windows, maybe, where reboots are common and frequent, shaving time off of booting is important, but I almost never reboot Linux, and if I did, I really don't mind a few extra seconds.

---

I've never understood this practice, it's wasteful of electricity.

Quote:

---

Originally Posted by jdong

I got bored and rewrite my entire system to boot up with Upstart. Yes, I rewrote all jobs, essentially wrapping them to call init scripts individually and tweaking the dependencies correctly.


Short Statistics:
Time spent hacking: 5+ hours. Ran into a lot of snags getting udev and filesystem mount ordered correctly, and each time I failed at that it meant needing a livecd!


The Victim: Me. Err, oh you meant the computer: Macbook Core 2 Duo 2.16, 2GB RAM, running Hardy.

Enjoyment: Infinite. This kind of nerdy thing keeps me going.

Stopwatch/Visual statistics:
I'll get to bootchart in a moment, but bootchart doesn't tell the whole story -- what bootchart calls a bootup time and what the user sees as the bootup time are two entirely different things. These boot times are from a stopwatch from GRUB to login chime:

Native Upstart: 18s

SysV Upstart: 30s

Bootchart Stats:

Native Upstart: (22s) http://jdong.mit.edu/~jdong/macbook/bootchart-upstart.png

SysV Upstart: (27s) http://jdong.mit.edu/~jdong/macbook/bootchart-sysv.png


Observations:

(1) Note how slow my laptop hard drive is. Around 12MB/s is its peak read speeds. If I tried this with a powerful desktop hard drive, the 10 second long "readahead-list" could be shaved to 5 seconds or less.

(2) Ignore the first 17s of both bootup charts for the time being: Time spent probing modules and slurping readahead cannot be optimized by parallelized startup and are more or less hardware and kernel limitations on bootup speed. Focus on what's after the big blue trapezoid @ 15s: The actual time it took to set up dbus, hal, avahi, GDM, anacron, sysklog, and other system services. It took 5 seconds to do all of that on Upstart while SysV took over 10s. That's effectively doubling the speed at which services are coming up. In addition, notice how Upstart was more consistently able to break the one-core barrier of CPU usage.


The event.d:
For educational purposes, here's how I wrote my upstart events: http://jdong.mit.edu/~jdong/event.d.tar.gz

You're welcome to play with it all you want, but DO IT ON AN EXPERIMENTAL BOX. It's extremely easy for one typo or logic error to result in an unbootable system.


EDIT

As a proof-of-concept, I "defragged" (rewrote rapidly) all the files readahead identified as being used at bootup, and got peak IO to around 15MB/s and upstart boot time to 19s by bootchart

EDIT 2: Youtube video of bootup: (note how rough around the edges it is :D)
http://www.youtube.com/watch?v=cY8MWwbESy4

---

Awesome work John!

I should also add, there's still plenty of room for improvement:

(1) The dependencies were originally tweaked to get the keyboard up and working ASAP because I ran into a lot of *cough* issues recovering from failed bootups during testing phase. Holding up bootup until udev is triggered is probably costing at least a second or two.

(2) Readahead takes nearly 10secs by itself. I'm SURE I can trim that down a bit.

(3) I'm still calling init scripts from upstart. In other words, rather than upstart telling sysvinit to "start up everything in runlevel2 in order" as by default, I am having upstart control the starting of each job individually, and the way I've set up how one job relates to another means that over 10 jobs can boot in parallel at a time. However, If I further switch to a fully upstart based solution, I can bypass starting a shell and sourcing all those init files.

Quote:

---

Originally Posted by DoktorSeven

Honestly, I never was interested in this sort of thing. For Windows, maybe, where reboots are common and frequent, shaving time off of booting is important, but I almost never reboot Linux, and if I did, I really don't mind a few extra seconds.

---

laptops.

Quote:

---

Originally Posted by macogw

laptops.

---

Exactly.