Results 1 to 10 of 85

Thread: HOWTO: Share updates across multiple machines

Threaded View

  1. #1
    Join Date
    Oct 2006
    Location
    Second star to the right
    Beans
    607
    Distro
    Ubuntu 7.10 Gutsy Gibbon

    HOWTO: Share updates across multiple machines

    NOTE:
    Apt-cacher does not seem to work with Lucid. This guide now use apt-cacher-ng. If you followed this guide before and are having troubles on Lucid, then run the following two commands:
    Code:
    sudo apt-get remove apt-cacher
    sudo apt-get install apt-cacher-ng
    If you have any troubles, check that configuration files are present and correct. They should remain intact, but you never know.



    So, you have Ubuntu, you love Ubuntu, and you've spread the love throughout your home. Perhaps you have a few family members using it happily, or maybe you simply enjoy having a different computer in every room of the house.

    Whatever the reason, you have multiple machines running this fine operating system. One problem, though: all those updates! You love getting upgraded software, but having to download it so many times really eats away at your bandwidth.

    Luckily for you, you're using open source software. You're not alone in this concern, and a solution already exists.

    This guide will get you setup with apt-cacher, a tool that caches Debian package files for your home network. When one of your machines requests a package, apt-cacher will check to see if it has it, and if it does, send it back to the client without downloading a single byte. If not, then it goes to the repository and grabs the package, streaming it back to the client as it does, and keeps it for future use.

    Things to note:
    1. A repository needn't be specified on the server machine for a client to access it. The server acts as a proxy which keeps a copy for itself.
    2. This is not Ubuntu-specific. Any Debian-based distro should be able to act as both a server and a client, although you may have to adapt some of the steps for yourself.


    Part Zero: The Formalities
    This has been confirmed as working on very diverse networks, with machines running different distros (not merely Ubuntu derivative) with very different architectures. I've tested this myself on versions up to, but not including, Lucid. The use of apt-cacher-ng yields similar results according to other users.

    Information for this guide was gathered primarily from the apt-cacher man page. I consulted this article (http://www.linuxjournal.com/article/1058) for help with inetd. It's rather dated, but quite useful.

    Thanks to the following users:
    omegamormegil, for providing a much easier method of redirecting APT.
    bugmenot_user, for figuring out how to clean the regular APT cache.
    mssever, for providing the workaround for upgrades.
    Skip Da Shu, for being generally awesome and providing a lot of testing.


    Part One: The Server
    You'll need to select a server for this setup. It should have a fair bit of space to spare, particularly if the client machines belong to an eclectic group. For those with more advanced partitioning schemes, the cache is kept in /var/cache/apt-cacher by default, but this can be changed.

    To get the ball rolling, we install the software in question:
    Code:
    sudo apt-get install apt-cacher-ng
    With it successfully installed, we have to tell it how to start. There are two options for this. You can have it run in daemon mode (always on), or you can get inetd, which is a program to launch server applications on request. The obvious advantage is that you save memory and CPU power, but CPU usage will spike on startup when using inetd, and you may experience a delay.

    If you're expecting a lot of traffic, go with the daemon mode. If you're a home user, chances are you'll want to go with inetd.

    Daemon Mode
    This is really quite easy. First, we'll need to edit our services file:
    Code:
    sudo nano /etc/services
    The following lines can go anywhere really, but the end of the file is convenient, particularly if you're planning to have many services running from this machine:
    Code:
    apt-cacher           3142/tcp
    apt-cacher           3142/udp
    3142 is the default port. I see no reason to change it, but if you have one, you may. If so, you'll need to change the daemon_port option in /etc/apt-cacher/apt-cacher.conf

    One more file to edit:
    Code:
    sudo nano /etc/default/apt-cacher
    Set AUTOSTART to 1 to get the daemon to launch at boot time.

    To launch it now, simply type the following:
    Code:
    sudo /etc/init.d/apt-cacher start
    Now, we're all done for the server side of things. Skip down to Part Two

    Inetd Mode
    To kick things off, we'll need to install the service:
    Code:
    sudo apt-get install inetutils-inetd
    Next, we edit a couple of files:
    Code:
    sudo nano /etc/services
    Add these two lines to the end of the file:
    Code:
    apt-cacher           3142/tcp
    apt-cacher           3142/udp
    Now, we have to setup inetd:
    Code:
    sudo nano /etc/inetd.conf
    Add this line:
    Code:
    apt-cacher stream tcp nowait www-data /usr/sbin/tcpd /usr/sbin/apt-cacher -i
    Now, restart the inetd server:
    Code:
    sudo /etc/init.d/inetutils-inetd restart
    You're done!


    Part Two: The Client
    To use apt-cacher, your clients need to redirect their requests to the server machine. APT has its own proxy configuration. Thanks to omegamormegil for this tip
    Type the following in a terminal:
    Code:
    echo 'Acquire::http::Proxy "http://hostname:3142";' | sudo tee /etc/apt/apt.conf.d/01proxy
    Where hostname is a valid identifier for your server (for home users, an IP address will probably be convenient).

    For laptop users, this means that you'll be unable to update using this method while on the road (unless you have a global route to your server, although the speed will be greatly reduced). To switch back to direct updates, type the following command:
    Code:
    echo 'Acquire::http::Proxy "http://";' | sudo tee /etc/apt/apt.conf.d/01proxy
    Then once you get home, the command given above will get you using shared updates again.

    If you prefer, APT will look for the last line in the file. You can cut/paste them as needed, but the above commands can be easily written into a script to automate the process for you.

    From ace214, the following command will tell apt-get to use direct fetching methods:
    Code:
    export http_proxy=http://
    As far as I know, this will only work for apt-get until you end your session (by logging out, or exiting/closing the terminal). If you're not tied to the GUI, this may serve as a useful solution for you.

    Now, update!
    Code:
    sudo apt-get update
    If all goes well, you should download packages as usual. Try downloading some software, or getting some updates. It should work pretty much as before.

    If you are upgrading, these steps will not work. This appears to be a bug in Update Manager.

    To upgrade, you must edit the 01proxy file to use direct updates as described above. Now, run the following command:
    Code:
    sudo sed --in-place s_http://_http://hostname:3142/_ /etc/apt/sources.list
    The syntax may seem a little daunting, but this simply replaces any occurance of http:// with http://hostname:3142/. Once again, hostname must be a valid identifier for your server.

    Upgrades should now work normally through the Update Manager. Once you've finished the upgrade, run the following command:
    Code:
    sudo sed --in-place s_http://hostname:3142/_http://_ /etc/apt/sources.list
    Now edit the 01proxy file to set yourself to shared updates as described above.

    So far, this is the only workaround I know of. I apologise to anyone who may have suffered this bug. Once more, thank you mssever for bringing this to my attention and providing a fix.


    Part Three: The Transition
    Chances are your server's up to date,and has a fair package cache of its own. Why duplicate all of this fine work? Apt-cacher comes with a migration helper for pre-caching. Type the following in a terminal to copy your cache over:
    Code:
    sudo cp /var/cache/apt/archives/*.deb /var/cache/apt-cacher/import
    sudo /usr/share/apt-cacher/apt-cacher-import.pl
    At the moment, the server machine itself will go downloading its own packages, rather than reusing those in the cache. To fix this, use the same proxy setup as used above on the clients, using "localhost" as the hostname.

    The final step is optional, but it will help you free up space. The server is currently maintaining two caches: the regular apt one, and the shared cache.

    NOTE: If you omit this step, APT will keep building its own cache. This may be of benefit to you when you uninstall apt-cacher if have limited bandwidth, but APT's operation will not be affected.

    If your server uses Synaptic at all, there's an option under Settings -> Preferences -> Files to delete packages after installation. Thanks to Skip Da Shu for pointing this out to me.

    This will only apply (in Ubuntu and Xubuntu) to packages fetched through the GUI. It will not work in Kubuntu, and will not work from the command line. For a more universal solution, enter the following command:
    Code:
    echo 'DPkg::Post-Invoke {"/bin/rm -f /var/cache/apt/archives/*.deb || true";};' | sudo tee /etc/apt/apt.conf.d/clean
    This command is carried out just before APT quits, and will clean the cache of any .deb files. Thanks to bugmenot_user for this command.


    Part Four: The Removal
    To undo the changes made in this guide, you must remove the proxy configuration lines. You must also remove the file /etc/apt/apt.conf.d/clean, or set the option in Synaptic back to its original value.

    Purging inetutils-inetd, inetutils-tcpd, and apt-cacher-ng will undo most of the remaining changes. Remove the two lines from the end of /etc/services, and your system should be back to the way it was.
    Last edited by coolen; August 23rd, 2010 at 01:41 AM.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •