Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: Backup for Snap configurations?

  1. #1
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Backup for Snap configurations?

    I hope this is something simple, but I haven't found it.
    As snap packages are used for more complex needs, how can we capture the special configurations we've made using the snap set commands?

    For example, I bet most people connect removable media and limit the total copies of snaps to 2, instead of the default 3, and perhaps they limit how often snaps can be updated to specific days of the week only to prevent interruptions to their work. How can we capture this data for our backup/restore needs in a flexible way. Flexible means on a per-snap basis, since during restore, we probably won't put all the same services onto the say systems.

    Code:
    $ sudo snap get system --all
    doesn't work.
    Code:
    $ sudo # snap get system refresh
    Key             Value
    refresh.retain  2
    refresh.timer   sat
    requires knowing the setting to retrieve it. That's beyond cumbersome. How can I get a list for all of them.

    I have to be missing something. Certainly, they don't expect users to create JSON to query every possible parameter using the REST API.
    Needed for 20.04 and later desktops and servers.

    This is also needed to backup lxd managed lxc containers, since many will have external storage connections specific to the container or special network settings.
    Last edited by TheFu; February 28th, 2023 at 06:20 PM.

  2. #2
    Join Date
    Feb 2019
    Location
    Virginia
    Beans
    386
    Distro
    Xubuntu 22.04 Jammy Jellyfish

    Re: Backup for Snap configurations?

    Does a read through this give any help. A bit over my head, sorry I have no real knowledge in this space. It makes sense though to want to control snap activity in order to prevent unwanted interruptions during backups or other processes.
    https://ubuntu.com/blog/how-to-manage-snap-updates

  3. #3
    Join Date
    Jul 2013
    Location
    Wisconsin
    Beans
    4,967

    Re: Backup for Snap configurations?

    Hmmm. See if https://snapcraft.io/docs/snapshots adds any light.

    Particularly this section on using snapshots for backup/restore:

    Inside a snapshot

    Snapshots are stored as a zip file for each snap, and each zip file contains the following:

    • meta.json: describes the contents of the snapshot, alongside its configuration and checksums for the archives.
    • archive.tgz: contains system data.
    • user/<username>.tgz: contains any user data (for each system user).


    On Ubuntu-based systems, snapshots are stored in the /var/lib/snapd/snapshots directory.

    Snapshots can also be exported/imported, specifically for backup purposes.

    Also, according to https://snapcraft.io/docs/data-locations ,

    - System configuration data should be at /var/snap/<snap name>/common
    - User configuration data should be at /home/<username>/snap/<snap name>/common
    Last edited by ian-weisser; February 28th, 2023 at 08:13 PM.

  4. #4
    Join Date
    Dec 2014
    Beans
    2,658

    Re: Backup for Snap configurations?

    You might want to try
    Code:
    sudo snap get system -d
    The '-d' option forces snap to return a JSON document and 'snap set' has an option (-t) to parse it's parameter as a JSON document (so that should take care of how to restore backed up settings, I hope)

    Or you could use the REST-API as described in https://snapcraft.io/docs/snapd-api :
    Code:
    sudo curl -sS --unix-socket /run/snapd.socket http://localhost/v2/system-info | jq
    I'm rather certain that there's a database or some files where snap stores all these settings, but up to now I haven't looked for them ...

    Holger

  5. #5
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Backup for Snap configurations?

    Quote Originally Posted by ian-weisser View Post
    Hmmm. See if https://snapcraft.io/docs/snapshots adds any light.

    Particularly this section on using snapshots for backup/restore:
    Snapshots can also be exported/imported, specifically for backup purposes.

    Also, according to https://snapcraft.io/docs/data-locations ,

    - System configuration data should be at /var/snap/<snap name>/common
    - User configuration data should be at /home/<username>/snap/<snap name>/common
    Thanks. "Snapshots" that result in tgz files are a failure to me. I have a backup solution that understands versioned files and would like to use that. I don't want new files created with time-stamps or directories named with a timestamp having all the data included each time. To me, that's like using ZIP for system backups. For people who don't do backups all the time, I can understand the good feeling that each new backup being 100% stand-alone would bring. That isn't me.

    The snaps I want to backup are system-level, not user-level programs. For example, I have the nextcloud snap running with the programs and backups in /var/snap/nextcloud/common/.
    Backups created by nextcloud.export look like this with time-stamped directories.
    Code:
    # tree -L 3 -d /var/snap/nextcloud/common/
    /var/snap/nextcloud/common/
    ├── backups
    │** ├── 20230228-110921
    │** │** ├── apps
    │** │** └── data
    │** └── 20230228-162248
    │**     ├── apps
    │**     └── data
    └── nextcloud
        ├── data
        │** ├── appdata_zyxabctmyydv
        │** ├── files_external
        │** ├── ann
        │** ├── dad
        │** ├── eileen
        │** ├── mom
        │** ├── rachel
        │** └── thefu
        └── tmp
            └── news
    The exported data isn't suitable for my backup requirements due to the timestamp directories. Fortunately, getting the NC applications, NC settings, NC data and NC dumped DBMS isn't hard. I'll probably end up scripting renaming the time-stamped directory name to a unique name to be included by my normal backup tool and let it handle the versioning. Would be nice of the nextcloud.export tool allowed forcing the same output directory to be used. Sigh.

    Thanks for the information. It does help - for the typical stuff, but not the container environment stuff.

    More stuff to research ... and need to read the next post carefully.

  6. #6
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Backup for Snap configurations?

    Just to be clear, I don't care to backup programs or packages. Those are trivial to replace.

    It is the customized settings and local data that matter.

    So the lxc command to add 3 external storage locations to a container, THAT's the information I'd like to capture for backups and have part of the restore process.

    Spent the last 2 hrs trying different commands. Looks like
    Code:
    lxc config show nextcloud-lxd --expanded
    is the per-instance command.

    If this seems unclear, it is partially because I remember entering about 10 lxc config set commands, but don't recall exactly what they were. Some were required ... and I was careful to NOT remove the non-root running aspect for any containers.
    Code:
    lxc  config show --debug
    Has lots of stuff, but not the ALL stuff I entered, just some if it from the lxd init command.

    lxc config show or
    Code:
    lxc config device list  nextcloud-lxd
    Found it!
    Code:
    $ lxc config device show nextcloud-lxd
    I'm sure there's a subtle difference between "show" and "list" options that I'm not getting. Now I just need to figure out the other 9 custom settings and how to get them back out. I'd hate to have to manually track these things. That's like asking someone to track manually installed deb packages.

    Part of the problem is lxd is a snap too. So the lxc containers are managed by lxc command included in the snap and the container is running a nextcloud snap inside it.

    I didn't want nextcloud polluting my hostOS, hence the use of an lxd managed container to have each major application separate. I'd not needed any special settings in the other lxc instances running here for the last few years, beyond using bridged networking. Perhaps I'll go with the docker method of port forwarding specific ports in the future. The snap for nextcloud forced non-default settings.

    Really want to dump all settings for all container configs 1 time. I suppose
    Code:
    Usage:
      lxc export [<remote>:]<instance> [target] [--instance-only] [--optimized-storage] [flags]
    may be the only way. Hard to tell. It has some caveats. First, lxc export isn't supported in lxc v3.x, which means it doesn't work for most of my lxc containers. I haven't researched why lxc is v3.0.3 on the system with lxd 5.11 installed. It is in the 20.04 lxc package, v5.11. Appears that lxc was separate in 18.04, but was merged into the lxd snap package sometime later.
    Yet another rabbit hole to follow.

    The manpages for lxd and lxc have the most concise information I've found all in one place, but it is huge and somehow there aren't any manpages for either lxc or lxd on my 20.04 install. Rabbit hole #2.

  7. #7
    Join Date
    Jul 2013
    Location
    Wisconsin
    Beans
    4,967

    Re: Backup for Snap configurations?

    Quote Originally Posted by TheFu View Post
    Now I just need to figure out the other 9 custom settings and how to get them back out.
    My own method is to keep notes about exactly how to reinstall and reconfigure everything from bare metal.

    When I was young, I thought "Aw, I will remember it". That turned out exactly as one might expect: I remembered nothing.

    Now, if I successfully install something, I promptly tear it back out by the roots.
    Then deliberately start over with my notes file handy, diligently recording each step, each test, each web page or source file or reference.
    Then I tear it out again and follow my notes, step-by-step, to install it a third time.

    I have used those notes to reinstall containers a couple times. But turns out I have used those notes for maintenance and repair far more often than for recovery.

    Younger me would laugh at doing it three times. But then younger me would be impressed by the glowing health of my maintained systems, and the lack of data loss for over a decade.
    Last edited by ian-weisser; March 1st, 2023 at 04:02 AM.

  8. #8
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Backup for Snap configurations?

    https://www.cyberciti.biz/faq/how-to...xd-containers/ does the lxd/lxc stuff, I hope. Ah ... it doesn't work in the lxc version I need to migrate off. Sigh.

    This afternoon, my screwing around took down the LAN DNS and email gateway and a few other containers. Bringing those backup up are priority now and lxd refuses to start ... spent 2 yrs trying to solve the issue and decided to rebuild them on a different system since the other host is on 18.04 and needs to be pushed to a different hostOS shortly.
    Last edited by TheFu; March 1st, 2023 at 03:19 AM. Reason: URL link is for newer LXC versions than I have.

  9. #9
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Backup for Snap configurations?

    Seems /var/snap/lxd/common/lxd has good stuff in it, but also some images/ ... which I'm trying to avoid. The lxd DBs are sqlite3 and those are in /var/snap/lxd/common/lxd/database/. The security/ and perhaps devices/ directories there could be useful as well. How? IDK. If the DBs become corrupt, having a backup is important. Still looking at the other things below there that are needed to recreate a setup 100%, preferably via automation. On the old lxd/lxc version, I'm not hopeful.

    Added zfs information to the backup data capture too. Doubt I have all that is needed, but when it is scripted, it can be incrementally improved.

    Got my "pull" backups working for the new lxc instances created yesterday. Still need to migrate some less important lxc containers off the 18.04 box, which is hard since the entire lxd/lxc subsystem is broke due to a corrupted DB.
    The system receiving this added workload is running short on RAM. I have 16G more around here somewhere. Just need to find it.
    Happy to say the steps to setup a root-equiv account are 100% working every time. That's needed for the pull backups to capture everything.

    When I get this working and it isn't embarrassing code, I'll post it somewhere. Unfortunately, it isn't trivial code, so it won't help most users there. Perhaps just the people using LVM and/or ZFS snapshots as part of their backup methodology would care for the complexity my backups have become. I have little confidence that I could restore anything besides the data for the lxc backups at this point. Suppose that's a start. Wish I'd be happy using ZFS snapshots + zsend to another machine as a viable versioned backup solution. I don't think that's sufficient and the space requirements would likely explode.

  10. #10
    Join Date
    Mar 2010
    Location
    USA
    Beans
    Hidden!
    Distro
    Ubuntu Development Release

    Re: Backup for Snap configurations?

    Please do post what you come up with!!! Pretty please? I am following this with very much interest in what you find out, and what you find that works. That would definitely make life easier.

    Like Ian, I 'try' to keep good notes, but after re-reading them from time to time, I have to do it again to figure out what I was thinking at the time (LOL).

    "Concurrent coexistence of Windows, Linux and UNIX..." || Ubuntu user # 33563, Linux user # 533637
    Sticky: Graphics Resolution | UbuntuForums 'system-info' Script | Posting Guidelines | Code Tags

Page 1 of 2 12 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •