Page 1 of 2 12 LastLast
Results 1 to 10 of 18

Thread: rsyncing rsnapshots takes up too much space

  1. #1
    Join Date
    Aug 2005
    Beans
    28

    rsyncing rsnapshots takes up too much space

    Hi everybody

    I use rsnapshot for my backups. I don't directly back up to an external disk. I backup my data locally via rsnapshot and then rsync the rsnapshot to the usb disk. The usb disk is always full though. I think it doesn't back up the rsnapshots like they are, instead backs up all of the data every time. Is that possible ? If yes, what could I do instead? I don't want to rsnapshot to the usb disk directly.

    Thanks for all answers

    cheers

  2. #2
    Join Date
    Apr 2008
    Location
    San Diego, CA
    Beans
    162
    Distro
    Ubuntu

    Re: rsyncing rsnapshots takes up too much space

    Why don't you want to do the rsnapshot to the drive directly? I've been using rsnapshot for several years now to a USB connected drive without issues.

  3. #3
    Join Date
    Aug 2005
    Beans
    28

    Re: rsyncing rsnapshots takes up too much space

    The reason I don't want to do that is because I will be switching the USB disk and I want the Backup of my data to be local as well.

  4. #4
    Join Date
    Aug 2005
    Beans
    28

    Re: rsyncing rsnapshots takes up too much space

    Ok I believe this was my problem (yes I'm ashamed of myself):

    * 04 * * * /root/backup/rsynctousb.sh

    instead of

    0 04 * * * /root/backup/rsynctousb.sh



    If it works now (which I believe I will close the thread)

  5. #5
    Join Date
    Aug 2005
    Beans
    28

    Re: rsyncing rsnapshots takes up too much space

    rsyncing still fills up all the space on my usb disk. Anyone have any ideas ?

  6. #6
    Join Date
    Sep 2011
    Location
    Behind you!
    Beans
    531
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: rsyncing rsnapshots takes up too much space

    I have to ask....how large is the total amount of space you are wanting to sync to the USB and what size is the USB...and is it dedicated for just that backup?

  7. #7
    Join Date
    Aug 2005
    Beans
    28

    Re: rsyncing rsnapshots takes up too much space

    Hi LHammonds,

    Thanks for your reply.

    The rsnapshot backup is 213GB. My usb disk is 1.8 TB and yes it is only used for backups. df -T outputs this:

    /dev/sdi1 fuseblk 1953511420 1952793420 718000 100% /usb_backup

    Could it be that the problem is the type? (fuseblk). I think i formatted it as ntfs so I could just plug it into either linux or windows if I need a backup.

  8. #8
    Join Date
    Sep 2011
    Location
    Behind you!
    Beans
    531
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: rsyncing rsnapshots takes up too much space

    fuseblk is how Linux reports NTFS mounted drives so that is probably no problem at all. However, a big part of the puzzle is exactly how much space is being consumed. The block size being used and the amount/size of the files being backed up may not be matching up very well (e.g. very large block size but tons of super tiny files)

    Type the following so we can see how big the folder is:

    Code:
    du -sh /path/to/usb
    Then list the contents of the drive where the snapshot files are being stored in order to get a feel for the size of each file over time.

    Code:
    ls -lh /path/to/usb
    or if all the files are over a GB,
    Code:
    ls -l --blocksize=g /path/to/usb

  9. #9
    Join Date
    Aug 2005
    Beans
    28

    Re: rsyncing rsnapshots takes up too much space

    Hi LHammonds

    /data1/backup/rsnapshot is where the rsnapshot writes to locally

    Code:
    sudo du -sh /data1/backup/rsnapshot/
    213G    /data1/backup/rsnapshot/
    Code:
    du -sh /data1/backup/rsnapshot/daily.0
    201G    /data1/backup/rsnapshot/daily.0
    
    du -sh /data1/backup/rsnapshot/daily.1
    203G    /data1/backup/rsnapshot/daily.1
    
    du -sh /data1/backup/rsnapshot/daily.2
    203G    /data1/backup/rsnapshot/daily.2
    
    du -sh /data1/backup/rsnapshot/daily.3
    203G    /data1/backup/rsnapshot/daily.3
    
    du -sh /data1/backup/rsnapshot/daily.4
    203G    /data1/backup/rsnapshot/daily.4
    
    du -sh /data1/backup/rsnapshot/daily.5
    203G    /data1/backup/rsnapshot/daily.5
    
    du -sh /data1/backup/rsnapshot/daily.6
    203G    /data1/backup/rsnapshot/daily.6
    Code:
    ls -lh /data1/backup/rsnapshot/
    total 208K
    drwxr-xr-x 6 root root 4.0K 2012-09-11 03:30 daily.0
    drwxr-xr-x 6 root root 4.0K 2012-09-10 03:30 daily.1
    drwxr-xr-x 6 root root 4.0K 2012-09-09 03:30 daily.2
    drwxr-xr-x 6 root root 4.0K 2012-09-08 03:30 daily.3
    drwxr-xr-x 6 root root 4.0K 2012-09-07 03:30 daily.4
    drwxr-xr-x 6 root root 4.0K 2012-09-06 03:30 daily.5
    drwxr-xr-x 6 root root 4.0K 2012-09-05 03:30 daily.6
    drwxr-xr-x 6 root root 4.0K 2012-07-30 03:30 monthly.0
    -rw-r--r-- 1 root root 156K 2012-09-11 03:30 rsnapshot.log
    drwxr-xr-x 6 root root 4.0K 2012-09-03 03:30 weekly.0
    drwxr-xr-x 6 root root 4.0K 2012-08-27 03:30 weekly.1
    drwxr-xr-x 6 root root 4.0K 2012-08-20 03:32 weekly.2
    drwxr-xr-x 6 root root 4.0K 2012-08-10 03:30 weekly.3
    /usb_backup/backup is where the local rsnapshots I made are being rsynced to

    Code:
    du -sh /usb_backup/backup/rsnapshot/
    1.6T    /usb_backup/backup/rsnapshot/
    Code:
    ls -lh /usb_backup/backup/rsnapshot/
    total 160K
    drwxrwxrwx 1 root root    0 2012-09-11 03:30 daily.0
    drwxrwxrwx 1 root root    0 2012-09-10 03:30 daily.1
    drwxrwxrwx 1 root root    0 2012-09-09 03:30 daily.2
    drwxrwxrwx 1 root root    0 2012-09-08 03:30 daily.3
    drwxrwxrwx 1 root root    0 2012-09-07 03:30 daily.4
    drwxrwxrwx 1 root root    0 2012-09-06 03:30 daily.5
    drwxrwxrwx 1 root root    0 2012-09-05 03:30 daily.6
    drwxrwxrwx 1 root root 4.0K 2012-07-30 03:30 monthly.0
    -rwxrwxrwx 1 root root 156K 2012-09-11 03:30 rsnapshot.log
    drwxrwxrwx 1 root root    0 2012-09-04 04:00 weekly.0
    drwxrwxrwx 1 root root    0 2012-09-04 04:00 weekly.1
    drwxrwxrwx 1 root root    0 2012-09-04 04:00 weekly.2
    drwxrwxrwx 1 root root    0 2012-09-04 04:00 weekly.3
    Code:
    du -sh /usb_backup/backup/rsnapshot/daily.0/
    201G    /usb_backup/backup/rsnapshot/daily.0/
    
    du -sh /usb_backup/backup/rsnapshot/daily.1
    203G    /usb_backup/backup/rsnapshot/daily.1
    
    du -sh /usb_backup/backup/rsnapshot/daily.2
    203G    /usb_backup/backup/rsnapshot/daily.2
    
    du -sh /usb_backup/backup/rsnapshot/daily.3
    203G    /usb_backup/backup/rsnapshot/daily.3/
    
    du -sh /usb_backup/backup/rsnapshot/daily.4
    203G    /usb_backup/backup/rsnapshot/daily.4
    
    du -sh /usb_backup/backup/rsnapshot/daily.5
    203G    /usb_backup/backup/rsnapshot/daily.5
    
    du -sh /usb_backup/backup/rsnapshot/daily.6
    203G    /usb_backup/backup/rsnapshot/daily.6
    I'm really starting to think, that rsyncing the rsnapshot somehow always copies all the files instead of just the links.

  10. #10
    Join Date
    Jun 2006
    Location
    Brisbane Australia
    Beans
    713

    Re: rsyncing rsnapshots takes up too much space

    Quote Originally Posted by silbro View Post
    I'm really starting to think, that rsyncing the rsnapshot somehow always copies all the files instead of just the links.
    Remember that rsnapshot works by using hardlinks to save space. Note that rsync does not preserve hardlinks by default with "-a". You must also specify "-H". See man rsync.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •