How to synchronize a networked drive and an external drive (Ubuntu & OS X)?

[pedrogent]

Firstly, I've posted something similar here. I'm not sure the answer there goes into enough detail for me to understand. Furthermore, I'm not sure it deals with my concerns about resource forks being handled properly.

OK:

In light of a recent hardware failure, I've decided that it's time to be a little less lax regarding back-ups.

What I have:

• An Ubuntu 10.10 'server/NAS'
• A MacBook 5,2
• A WD external HDD

The server and the MacBook are permanently connected by Ethernet.

I have a directory on the server that I'd like to keep synchronized with the external drive that I plug in from time to time. The Ubuntu filesystem is ext4 and the external drive is HFS+ (Mac OS Extended [Journaled]). It should be noted that this is not case-sensitive. I'm unsure if this is a problem. If it is I can reformat the external HDD so that it is.

It should also be noted that OS X 10.6 ships with a fairly old rsync, ver. 2.6.9. I believe there are some problems with this concerning the handling of resource forks and other things.

So then, what is the best way to proceed?

• Should I be doing rsyncs across the network? If so some guidance re: how to achieve this using the CLI would be appreciated (I've read the rsync man page numerous times and can't quite get my head around it).
• Or should I be installing a kernel module so that I can just plug the HFS+ formatted drive into the server and it be read & writeable in Linux? Again, some rsync CLI guidance would be great.
• Or should I be doing something else entirely?

Thanks in advance and sorry about this duplication, but I really want to get my head around rsync. I feel if I do, my computing life will be made a lot easier.

[MrEgg]

If Ubuntu is your server, then can we assume that it's Ubuntu you want to rsync to your external drive - and that you manually take care of synchronizing your local files on the Mac with your server?

Additionally, are you going to remove the external drive from the server, or can you afford to leave it plugged in and turned on all the time?

If so, here's what I would do.

1. Format the external HD to ext4.
2. On the Ubuntu server, write a script that :
2.1. manually detects if your external is plugged into the server
2.2. rsyncs your server directory to the external drive
2.3. unmount the external drive (optional)
3. Set the script to run periodically

Let us know if this would do the trick for you so we can work on the script.

[pedrogent]

Thanks for the quick reply MrEgg.

For clarification purposes first:

• The external HDD is currently attached to the MacBook. So it looks something like this: Ubuntu --> MacBook --> external HDD.
• You're correct that I wish to rsync the server to the external HDD, and that I can manually handle the MacBook side of things.

With that in mind, it seems as though the best course of action is for me to format the external HDD to ext4 and thus avoid rsync'ing across Ethernet. Do you happen to know the state-of-play vis-à-vis FUSE-type utilities for OS X. I want to have this external HDD read & writeable in OS X.

After this I'll need a bit of a lesson in writing rsync scripts.

Cheers again for the rapid reply.

[MrEgg]

I'm not able to help from the Mac end, unfortunately.

In your configuration, would it be possible to plug the external HD directly to the server? If you format it in ext4, I do not know whether you'll be able to read it directly from the Mac or not -- ie you'll have to check for yourself. However, if you're okay with that configuration, we'll work on the script together, of course.

[pedrogent]

Thanks again MrEgg. I'll do some research re: having ext4 read/writeable in OS X. Once I've worked this out let's get down to that script.

I have a few things to do right now, but I'll report back ASAP.

[MrEgg]

Just in case OSX and ext4 don't play together well, check OSX and xfs as another possibility.

[pedrogent]

Ah cool will do. Am I hallucinating or is exFAT now supported on both ends? I'll looks into it.

[MrEgg]

Ah cool will do. Am I hallucinating or is exFAT now supported on both ends? I'll looks into it.

WHAT?? You want to use a msft filesystem?? You're kidding, right???

It won't support file ownership nor file permissions. It will fragment. You said you wanted to use your external HD for backup purposes - so use a reliable filesystem, please!

For ext4 on OSX, check this out (it's in French, you may want to google translate it) : http://www.digitalspirit.org/blog/index.php/post/2009/10/12/Monter-une-partition-ext2,-ext3-ou-ext4-sur-MacOs


Now, in your situation is it absolutely mandatory that OSX can mount the external HD directly? Even in ext4, it can be accessed by the mac through the ubuntu server via a proper network share.

[pedrogent]

Haha, yes... Moving on... But before we do, why can't we all get along and all use ZFS? OK, moving on now...

What you describe is fine: i.e. external HDD plugged into Ubuntu and I'll access that over the network. It'd be better if the drive was OS X read/writeable but it may not be possible under these circumstances.

I've been having odd AFP (netatalk/avahi-daemon) issues from time to time. Perhaps I should be using SMB?

Thanks for the link. I'll take a look.

[MrEgg]

You have at least 2 options :

SMB is easy and works just fine. How do you currently send your documents from the mac to the server? It could be just the same with the external drive. Remember that it's the server that'll be sending stuff to it, not the mac. Access through the mac is only for checking purposes, really. And besides, you'll always be able to ssh to the server and check the contents of the drive from there. Quick, fast, easy.

You can also mount the server drive on your mac through SSH, using macfusion. It works. Even remotely through the internet. Check this : http://www.simplehelp.net/2008/07/25...drive-in-os-x/

Just settle for a filesystem already, and then we can move on