Where are inode tables for USB drives?

[Iggy64]

As a Linux beginner, I am trying to comprehend how file permissions are created and saved. Of particular interest to me are permissions for files on detachable USB devices. With my very limited knowledge of how these file systems work, I get the general idea that a file's permissions (and ownership and block location and so on) is stored in its inode. So, the file's name can be looked up in its directory, which provides the inode number, which can then be used to look up the permissions in the inode.

Next, I have this vague notion that the inodes are stored in tables that are distributed throughout the physical storage devices, so that an inode table is reasonably close to the file(s) it describes. I don't know if these means it is in the same group block, or just the same disk cylinder, or what. Too many opinions out there.

Anyway, when I attach a USB drive and then mount it, I assume the mounting process must be creating inode mappings to all the files on the USB drive. I would really like to know if the inode tables for the files on the USB drive are actually written onto the USB drive. If so, what happens when I unmount it? Does the system take all those inodes back into the pool, and the permissions info about the files is simply gone? When I re-attach and remount the drive later, is a fresh batch of inode tables created? And therefore do I have to re-specify the desired permissions?

Maybe I'm asking the wrong questions because my knowledge of all this is so rudimentary. But I find file permissions to be rather bewildering and challenging. If I could better understand what is stored where, and how persistent or temporary it is, it would be very helpful

[VMC]

I'm not sure how deep you want to pursue this, but

chgrp --help,chmode --help

, and

ls --help, ls -i (inodes)

Also read up on coreutils

info coreutils

[Bashing-om]

Iggy64;

A different perspective but highlights file structure and the role of inodes:
http://linux.sys-con.com/node/117909?page=0,0

inquiring minds want to know

[mcduck]

It's same as with any other drive, really. If the USB drive is formatted into Ext4 or some similar Unix filesystem, then the inodes are stored on the drive itself as they are part of the filesystem.

And on the other hand if the drive is formatted to FAT, for example, there are no inodes as FAT doesn't use them. (And in the same way, the permissions & ownerships can't be stored at all since FAT doesn't support them either)

[Iggy64]

I do want to learn as much as I can about how GNU/Linux systems work, so that I can fully break free from Windows. I very much appreciate the helpful resources suggested by VMC and Bashing-om. I have "bookmarked" those pages (and commands) and have starting drilling down through them. I can see how much more I need to learn. But I am grateful to know where to find this information. Indeed, the link at http://linux.sys-con.com/node/117909?page=0,0 definitely provided some "glue" to confirm some of my suspicions and then go a step beyond. Thanks much for that.

Thanks, too, to mcduck for your "take" on where the inode information is stored. It is consistent with the ideas in the above-mentioned link. If - in fact - the inodes for the files on a USB drive formatted ext4 are stored in the group blocks on that drive, then I would expect the permissions (which are contained in those inodes) to continue to exist when I detach the USB drive and store it elsewhere. If I re-attached it a week later, those permissions should still be there, and I won't have to reset them. I'll have to try to confirm this with some more experiments later this week. Presumably, I could similarly attach the USB drive to a completely different Linux machine, and still have it recognize those permissions? Again, something else to try.

I suppose if you had lots of different USB drives and took turns connecting them up to one Linux box, you might use up all the inodes that the system allocated at system creation time? If so, how would you go about freeing up those inodes? There is so much to learn.

I understand the business about how FAT drives are different. But I wonder about an NTFS USB drive that has been mounted with the --permissions option. Are inodes created on such a drive? If not, where are the permissions stored? How persistent are they, compared with those on an ext4 drive?

Why do I want to know all this? So far, I am much more of a user than a OS expert (as if you couldn't tell). A main hobby is collecting, processing, and playing music files. Some of my legacy collections are on NTFS drives. (Sure, I could copy them all to ext4 drives; but there is only so much time.) Over the years, I have had various overzealous software packages mess with the metadata in my music files ("helping me out" by updating stuff I don't want to change). In my Windows days, I soon learned to make all the music files "read only" and leave them that way, except when I really wanted to make changes. So no more costly "accidents" happened. Now, when moving to Linux, I hear about all the added security features. But I quickly found that these features come with a price: they can be very complicated for someone who is mainly a "user." In Windows, the "read only" attribute was simply stored with the file -- easy to visualize. In Linux, the permissions are broken into three groups (user, group, other), and not stored with the file, but spread around in various inode tables. The directory links the file to the inode table which is somewhere in the group block, which is .... . So there seems so much more to learn if you don't want to screw up, and you want to feel comfortable that you are actually protecting your files.

Therefore, I thank those of you who responded because your help is invaluable. I have read for many hours, and made little diagrams all over my notebook. I am beginning to understand how this works, but am not super confident. Without your help, this would take me forever. I really, really appreciate the support offered by this forum.

[mcduck]

As inodes are not an opertaing system feature but instead a file system feature, there is no one pool of inodes the OS would assign from when you mount a drive. INstead each filesystem has it's own inodes, created at the time when the drive(or partition) was formatted.

So you'll never run out of inodes by mounting drives, only by using more inodes in a specific filesystem (partition or a drive) than what it was set to have when it was formatted. And filesystems that don't use inodes don't get assigned any when they are mounted either. Different filesystems have their own ways for managing files and their ownerships, permissions and other metadata.

What comes to permissions on removable drives formatted with Unix filesystems, they are indeed stored on the filesystem itself and will work after removing and reconnecting the drive, or when moved to another machine. The only limitation is that ownership is not tied to user name, but the UID which is the numerical version of user name, and same UID might be used by different user account on another computer.

[Iggy64]

mcduck, that is extremely helpful stuff! I have been reading one tutorial after another for days on this subject, I haven't come across such a straightforward, logical explanation. Now I feel like I have a decent idea of how to protect my files on an ext4 drive/partition. I am saving your explanation to my KeepNote notebook.

Do you by chance have any thoughts on my other question, concerning NTFS drives? I probably won't be using them much longer, since I will be working on transfering all that old content onto newly formatted ext4 drives. Meanwhile, though, it would be nice to understand how permissions are stored for files on those NTFS drives. To the best of my knowledge (which isn't much, yet), I can enable Linux-type permissions on an NTFS drive by mounting the drive with the --permissions option. What I wonder now is: does Linux somehow write inode tables into the blocks on an NTFS drive, even though it is not a native Linux file system? Or are the permissions stored some other way/place? Bottom line: Are they stored on the NTFS disk (as in the case of an ext4 drive), so they will persist when the drive has been detached and then re-attached and remounted?

Thanks very much for your continuing support. This has been one of the most helpful discussions I've had in trying to learn Linux basics.

[mcduck]

NTFS has it's own way of storing file permissions and file metadata, and when you mount an NFTS drive with the --permissions the filesystem driver in Linux simply remaps the UNIX permissions into NFTS permissions, and then the filesystem is able to store them just like if you were setting the permissions while running Windows. So no inodes are created or used to store permissions on NFTS drives.


Anyway, like I said the ownership of files is based on UID, not a specific user, so file ownership and permissions are not a secure way to restrict access to your file. They are intended to protect files between users of a running system, but if somebody is able to move the drive to another computer, or even boot the same computer with another operating system, the ownerships won't help you. To secure your files in that kind of situations encryption is the only option.

(if you want a simple to use and flexible way of encrypting files & directories, I recommend taking a look at Gnome EncFS Manager, which allows you to create encrypted directories and mount & unmount them as you wish rather than having to encrypt the complete filesystem. Works nice with removable drives and even with Dropbox and other cloud storage services )

[Iggy64]

Thank you, mcduck, for the additional info.

From my reading, I get the impression that NTFS systems store their file metadata (permissions, dates, block locations, etc.) in a "Master File Table," with an entry for each file on the system. Each record in the MFT is therefore equivalent to an inode in Linux. However, it sounds like all the records are in one large file (MFT), rather than distributed about the disk in smaller tables, as they are in Linux.

So, if I mount the NTFS drive using the permissions option, does Linux use my current Linux default permissions to overwrite the entire MFT, or does Linux simply read the permissions that are already present in the MFT and interpret them in Linux terms? And if I use chmod to change permissions on the NTFS drive, does Linux "do its best" to create the Windows-equivalent permissions in the MFT? (I realize that I should be simply trying this out, but I really don't want to play games with any of valuable NTFS data right now. Perhaps I can find an old drive to use for experimenting.)

Thanks for the advice on encryption. Right now, that is not my concern. I'm virtually the only one with access to any of these disks. My worry is the accidental modification of files that are not write protected. As I may have noted earlier, I have had certain music-playing software automatically changed the metadata in hundreds of my files without my knowledge, only to be discovered sometime later --- causing me to spend hours/days repairing them. Therefore I want these music files to be read-only, unless I want to switch them briefly to writable, when I WANT to change/update the tags. So I want to make the NTFS disks read only most of the time, but also want to be able to temporarily chmod to writable when I need to update at tag.

With your help (and patience), I feel that I am getting closer to understanding how this works. If I can understand it, I can probably avoid making dumb mistakes.

[JKyleOKC]

So, if I mount the NTFS drive using the permissions option, does Linux use my current Linux default permissions to overwrite the entire MFT, or does Linux simply read the permissions that are already present in the MFT and interpret them in Linux terms? And if I use chmod to change permissions on the NTFS drive, does Linux "do its best" to create the Windows-equivalent permissions in the MFT? (I realize that I should be simply trying this out, but I really don't want to play games with any of valuable NTFS data right now. Perhaps I can find an old drive to use for experimenting.)

Since McDuck seems to be offline at the moment, I'll jump in and possibly confuse the issue. While I've disassembled the original FAT file system code from MS-DOS days, and learned the major differences for FAT32 (known to Linux as vfat), my knowledge of the internals of NTFS is limited to some semi-educated guesswork.

That said, I do know that Linux will never overwrite the MFT, although when it writes to an NTFS system it may have to modify a few entries. For many years, NTFS filesystems were only mounted in read-only mode because the Linux drivers of the day were not reliable in the MFT modification. However, for the past 6 years or more NTFS systems have been just as reliable as EXT2, 3, or 4.

What does happen is that when an NTFS filesystem is mounted, the driver establishes the Linux permissions for everything to the same set of NTFS permissions. This is controlled by the "options" parameter of the "mount" command or the /etc/fstab file. The default permissions, as I recall, are read-only and execute (even if the file is not executable!) and the file owner defaults to the user that mounted the file -- usually "root" since by default normal users cannot mount anything. However, options can be used to set read-write mode, and the user and group owner values.

These permissions, set at mount time, apply to every file and directory within the mounted system and cannot be changed. The "chmod" and "chown" commands do not work on NTFS systems -- and this often confuses newcomers. However there's a "remount" option to the mount command that you can use to change things if need be. I use it so often to make USB drives writeable that I've created an alias for it!

I hope this helps some... Keep asking questions and you'll find many of us eager to help.