Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: How do I know which applications are not POSIX-compliant?

  1. #1
    Join Date
    May 2005
    Location
    US
    Beans
    Hidden!
    Distro
    Ubuntu

    How do I know which applications are not POSIX-compliant?

    I'm interested in this new Ext4 thing.

    I've heard about possible data loss. And I've also heard if you use certain workarounds to prevent data loss, the performance gains of Ext4 are only minimal.

    The conventional wisdom seems to be that the data loss isn't Ext4's "fault" but the fault of applications that are not POSIX-compliant (whatever that means).

    So I'm just curious: how do I know which applications are not POSIX-compliant? And if I don't use any of those applications, is data loss pretty unlikely?

  2. #2
    Join Date
    Nov 2007
    Beans
    465

    Re: How do I know which applications are not POSIX-compliant?

    This may be part of the POSIX standards, but the data loss that I have heard of mainly occurs when a person kills the power to the computer (primarily by shutting it down with the power button, but other ways will do it too). That comes from ext4's vaunted speed increases: caching into RAM.

    Anyhow, there are sure to be bugs in the code that we have not discovered yet that could cause data loss. Ext4 is new, and there are many things that could go wrong. Even though I use it and have not encountered data loss, I may be lucky.

    Got a little off-topic there, I /think/ these are the POSIX standards: http://www.opengroup.org/onlinepubs/009695399/
    Last edited by collinp; June 4th, 2009 at 09:05 PM.

  3. #3
    Join Date
    Mar 2008
    Beans
    1,076
    Distro
    Ubuntu

    Re: How do I know which applications are not POSIX-compliant?

    Quote Originally Posted by aysiu View Post
    I'm interested in this new Ext4 thing.

    I've heard about possible data loss. And I've also heard if you use certain workarounds to prevent data loss, the performance gains of Ext4 are only minimal.

    The conventional wisdom seems to be that the data loss isn't Ext4's "fault" but the fault of applications that are not POSIX-compliant (whatever that means).

    So I'm just curious: how do I know which applications are not POSIX-compliant? And if I don't use any of those applications, is data loss pretty unlikely?
    links on the posix thinghy.
    http://en.wikipedia.org/wiki/POSIX

    the open groups singel specification page that is linked to at the bottom is very interesting reading also.

    i am simly too tired to refflect upon this but i think it sounds strange that a application will have greater chance at producing data loss if it is not posix than if it is. it sounds like the file format expects a certain way of programs handling files to work right.
    please buy this online only compilation to support victims of japan. http://www.punk4japan.com/

  4. #4
    Join Date
    Sep 2007
    Beans
    Hidden!
    Distro
    Ubuntu 9.04 Jaunty Jackalope

    Re: How do I know which applications are not POSIX-compliant?

    Do a search for "fsync ext4", this is one of the major concerns I heard leveled at ext4.

  5. #5
    Join Date
    May 2009
    Beans
    104

    Re: How do I know which applications are not POSIX-compliant?

    why not give zfs a try?

  6. #6
    Join Date
    Jun 2006
    Beans
    Hidden!

    Re: How do I know which applications are not POSIX-compliant?

    Quote Originally Posted by baseface View Post
    why not give zfs a try?
    Isn't ZFS only faster with larger files?

    As for data loss with Ext4, improper shut-down seems to be what is causing the majority of it. I haven't had any problems with it and I've been using it since it went to Arch's 2009.02 build. I also haven't cut the power to my computer without actually issuing 'halt' though.
    arch 2008.06
    Intel Core 2 Extreme QX6850 | Asus Maximus II Formula | EVGA 8800GT Superclocked | 4GB G.Skill DDR2-1000 | Zalman ZM600-HP

  7. #7
    Join Date
    May 2005
    Location
    US
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: How do I know which applications are not POSIX-compliant?

    I can't really think of a situation in which I'd have an improper shut down.

    I have a netbook (and, yes, that is my main computer), so if the power cuts out from the plug, it'll just go to battery.

    Maybe I will give it a go. I always back up my data anyway, I guess.

  8. #8
    Join Date
    Dec 2006
    Location
    Australia
    Beans
    1,097
    Distro
    Xubuntu 15.10 Wily Werewolf

    Re: How do I know which applications are not POSIX-compliant?

    POSIX is an operating system standard, not an application programming standard.

    The criticisms that Ted Tso levelled at application programmers are criticisms about lazy programming, rather than any break from standards. Unfortunately, lazy programming occurs just about everywhere.

    Not all fault can be levelled at application programmers. The POSIX standards do not specify behaviour for system crashes, nor does it dictate how files should be safely handled. All that application programmers have done is use the assumption (in the absence of specifications in POSIX) that their code is OK because ext3 seems to work well with their code. Unfortunately, system programmers who create file systems have gone on to enhance performance using the delayed allocation feature and assuming that application programmers have written robust code. Well, applications are not robustly written, and therefore: data loss.

    The thing is, ext4 is not alone in using the 30 seconds delays for its delayed allocation feature. XFS, and probably many others, use similar periods. It's just that ext4 has had more publicity due to it being the next flagship file system for Linux. Ted Tso has submitted his patch to reduce the delay to 5 seconds, which will alleviate the problem. That patch has been backported to Jaunty (as well as Fedora 11's kernel, and I suspect other newly-released or soon-to-be-released distro kernels).

  9. #9
    Join Date
    May 2005
    Location
    US
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: How do I know which applications are not POSIX-compliant?

    Thank you, samjh. That makes things a little clearer.

  10. #10
    Join Date
    Dec 2005
    Location
    Tucson, Arizona
    Beans
    Hidden!
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: How do I know which applications are not POSIX-compliant?

    without looking at the code, there really is no way to tell. Hopefully in time people will start writing their code to anticipate delayed allocation and fsync when necessary.
    Jabber: markgrandi[at]gmail.com

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •