Page 6 of 10 FirstFirst ... 45678 ... LastLast
Results 51 to 60 of 99

Thread: How NOT to write a shared library

  1. #51
    Join Date
    Feb 2007
    Beans
    236

    Re: How NOT to write a shared library

    Quote Originally Posted by aks44 View Post
    before it becomes unpleasant
    No, stick around. After Wy completes the tests and posts the results, things are going to get even funnier.

  2. #52
    Join Date
    May 2007
    Location
    Paris, France
    Beans
    927
    Distro
    Kubuntu 7.04 Feisty Fawn

    Re: How NOT to write a shared library

    Quote Originally Posted by j_g View Post
    Quote Originally Posted by aks44 View Post
    before it becomes unpleasant
    No, stick around.
    Precision: before my previous rant became unpleasant (or even rude, knowing myself) to the people that believe there can be excuses for killing an application from a library...


    No way I'll stop watching this thread, it is way more funny than any comedy.
    Not even tinfoil can save us now...

  3. #53
    Join Date
    May 2007
    Beans
    245
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: How NOT to write a shared library

    FWIW -- I have really enjoyed these threads... so much so that I've extended the conversation onto Usenet:

    http://groups.google.com/group/alt.o...a163b7b641a052

  4. #54
    Join Date
    Feb 2007
    Beans
    236

    Re: How NOT to write a shared library

    Omg! That is funny! A person on the newsgroup actually asks the question:

    Do you really think a program can carry on and do anything reasonable when it runs out of memory?

    Um... YES! That's the whole point of checking the error return from malloc, and if it's zero, doing something besides keeling over dead into a call to exit(). Yes Virginia, your app can continue to do something.

    Is this guy for real??? And he's calling you "an idiot"???

    I tell you. We've got to get computer science courses to teach error handling. Some graduates appear not to know what it entails. (And while we're at it, can we please teach them that a failed call to malloc is not the end of the world, assuming, unlike the Pulse Audio author, you don't believe that it's a waste of time trying to recover from it? On the other hand, a call to exit() is definitely the end of your app. The former is recoverable if you actually know how to write error handling. The latter is fatal).
    Last edited by j_g; November 17th, 2007 at 04:08 PM.

  5. #55
    Join Date
    Dec 2006
    Location
    Uk
    Beans
    109

    Re: How NOT to write a shared library

    Quote Originally Posted by Wybiral View Post
    I decided to test this out on my own computer using this code:

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    #define MB (1024 * 1024)
    
    int main(int argc, char *argv[])
    {
        char *mem = NULL;
        long long unsigned int size = MB * 512;
        for(;;)
        {
            if((mem = malloc(size)) != NULL)
            {
                free(mem);
                printf("Successfully allocated and freed %llu bytes!\n", size);
            }
            else
            {
                printf("Unable to allocate %llu bytes. Resize and try again!\n", size);
                break;
            }
            size += MB * 512;
        }
        return 0;
    }
    Read this article to understand why your test doesn't really test what you think it does.
    http://www.linuxdevcenter.com/pub/a/...of-memory.html

    Malloc is an abstraction. Like many abstractions it has edge cases where the underlying details leak through. In this case the leak is malloc can allocate memory that doesn't exist. When you try and use the memory there is a problem which the malloc abstraction has no way of dealing with.
    Last edited by winch; November 17th, 2007 at 05:30 PM.
    OpenStreetMap - Free editable map of the whole world

  6. #56
    Join Date
    May 2007
    Location
    Paris, France
    Beans
    927
    Distro
    Kubuntu 7.04 Feisty Fawn

    Re: How NOT to write a shared library

    Quote Originally Posted by winch View Post
    Read this article to understand why your test doesn't really test what you think it does.
    http://www.linuxdevcenter.com/pub/a/...of-memory.html
    Good point.


    Back on topic, in this very article the author says (page 2):

    Check for NULL Pointer after Memory Allocation and Audit for Memory Leak

    This is a simple rule, but it sometimes goes omitted. By checking for NULL, at least you know that the allocator could extend the memory area, although there is no obvious guarantee that it will allocate the needed pages later. Usually, you need to bail out or delay the allocation for a moment, depending on your scenarios. Together with overcommit tunables, you have a decent tool to anticipate OOM because malloc() will return NULL if it believes that it cannot acquire free pages later.
    Later:

    Use ulimit() for User Processes

    With ulimit -v, you can limit the address space a process can allocate with mmap(). When you reach the limit, all mmap(), and hence malloc(), calls will return 0 and the kernel's OOM killer will never start. This is most useful in a multi-user environment where you cannot trust all of the users and want to avoid killing random processes.
    The point being: way before any hard, unrecoverable OOM condition is hit, the kernel gives any well behaved application (or library ) plenty of time (due to malloc returning NULL) to scale down and/or gracefully clean up its mess.

    For those who still doubted, this confirms again that a library killing its host application at the first NULL malloc is just badly designed / behaved. Now when the author of such library says that people making a point about good programming practices have "no clue about memory management" and evokes the kernel's OOM killer to support his claims, oh well, I wonder (not!) who really is clueless...
    Last edited by aks44; November 17th, 2007 at 05:47 PM.
    Not even tinfoil can save us now...

  7. #57
    Join Date
    Feb 2007
    Beans
    236

    Re: How NOT to write a shared library

    Wy, I just want to know how putting

    Code:
    if (!mem) exit();
    compares to what happens with your original, "proper" memory checking. That's the real point. You have to put that exit() in there, just like pa_xalloc() does. And now you have an actual working example of how the two different approaches yield two entirely different outcomes, one useful, and the other bad. The minute details of what happens inside that call to malloc are irrelevant. We're talking only about the code around it. That's the error handling part.

  8. #58
    Join Date
    Dec 2006
    Location
    Uk
    Beans
    109

    Re: How NOT to write a shared library

    Quote Originally Posted by aks44 View Post
    The point being: way before any hard, unrecoverable OOM condition is hit, the kernel gives any well behaved application (or library ) plenty of time (due to malloc returning NULL) to scale down and/or gracefully clean up its mess.
    The point being is that the malloc abstraction alone doesn't provide the ability to handle the problem. If you want to handle the problem you have to leave the nice simple world provided by malloc and start worrying about lots of complicated details.

    The whole point of malloc is to enable people to write software without having to deal with those details.
    If malloc alone is the right answer or not is a question that often doesn't have a right answer. People use computers for different tasks and so often have opposing needs. How do you keep audio running for as long as possible for one user and sacrifice it as quickly as possible for another?
    OpenStreetMap - Free editable map of the whole world

  9. #59
    Join Date
    Jan 2007
    Location
    the third world
    Beans
    Hidden!

    Re: How NOT to write a shared library

    It's a mistake. Probably introduced by a not-very-experienced programmer and overlooked by others.
    Happens to the best of projects.

    Hopefully, it'll be fixed soon.
    IESVS FELLAT IN INFERNVM

  10. #60
    Join Date
    Oct 2006
    Location
    Austin, Texas
    Beans
    2,712
    Distro
    Ubuntu 7.10 Gutsy Gibbon

    Re: How NOT to write a shared library

    Quote Originally Posted by winch View Post
    Read this article to understand why your test doesn't really test what you think it does.
    http://www.linuxdevcenter.com/pub/a/...of-memory.html

    Malloc is an abstraction. Like many abstractions it has edge cases where the underlying details leak through. In this case the leak is malloc can allocate memory that doesn't exist. When you try and use the memory there is a problem which the malloc abstraction has no way of dealing with.
    But in C it is considered best practice to check your malloc return. And if this is the case for most oom situations involving malloc, then the pa_malloc is silly anyway (it's checking for a NULL to bail out, why do that if it's not going to bail out anyway?)

    But in my experience I haven't found a situation (on my computer) that doesn't result in a NULL return from malloc. It's been consistent. I'd like others to do the same. Maybe it varies from gcc to gcc or kernel to kernel... I don't know. But it's always worked for me.

    People don't do it because it's easier to use exit (ie: they're lazy) which may be fine for small applications that aren't really important, but if a useful app might be built with your code, DON'T DO THAT. You are going to screw some users out of their data.

    I would be devastated if I had just managed to record that perfect guitar solo I'd been working on for the past month and I try to play it back before saving it and... POOF. It all disappears. I can't play that again! One time thing... But I will say once again: I don't know when PA uses it or when PA will be used. But it doesn't look good.

Page 6 of 10 FirstFirst ... 45678 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •