Results 1 to 9 of 9

Thread: Opening and closing a file to append to it?

  1. #1
    Join Date
    Oct 2012
    Beans
    13

    Opening and closing a file to append to it?

    I've got a loop that goes through an array and appends the elements to a file and adds a field onto the end of the line to denote what type the element is.

    However should I open the file before the loop and then close it after the loop or open and close it on each iteration?

    Speed isn't that crucial since the loop waits on user input before moving on.

    Would opening and closing it on each iteration make the data safer if the program was to crash or be closed in the middle of execution?
    Last edited by arcsin; April 2nd, 2013 at 09:38 PM.

  2. #2
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,273
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Opening and closing a file to append to it?

    it would be nice to know what language we are talking about
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  3. #3
    Join Date
    May 2007
    Location
    Leeds, UK
    Beans
    1,664
    Distro
    Ubuntu 13.10 Saucy Salamander

    Re: Opening and closing a file to append to it?

    Which language are you using?
    Please create new threads for new questions.
    Please wrap code in code tags using the '#' button or enter it in your post like this: [code]...[/code].

  4. #4
    Join Date
    Oct 2012
    Beans
    13

    Re: Opening and closing a file to append to it?

    Python 2.7

  5. #5
    Join Date
    Oct 2007
    Beans
    1,914
    Distro
    Lubuntu 12.10 Quantal Quetzal

    Re: Opening and closing a file to append to it?

    Quote Originally Posted by arcsin View Post
    Would opening and closing it on each iteration make the data safer if the program was to crash or be closed in the middle of execution?
    Yes and no. In principle it might make your data safer. However, you can achieve the same at runtime by calling the "flush()" method of your file object after writing to the file.

    There is also the issue of unmounting a filesystem. If the file that you write to is on some USB pen drive or external hard drive, then you can assume that the device doesn't get unmounted until you close the file. So you have to take into account that opening the file might fail at any time, just like writing to the file may fail at any point (e.g., when the drive has no more free space). Finally, depending on the file system, if you want your changes to be visible to the outside world immediately after writing, you might want to close the file so that other applications can read the changes already. Some file systems may or may not make these changes visible before you close the file.

  6. #6
    Join Date
    Oct 2012
    Beans
    13

    Re: Opening and closing a file to append to it?

    Theres no issues with unmounting filesystems since the output file is in the same directory as the python script.

    If I make the code like this:

    Code:
    f = open("output.txt","a")
    
    for element in array:
        #Do stuff
        f.write(element)
        f.flush()
        
    f.close()
    Will that be pretty much the same as:
    Code:
    for element in array:
        #Do stuff
        
        f = open("output.txt","a")
        f.write(element)
        f.flush()
        f.close()
    Which one of those 2 is better practice / code?

  7. #7
    Join Date
    May 2007
    Location
    Leeds, UK
    Beans
    1,664
    Distro
    Ubuntu 13.10 Saucy Salamander

    Re: Opening and closing a file to append to it?

    A close must do a flush, so the flush in the second case is redundant isn't it?

    Wouldn't you want exception handling or a with block in here too, so that you can close the file neatly, report the error and continue?

    Looking at the docs, they suggest that you should use os.fsync in addition to flush to force a disk write: http://docs.python.org/2/library/os.html#os.fsync

    I'm happy for this to be pulled apart but I would suggest:

    Code:
    with open("output.txt") as f:
        for element in array:
            f.write(element) # exception here would call close
            f.flush()
            os.fsync(f.fileno())
    I don't know whether close() does os.fsync, so an exception block with flush, os.fsync and close in the finally block could be better?

    Code:
    f = open("output.txt")
    try:
        for element in array:
            f.write(element) # exception here would call flush, fsync and close
            f.flush()
            os.fsync(f.fileno())
    except:
        pass # do something useful here
    finally:
            f.flush()
            os.fsync(f.fileno())
            f.close()
    Please create new threads for new questions.
    Please wrap code in code tags using the '#' button or enter it in your post like this: [code]...[/code].

  8. #8
    Join Date
    Oct 2012
    Beans
    13

    Re: Opening and closing a file to append to it?

    Quote Originally Posted by r-senior View Post
    A close must do a flush, so the flush in the second case is redundant isn't it?

    Wouldn't you want exception handling or a with block in here too, so that you can close the file neatly, report the error and continue?

    Looking at the docs, they suggest that you should use os.fsync in addition to flush to force a disk write: http://docs.python.org/2/library/os.html#os.fsync

    I'm happy for this to be pulled apart but I would suggest:

    Code:
    with open("output.txt") as f:
        for element in array:
            f.write(element) # exception here would call close
            f.flush()
            os.fsync(f.fileno())
    I don't know whether close() does os.fsync, so an exception block with flush, os.fsync and close in the finally block could be better?

    Code:
    f = open("output.txt")
    try:
        for element in array:
            f.write(element) # exception here would call flush, fsync and close
            f.flush()
            os.fsync(f.fileno())
    except:
        pass # do something useful here
    finally:
            f.flush()
            os.fsync(f.fileno())
            f.close()
    Thanks the first code example works perfectly, I didn't try the second as I don't understand try and except blocks so I just used the more simple one that I understood.

  9. #9
    Join Date
    May 2007
    Location
    Leeds, UK
    Beans
    1,664
    Distro
    Ubuntu 13.10 Saucy Salamander

    Re: Opening and closing a file to append to it?

    Please don't take that code as the final solution, it was just a suggestion.

    If the code in the try block generates an exception, execution jumps to the except block for you to report the error. In all cases, whether an exception is thrown or not, the finally block runs. The key, therefore, is whether the flush and fsync happens if an exception is thrown, i.e. whether you want a finally block that does fsync.

    I'm not completely sure that fsync is a good idea in a loop because of all the disk writes it would generate. It might be better to have the finally block do the fsync but not have it in the loop. Or it might be better to just do standard file operations and not worry about all of this. I'm just offering an idea here, not a definitive solution.
    Please create new threads for new questions.
    Please wrap code in code tags using the '#' button or enter it in your post like this: [code]...[/code].

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •