Results 1 to 5 of 5

Thread: Python doesn't write when log files are rotated

  1. #1
    Join Date
    Oct 2020
    Beans
    8

    Python doesn't write when log files are rotated

    Hi, I'm trying to find a way to rotate my logfiles. What I'm doing is correctly rotating my logfile, but Python no longer writes to the file after it is rotated. I'm rotating the files using a simple bash script, and I'm calling that script once per minute using crontab.

    I'm pretty sure that I can fix this in Python using the os library, something like test for my logfile, if present use it, if not create it...and then take the "touch" function out of my bash, but I really want to understand what is happening here.

    Also: I can tail -f my example.log and I get updates, but when I look in the file, it is empty. This is the confusing part!

    Here is my quick Python script:

    Code:
    import logging as templog
    import time
    
    
    templog.basicConfig(filename='example.log',level=templog.INFO)
    
    
    def basicLoopFunction():
        print("Starting Function....")
        templog.info("Starting Function....")
        while True:
            print("Executing....")
            templog.info("Executing....")
            time.sleep(10)
    
    
    basicLoopFunction()
    My log rotator looks like this:

    Code:
    
    #!/bin/bash
    #
    # Script to manage log files
    #
    
    
    rm /home/shadow/dev/logTest/example.log.1 &> /dev/null
    mv /home/shadow/dev/logTest/example.log   /home/shadow/dev/logTest/example.log.1 &> /dev/null
    touch /home/shadow/dev/logTest/example.log
    and my crontab line is:

    Code:
    * * * * * /home/shadow/dev/logTest/logrotate
    Last edited by shadowsaunter; December 24th, 2020 at 06:51 PM.

  2. #2
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Python doesn't write when log files are rotated

    close() the file before the rotation, then open() a new file after.
    open() files access the same inode, regardless of the filename.
    If the file inode is not closed, the new file will not be written into.

    The wikipedia article on inodes may be useful background.

    Once a minute seems obsessive for production.

  3. #3
    Join Date
    Nov 2007
    Location
    London, England
    Beans
    7,202
    Distro
    Xubuntu 20.04 Focal Fossa

    Re: Python doesn't write when log files are rotated

    Just re-wording what TheFu said (he is absolutely right):
    Once python has opened example.log for writing, something else renaming the file will make no difference. Python will continue writing to it not even realising that it has been renamed.

    One solution would be to have python open, write and close every time it has log messages to write (or close and re-open fairly frequently).
    Another solution would be to get python to do the rotating.
    Last edited by The Cog; December 24th, 2020 at 09:57 PM.

  4. #4
    Join Date
    Dec 2014
    Beans
    1,730

    Re: Python doesn't write when log files are rotated

    Why reinvent the wheel ? Python logging has a RotatingFileHandler. An example can be found here.

    Holger

  5. #5
    Join Date
    Oct 2020
    Beans
    8

    Re: Python doesn't write when log files are rotated

    Thank you all, (everyone.)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •