Results 1 to 8 of 8

Thread: Retrieve output from terminal while command is running (Python)

  1. #1
    Join Date
    Feb 2010
    Location
    Colorado
    Beans
    6
    Distro
    Ubuntu 9.10 Karmic Koala

    Retrieve output from terminal while command is running (Python)

    I'm trying to retrieve the output of a terminal command sent by python. I have used subprocess.Popen(command, stdout=subprocess.PIPE ).communicate()[0] in the past which gives me the output but only after the process is finished. I need to find a way to get the output as the command runs. I have also tried subprocess.Popen(command, stdout=subprocess.PIPE ).stdout.readline() but it stops working after approx. 50 lines. I think that this command uses a stdout buffer which is being filled.

    Thanks in advance for your suggestions.
    jrhughes

  2. #2
    Join Date
    Aug 2007
    Beans
    949

    Re: Retrieve output from terminal while command is running (Python)

    If it's possible (i.e. if you have the source of the other program and can recompile it) you should modify it to flush its buffers after every time it writes data, so you will be able to receive it. As for buffering on the Python end, I really don't know a way to solve this.

  3. #3
    Join Date
    Feb 2010
    Location
    Colorado
    Beans
    6
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: Retrieve output from terminal while command is running (Python)

    To be more specific the command that I'm trying to get the output for is mencoder so I don't think there is anything that I can do on the command (program) side. However I know there most be a way to get the information I want since other programs (ex. acidrip) can get it.

    thanks
    jrhughes

  4. #4
    Join Date
    Aug 2007
    Beans
    949

    Re: Retrieve output from terminal while command is running (Python)

    This is odd. I do know that by default stdout is line buffered, so if mencoder uses some kind of fancy output scheme (like ffmpeg does) then this might be messing with the buffering scheme on stdout which is in turn messing with Python. As from what I can tell running tests with various other UNIX programs, there doesn't seem to be a limit of output lines that can be collected by doing:

    Code:
    $ python
    >>> import subprocess
    >>> p = subprocess.Popen(['command'], stdout=subprocess.PIPE)
    >>> while p.returncode is None:
    ...    print p.stdout.readline()
    To solve the problem, there is probably an option for turning on script friendly output with mencoder. Have you checked already?

  5. #5
    Join Date
    Feb 2010
    Location
    Colorado
    Beans
    6
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: Retrieve output from terminal while command is running (Python)

    I have followed your excellent suggestion and reduced the output to only the information that I need. As a result I discovered that the output for the status line is printed over top of the previous line. Therefore, there are no endlines or new lines after the first status line which maybe the reason ___.readline() fails to work.

    I then quickly tried ___.read(1) which returns only one char at a time. A loop calling the read fcn allowed me to get all the output that I needed. However, I still need to figure out how to break the resulting string up into proper lines of output since there are no endlines. I will post again this evening once I figured it out to let you know the result.

  6. #6
    Join Date
    Feb 2010
    Location
    Colorado
    Beans
    6
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: Retrieve output from terminal while command is running (Python)

    It turns out I only need to add universal_newlines=True. It seems that mencoder uses '\r' (carriage return) instead of '/n' (endline). The universal_newlines=True causes file objects stdout to be opened as text files and enables the readline fcn to use '\r', '\n', and '\r\n'.
    see: http://docs.python.org/library/subpr...ule-subprocess
    and http://docs.python.org/library/io.ht...OBase.readline

    Code:
    s=subprocess.Popen(command, stdout=subprocess.PIPE, universal_newlines=True )
    while 1:
          print s.stdout.readline()[:-1]
          if s.stdout.readline()[:-1]=='':
                break
    Special thanks to soltanis. Your suggestion to have mencoder produce script friendly output was the only reason I discovered the '\r'.

  7. #7
    Join Date
    May 2008
    Location
    UK
    Beans
    1,448
    Distro
    Ubuntu 8.04 Hardy Heron

    Re: Retrieve output from terminal while command is running (Python)

    Your script fragment will skip every other line - is that intended ?
    Tony - Happy to try to help.
    Unless otherwise stated - all code posted by me is untested. Remember to Mark the Thread as Solved.
    Ubuntu user number # 24044 Projects : TimeWarp - on the fly Backups

  8. #8
    Join Date
    Feb 2010
    Location
    Colorado
    Beans
    6
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: Retrieve output from terminal while command is running (Python)

    No that was not intentional I just didn't notice. However, the code I'm actually using forces the readline to get only every fifth line so it does not matter to me. Out of curiosity, do you know what is causing that?

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •