Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: terminal output to object for math operations

  1. #1
    Join Date
    Jan 2013
    Beans
    48

    terminal output to object for math operations

    I parse files for certain lines I would like to perform math on. To do this now I pass the output into a ruby terminal or a to a .csv file for me to work without outside of the terminal session that I'm already in.

    For example I have 2 files which each contain a line documenting the amount of time it took for some software to complete it's job. I want to then make a ratio of the times so I can say "the second one took 3 times longer than the first".


    I find the entries with the following:

    Code:
    $ find . -name *.sum | xargs -I {} grep -iH -A5 analysis\ statistics {} | grep -i total
    ./solver/beast.sum- Total:             2315 seconds
    ./solver/vm.sum- Total:             6615 seconds
    From here I can parse out everything but the numbers or even make a .csv file...

    to just output numbers:
    Code:
    $ find . -name *.sum | xargs -I {} grep -i -A5 analysis\ statistics {} | grep -i total | sed 's/[^0-9]*//g'
    2315
    6615

    BUT how can I go from here to doing some simple math on those numbers? I tried crazyness like this:
    Code:
    $ find . -name *.sum | xargs -I {} grep -i -A5 analysis\ statistics {} | grep -i total | sed 's/[^0-9]*//g' | echo (( $1/$2 ))
    but I know I'm not passing my lines as variables. Any help or thoughts? I'm preferably looking for a simple one liner.

    Thank you!

  2. #2
    Join Date
    Apr 2012
    Beans
    5,814

    Re: terminal output to object for math operations

    You could read them into an array maybe?

    Code:
    $ while read -r key val units; do mydata+=( "$val" ); done < <(find . -name '*.sum' -exec grep -iA5 'analysis statistics' {} + | grep -i 'total:')
    Then
    Code:
    $ echo $(( ${mydata[1]}/${mydata[0]} ))
    2
    or if you need more precision
    Code:
    $ echo "scale=2; ${mydata[1]}/${mydata[0]}" | bc
    2.85
    If you want to get fancier, you could use an associative array, which would allow you to refer to the values using the filename as a key, e.g.

    Code:
    $ declare -A myAssocData
    $ while read -r key val units; do myAssocData[${key%-?otal:}]="$val"; done < <(find . -name '*.sum' -exec grep -iA5 'analysis statistics' {} + | grep -i 'total:')
    $ 
    $ echo "scale=2; ${myAssocData[./file2.sum]}/${myAssocData[./file1.sum]}" | bc
    2.85

  3. #3
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,430
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: terminal output to object for math operations

    assuming that after grep 'total' you get something like this:
    Code:
    $ echo $'Total: 2232 seconds \nTotal: 4565 seconds'
    Total: 2232 seconds 
    Total: 4565 seconds
    you can divide these numbers with awk
    Code:
    $ echo $'Total: 2232 seconds \nTotal: 4565 seconds' | awk 'x{print x/$2} !x{x=$2}'
    0.488938

    that said, oneliners are overrated. Outside of simplest of cases, having a well written, maintainable script or a shell function is much better.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  4. #4
    Join Date
    Jan 2013
    Beans
    48

    Re: terminal output to object for math operations

    steeldriver: Thank you for this. I really like the associate array with file name key. Lots of knowledge here for me to absorb! I appreciate it!
    vaphell: I really like the awk method, I was trying to do that but couldn't get it right. Thank you! I don't entirely understand your awk arguments. Mind quickly walking me through them? I've been playing with it now for 20 minutes and almost grasp it! The reason I'm looking for 1 liners here is for flexibility, I often don't know what I'm searching for and how to manipulate it until I see it live.
    Last edited by twinturbotom; October 9th, 2013 at 12:16 PM.

  5. #5
    Join Date
    Sep 2006
    Beans
    7,797
    Distro
    Lubuntu Development Release

    spreadsheet

    An alternate method, espcially if you want more complex calculations, is to use a script to parse the data into tab-delimited text and then import it into a spreadsheet like LibreOffice Calc. All that's needed is that the columns of data are separated by tabs and that the rows by new lines and the resulting file gets the extension .csv After the import it is easy to work on the data and make any calculations you decide on, both now and later. Using that method, it would be fairly easy to calculate standard deviation, standard error, and many other things.

  6. #6
    Join Date
    Jan 2013
    Beans
    48

    Re: spreadsheet

    Lars, that's basically what I'm doing now but once in a while I want to make a few quick comparisons and figured a savvier command line ninja would know the way! Thanks

  7. #7
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,430
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: terminal output to object for math operations

    this awk part works like this: for every line of input awk goes through all blocks of code optionally guarded by a condition.
    Awk treats 0 and null value as false so if you put uninitialized variable as a condition it will skip the block until a non-zero value is set.
    assuming format of 'total: <number> seconds' $2 is what you want

    Code:
    line 1:  x{print x/$2}  # skip because uninitialized x -> false
            !x{x=$2}        # execute because !x -> true, { save 2nd field of the current line as x }
    line 2:  x{print x/$2}  # execute because x!=null/0 -> true { print x/(2nd field of the current line) }
            !x{x=$2}        # skip because !x -> false
    Last edited by Vaphell; October 9th, 2013 at 03:19 PM.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  8. #8
    Join Date
    Jan 2013
    Beans
    48

    Re: terminal output to object for math operations

    Vaphell - Thank you! I almost grasp this; very helpful. I'll play with it and awk a bit more.

    Awk treats 0 and null value as false so if you put uninitialized variable as a condition it will skip the block until a non-zero value is set.
    Great stuff there!

    Looks like you
    LINE 1:
    x{print x/$2} - skip this block and move to the next for line 1 as your calling for an x that hasn't been defined
    !x{x=$2} - save x as the 2nd field of the current line, which is still the first line

    LINE 2:
    x{print x/$2} - first block can be evaluated as x/$2 exists
    !x{x=$2} - I don't get why this doesn't just redefine x as the current $2. I need to play with that more. Is it the exclamation or can I not redefine the variable?

  9. #9
    Join Date
    Sep 2006
    Beans
    7,797
    Distro
    Lubuntu Development Release

    Re: terminal output to object for math operations

    The first part of an awk statement is always a pattern to be evaluated. You can think of the above two-statement awk script like this pseudo-code:

    Code:
         if ( $x ) { print $x/$2 }; 
         if ( ! $x ) { $x=$2 };
    The part before the braces { } can be any formula or regex pattern. awk reads the input one line at a time, but the variables it has internally stay set while the whole file is processed.
    Last edited by Lars Noodén; October 10th, 2013 at 03:01 PM.

  10. #10
    Join Date
    Jan 2013
    Beans
    48

    Re: terminal output to object for math operations

    LARS, this is a very helpful explanation. Thank you!

Page 1 of 2 12 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •