Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: String operations

  1. #1
    Join Date
    Nov 2008
    Beans
    227

    String operations

    Hi.

    I have a bunch of files, where I would like to truncate the last 2 bytes (0d 0a) and replace them by 0a.
    It's basically a string operation, but I didn't find any information on how to do that.

    It should be something like ...

    newfile = (oldfile,len(oldfile)-2) + "0a"

    How shall I do that?

    Thanks,
    Cu, geohei

  2. #2
    Join Date
    Sep 2010
    Beans
    898

    Re: String operations

    There are some packages in the repositories that make this easier:
    dos2unix
    tofrodos

  3. #3
    Join Date
    Nov 2007
    Location
    London, England
    Beans
    6,327
    Distro
    Xubuntu Development Release

    Re: String operations

    Don't know how to do it on only the last line of the file off the top of my head, although I 'm pretty sure you can use sed to only modify the last line.
    But there is a utility called dos2unix that will convert all line endings (0d0a to 0a) in a file.
    Code:
    sudo apt-get install dos2unix
    man dos2unix

  4. #4
    Join Date
    Feb 2008
    Location
    Oblivion
    Beans
    Hidden!
    Distro
    Xubuntu

  5. #5
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: String operations

    Code:
    sed 's/\r$//'
    Don't know how to do it on only the last line of the file off the top of my head, although I 'm pretty sure you can use sed to only modify the last line.
    Code:
    sed '$s/\r$//'
    -i switch will change files in place
    Last edited by Vaphell; February 11th, 2014 at 01:30 PM.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  6. #6
    Join Date
    Nov 2008
    Beans
    227

    Re: String operations

    Code:
    root@boxy:/media/raid-main/test# sed '$s/\r$/' < aaa.md5 > test.md5
    sed: -e expression #1, char 7: unterminated `s' command
    root@boxy:/media/raid-main/test# sed 's/\r$/' < aaa.md5 > test.md5
    sed: -e expression #1, char 6: unterminated `s' command
    Is there any possibility to to string operations more generalized, e.g. left($s,x), right($s,y), len($s), ...
    Cu, geohei

  7. #7
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: String operations

    my bad, i ate 1 / like a noob. Fixed.

    yes, there is
    Code:
    $ while read -r ln; do echo "$ln"; echo "${ln: 0:5}"; echo "${ln: -5:5}"; done <<< $'abcdefghijklmnopqrstuvwxyz\nABCDEFGHIJKLMNOPQRSTUVWXYZ' 
    abcdefghijklmnopqrstuvwxyz
    abcde
    vwxyz
    ABCDEFGHIJKLMNOPQRSTUVWXYZ
    ABCDE
    VWXYZ
    
    $ cut -c 10-15 <<< $'abcdefghijklmnopqrstuvwxyz\nABCDEFGHIJKLMNOPQRSTUVWXYZ' 
    jklmno
    JKLMNO
    
    $ sed -r 's/^(.{5}).*/\1/' <<< $'abcdefghijklmnopqrstuvwxyz\nABCDEFGHIJKLMNOPQRSTUVWXYZ' 
    abcde
    ABCDE
    $ sed -r 's/.*(.{5})$/\1/' <<< $'abcdefghijklmnopqrstuvwxyz\nABCDEFGHIJKLMNOPQRSTUVWXYZ' 
    vwxyz
    VWXYZ
    learn 2 bash parameter expansion, cut and regex.
    Last edited by Vaphell; February 11th, 2014 at 01:45 PM.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  8. #8
    Join Date
    Nov 2008
    Beans
    227

    Re: String operations

    Quote Originally Posted by Vaphell View Post
    Code:
    sed '$s/\r$//'
    I did a typo. Works fine now. Sorry!

    Something else (more bash related than sed).

    I'd like to create a list of all *.md5 files with above mentioned modification.
    Code:
    root@ganymede:~/# sed '$s/\r$//' < *.md5
    -bash: *.md5: ambiguous redirect
    root@ganymede:~/# sed '$s/\r$//' < test.md5"
    8137b27a378323da97ee0f27023de072 *test.mpg
    *.md5 doesn't work.
    A given filename works!

    What's wrong with my line?

    BTW ... I'll look later in detail into your reply from 11.02.2014. Thanks!
    Last edited by geohei; April 21st, 2014 at 12:13 PM.
    Cu, geohei

  9. #9
    Join Date
    Apr 2012
    Beans
    7,190

    Re: String operations

    You don't need an input redirect (<) at all - sed will read, concatenate, and process a list of named files (not just standard input)

    Code:
    sed '$s/\r$//' *.md5
    HOWEVER it will almost certainly not do what you want since $ really means 'last line of input' not 'last line of each file' i.e. it will only make the substitution on the last line of the last file in the *.md5 glob expansion. Probably what you need is a loop:

    Code:
    for file in *.md5; do '$s/\r$//' "$file"; done
    EDIT: just realized you can use the -s ('separate') switch to make sed interpret $ to mean 'last line of each file'

    Code:
    sed -s '$s/\r$//' *.md5
    The loop is probably safer if there are MANY (thousands of) files because of limits on max argument length.
    Last edited by steeldriver; April 21st, 2014 at 01:54 PM.

  10. #10
    Join Date
    Nov 2008
    Beans
    227

    Re: String operations

    Quote Originally Posted by steeldriver View Post
    Code:
    for file in *.md5; do sed '$s/\r$//' "$file"; done
    sed -s '$s/\r$//' *.md5
    The loop is probably safer if there are MANY (thousands of) files because of limits on max argument length.
    Wow ... I'm impressed! Both methods work perfectly!
    And you were right ... the first syntax only deals with the 'last line of input'.
    But what do you mean with "... of limits on max argument length"?
    Many thanks so far!

    Next question. I'd like to ass the result to md5sum. This doesn't work. I don't understand the error message.
    Code:
    root@ganymede:~# for file in *.md5; do sed '$s/\r$//' "$file"; done -exec md5sum -c {} \;
    -bash: syntax error near unexpected token `-exec'
    Using an intermediate file works.
    Code:
    root@ganymede:~# for file in *.md5; do sed '$s/\r$//' "$file"; done > md5.md5
    root@ganymede:~# md5sum -c md5.md5
    ...
    ... later ...

    This works!
    Code:
    root@ganymede:~# for file in *.md5; do sed '$s/\r$//' "$file"; done | md5sum -c
    ...
    Thanks for the help!
    Last edited by geohei; April 22nd, 2014 at 03:36 AM.
    Cu, geohei

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •