Results 1 to 6 of 6

Thread: Batch Change ^M to New Line

  1. #1
    Join Date
    Apr 2008
    Beans
    76

    Batch Change ^M to New Line

    Hello,

    This feels like a very basic question but I can't seem to get it to work at all... I have a directory full of files from DOS, *NIX, MAC, whatever else. I want to go through each file, look for the ^M (control-m) character and replace it with a suitable line feed. I wrote the following batch file, but keep getting errors:

    Code:
    #/bin/ksh
    for fileName in $(ls *.txt): do
    cat $fileName | sed s/^M//g >tmp && mv tmp $fileName
    done
    Code:
    ./rename.sh: line 3: syntax error near unexpected token `cat'
    Seems simple enough... What am I missing? Thanks!

  2. #2
    Join Date
    May 2007
    Location
    STL, MO
    Beans
    1,283
    Distro
    Kubuntu Karmic Koala (testing)

    Re: Batch Change ^M to New Line

    ^ is a special character. It means "lock to beginning of line." Try this instead
    Code:
    sed s/\^M//g $filename > $filename
    Edit: That will destructively overwrite the original file. You may want to try outputting to another file to make sure it works properly first.

  3. #3
    Join Date
    Aug 2008
    Location
    WA
    Beans
    2,186
    Distro
    Ubuntu

    Re: Batch Change ^M to New Line

    Code:
    sudo apt-get install tofrodos
    dos2unix *

  4. #4
    Join Date
    Apr 2008
    Beans
    76

    Re: Batch Change ^M to New Line

    Quote Originally Posted by jbrown96 View Post
    ^ is a special character. It means "lock to beginning of line." Try this instead
    Code:
    sed s/\^M//g $filename > $filename
    Edit: That will destructively overwrite the original file. You may want to try outputting to another file to make sure it works properly first.
    Now I get the following error:

    Code:
    ./rename.sh: line 3: syntax error near unexpected token `sed'
    ./rename.sh: line 3: `sed s/\^M//g $fileName > $fileName'
    File looks like this:

    Code:
    #/bin/ksh
    for fileName in $(ls *.txt): do
      sed s/\^M//g $fileName > $fileName
    done
    I'm testing this in a subdirectory figuring a few attempts will destroy my files

    Thanks!

  5. #5
    Join Date
    May 2007
    Location
    STL, MO
    Beans
    1,283
    Distro
    Kubuntu Karmic Koala (testing)

    Re: Batch Change ^M to New Line

    Quote Originally Posted by scambro View Post
    Now I get the following error:

    Code:
    ./rename.sh: line 3: syntax error near unexpected token `sed'
    ./rename.sh: line 3: `sed s/\^M//g $fileName > $fileName'
    File looks like this:

    Code:
    #/bin/ksh
    for fileName in $(ls *.txt): do
      sed s/\^M//g $fileName > $fileName
    done
    I'm testing this in a subdirectory figuring a few attempts will destroy my files

    Thanks!
    I just created a sample file, and I'm not getting any syntax errors; however I did have to put single quotes around the regex
    Code:
    sed 's/\^M//g' $fileName > $fileName
    This won't work if ^M is actually a single character instead of ^ and M. That still wouldn't cause syntax errors though; you just wouldn't get any matches.
    Here are my two lame files
    ^M lkdsfjlkj lkjdlfsj ^lkjlkj lkjalsdjf
    kljaldsjfl lsdkjflj lkjljfm^ ljsladjf ^M
    lkasdlfkj ^M
    and the output
    lkdsfjlkj lkjdlfsj ^lkjlkj lkjalsdjf
    kljaldsjfl lsdkjflj lkjljfm^ ljsladjf
    lkasdlfkj
    You may want to check out an ascii table to see if ^M is two characters or one special one.

  6. #6
    Join Date
    Apr 2008
    Beans
    76

    Re: Batch Change ^M to New Line

    I think I got it with the following (you were right though, ^M is not two characters, so I used the octal number to get it):

    Code:
    #/bin/ksh
    for fileName in $(ls *.txt)
    do
      cat $fileName | sed 's/\o015/\n/g' > tmp && mv tmp $fileName
    done
    This appears to work. The dos2unix wasn't working at all for me, I tried that a few times before figuring I'd just have to deal with all of these files with my own script.

    Thanks for the help!

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •