Results 1 to 3 of 3

Thread: Removing duplicate words from file?

  1. #1
    Join Date
    Mar 2008
    Beans
    47

    Removing duplicate words from file?

    Hello,
    I want to merge file1 with file2, remove and duplicate words and then make file3 with the contents of file1 and file2 without the duplicates.

    ie.

    File1:
    A
    B
    C
    D

    File2:
    A
    B
    E
    F

    File3:
    A
    B
    C
    D
    E
    F

    I know you can merge files with the "paste" command, however dont know how to remove the duplicates.

    Any help apretiated, thanks in advance.

  2. #2
    Join Date
    May 2008
    Beans
    80

    Re: Removing duplicate words from file?

    Try this:

    Code:
    cat file1 file2 | sort | uniq
    This will print to the screen. When you're satisfied with the output, you can run this to send the output to file3:

    Code:
    cat file1 file2 | sort | uniq > file3

  3. #3
    Join Date
    May 2008
    Location
    Eugene, OR, USA
    Beans
    435

    Re: Removing duplicate words from file?

    Or just:

    Code:
    sort -u file1 file2 >file3
    Hal Pomeranz, Deer Run Associates
    [[ Various Linux/Unix related documents ]]
    [[ Command-Line Kung Fu blog ]]

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •