Page 1 of 2 12 LastLast
Results 1 to 10 of 16

Thread: grep wont search multiple patterns sepperated by a newline

  1. #1
    Join Date
    Apr 2020
    Location
    where the work takes me
    Beans
    237
    Distro
    Ubuntu 20.04 Focal Fossa

    grep wont search multiple patterns sepperated by a newline

    Hello all,

    This ones been annoying me for a while. grep is supposed to accept a list of patterns to search for with each separated by a newline, so I've tried the following syntax(s)...
    Code:
    grep [PATTERN1\nPATTERN2\nPATTERN3] [INPUTFILE.TXT]
    grep ["PATTERN1\nPATTERN2\nPATTERN3"] [INPUTFILE.TXT]
    grep ["PATTERN1"\n"PATTERN2"\n"PATTERN3"] [INPUTFILE.TXT]
    ...plus various other syntax(s) but none of them seem to work, I can only search for multiple strings using the '-E' option, as in...
    Code:
    grep -E "PATTERN1|PATTERN2" [INPUTFILE.TXT]
    ...can anyone guide me here?

  2. #2
    Join Date
    Dec 2014
    Beans
    1,729

    Re: grep wont search multiple patterns sepperated by a newline

    That only works for "grep -F", which will search for fixed strings (so the 'patterns' are not regular expressions). Example:
    Code:
    ls -l|grep -F 'lrwx
    public
    Bilder'
    which - if executed in my $HOME - would list one symbolic link, my public_html directory and my pictures directory.

    You can also use the option '-f file_full_of_patterns' where file _full_of_patterns is a text file with multiple patterns one per line.

    Holger

  3. #3
    Join Date
    Nov 2007
    Location
    London, England
    Beans
    7,201
    Distro
    Xubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    The top answer here might be of help: https://stackoverflow.com/questions/...-search-needed
    In brief, use grep -zo to convert newlines to nulls and only output the found string.

  4. #4
    Join Date
    Apr 2020
    Location
    where the work takes me
    Beans
    237
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    hmmm very annoying, my grep doesn't seem to respond to this!
    Code:
    perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$ ls|grep -F 'owl\n
    > alpha\n
    > grep'
    grepfile
    
    perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$ ls|grep -E 'owl|alpha|grep'
    alphazulu
    grepfile
    theowlandthepussycat
    perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$
    As you can see, when I use 'grep -F' followed by a newline separated list, it will only return matches for the final item in the list, whereas when I use 'grep -E' it will return matches for all...

  5. #5
    Join Date
    Apr 2020
    Location
    where the work takes me
    Beans
    237
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    Thanks, I've just given it a go but still grep will only print the strings which match the final line. I just find it really weird because the info pages literally state this is a feature of grep...
    grep [OPTION...] [PATTERNS] [FILE...]

    There can be zero or more OPTION arguments, and zero or more FILE
    arguments. The PATTERNS argument contains one or more patterns
    separated by newlines, and is omitted when patterns are given via the
    ‘-e PATTERNS’ or ‘-f FILE’ options. Typically PATTERNS should be quoted
    when ‘grep’ is used in a shell command.

    Since newline is also a
    separator for the list of patterns, there is no way to match newline
    characters in a text.

  6. #6
    Join Date
    Nov 2007
    Location
    London, England
    Beans
    7,201
    Distro
    Xubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    Grep normally works on single lines. You will never find a \n with grep because it splits the input into lines before searching. Unless you use -z to convert \n to null.

  7. #7
    Join Date
    Apr 2020
    Location
    where the work takes me
    Beans
    237
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    sorry there's a misunderstanding, I'm not trying to find newlines in the input data, but rather trying to specify a list of patterns to be searched for with each pattern in the list separated by a new line, which is what the grep documentation seems to suggest is the done thing.

  8. #8
    Join Date
    Dec 2014
    Beans
    1,729

    Re: grep wont search multiple patterns sepperated by a newline

    I had previously just looked at the man page (which doesn't mention this way of specifying multiple patterns except for use in fixed string mode, though it mentions giving multiple -e PATTERN options). You're right, the info documentation mentions this. So I tried
    Code:
    ls -l|grep -E '^lrwx.*
    \.db$
    w{3}-data'
    and got the expected results
    Code:
    -rw-r--r--   1 holgeh holgeh   145408 Sep 16  2017 ereader.db
    lrwxrwxrwx   1 holgeh holgeh       38 Nov  8  2019 PlayOnLinux's virtual drives -> /home/holgeh/.PlayOnLinux//wineprefix/
    drwxrwxr-x  19 holgeh www-data   4096 Jul 10 15:19 public_html
    The one thing I did differently from you was that I didn't put in '\n'. To grep '\n' doesn't have any special meaning, it would search for a literal 'n'.

    Holger
    Last edited by Holger_Gehrke; January 2nd, 2021 at 01:18 AM.

  9. #9
    Join Date
    Nov 2007
    Location
    London, England
    Beans
    7,201
    Distro
    Xubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    You are right, I misunderstood. Try this:
    Code:
    grep 'PATTERN1
    PATTERN2
    PATTERN3' INPUTFILE.TXT
    or this which places newlines into the search string (note the dollar in front to fix up interpreting \n by bash)
    Code:
    grep $'PATTERN1\nPATTERN2\nPATTERN3' INPUTFILE.TXT
    Last edited by The Cog; January 2nd, 2021 at 01:34 AM.

  10. #10
    Join Date
    Apr 2020
    Location
    where the work takes me
    Beans
    237
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: grep wont search multiple patterns sepperated by a newline

    Quote Originally Posted by Holger_Gehrke View Post
    I had previously just looked at the man page (which doesn't mention this way of specifying multiple patterns except for use in fixed string mode, though it mentions giving multiple -e PATTERN options). You're right, the info documentation mentions this. So I tried
    Code:
    ls -l|grep -E '^lrwx.*
    \.db$
    w{3}-data'
    and got the expected results
    Code:
    -rw-r--r--   1 holgeh holgeh   145408 Sep 16  2017 ereader.db
    lrwxrwxrwx   1 holgeh holgeh       38 Nov  8  2019 PlayOnLinux's virtual drives -> /home/holgeh/.PlayOnLinux//wineprefix/
    drwxrwxr-x  19 holgeh www-data   4096 Jul 10 15:19 public_html
    The one thing I did differently from you was that I didn't put in '\n'. To grep '\n' doesn't have any special meaning, it would search for a literal 'n'.
    I still can't get this to work but it's probably my syntax. I can't see any '\' at the end of your first line, mine looks like this...
    Code:
    perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$  grep -i 'owl\
    > *****-cat' theowlandthepussycat
    I know something is wrong because I cant tab to complete the name of the input file also rather amused to see I've been censored! (the asterisks aren't actually part of my syntax)

    Quote Originally Posted by The Cog View Post
    You are right, I misunderstood. Try this:
    Code:
    grep 'PATTERN1
    PATTERN2
    PATTERN3' INPUTFILE.TXT
    or this which places newlines into the search string (note the dollar in front to fix up interpreting \n by bash)
    Code:
    grep $'PATTERN1\nPATTERN2\nPATTERN3' INPUTFILE.TXT

    This actually works! it was the '$' that I was missing. I thought that wrapping a string in quotes was sufficient to protect the meta-characters in it from being expanded by the shell? why is the '$' necessary? (or does the '$' ensure that '\n' is expanded by the shell??)
    Last edited by jcdenton1995; January 3rd, 2021 at 02:09 AM. Reason: added itallics

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •