grep wont search multiple patterns sepperated by a newline

[jcdenton1995]

Hello all,

This ones been annoying me for a while. grep is supposed to accept a list of patterns to search for with each separated by a newline, so I've tried the following syntax(s)...

grep [PATTERN1\nPATTERN2\nPATTERN3] [INPUTFILE.TXT]
grep ["PATTERN1\nPATTERN2\nPATTERN3"] [INPUTFILE.TXT]
grep ["PATTERN1"\n"PATTERN2"\n"PATTERN3"] [INPUTFILE.TXT]

...plus various other syntax(s) but none of them seem to work, I can only search for multiple strings using the '-E' option, as in...

grep -E "PATTERN1|PATTERN2" [INPUTFILE.TXT]

...can anyone guide me here?

[Holger_Gehrke]

That only works for "grep -F", which will search for fixed strings (so the 'patterns' are not regular expressions). Example:

ls -l|grep -F 'lrwx
public
Bilder'

which - if executed in my $HOME - would list one symbolic link, my public_html directory and my pictures directory.

You can also use the option '-f file_full_of_patterns' where file _full_of_patterns is a text file with multiple patterns one per line.

Holger

[The Cog]

The top answer here might be of help: https://stackoverflow.com/questions/...-search-needed
In brief, use grep -zo to convert newlines to nulls and only output the found string.

[jcdenton1995]

hmmm very annoying, my grep doesn't seem to respond to this!

perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$ ls|grep -F 'owl\n
> alpha\n
> grep'
grepfile

perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$ ls|grep -E 'owl|alpha|grep'
alphazulu
grepfile
theowlandthepussycat
perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$

As you can see, when I use 'grep -F' followed by a newline separated list, it will only return matches for the final item in the list, whereas when I use 'grep -E' it will return matches for all...

[jcdenton1995]

Thanks, I've just given it a go but still grep will only print the strings which match the final line. I just find it really weird because the info pages literally state this is a feature of grep...

grep [OPTION...] [PATTERNS] [FILE...]

There can be zero or more OPTION arguments, and zero or more FILE
arguments. The PATTERNS argument contains one or more patterns
separated by newlines, and is omitted when patterns are given via the
‘-e PATTERNS’ or ‘-f FILE’ options. Typically PATTERNS should be quoted
when ‘grep’ is used in a shell command.

Since newline is also a
separator for the list of patterns, there is no way to match newline
characters in a text.

[The Cog]

Grep normally works on single lines. You will never find a \n with grep because it splits the input into lines before searching. Unless you use -z to convert \n to null.

[jcdenton1995]

sorry there's a misunderstanding, I'm not trying to find newlines in the input data, but rather trying to specify a list of patterns to be searched for with each pattern in the list separated by a new line, which is what the grep documentation seems to suggest is the done thing.

[Holger_Gehrke]

I had previously just looked at the man page (which doesn't mention this way of specifying multiple patterns except for use in fixed string mode, though it mentions giving multiple -e PATTERN options). You're right, the info documentation mentions this. So I tried

ls -l|grep -E '^lrwx.*
\.db$
w{3}-data'

and got the expected results

-rw-r--r--   1 holgeh holgeh   145408 Sep 16  2017 ereader.db
lrwxrwxrwx   1 holgeh holgeh       38 Nov  8  2019 PlayOnLinux's virtual drives -> /home/holgeh/.PlayOnLinux//wineprefix/
drwxrwxr-x  19 holgeh www-data   4096 Jul 10 15:19 public_html

The one thing I did differently from you was that I didn't put in '\n'. To grep '\n' doesn't have any special meaning, it would search for a literal 'n'.

Holger

[The Cog]

You are right, I misunderstood. Try this:

grep 'PATTERN1
PATTERN2
PATTERN3' INPUTFILE.TXT

or this which places newlines into the search string (note the dollar in front to fix up interpreting \n by bash)

grep $'PATTERN1\nPATTERN2\nPATTERN3' INPUTFILE.TXT

[jcdenton1995]

I had previously just looked at the man page (which doesn't mention this way of specifying multiple patterns except for use in fixed string mode, though it mentions giving multiple -e PATTERN options). You're right, the info documentation mentions this. So I tried

ls -l|grep -E '^lrwx.*
\.db$
w{3}-data'

and got the expected results

-rw-r--r--   1 holgeh holgeh   145408 Sep 16  2017 ereader.db
lrwxrwxrwx   1 holgeh holgeh       38 Nov  8  2019 PlayOnLinux's virtual drives -> /home/holgeh/.PlayOnLinux//wineprefix/
drwxrwxr-x  19 holgeh www-data   4096 Jul 10 15:19 public_html

The one thing I did differently from you was that I didn't put in '\n'. To grep '\n' doesn't have any special meaning, it would search for a literal 'n'.

I still can't get this to work but it's probably my syntax. I can't see any '\' at the end of your first line, mine looks like this...

perrywinklesnoop@perrywinklesnoop-System-Product-Name:~$  grep -i 'owl\
> *****-cat' theowlandthepussycat

I know something is wrong because I cant tab to complete the name of the input file also rather amused to see I've been censored! (the asterisks aren't actually part of my syntax)

You are right, I misunderstood. Try this:

grep 'PATTERN1
PATTERN2
PATTERN3' INPUTFILE.TXT

or this which places newlines into the search string (note the dollar in front to fix up interpreting \n by bash)

grep $'PATTERN1\nPATTERN2\nPATTERN3' INPUTFILE.TXT

This actually works! it was the '$' that I was missing. I thought that wrapping a string in quotes was sufficient to protect the meta-characters in it from being expanded by the shell? why is the '$' necessary? (or does the '$' ensure that '\n' is expanded by the shell??)