Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: Bash script help - appending lines to files

  1. #1
    Join Date
    Apr 2007
    Location
    chicago
    Beans
    Hidden!
    Distro
    Ubuntu Development Release

    Bash script help - appending lines to files

    Background: I have a bunch of filenames named username.sub in single letter directories under script_testing (first letter of username is the folder name). For every username.sub, I need to check if the line user.$username.contacts\t exists and, if not, append the line. I am having troubles with my code thus far. This should be a rather simple thing, I think. I am currently testing with 4 username.sub files in subdirectory "g." I have gone through many changes, this is how the script currently stands:

    Code:
    Code:
    #!/bin/bash
    Path_to_files=$Path_to_files$( find $HOME/script_testing/ -type d -printf ":%p" )
    sub_files=$Path_to_files$( find . -type f *.sub)
    FULLNAME="${sub_files##*/}"
    username="${FULLNAME%%.*}"
    
    if grep user.$username.Contacts $username.sub; then
        echo "Contacts already subscribed"
    else
        echo "subscribing to contacts"
        echo -e "user.$username.Contacts\t" >> $username.sub
    fi

    Debug output:
    Code:
    ++ find /home/user/script_testing/ -type d -printf :%p
    + Path_to_files=:/home/user/script_testing/:/home/user/script_testing/g
    ++ find . -type f g.sub '^*.sub'
    find: paths must precede expression: g.sub
    Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]
    + sub_files=:/home/user/script_testing/:/home/user/script_testing/g
    + FULLNAME=g
    + username=g
    + grep user.g.Contacts g.sub
    user.g.Contacts	
    + echo 'Contacts already subscribed'
    Contacts already subscribed
    Questions:
    1. Why is it using the folder name ("g" in this instance) instead of the username?
    2. Why is it not appending to the file? If I hardcode everything, it works correctly for one file. I think I am missing something simple.
    3. If I were to get this working, how would I get it to do so for multiple subfolders? To clarify, there are 26 subfolders (one for each letter of the alphabet) containing multiple username.sub files (along with other files)
    Last edited by gforster; June 12th, 2013 at 02:18 PM. Reason: solved - thank you!

  2. #2
    prodigy_ is offline May the Ubuntu Be With You!
    Join Date
    Mar 2008
    Beans
    1,219

    Re: Bash script help - appending lines to files

    Code:
    #!/bin/bash
    
    root_dir="/home/script_testing"
    
    find $root_dir -type f -regex "${root_dir}/\([a-zA-Z]\)/\1.*\.sub" -print0 | \
    	while read -r -d $'\0' full_name; do
    		file_name=$(basename "$full_name")
    		user_name="${file_name%%.*}"
    		if grep "user.${user_name}.Contacts" "$full_name"; then
    			echo "Contacts already subscribed"
    		else
    			echo "subscribing to contacts"
    			echo -e "user.${user_name}.Contacts\t" >> "$full_name"
    		fi
    	done
    Last edited by prodigy_; June 11th, 2013 at 09:42 PM.

  3. #3
    Join Date
    May 2009
    Location
    Courtenay, BC, Canada
    Beans
    1,661

    Re: Bash script help - appending lines to files

    Quote Originally Posted by gforster View Post
    Questions:
    1. Why is it using the folder name ("g" in this instance) instead of the username?
    2. Why is it not appending to the file? If I hardcode everything, it works correctly for one file. I think I am missing something simple.
    3. If I were to get this working, how would I get it to do so for multiple subfolders? To clarify, there are 26 subfolders (one for each letter of the alphabet) containing multiple username.sub files (along with other files)
    1.because you're using find and /home/user/script_testing/g is the last result. specify
    Code:
    -maxdepth
    to prevent find from going too deep, however, it would be much simpler to just
    Code:
    #!/bin/bash
    p=$HOME/script_testing
    for d in $p/*; do
     # if this is a folder
     if [ -d "$d" ]; then
      # check its contents
      ls "$d"| while read -r f; do
       # if this is a file
       if [ -f "$f" ]; then
        # define variables
        F="${f##*/}"
        u="${F%%.*}"
        yes=$(grep "user.$u.Contacts" "$f")
        # if our file doesn't contain our string
        if [ -z "$yes"]; then
         # add it
         echo -e "user.$u.Contacts\t" >> "$f"
        fi
       fi
      done
     fi
    done
    might want to double check it (I'm on my tablet) but it should be fine
    Last edited by HiImTye; June 11th, 2013 at 09:23 PM.

  4. #4
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,431
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Bash script help - appending lines to files

    why would you want to use unreliable ls "$d" | while read f when you can use the same trick as with d - for f in "$d"/* ?
    how about for f in "$p"/*/* ?
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  5. #5
    Join Date
    May 2009
    Location
    Courtenay, BC, Canada
    Beans
    1,661

    Re: Bash script help - appending lines to files

    considered switching it to the second but didn't feel like editing it beyond correcting tablet typing issues

    why is
    Code:
    ls | while read
    unreliable? I googled but no beans

  6. #6
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,431
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Bash script help - appending lines to files

    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  7. #7
    Join Date
    May 2009
    Location
    Courtenay, BC, Canada
    Beans
    1,661

    Re: Bash script help - appending lines to files

    so it's only an issue if you use weird filenames? good to keep in mind, but given the formatting implied in post#1 it isn't an issue

  8. #8
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,431
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Bash script help - appending lines to files

    ugly non-kosher shortcuts reinforce bad, potentially dangerous habits and not always you have processed filenames under full control. Let's you run some malformed script that creates a lot of garbage files everywhere due to a simple typo. How would you clean up these files with non-printable chars all over the place with ls that is unable to reproduce their names correctly?

    When it comes to filenames the order would be as follows: 100% rock solid native globs, then 100% rock solid find -print0 + while read -d $'\0', then dirty hacks if there is absolutely no other choice. If you do the right thing right off the bat, you don't have to fix anything when it comes to more complicated inputs and it's not like you need to write an additional page of code to get proper solution.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  9. #9
    Join Date
    May 2009
    Location
    Courtenay, BC, Canada
    Beans
    1,661

    Re: Bash script help - appending lines to files

    I don't know how prevalent newlines are in filenames, but it stands to reason that if they were as common as you make them out to be, everyone would have a filesystem completely destroyed by
    Code:
    ls | while read
    sounds like a linux boogeyman to me

  10. #10
    Join Date
    Sep 2006
    Beans
    7,808
    Distro
    Lubuntu Development Release

    Re: Bash script help - appending lines to files

    I'd defer to Vaphell's experience there. In general it's always best to learn the right way when starting something new so that there is less to unlearn along the way. It's really hard to unlearn bad habits like relying on ls once they get set. So it's less work not to go down that path in the first place.

    If you need anecdotal support, I've gotten stung more than a few times in the distant past by (mis-)using ls in place of globbing. Mostly it was caused by spaces in the file names and paths.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •