Page 1 of 2 12 LastLast
Results 1 to 10 of 19

Thread: batch photo management script

  1. #1
    Join Date
    Nov 2012
    Beans
    15

    Question batch photo management script

    I recently bought a nice homeserver that I am trying to customise to my needs. The problem I've got has to do with photos management.

    What I am trying to do is creating a script that daily checks for changes in the /mount/photos folder and subfolders for newly added pics and resize and move the resized copy into /mount/photos/resized.

    Useless to say that I am not 'big' in programming . I guess I am stuck because
    I cannot tackle the first item on the list. I checked FAQs and searched for similar queries in this forum but to no avail.

    Automatic Daily running script that does the following:


    • SCAN - /mount/photos sub-directories for photos (jpg) more recent than this script log file
    • RENAME - <folder_name>+<date>+<sequential>.jpg sequential=picking up from the last one
    • RESIZE - selected files [convert example.jpg -resize 1920 example.jpg]
    • MOVE - resized pictures into /mount/images/resized/2012
    • CREATE - log file with outcome of the script


    Any help is appreciated.

    Cheers
    mcapri

  2. #2
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: batch photo management script

    Code:
    #!/bin/bash
    
    src_dir=.                    # main dir (SCAN)
    dest_dir=../new_pics         # dir for resized stuff (RESIZE+MOVE)
    log_dir=.                    # 
    log=pics_$( date +%F ).log   # log name, not used yet
    year=$( date +%Y )           # not used yet (MOVE)
    
    # find latest log file
    read _ lastlog < <( find "$log_dir" -iname '*.log' -printf "%T@\t%f\n" | sort -nr )
    if [ "$lastlog" ]
    then
      log_ts=$( stat -c %y "$lastlog" )   # log timestamp for debugging
      echo "latest log file: $lastlog ($log_ts)"
      # prepare parameter list
      scan_options=( "-newer" "$lastlog" "-iname" "*.jpg" )
    else
      scan_options+=( "-iname" "*.jpg" )
    fi
    
    # print find options
    # for s in "${scan_options[@]}"; do echo "'$s'"; done
    
    # find files newer than the log file (all files if there is no .log file)
    files=() 
    while read -r -d $'\0' f
    do
      files+=( "$f" )
    done < <( find "$src_dir" "${scan_options[@]}" -print0 )
    
    # print files etc
    for f in "${files[@]}"
    do
      echo "file: $f   ($( stat -c %y "$f" )) > ($log_ts)"
    # test if in proper format
    #  [[ $f =~ .+-.+-[0-9]+.jpg ]] && echo proper format || echo rename
    done  #> "$log_dir/$log"
    that should find the files newer than the newest existing log file. Obviously variables at the top should be filled with proper values to point where they should.
    It doesn't produce log by itself, i skipped that part as it would make debugging annoying (every run would create newer log and i'd have to roll its timestamp back or touch all dummy files - pain in the back when you have to do this every 10 seconds)
    Create whatever .log file in proper location, use touch command to set it to some time, eg.
    Code:
    touch -d 'now -2 days' test.log
    and see if the list is ok.

    Could you elaborate more on the renaming part? What would the date format be exactly? And is the sequence global in the directory or maybe it is only related to subsets based on days?

    Code:
    global:                     day based:
    abc-(yesterday)-001.jpg     abc-(yesterday)-001.jpg
    abc-(yesterday)-002.jpg     abc-(yesterday)-002.jpg
    abc-(today)-003.jpg         abc-(today)-001.jpg
    abc-(today)-004.jpg         abc-(today)-002.jpg
    Last edited by Vaphell; November 7th, 2012 at 12:13 PM.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  3. #3
    Join Date
    Nov 2012
    Beans
    15

    Re: batch photo management script

    Thanks you for your help.

    Sorry not to have been more specific about the rename part. Looking back I should have removed the <date> bit. I normally include the year in the folder name to make it unique. So the script should just rename to <foldername>+<sequential>.jpg

    mcapri

  4. #4
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: batch photo management script

    so you throw some files in and expect them to be renamed to fit the rest of files in that dir?
    abc/abc-0001.jpg
    abc/abc-0002.jpg
    abc/some-junk.jpg -> abc-0003.jpg

    should the index width be fixed? 4digits or something?
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  5. #5
    Join Date
    Nov 2012
    Beans
    15

    Re: batch photo management script

    exactly!

    It makes sense to me to have all files in a photo directory (e.g: Lanzarote2011) having the name of the folder plus the sequential. Three digits are more than enough.

    Just out of curiosity, do you have any photo files naming standard that you use?

    Cheers

  6. #6
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: batch photo management script

    i don't collect photos, so no
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  7. #7
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: batch photo management script

    Code:
    #!/bin/bash
    
    src_dir=.                    # main dir (SCAN)
    resized=../resized          # RESIZE+MOVE
    log_dir=.                    # 
    log=pics_$( date +%F ).log   # log name, not used yet
    year=$( date +%Y )           # not used yet (MOVE)
    w=3                          # index width
    
    # exit if any of dirs is missing
    [ -d "$src_dir" ] || exit 1
    [ -d "$log_dir" ] || exit 1
    [ -d "$resized" ] && mkdir -p "$resized/$year" || exit 1
    
    # find latest log file
    read _ lastlog < <( find "$log_dir" -iname '*.log' -printf "%T@\t%f\n" | sort -nr )
    
    if [ "$lastlog" ]
    then
      log_ts=$( stat -c %y "$lastlog" )   # log timestamp for debugging
      echo "latest log file: $lastlog ($log_ts)"
      echo "new log file: $log_dir/$log"
      # prepare parameter list
      scan_options=( "-newer" "$lastlog" "-iname" "*.jpg" )
    else
      scan_options+=( "-iname" "*.jpg" )
    fi
    echo "-----------------------------------"
    # find files newer than the log file (all files if there is no .log file)
    files=() 
    while read -r -d $'\0' f
    do
      files+=( "$f" )
    done < <( find "$src_dir" "${scan_options[@]}" -print0 ) 
    
    echo "${#files[@]} new file(s) found!"
    
    # print files etc
    for f in "${files[@]}"
    do
    #  echo "*** file: $f ($( stat -c %y "$f" )) newer than $log_dir/$lastlog ($log_ts)"
      echo "*** file: $f"
      pdirpath="${f%/*}";
      pdir="${pdirpath##*/}";
    
      # find first unused fileXXX.jpg
      i=0
      while (( ++i ))
      do
        newfpath="$pdirpath/$pdir $( printf "%0${w}d" $i ).jpg"
        [ "$newfpath" = "$f" ] && break
        [ -f "$newfpath" ] || break
      done
    
      # rename if 
      [ "$newfpath" != "$f" ] && mv -v -- "$f" "$newfpath"
      # resize if the resized version doesn't exist
      resfpath=$resized/$year/${newfpath##*/}
      [ ! -f "$resfpath" ] && echo "resized: $resfpath" && convert "$newfpath" -resize 1920 "$resfpath"
      echo
    done >> "$log_dir/$log"

    i haven't used real jpg files so i simply echoed that convert line. Either way, test it hard on some dummy set of files and locations. I wouldn't want to be responsible for a heavy mess caused by improper mv calls or something.

    If there is no log file in provided location, all files are caught. If the log files (name based on date) exist, newest one is picked for comparison.
    Files that match the name pattern (in case it's possible to have good name on a file newer than the latest log file) are skipped.
    script is not smart in case there are gaps. If the name is in proper format but the first empty slot is before that index, file will be renamed.
    proper format 001.jpg
    proper format 002.jpg
    proper format 004.jpg -> proper format 003.jpg

    resized/$year currently is automatically calculated (YEAR of date='now'), but do you need that? i think it's nice but what should happen when you drop something into the 'NewYearsEve 2012' photo album in 2013?

    the naming convention - after some thinking i'd say that 'year(+month?)+name' format of directory names would be best to preserve chronology. Of course it all depends on what logic the format is supposed to represent.
    Last edited by Vaphell; November 8th, 2012 at 05:47 AM.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  8. #8
    Join Date
    Nov 2012
    Beans
    15

    Re: batch photo management script

    After many hiccups I was able to test the script above and it works a treat!

    Thank you.

    The test highlighted one thing I did not think about; is it possible to modify the script so it replicates the original photo folder structure into the /resized/$year folder.

    Photos > Folder1 > Photo1 > Photo2 > Photo3
    > Folder2 > PhotoA > PhotoB > PhotoC
    > Folder3

    Resized/2013/Folder1 > Photoresized1 > Photoresized2 > Photoresized3
    /Folder2 > PhotoresizedA > PhotoresizedB > PhotoresizedC
    /Folder3

    I honestly tried to work it out myself but I am New to this whole thing (with the capital 'N').

    Cheers

  9. #9
    Join Date
    Jul 2007
    Location
    Poland
    Beans
    4,499
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: batch photo management script

    Code:
    #!/bin/bash
    
    src_dir=.                    # main dir (SCAN)
    resized=../resized           # RESIZE+MOVE
    log_dir=.                    # log dir
    log=pics_$( date +%F ).log   # log name
    w=3                          # index width
    year=$( date +%Y )
    
    # exit if any of dirs is not available
    [ -d "$src_dir" ] || exit 1
    [ -d "$log_dir" ] || exit 1
    [ -d "$resized" ] && mkdir -p "$resized/$year" || exit 1
    
    # find latest log file
    read _ lastlog < <( find "$log_dir" -iname '*.log' -printf "%T@\t%f\n" | sort -nr )
    
    if [ "$lastlog" ]
    then
      log_ts=$( stat -c %y "$lastlog" )   # log timestamp for debugging
      echo "latest log file: $lastlog ($log_ts)"
      echo "current log file: $log_dir/$log"
      # prepare parameter list
      scan_options=( "-cnewer" "$lastlog" "-iname" "*.jpg" )
    else
      scan_options=( "-iname" "*.jpg" )
    fi
    echo "-----------------------------------"
    # find files newer than the log file (all files if there is no .log file)
    files=() 
    while read -r -d $'\0' f
    do
      files+=( "$f" )
    done < <( find "$src_dir" "${scan_options[@]}" -print0 ) 
    
    echo "${#files[@]} new file(s) found!"
    
    # print files etc
    for f in "${files[@]}"
    do
    #  echo "*** file: $f ($( stat -c %y "$f" )) newer than $log_dir/$lastlog ($log_ts)"
      echo "*** file: $f"
      fdir="${f%/*}";
      album="${fdir##*/}";
    
      # find first unused fileXXX.jpg
      i=0
      while (( ++i ))
      do
        fnew="$fdir/$album $( printf "%0${w}d" $i ).jpg"
        [ "$fnew" = "$f" ] && break
        [ -f "$fnew" ] || break
      done
    
      # rename if 
      [ "$fnew" != "$f" ] && mv -vi -- "$f" "$fnew"
      # resize if the resized version doesn't exist
      res_fnew="$resized/$year/${fnew##$src_dir/}"
      if [ ! -f "$res_fnew" ]
      then
        mkdir -p "${res_fnew%/*}"
        echo "resized: $res_fnew"
        convert "$fnew" -resize '1920x1920>' "$res_fnew"
      fi
      echo
    done | tee -a "$log_dir/$log"
    again, try on a set of dummy images first. I made some minor changes, i didn't like old variable names
    also, out of curiosity: what should happen when you put some image that scales to let's say 1920x1440? Do you consider it a valid output format?
    Last edited by Vaphell; April 11th, 2013 at 05:38 PM.
    if your question is answered, mark the thread as [SOLVED]. Thx.
    To post code or command output, use [code] tags.
    Check your bash script here // BashFAQ // BashPitfalls

  10. #10
    Join Date
    Aug 2011
    Location
    47°9′S 126°43W
    Beans
    2,172
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: batch photo management script

    In the "find" option, you should really use -cnewer instead of "-newer". With -newer, if you import old photos (with modification date older than last run), they won't be taken in account. But their inode change date should still tell when they were imported.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •