Results 1 to 6 of 6

Thread: Need Help Executing commands inside of Bash Script

  1. #1
    Join Date
    Jan 2021
    Beans
    2

    Need Help Executing commands inside of Bash Script

    Hello all - Scripting n00b here. I'm setting up the UFW and I have a file with about 80,000 commands I need to run. I figured I would just loop through the file and execute the commands, but I'm have an issue doing it. Need some help. Here's what I built so far:

    SetupUFW.sh
    Code:
    #!/bin/bash
    while read p; do
      echo "$p"
    done < BlockList.txt
    BlockList.txt (80,000 lines that look like this
    Code:
    sudo ufw deny from 1.0.1.0/24
    sudo ufw deny from 1.0.2.0/23
    sudo ufw deny from 1.0.8.0/21
    As you can see, I'm trying to loop through the file called BlockList.txt as I have already built it with the commands necessary. How can I loop through each line and execute the lines? Any help you can provide would be appreciated.

  2. #2
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Need Help Executing commands inside of Bash Script

    ufw can't handle this.
    iptables can't handle this.

    If you have more than a few hundred rules, use ipset.

    But so you see I'm right, just run the BlockList.txt file, directly.
    Code:
    chmod +x /path/to/BlockList.txt
    /path/to/BlockList.txt
    Give it 10 minutes to run. Maybe more.

    ipset will use 1 firewall rule for all the subnets, but match that single rule extremely efficiently.
    There are a number of how-to guides.

  3. #3
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Need Help Executing commands inside of Bash Script

    Would be smarter to only have 1 sudo too. Remove it from every command in the script, call the script:
    Code:
    sudo /path/to/BlockList.txt
    spawning an extra process for every line is bad. That's what sudo does.
    Last edited by TheFu; January 31st, 2021 at 09:07 PM.

  4. #4
    Join Date
    Jan 2021
    Beans
    2

    Re: Need Help Executing commands inside of Bash Script

    Thank you! This is definitely a better way to go. I found a great tutorial online for how to implement. Thank you so much for your suggestion!

    Tutorial Link: How to block IP addresses from a country using IPset - IP2Location.com

    Quote Originally Posted by TheFu View Post
    ufw can't handle this.
    iptables can't handle this.

    If you have more than a few hundred rules, use ipset.

    But so you see I'm right, just run the BlockList.txt file, directly.
    Code:
    chmod +x /path/to/BlockList.txt
    /path/to/BlockList.txt
    Give it 10 minutes to run. Maybe more.

    ipset will use 1 firewall rule for all the subnets, but match that single rule extremely efficiently.
    There are a number of how-to guides.

  5. #5
    Join Date
    Nov 2008
    Location
    Boston MetroWest
    Beans
    16,326

    Re: Need Help Executing commands inside of Bash Script

    I run custom iptables scripts. I put the banned addresses in a file, then use some code like this:
    Code:
    EVIL=$(cat /path/to/banned/list)
    for h in $EVIL
    do
        /sbin/iptables -A INPUT -j REJECT -s $h
    done
    This is usually among the first commands I run in the firewalling script since we just want to drop any packets from these hosts before moving on to any other rules.
    If you ask for help, do not abandon your request. Please have the courtesy to check for responses and thank the people who helped you.

    Blog · Linode System Administration Guides · Android Apps for Ubuntu Users

  6. #6
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Need Help Executing commands inside of Bash Script

    When the number of rules becomes large and many are really just 1 rule, just with hundreds or thousands of subnet matches, using ipset is exponentially more efficient.
    Code:
    -A INPUT --match set --match-set countryblock  src --jump DROP
    The OP has 80K rules. I only have 6900:
    Code:
    $ wc -l /etc/ipset.up.rules
    6926 /etc/ipset.up.rules
    Somewhere around 200, it was noticeably slower both to load and while running.
    Put in the single ipset "match-set" and the performance improvements are great.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •