LinuxQuestions.org - Pinging numerous addresses at the same time and collecting results

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Pinging numerous addresses at the same time and collecting results (https://www.linuxquestions.org/questions/programming-9/pinging-numerous-addresses-at-the-same-time-and-collecting-results-587265/)

Pinging numerous addresses at the same time and collecting results

Hi there,

I'm currently facing a problem that I can't really think of solving. The problem is as follows:

I need to write a script that figures out what address the proxy server in that location has. The way I can find this out is by pinging a set number of IP-addresses.

For example if 192.168.100.120 answers I know that the proxy address is 10.120.10.10, if 192.168.100.170 answers I know that the proxy address is 10.170.10.10.

This would be ok if there only was two or three possible locations but my problem is that there is already about 30 locations which could rise into the 100's soon.

I can ping them one after another but going through 100 pings makes it quite slow.

What I'd want to happen is the following.

- The script spawns/forks as many simultaneous pings as possible
- If one of these pings get a 'hit' it writes it to result file
- The script reads this value and pairs it with a proxy address and writes this to the setting for Firefox.

The theory of this looks well cool and should work but as usual for me I have no real idea of how to make this into a working script.

Does anyone have any bright ideas about how to implement this?

Thanks in advance, Mike

You could use nmap, which will ping ranges of IPs with the -sP option. For example:

Code:

nmap -sP 192.168.1-3.*

...will search for ping-able hosts in the three /24 ranges 192.168.1.anything, 192.168.2.anything and 192.168.3.anything.
You might want to add the -n option to prevent resolution to names so you always get IPs back, and filter the output with awk or something to get a simple list of IPs, i.e.

Code:

nmap -n -sP 192.168.1-3.* |awk '/^Host/ { print $2 }'

You should carefully consider the use of nmap if you are in Germany - recent legislation outlawing "hacking" tools might put you in a legal grey area with this program, despite the legitimate use. Silly law!

Thanks for your reply but it wouldn't work for this special scenario.

I need to use ping since we can only use what's being installed as standard with the distribution we are using.

Besides, using Nmap kills the firewalls/proxies that we have (read: Bordermanager) so that's not an option.

Once again, thanks for your suggestion, I'm really grateful for any ideas.

// Mike

The most ubiquitous tools are going to be a simple borne style shall and ping I guess. You'd have to watch out for different ping versions on Linux, the BSDs, other Unixes (things like return values have a nasty trick of changing subtly across the different *nixes...

Components of the script would be a function to turn an IP spec with ranges into a list of IPs, i.e. which takes a parameter like "192.168.1.1-3" and returns "192.168.1.1 192.168.1.2 192.168.1.3". Then you can iterate over the list and call another function which does the actual ping.

ping itself would end up being called and the return value tested something like this:

Code:

ping -W 3 -c 3 $IP > /dev/null

if [ $? -eq 0 ]; then

    echo $IP >> success.log

fi

And then at the end of it all, the file success.log contains the IPs which were reachable.

Clearly if you do all the pings one after the other it's going to be very slow, so you'll want to use shell job control to achieve some degree of parallelism. Launching off groups of ping in the background and calling "wait" is the way to do this.

To my mind easiest (if not optimal) way to do it would be split the list of IPs into groups of some number, launch that number of IP checks in parallel, wait for the result, and then do the next group.

If you have multiple ping appending to some log file in parallel, you might have trouble with un-ordered writing to the log file... this is probably the hardest part to work around. I am not sure if the shell append re-direction operator >> will suffer from this problem or not - perhaps. If this is the case, you might create a directory for the results and touch a file for each positive result, deleting them once you've finished the check.

How about this:

Code:

#!/bin/bash

#

# Ping ranges of hosts and output a list of the hosts which responded on stdout

# usage:

#

# multiping range [range] ...

#

# Where range is an IP range.  Examples of ranges:

#  192.168.1.1        - just 1 IP

#  192.168.1-5.*      - 192.168.1.1 .. 192.168.5.255



# So basically you have two wildcards m..n ranges and * meaning 0..255

# (C) 2007 M N Gates

# License: GNU GPL v2



LOG=$(mktemp)



# change this to the number of pings you want to do in parallel at a given time

PARALLELISM=20



main () {

        declare -a group=()

        for range in "$@"; do

                for ip in $(ip_range_to_list "$range"); do

                        if [ ${#group[*]} -lt $(( $PARALLELISM - 1 )) ]; then

                                group=(${group[*]} $ip)

                        else

                                group=(${group[*]} $ip)

                                ping_group ${group[*]}

                                group=()

                        fi

                done

                if [ ${#group[*]} -gt 0 ]; then

                        ping_group ${group[*]}

                        group=()

                fi

        done



        echo "IPs which are up:"

        cat $LOG

        rm -f $LOG

}





ip_range_to_list () {

        oIFS="$IFS"

        IFS=.

        # prevent * getting expanded to file list in array set statement...

        full=${1/\*/+}

        declare -a ip_parts=($full)

        IFS="$oIFS"

        for p1 in $(expand_range ${ip_parts[0]}); do

                for p2 in $(expand_range ${ip_parts[1]}); do

                        for p3 in $(expand_range ${ip_parts[2]}); do

                                for p4 in $(expand_range ${ip_parts[3]}); do

                                        echo "$p1.$p2.$p3.$p4"

                                done

                        done

                done

        done

}



expand_range () {

        if [ "$1" = "+" ]; then

                for ((i=0; i<256; i++)); do

                        echo $i

                done

                return

        fi



        low=${1%%-*}

        high=${1##*-}

        for ((i=$low; i<=$high; i++)); do

                echo $i

        done

}



ping_group () {

        echo "$@"

        for f in "$@"; do 

                pinger "$f" &

        done

        wait

}



pinger () {

        ping -W 3 -c 3 $1 > /dev/null

        if [ $? -eq 0 ]; then

                echo "$1" >> $LOG

        fi

}



main "$@"

This looks awesome! I'll cut/paste and test straight away! Many thanks for taking the time to help me with this.

// Mike