RegEx is not matching

blason · 08-26-2020, 12:03 AM

Hi there,

Can someone please help me on regex pattern matching with my below function?

This is matching IPv4 addresses but somehow my IPv6 addresses are not getting matched

Code:

212.129.46.32
213.163.64.208
217.61.0.97
223.5.5.5
223.6.6.6
2001:1398:1:0:200:1:123:46
2001:19f0:5:6012:5400:2ff:fe60:7579
2001:19f0:5801:b7c::1
2001:19f0:6401:b3d:5400:2ff:fe5a:fb9f

Code:

function convert {
    while read ip; do
        if ! [[ "$ip" =~ [^0-9.-] ]];
        then
                                todo[$y]+=" $ip $ip"
                                        if [ $z -eq 2000 ]
                                        then
                                                z=0
                                                let y=$y+1
                                        else
                                                let z=$z+1
                                        fi
        fi
     done

        dynamic_objects -do ti_$type
        dynamic_objects -n ti_$type

        for i in "${todo[@]}" ;
        do
                dynamic_objects -o ti_$type -r $i -a
        done
}

blason · 08-26-2020, 12:12 AM

So I removed the caret which negates but this is still not working. Am I doing anything wrong with RegEx

if ! [[ "$ip" =~ [^0-9.-] ]];

astrogeek · 08-26-2020, 12:40 AM

Your ipv6 addresses contain characters not covered by your regex - hexadecimal digits and the colon separator.

Add those to your regex and it will work such as it is.

Of course, it will also match many things other than IP addresses as well and will not differentiate ipv4 from ipv6, but those are not mentioned as considerations so may not be important to you.

It is probably important that you at least consider a few patterns you do not want to match in order to help specify your regex more precisely. For example, your existing regex will also match phone numbers in almost any format, and with the addition of hex digits it will match many other things including "deadbeef" and 1-800-bad-deed, among a great many other things!

syg00 · 08-26-2020, 01:46 AM

Ah the joys of corner cases ...

I must admit I've never had the temerity to use double negatives with regex. Life has enough angst as it is.

MadeInGermany · 08-26-2020, 02:15 AM

Put all allowed characters into the [char set]

Code:

if [[ "$ip" !~ [^0-9a-fA-F:.] ]]

A more precise match

Code:

if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{,4}){4,8}$ ]]

blason · 08-26-2020, 02:27 AM

Quote:

Originally Posted by MadeInGermany

Put all allowed characters into the [char set]

Code:

if [[ "$ip" !~ [^0-9a-fA-F:.] ]]

A more precise match

Code:

if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{,4}){4,8}$ ]]

Let me try that..

blason · 08-26-2020, 02:35 AM

Quote:

Originally Posted by MadeInGermany

Put all allowed characters into the [char set]

Code:

if [[ "$ip" !~ [^0-9a-fA-F:.] ]]

A more precise match

Code:

if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{,4}){4,8}$ ]]

any issues?

line 11: syntax error in conditional expression: unexpected token `('
line 11: syntax error near `^(['
line 11: `if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{,4}){4,8}$ ]];'

Code:

function convert {
    while read ip; do
#        if ! [[ "$ip" =~ [^0-9.-] ]];
if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{,4}){4,8}$ ]];
        then
                                todo[$y]+=" $ip $ip"
                                        if [ $z -eq 2000 ]
                                        then
                                                z=0
                                                let y=$y+1
                                        else
                                                let z=$z+1
                                        fi
        fi
     done

        dynamic_objects -do ti_$type
        dynamic_objects -n ti_$type

        for i in "${todo[@]}" ;
        do
                dynamic_objects -o ti_$type -r $i -a
        done
}

boughtonp · 08-26-2020, 06:45 AM

When using Bash regex, predefine the patterns as variables with single-quotes around them, and then use those variables in the if statement.

Also, note that "::1" is a valid IPv6 address, which is not matched by that pattern, and I just noticed that the IPv4 pattern is requiring a "." at the start, which is just wrong.

The IPv6 pattern below may or not be good enough for your needs (I don't see any mention of what those are).
Likewise, IPv4 only allows each octet to be up to 255 - you may or not need separate validation to check that.

Code:

IPv4='^[0-9]{1,3}(\.[0-9]{1,3}){3}$'
IPv6='^[0-9A-Fa-f:]{3,39}$'
if [[ "$ip" =~ $IPv4 || "$ip" =~ $IPv6 ]]
then
...
fi

syg00 · 08-26-2020, 06:58 AM

Quote:

Almost only counts in horseshoes and hand grenades.

Certainly not regex. Precision, as in exact specification, is absolute. The more you use it, the more you see the truth in this. You can only learn it by doing.

MadeInGermany · 08-26-2020, 07:23 AM

Corrections:
1. the !~ operator might be unknown

Code:

if ! [[ "$ip" =~ [^0-9a-fA-F:.] ]]

2. a {,4} might be interpretet as {1,4}

Code:

if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{0,4}){4,8}$ ]]

@boughtonp, note the prepended "." or ":"

Quote:

line 11: syntax error in conditional expression: unexpected token `('
line 11: syntax error near `^(['
line 11: `if [[ ".$ip" =~ ^([.][0-9]{1,3}){4}$ ]] || [[ ":$ip" =~ ^(:[0-9a-fA-F]{,4}){4,8}$ ]];'

Ensure the executing shell is bash or zsh! E.g. at the very beginning of the script have the shebang

Code:

#!/bin/bash

blason · 08-26-2020, 07:30 AM

Quote:

Originally Posted by boughtonp

When using Bash regex, predefine the patterns as variables with single-quotes around them, and then use those variables in the if statement.

Also, note that "::1" is a valid IPv6 address, which is not matched by that pattern, and I just noticed that the IPv4 pattern is requiring a "." at the start, which is just wrong.

The IPv6 pattern below may or not be good enough for your needs (I don't see any mention of what those are).
Likewise, IPv4 only allows each octet to be up to 255 - you may or not need separate validation to check that.

Code:

IPv4='^[0-9]{1,3}(\.[0-9]{1,3}){3}$'
IPv6='^[0-9A-Fa-f:]{3,39}$'
if [[ "$ip" =~ $IPv4 || "$ip" =~ $IPv6 ]]
then
...
fi

Yep that seems to have worked!! Thanks for the help
I am testing it further let me see. Hopefully should not encounter any other issues.

valeoak · 08-26-2020, 08:56 AM

You may find Debuggex useful for testing regex and visualising the pattern matching. They also provide some handy cheatsheets.