How to exclude all speacial characters using regex?
 

Hi Folks,

I need to exclude special characters from file and only include
[a-zA-Z0-9] . -

In-fact I am just including domain names and exclude all special characters.

I am not able achieve the same.

~`!@#$%^&*()_+={}[]\|;:'"<,>/?

Can someone please help?

which language is it? do you have any written code already?

Need that in bash, dang I am not regex pro but giving my best and failing :(
Any hint?

My sample text would be

example.com
test.com
test123.com
123test.ocm
calid-domain.com
test-test.net
!def
@fsf
dafsrf#
fffgg$.net
%rrt.com
^testcom
asddf&.net
as*
(
)
_
+
=
\
;
:
'
"
<
,
>
?
/

You've been posting things like this for a good while now:
https://www.linuxquestions.org/quest...es-4175657403/
https://www.linuxquestions.org/quest...nd-4175656948/
https://www.linuxquestions.org/quest...rs-4175655180/
https://www.linuxquestions.org/quest...ng-4175648204/
https://www.linuxquestions.org/quest...es-4175641557/
https://www.linuxquestions.org/quest...pt-4175635666/
https://www.linuxquestions.org/quest...pt-4175616729/

Show your own efforts when posting, and do basic research. After three years, you should have SOME scripting/research skills.

Putting "bash regex strip out anything but letters and numbers" into Google pulls up a LOT of 'hints'. You've been told many times to post things in CODE tags, but don't seem to follow that advice either. The [:alnum:] is alpha-numeric.

I understand and I am definitely trying to get the answer and of course everyone first tries google which I also did and if that didnt resolve then come here.

Will definitely ensure to follow the code tags.

You need to escape certain characters inside the RegEx:
The above code takes care of the most problematic ones. Notice, that if you want to match a literal ']' inside the brackets then it must be the first character after the opening '['.
I will leave matching the remaining characters as an excercise.

PS:
You can also achieve this by using [:alnum:] by TB0ne but it has also a pitfall. I think, however, that doing it the "hard" way is more educational in the long run since you can learn how to handle certain characters in a RegEx.

I guess I am not able to exclude single quote

Read post #8 again.

Sorry, just don't believe that. Putting the search term I used into Google yielded 559,000 hits....hard to believe that out of all that there wasn't one 'hint' you could have used. And you've been asked about CODE tags for a LONG time, but don't use them.
And why is that, given the fact that I not only gave you a search-term that has your 'hints', but the **EXACT** thing you need to use for a regex to strip out anything but letters and numbers???

Better name the printable characters, and use the complement of it, either with tr and -c option, or with a negating ^ in a charset in a RE:

just a quick test of that one loop.
testfile
results
tells a story...

And what story would that be?

Thanks and nice option; however I am looking with Grep if possible.

Then why did you ask for a bash solution in a previous post?

Just a few modification

looks like someone needs to read the foot notes. it works..

my bad bash as in wanted in grep :(

Is the sample file you provided at least representative?

Those solutions will also keep invalid domain names, like '%rtt.com'. It will "transform" to 'rtt.com' but I am not sure if this is desired by OP.

@OP:
Please provide a sample output file of what you expect it to look like before we keep guessing.

If you want to not print lines that have a forbidden character, with grep:
Again, it is easier to name the allowed characters.
For the [a-zA-Z0-9] set there is [[:alnum:]], can be augmented with extra characters and of course with the ^ negation:

Yep, the alnum was given earlier, and promptly ignored.

On a side note, let me just point out for OP that if you want to include '-' inside '[]' then it must be the last character. This is the pitfall I was refering to earlier. Otherwise it will be interpreted as a range operator. This solution will almost bring you there. You still need to figure out the last missing part.

Yes, so adding more extra characters must be like this
or this
Actually it is possible to have the - character first (after the ^ of course), but there are other characters like a ] that must be first, so it is better to remember "- must be last".
Please promptly ignore any sarcasm ;)

Did not know that it can be also first, always had it last when needed and never questioned it. Makes sense, though, since on position one (ignoring ^) it cannot be mistaken to indicate a range.

It wasn't ignored but mistakenly it wasn't working as I was just using [[:alnum:]] hence was showing different results.

Hello,

That worked perfectly fine; however what I am trying to match here is and not sure if this can be achieved in the same line.
Since the above pattern is catching single dot as liternal and hyphen. Being a domain name those will be surrounded by alnum hence trying hard for validation to match . and - only if surrounded by \wfollowed by those two literals.

May be I am missing something?

The most important thing you are missing is to provide a representative sample file and the output you expect.
You have been presented with a solution that works for the sample file you provided. Now you are telling us that the sample file is not representing the actual input data, thus the solution is inappropriate. It is pointless to provide you with a solution if you keep changing the requirement.

wrong post.. oops

\w and [:alnum:] are almost the same, just different syntax.
^ will have an effect on everything inside [ and ] (means exclusion instead of inclusion).
Also would be nice to check www.regex101.com because you can construct and check any regexp yourself

From the man page: The symbol \w is a synonym for [_[:alnum:]] and \W is a synonym for [^_[:alnum:]], so they extend the class [:alnum:] with the character _ (in any place for \w) or with _ as FIRST character (for \W).
As _ is not a alphanumeric character, both thus are an extension on [:alnum:]

The OP asked for 'hints' (post #4). They've been given a LOT of hints, but don't seem to have actually worked on/thought about them.

OP, you have been given a LOT of advice that you could act on and research, to solve your own problem.

Here is my Input file
And here is the answer I am getting. Now if you see this is matching a single dot and hyphen as well. I am working include only if dot/hyphen has surrounded by [:alnum:]

Note: if this is about "filename like" strings:
1) Quite often filenames start with a dot (so-called hidden files).
Use "la -A" in your home dir to see a lot of them.
2) Filenames with multiple dots and/or dashes are common too, i.e. php-5.6.40-x86_64-1.txz
3) The _ is often used to substitute spaces, like George_Harrison-What_Is_Life.mp3

Nah I am not trying to match files but trying to include the data in that file and those are domain names. Only valid characters in domains names are . and - but should have alphanumeric around it.

so I would create something like this:
1. one alphanumeric
2. any alnum and -
3. one alnum
4. dot
5. any alnum (but at least one).

You can construct a regexp for this (or anything similar)

And how do you handle then hostnames like www.debian.org or en.wikipedia.org (that is, with multiple dots)?
Those steps 4 and 5 have to be repeated until the end of the string is reached.

So what have you done with any/all of the 'hints' you asked for thus far, to try to accomplish this?? Haven't seen your efforts thus far.

And if all you're looking to do (since the goal has apparently changed again), is to get domain names, why can't you just grep for things like .net,.com,.edu, etc. into another file??

This eventually should suffice my need? Please correct me if I am wrong.

this was only an example and anyone can improve. just a way to build a correct regexp

Did you check it? It looks [almost] identical/similar to:
Is this what you need?

Yes it is now showing me the desired results.

in that case probably you can mark the thread solved
and again, you may check www.regex101.com to improve your skills and to check your regexps.

The grep runs in a shell; many special characters have a special meaning in the shell, too.
Use quotes, so the shell does not try special substitutions!
I am unsure if perl-style extensions like \w work within a character set [ ], so I would go for only Posix classes [:xxxx:]
Here it is sufficient to add the _ character to the [:alnum:] class