Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
02-10-2017, 02:16 PM
|
#16
|
LQ Newbie
Registered: Feb 2017
Posts: 8
Original Poster
Rep: 
|
Quote:
Originally Posted by TB0ne
No worries, and yes, research is always the key to learning better. The options can get confusing, and we're always happy to explain things if you're stuck. The man pages are always a good starting point...unless you try looking at the ones for sed and awk, both of which can be hideously complicated, and are both very powerful commands. I think there's even an entire book written on them:
http://shop.oreilly.com/product/9781565922259.do
As a hint, look at the "-F" flag for awk, then look at your input string. See anything common at the beginning/end of what you're after that you can use as a field-separator?
|
Haha wow, I bet that's a fun read
I suppose the forward slashes would be useable as a field separator? I know I could use something like the below in order to define what is a valid IP address:
(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)
|
|
|
02-10-2017, 02:33 PM
|
#17
|
Moderator
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=15, FreeBSD_12{.0|.1}
Posts: 6,385
|
Not as bad as it looks, and very much worth learning.
As you are actually learning and paying attention, perhaps a working example will encourage you to follow through.
Here is a simple awk that does the trick:
Code:
echo 'https://192.168.3.4/random/directories/here/' |awk -F\/ '{print $3}'
192.168.3.4
Hopefully you can see that it does work, and then when you figure out why it works you'll feel empowered to learn more.
Awk is the go-to tool for most text extractions and manipulations.
Regular expressions are the foundation for most of the real power of text manipulations, and sed puts those right at your fingertips. Here is a sed that produces the same result.
Code:
echo 'https://192.168.3.4/random/directories/here/' |sed 's/.*\/\/\([^\/]*\).*/\1/'
192.168.3.4
Not so many slashes, but it does not enforce " valid IP address" and works equally well to extract a host name:
Code:
echo 'https://some.host.com/random/directories/here/' |sed 's/.*\/\/\([^\/]*\).*/\1/'
some.host.com
Try to figure why it works, it is not so difficult to understand!
And just to expand on TB0ne's comments - the rules are there to keep things helpful and friendly here at LQ. If you are looking for someone to do your homework for you, this is not the place.
On the other hand, if you are learning and growing and want to share in the experience - it is all good!
Welcome to LQ and good luck!
|
|
|
02-10-2017, 02:50 PM
|
#18
|
LQ Newbie
Registered: Feb 2017
Posts: 8
Original Poster
Rep: 
|
Quote:
Originally Posted by astrogeek
Not as bad as it looks, and very much worth learning.
As you are actually learning and paying attention, perhaps a working example will encourage you to follow through.
Here is a simple awk that does the trick:
Code:
echo 'https://192.168.3.4/random/directories/here/' |awk -F\/ '{print $3}'
192.168.3.4
Hopefully you can see that it does work, and then when you figure out why it works you'll feel empowered to learn more.
Awk is the go-to tool for most text extractions and manipulations.
Regular expressions are the foundation for most of the real power of text manipulations, and sed puts those right at your fingertips. Here is a sed that produces the same result.
Code:
echo 'https://192.168.3.4/random/directories/here/' |sed 's/.*\/\/\([^\/]*\).*/\1/'
192.168.3.4
Not so many slashes, but it does not enforce " valid IP address" and works equally well to extract a host name:
Code:
echo 'https://some.host.com/random/directories/here/' |sed 's/.*\/\/\([^\/]*\).*/\1/'
some.host.com
Try to figure why it works, it is not so difficult to understand!
And just to expand on TB0ne's comments - the rules are there to keep things helpful and friendly here at LQ. If you are looking for someone to do your homework for you, this is not the place.
On the other hand, if you are learning and growing and want to share in the experience - it is all good!
Welcome to LQ and good luck!
|
Ah thankyou very much. I'll admit awk and sed are commands I'm not familiar with much, but clearly I should look into it some more! I'm definitely very much a noob, have only been learning linux for a few months, as I'm self teaching skills for pentesting
Definitely here to learn/grow/share in the experience so seems like I came to the right place!
Thanks all for your input
|
|
|
02-10-2017, 03:34 PM
|
#19
|
LQ Guru
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 27,805
|
Quote:
Originally Posted by herpplederppleston
Ah thankyou very much. I'll admit awk and sed are commands I'm not familiar with much, but clearly I should look into it some more! I'm definitely very much a noob, have only been learning linux for a few months, as I'm self teaching skills for pentesting
Definitely here to learn/grow/share in the experience so seems like I came to the right place!
Thanks all for your input
|
Astrogeek nailed it. Your regex for identifying a valid IP address is a great start...and regex'es can be VERY daunting. The plus is, once you get comfortable with them, you can start using them in all sorts of ways, including in sed/awk...even in CRON jobs:
http://www.linuxquestions.org/questi...2/#post4654486
The complexity and flexibility of *nix is truly awesome, compared to other OS'es.
|
|
|
02-10-2017, 03:44 PM
|
#20
|
LQ Newbie
Registered: Feb 2017
Posts: 8
Original Poster
Rep: 
|
Quote:
Originally Posted by TB0ne
Astrogeek nailed it. Your regex for identifying a valid IP address is a great start...and regex'es can be VERY daunting. The plus is, once you get comfortable with them, you can start using them in all sorts of ways, including in sed/awk...even in CRON jobs:
http://www.linuxquestions.org/questi...2/#post4654486
The complexity and flexibility of *nix is truly awesome, compared to other OS'es.
|
Yeah I wish I could claim I figured that out on my own, but that was one clue I found via google to potentially help formulate the correct command.. Even though I understand how it works and allows for a valid IP
echo 'https://192.168.3.4/random/directories/here/' |awk -F\/ '{print $3}'
I completely understand this command up until $3
I know $ signs signify variables (?), but I don't understand why the number 3 is used?
|
|
|
02-11-2017, 12:18 AM
|
#21
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,756
|
Quote:
Originally Posted by herpplederppleston
I completely understand this command up until $3
I know $ signs signify variables (?), but I don't understand why the number 3 is used?
|
The $3 stands for the third field in the line, fields being delimited by the pattern specified by -F. In this case you've set awk to define fields as spans of characters that are separated by a single slash. That is because you are using a single slash / as the delimiter and the span after the second occurrence is the third such span. If you use a pattern instead you can grab a span of slashes as the delimiter. The -F can be an actual pattern in most (all?) versions of awk.
Once you've identified the field, you can then work on the field further.
Keep checking the manual page for awk, it's a good reference work for the language and will make sense more and more. Also there are some good books on sed and awk, check your local college technical or engineering library. IFF the library is any good then there will be at least one such book available.
Some wiggle room in that task is that what is allowed in a URL is not universally agreed upon.
Regarding sed, the characters delimiting the search and replace pattern only need to be three of a kind. So if you are working with a lot of slashes in your patterns then you can use something else like a pound sign # or pipe | or an exclamation mark:
Code:
echo $URL | sed -e 's#^.*//##; s#/.*$##;'
|
|
|
02-11-2017, 03:16 AM
|
#22
|
LQ Newbie
Registered: Feb 2017
Posts: 8
Original Poster
Rep: 
|
Quote:
Originally Posted by Turbocapitalist
The $3 stands for the third field in the line, fields being delimited by the pattern specified by -F. In this case you've set awk to define fields as spans of characters that are separated by a single slash. That is because you are using a single slash / as the delimiter and the span after the second occurrence is the third such span. If you use a pattern instead you can grab a span of slashes as the delimiter. The -F can be an actual pattern in most (all?) versions of awk.
Once you've identified the field, you can then work on the field further.
Keep checking the manual page for awk, it's a good reference work for the language and will make sense more and more. Also there are some good books on sed and awk, check your local college technical or engineering library. IFF the library is any good then there will be at least one such book available.
Some wiggle room in that task is that what is allowed in a URL is not universally agreed upon.
Regarding sed, the characters delimiting the search and replace pattern only need to be three of a kind. So if you are working with a lot of slashes in your patterns then you can use something else like a pound sign # or pipe | or an exclamation mark:
Code:
echo $URL | sed -e 's#^.*//##; s#/.*$##;'
|
Got it, thanks for explaining that
|
|
|
All times are GMT -5. The time now is 06:56 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|