LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)
-   -   How to replace in an bash script variable? (https://www.linuxquestions.org/questions/linux-general-1/how-to-replace-in-an-bash-script-variable-4175574038/)

postcd 03-05-2016 09:42 AM

How to replace in an bash script variable?
 
This topic is a waste of time, no solution to the question.

Hello,

im having urlfile and each url in it contains $ip variable which i need to replace by actual IP that is actual for the present "while loop" cycle:

Code:

while read ip;do

while read url;do
curl -s "$url" >> output
done < urlfile

done < ipfile

# cat urlfile
http://domain1.com/?something&?ipp=$ip
http://domain2.com/?ipaddr=$ip&something=something
http://domain2.com/?a=lldd?ipaddr=$ip#something

But i dont know what is the clever/low resource usage/low amount of code way to replace that $ip variable inside the URLs that i have either in urlfile or in variable that i populate somehow out of urlfile? Thanks in advance


----

Update: Solution for my case:

Quote:

while read ip;do
cp /mypath/urlfile /mypath/urlfile2
# do replacing in the temporary urlfile2 so the urls reflect current IP
find /mypath -name 'urlfile' -type f -print0 | xargs -0 sed -i "s|IPHERE|$ip|g"
while read webpage;do
# my command here
done < /mypath/urlfile2
done < /mypath/ipfile
rm -f /mypath/urlfile2
(it may need tweaking as i needed to rewrite it for this example purpose, it basically duplicate original urlfile with IP placeholders IPHERE and do replacing in it (sed need to use double quotation marks " to work with $ip variable))

TB0ne 03-05-2016 11:03 AM

Quote:

Originally Posted by postcd (Post 5510635)
Hello,
im having urlfile and each url in it contains $ip variable which i need to replace by actual IP that is actual for the present "while loop" cycle:
Code:

while read ip;do

while read url;do
curl -s "$url" >> output
done < urlfile

done < ipfile

# cat urlfile
http://domain1.com/?something&?ipp=$ip
http://domain2.com/?ipaddr=$ip&something=something
http://domain2.com/?a=lldd?ipaddr=$ip#something

But i dont know what is the clever/low resource usage/low amount of code way to replace that $ip variable inside the URLs that i have either in urlfile or in variable that i populate somehow out of urlfile? Thanks in advance

What have you tried to do this? Have you checked ANY of the bash scripting tutorials? Setting variables is very well documented. You can use either xargs, or any of the 'standard' methods of assigning a variable.

A brief Google search turns up lots. From just ONE is this sample:
Code:

saveIFS="$IFS"
IFS=$'\n'
array=($(<file))
IFS="$saveIFS"
echo ${array[0]}    # output: EXAMPLEfoo
echo ${array[1]}    # output: EXAMPLEbar
for i in "${array[@]}"; do echo "$i"; done    # iterate over the array

Did you try to do research of your own?

postcd 03-05-2016 11:09 AM

yes, please anyone else how to do it please?

TB0ne 03-05-2016 12:35 PM

Quote:

Originally Posted by postcd (Post 5510671)
yes, please anyone else how to do it please?

So again, you're handed a solution but ignore it, don't post any of your own efforts, and wonder why people don't help you more??

postcd 03-05-2016 01:40 PM

Sorry, im newbie i do not understand your copy paste "solutions" that do not fits my case. That is why im asking for other people oppinion. I ask you to stop answering in my topics. If you do not provide solutions and instead you will be bashing offtopic & wasting my time and other readers time, i will just report your post.

michaelk 03-05-2016 02:03 PM

Did you post the entire urlfile? And is it exactly as shown? Does the IP file and url file correspond one for one? Or as in your loop do you apply each IP to every URL?

postcd 03-05-2016 02:23 PM

Quote:

Originally Posted by michaelk (Post 5510730)
Did you post the entire urlfile? And is it exactly as shown? Does the IP file and url file correspond one for one? Or as in your loop do you apply each IP to every URL?

Thx, no, i didnt posted real urlfile as i wanted the URLs to stay private.

So far i have 3 URLs in urlfile and there are parts of that URLs, parts where should be IP assigned..:
...=$ip
...=$ip&...
...=$ip&...

michaelk 03-05-2016 03:14 PM

And the other questions?

Does the IP file and url file correspond one for one? Or as in your loop do you apply each IP to every URL?

postcd 03-05-2016 03:24 PM

Quote:

Originally Posted by michaelk (Post 5510761)
in your loop do you apply each IP to every URL?

simply said, i have 3 URLs to process, and for each URL i need to use all IPs in my list.

So for each IP i will process all 3 URLs, and i need to replace the $ip variable in them by actual IP (actual for that "while loop" i mentioned in my initial post). Is it clearer please?

TB0ne 03-05-2016 03:54 PM

Quote:

Originally Posted by postcd (Post 5510721)
Sorry, im newbie i do not understand your copy paste "solutions" that do not fits my case.

First, stop playing the "im newbie" card...you've been here THREE YEARS. Secondly, I *DID* provide you with a solution that did exactly what you were after. The 'problem' with it, is that it required YOU to actually think and learn something. It was up to YOU to modify it to fit your needs.

That, is how you stop being a 'newbie' and start to be able to do something for yourself.
Quote:

That is why im asking for other people oppinion. I ask you to stop answering in my topics. If you do not provide solutions and instead you will be bashing offtopic & wasting my time and other readers time, i will just report your post.
Too bad...this is a public forum. ANYONE can respond...if you don't like it, then post somewhere else. And don't bother reporting my post (which was not only on topic, but gave you a solution)...I did it for you, since you are AGAIN ignoring the question guidelines and LQ Rules.
Quote:

Originally Posted by postcd
Thx, no, i didnt posted real urlfile as i wanted the URLs to stay private.

..which makes NO SENSE at all, since you could easily replace any URL's with samples...same with addresses.
Quote:

simply said, i have 3 URLs to process, and for each URL i need to use all IPs in my list.

So for each IP i will process all 3 URLs, and i need to replace the $ip variable in them by actual IP (actual for that "while loop" i mentioned in my initial post). Is it clearer please?
Your post was clear to start with. What you STILL haven't provided is examples of your input data, what you're looking for as output data, and ANY EFFORT ON YOUR PART TO ACTUALLY PERFORM THIS TASK. Again, what have YOU written/done/tried on your own???

Since you're a 'newbie' (who has been here 3 years), you probably haven't read the question guidelines. Quoting the relevant parts here, bolded parts are mine:
Quote:

Originally Posted by Question Guidelines from LQ
Here are a couple tips that will enable us to help you moving forward:
  • When asking a question, be sure to provide as many relevant details as possible. You should include the distribution and version you're using, along with hardware details, application version and exact error messages where applicable. ..this is where you post your sample data
  • If you're actively troubleshooting an issue you should also include any steps you've already taken. ..this is where you post what YOU have written
  • Before posting, have you used the search function to ensure your question hasn't been asked before? ...this is where, if you had tried, you would have found THOUSANDS of bash looping examples, which you COULD have modified on your own
If you are unwilling or unable to ask questions in a manner that allows us to help you, it's unlikely our community will be able to provide you a solution. Unfortunately, serial offenders who show wanton disregard for this request after multiple pointers may be asked to seek help elsewhere.

And given your nasty private messages to me, which contain gems like:
Quote:

Originally Posted by postcd Private Messages
  • Im not looking to waste hours of time reading some technical data to get answer to my question.
  • if you see im looking for solution, please do not fill my thread repeatedly with something im not interested in. It just wasting my precious time and we already been reading from you that you want me/us read the readme so i assume it is a waste of time also for other future readers.
  • you seems to be too stupid to understand that threads are created in aim to get solution not to be isntructed to read the README
  • Im not interested to learn from you if that is advice of this kind: "read software developer website"

Again, if you want to LEARN, this is the place. If you want someone to hand you an exact command, write your scripts for you, etc., then HIRE SOMEONE.

michaelk 03-05-2016 08:14 PM

I think string expansion by using the eval command should work but there are many ways.
I can not provide examples at the moment

TB0ne 03-05-2016 09:12 PM

Quote:

Originally Posted by michaelk (Post 5510869)
I think string expansion by using the eval command should work but there are many ways. I can not provide examples at the moment

Good luck. The OP only want a pre-written script and isn't interested in learning or doing for themselves, if their previous posts are any indication. This thread is a good example.

In the first reply, I gave the OP something that would work, but they flat out said they didn't want to modify anything, and wanted a solution to 'fit their case'

TobiSGD 03-06-2016 05:43 AM

Things you do in a public forum:
- Be kind to each other
- Be on topic
- When you don't understand an answer ask for clarification, in the same way: If the question is not clear enough ask for clarification
- Use all the options the forum software provides you. This includes the Ignore List function.

Things you never do on a public forum (and even real life):
- releasing private messages into public without explicit permission from all involved parties. There is a reason they are called private messages. If you feel you are harassed, insulted, ..., in private messages contact a mod or Jeremy.

postcd 03-06-2016 07:15 AM

Please anyone else know the solution for this (see initial mesaage)? I cant believe it can be so difficult to come with simple, clever solution.

michaelk 03-06-2016 08:21 AM

Check out string replacement

http://www.tldp.org/LDP/abs/html/str...ipulation.html

TB0ne 03-06-2016 10:45 AM

Quote:

Originally Posted by postcd (Post 5511085)
Please anyone else know the solution for this (see initial mesaage)? I cant believe it can be so difficult to come with simple, clever solution.

You've been handed:
  • A tip to look at xargs
  • A sample script which will work with a SMALL amount of thought and effort on your part
  • A tip to look at string replacement/eval command (with a sample script)
  • Links to bash scripting tutorials
...and you STILL can't/won't/aren't showing us anything YOU, personally, have done to make this happen. You STILL haven't shown us your sample input data, or what you're after for output data exactly...despite being asked for these things numerous times. This is the same behavior you show in most of your other threads.

This seems to dovetail nicely with your previous comment of "Im not looking to waste hours of time reading some technical data to get answer to my question." We WILL NOT write your script for you. Show effort of your own.

postcd 03-06-2016 10:53 AM

That script was not helpfull alas.

michaelk 03-06-2016 11:02 AM

Which script?
Did you look at the link I posted?

postcd 03-06-2016 11:08 AM

Quote:

Originally Posted by michaelk (Post 5511177)
Which script?
Did you look at the link I posted?

Yes, this seems related

stringZ=abcABC123ABCabc
echo ${stringZ//abc/xyz}

But not sure how i can use it in my case. I already tried it, but it do not replace phrasse "$ip" by IP. I used something like:
echo ${url//1.1.1.1/$ip}
and
echo ${url//$ip/1.1.1.1}
but no change in variable

pan64 03-06-2016 11:13 AM

Quote:

Originally Posted by postcd (Post 5511085)
Please anyone else know the solution for this (see initial mesaage)? I cant believe it can be so difficult to come with simple, clever solution.

Looks like it is definitely difficult. In general it is something like "communication error". You do not understand the solution given and the others do not really understand what is your problem.
I have only one comment to this (see LQ rules)
"That script was not helpfull alas." is not an acceptable answer.
"your copy paste "solutions" that do not fits my case. That is why im asking for other people oppinion" is not an answer too.
If you want to improve the quality of this communication (and obviously it is you who need a solution) you need to give some information about: what was wrong with that? Otherwise others will not be able to help you, because they (even me) have no idea how to give a better tip...

postcd 03-06-2016 11:18 AM

pan64: i dont understand anything on one single script that was provided in this thread. Not sure how it can work in my case.

pan64 03-06-2016 11:26 AM

in that case ask how does this or that work, even line by line. Probably you will be able to modify it to fit your needs. But "repeating does not work" will not help

postcd 03-06-2016 11:31 AM

ok, then im asking for line by line explanation, if there is not any more simplistic way to achive what i want. i never thought it can be that difficult, wanted to learn some simple way.

TB0ne 03-06-2016 11:31 AM

Quote:

Originally Posted by postcd (Post 5511187)
pan64: i dont understand anything on one single script that was provided in this thread. Not sure how it can work in my case.

You do this by doing something you have flat out said you don't want to do: READ AND LEARN. Again, you STILL, despite being asked many times, do not provide sample input data, sample output data, or most importantly ANYTHING THAT YOU, PERSONALLY HAVE ACTUALLY DONE OR TRIED. You are expecting people to hand you a script that 'meets your case', without YOU having to do anything. How many times, over the past three years, have you done exactly this?

Sorry, the scripts provided WILL work, if you put any effort into them. If you actually posted something like "I modified the script in post #xxx, and gave it this input, but got this result, and I'm confused", you'd get help. Saying "it doesn't work" tells us nothing.

And looking at the link michaelk gave you (after suggesting you look at it, they also had to look up a link for you), the sample script tells you exactly what you need to do...but you didn't do it. You're not following the syntax that's very clearly laid out in that document. And whether you like it or not, the FIRST REPLY (post #2), will work perfectly for you. It not only shows you how to read a file into an array, but how to process that array into variables (which is EXACTLY WHAT YOU WANT), then perform operations on that array...such as print them out, or do whatever you want with them.

postcd 03-06-2016 11:33 AM

post reported & ignored, please be ontopic.

TB0ne 03-06-2016 11:44 AM

Quote:

Originally Posted by postcd (Post 5511197)
post reported & ignored, please be ontopic.

It was on topic, and as you've been told before, report whatever you want. AGAIN:
  • You are asked for sample input data. You have not provided it.
  • You are asked for sample output data. You have not provided it.
  • You are asked for examples of what YOU, PERSONALLY have done. You have not provided it
  • You say NOTHING about the problems you're having with the script(s)/example(s)
  • You keep asking for something to 'meet your case', yet show no effort of your own
NONE of these things are in keeping with LQ Rules or Question Guidelines, period.
Quote:

Originally Posted by postcd
ok, then im asking for line by line explanation, if there is not any more simplistic way to achive what i want. i never thought it can be that difficult, wanted to learn some simple way.

It isn't difficult...would take me about 5 minutes to write such a script, and any others that have replied to you could do it in about the same time. That is because instead of whining about "im newbie" (after three years), they actually learned and read the scripting tutorials, wrote scripts, and LEARNED. Asking for "line by line explanation" is the first sign of effort you have actually shown in any of your posts.
Code:

saveIFS="$IFS"  IFS defines what separates the fields. In this case, a newline. Replace with whatever is your separator
IFS=$'\n'
array=($(<file)) Open your input file, and read it all into the array. This would assume you have a file with foo and bar in it
IFS="$saveIFS"  Only needed if something in the loop depends on IFS
echo ${array[0]}    Array element 1 (starts at zero), output: EXAMPLEfoo
echo ${array[1]}    Array element 2 (second after separator) output: EXAMPLEbar
for i in "${array[@]}"; do echo "$i"; done  iterate over the array, until you're done reading the input file

So instead of echoing $i back out, put it into your output string wherever you want.

AGAIN...try reading the bash scripting tutorials.

postcd 03-06-2016 11:51 AM

post not read and ignored as it do not provide any helpfull solution

pan64 03-06-2016 12:35 PM

what do you think, what will be really helpful - for you?

postcd 03-06-2016 12:40 PM

you do not know what is helpfull when you ask the question?
an answer that can help you solve the issue.
prefferably example with comment on how that works. are you that stupid not to understand this. I hope i explained enough. And we can stop offtopic and time waste in this thread. If anyone of you do not wish to provide what i said, please be so kind and give me a favor to stay away from commenting in my topics and wasting everyones time.

pan64 03-06-2016 12:50 PM

I'm afraid here, at LQ everyone is so stupid, they cannot understand you. Thanks for your valuable input.

postcd 03-06-2016 01:16 PM

I hope there will be someone who can help answer initial question.


update: i updated my initial post with solution i found myself.

TB0ne 03-06-2016 04:34 PM

Quote:

Originally Posted by postcd (Post 5511246)
I hope there will be someone who can help answer initial question.

update: i updated my initial post with solution i found myself.

Wow..congratulations on being able to find a solution yourself.

Oh wait...I TOLD YOU about xargs in my VERY FIRST REPLY. Congratulations again on your efforts.

postcd 03-06-2016 04:43 PM

Yes, your post was useless for me. Fortunatelly i was able to find ready made command my own way.

TB0ne 03-06-2016 05:29 PM

Quote:

Originally Posted by postcd (Post 5511336)
Yes, your post was useless for me. Fortunatelly i was able to find ready made command my own way.

It was so useless, you wound up using the command that I handed you, to do it 'your own way'. Again, congratulations.

TB0ne 03-06-2016 05:34 PM

Quote:

Originally Posted by postcd (Post 5511226)
you do not know what is helpfull when you ask the question?
an answer that can help you solve the issue.
prefferably example with comment on how that works. are you that stupid not to understand this. I hope i explained enough. And we can stop offtopic and time waste in this thread. If anyone of you do not wish to provide what i said, please be so kind and give me a favor to stay away from commenting in my topics and wasting everyones time.

...except for the sample scripts provided, line-by-line explanations, requests for information which you NEVER provided...everyone was NO HELP at all to you, right???

Grow up. You showed no effort here, any more than you have in many of your other threads. When asked for information and examples of YOUR OWN EFFORTS, you showed none, and get hostile. I received yet another nasty private message from you, despite me asking you EIGHT TIMES now not to message me any further.

If you want someone to spoon-feed you an answer, with zero effort on your part, then just post things in the LQ Jobs forum, along with how much you'll PAY someone to do your work for you. Otherwise, if you have no interest in learning to stand on your own, and do things for yourself, there is no point in posting.

postcd 03-06-2016 05:37 PM

i will not waste my time reading your stories sorry.
No, i have not used any of your comands knowingly as i ignored your posts except first one that had an script in it which i did not understood as mentioned.

TB0ne 03-06-2016 05:40 PM

Quote:

Originally Posted by postcd (Post 5511357)
i will not waste my time reading your stories sorry.

And apparently won't 'waste your time' reading any documentation either.
Quote:

No, i have not used any of your comands knowingly as i ignored your posts except first one that had an script in it which i did not understood as mentioned.
Which is why you've NEVER LEARNED ANYTHING as of yet. You were told about xargs in post #2. Had you bothered to put ANY effort into learning anything, you'd have had the solution then. And I handed you a script, which you put NO EFFORT into understanding.

You're plain lazy. Period. Stop wasting peoples time here by asking for handouts like a beggar. Show effort, and we'll help you, as not only I have, but pan64, michaelk, and others in the MANY other threads you've opened, where you show no effort.

Grow up.

postcd 03-06-2016 05:41 PM

too long man )
if you need ot tell me something, please use PM, thanks

TB0ne 03-06-2016 05:43 PM

Quote:

Originally Posted by postcd (Post 5511359)
too long man )
if you need ot tell me something, please use PM, thanks

Too lazy to read/understand. And as I've told you many times, STOP PM'ing me. If you're not going to read a thread here, why bother PM'ing you? You'd ignore it, as you have the other messages where I've PM'ed you, telling you to stop.

Want it shorter? You're a lazy bum, asking for handouts. Stop it.

michaelk 03-06-2016 05:56 PM

I am happy you found the answer...
Not sure what you did not understand in the link but here it is plus a similar method using sed.

ip=1.2.3.4
url='http://domain1.com/?something&?ipp=$ip'

newurl=${url/'$ip'/$ip}
echo $newurl

newurl=$( echo $url | sed "s/\$ip/$ip/" )
echo $newurl

And go reread post #13

TB0ne 03-06-2016 08:38 PM

Quote:

Originally Posted by TobiSGD (Post 5511053)
Things you do in a public forum:
- Be kind to each other
- Be on topic
- When you don't understand an answer ask for clarification, in the same way: If the question is not clear enough ask for clarification
- Use all the options the forum software provides you. This includes the Ignore List function.

Things you never do on a public forum (and even real life):
- releasing private messages into public without explicit permission from all involved parties. There is a reason they are called private messages. If you feel you are harassed, insulted, ..., in private messages contact a mod or Jeremy.

I did...and nothing happened. The OP still continued, and his behavior still continues. Look at posting history and comments

jeremy 03-07-2016 11:07 AM

Please get this thread back on topic. postcd, please read http://www.linuxquestions.org/questi...#faq_lqwelcome and let me know if you have any questions or need clarification. While LQ aims to be friendly and welcoming to all, you do need to be able to follow some minimal guidelines if you'd like to continue participating.

--jeremy

TobiSGD 03-07-2016 11:08 AM

This is the last reminder, for both, postcd and TB0ne: Be kind to each other. Don't insult each other. If you feel that the other part of the discussion is unable or not willing to understand you ask yourself if you want to use the Ignore List, a function of the forum software that is there for a reason.
The behavior that both of you have shown in this thread is not wanted here on LQ and if I see something like that in future postings of yours moderator action will follow.

@TB0ne: I will ask Jeremy to look at your reports and to handle the situation.

Since this thread is marked as solved and further discussion seems to lead nowhere it will be closed.


All times are GMT -5. The time now is 03:48 AM.