LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Inserting a new line at a specific row with a tab in bash script using sed, along with env variable (https://www.linuxquestions.org/questions/linux-newbie-8/inserting-a-new-line-at-a-specific-row-with-a-tab-in-bash-script-using-sed-along-with-env-variable-4175672531/)

296.saurabh 04-04-2020 05:54 AM

Inserting a new line at a specific row with a tab in bash script using sed, along with env variable
 
I have a part of json which looks like below:
Code:

{
  "openstack": {
    "admin": {
      "username": "admin",
      "password": "password",
      "tenant_name": "test"
    },

I want to add specific env variables at row no 3, 4, and 5 like $auth_url, $region_name and $endpoint Something like:
Code:

{
  "openstack": {
    "auth_url": $auth_url,
    "region_name": $region_name,
    "endpoint_type": $endpoint,
    "admin": {
      "username": "admin",
      "password": "password",
      "tenant_name": "test"
    },

How can I do this in bash with sed, Here $auth_url and other $ are basically env variables that are to be replaced.
Using
set -ex
sed -e '3i\\t"auth_url":$AUTH,' -i account_2.json
sed -e '4i\\t"region_name":$REGION,' -i account_2.json
sed -e '5i\\t"endpoint_type":$ENDPOINT,' -i account_2.json

It is not replacing $auth_url with env variables

pan64 04-04-2020 06:06 AM

Please use code tags to post your script, otherwise hard to read and understand your post.
I would rather suggest you to do it in python, there is no reliable way to edit json files with sed.
From the other hand:
Code:

sed -e '3i\\t"auth_url":$AUTH,' -i account_2.json
# the quoting is not correct (that's why it does not work)
sed -e '3i\\t"auth_url":'$AUTH',' -i account_2.json

probably better

syg00 04-04-2020 06:09 AM

Single quotes inhibit interpolation - first suggestion would be to get rid of them, certainly as outer-most quotes. Things can get really ugly with quotes and escaped characters (that tab for example). Trial and error is the best teacher.

:doh: ... too slow again ... :eek:

Turbocapitalist 04-04-2020 06:15 AM

Python or perl would be the right way to do it. JSON is complex enough that it must be parsed in order to acheive correct results. Otherwise if a line or even sometimes a space is off, you'll just break the output.

However if you are writing a script in sed, and using the -i option, then you should feed the -i option a string so that a backup file is saved with that string tacked onto the end of the old name.

Code:

sed -i.orig -e '...'
Or

Code:

sed -i.bak -e '...'
Or anything. But if you leave it blank, then -i will overwrite the original file without leaving a copy of the original.

296.saurabh 04-04-2020 07:07 AM

I was able to get it going till I dont use, (comma). once I use it to add to sed like below:
Code:

sed "3i\    \"auth_url\":$OS_AUTH_URL,\n    \"region_name\":$OS_REGION_NAME,\n    \"endpoint_type\":$OS_INTERFACE," -i "${ACCOUNTS_JSON}"
It gives an error:
sed: -e expression #1, char 45: unknown command: `,'

Turbocapitalist 04-04-2020 07:19 AM

Quotes are read in series by the shell, not from the outside. Read them from left to right. The first one starts, the next one stops, the one after that starts again, then one after that stops again, and so on. So if you want to put double quotes inside of a pair of double quotes, the inner set has to be escaped:

https://mywiki.wooledge.org/Quotes

The dollar signs will also have to be escaped when inside double quotes.

Which shell are you using and which distro is this on?

296.saurabh 04-04-2020 07:30 AM

I am using bash and sed with versions
Code:

aq@9f766bdee2e2:~$ sed --version
sed (GNU sed) 4.4

aq@9f766bdee2e2:~$ bash --version
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.

I mistakenly left back slash while copying the sed command, corrected now

Turbocapitalist 04-04-2020 07:39 AM

What is inside the quotes and what is outside of them?

Code:

sed "3i\    "auth_url":$OS_AUTH_URL,\n    "region_name":$OS_REGION_NAME,\n    "endpoint_type":$OS_INTERFACE," -i "${ACCOUNTS_JSON}"
The parts outside of the quotes will be parsed by the shell. Any escapes \\ or dollar signs $ will also be interpreted by the shell inside double quotes.

296.saurabh 04-04-2020 07:56 AM

How to make this work without Error
Code:

sed "3i\    \"auth_url\":$OS_AUTH_URL,\n    \"region_name\":$OS_REGION_NAME,\n    \"endpoint_type\":$OS_INTERFACE," -i "${ACCOUNTS_JSON}"
Error:
Code:

+ sed 3i\    "auth_url":https://identity-nc,\n    "region_name":test,\n    "endpoint_type":public, -i /home/aqua/accounts_automated.json:
sed: -e expression #1, char 45: unknown command: `,'


shruggy 04-04-2020 08:05 AM

How about
Code:

jq "{openstack:{auth_url:\"$OS_AUTH_URL\",region_name:\"$OS_REGION_NAME\",endpoint_type:\"$OS_INTERFACE\"}}*." <"$ACCOUNTS_JSON"

pan64 04-04-2020 08:52 AM

probably you can get better result with (something like):
Code:

LINE="\    \"auth_url\":$OS_AUTH_URL,\n    \"region_name\":$OS_REGION_NAME,\n    \"endpoint_type\":$OS_INTERFACE,"
sed "3i$LINE" -i file

(not tested)
but as it was mentioned this is not really a task for sed. Use perl/python or jq as it was already suggested.

296.saurabh 04-04-2020 08:58 AM

@shruggy: This works, thank you


All times are GMT -5. The time now is 05:36 AM.