SED with a string

Entropy1024 · 09-12-2018, 07:07 AM

I can extract line one from a text file and place it into the string $LINE using:

Code:

LINE=$(sed -n 1p myfile.txt)

In my script I want to iterate through all the lines and extract one line at a time to do some work on them.
I can't figure out how to enter a string holding a variable into this bit of code. I was expecting something like the line below to work but it fails.

Code:

LINE=$(sed -n $LINENUMBER p myfile.txt)

What is the correct syntax?

Many thanks for any help.

Guttorm · 09-12-2018, 07:16 AM

Hi

You're close, but there will be a space between 1 and p. Try {} around the variable name, and remove the space.

Code:

LINE=$(sed -n ${LINENUMBER}p myfile.txt)

Entropy1024 · 09-12-2018, 07:19 AM

Quote:

Originally Posted by Guttorm

Hi

You're close, but there will be a space between 1 and p. Try {} around the variable name, and remove the space.

Code:

LINE=$(sed -n ${LINENUMBER}p myfile.txt)

Unfortunately that fails also. What should the {} do?

Cheers

Hko · 09-12-2018, 07:22 AM

However, if you want to iterate over all line in a file, it is not the best way to do it. Here's a much cleaner and faster way to iterate over all lines of a file:

Code:

while read line ; do
    echo "== work on this this line: $line"
done < myfile.txt

Guttorm · 09-12-2018, 07:27 AM

Are you sure LINENUMBER has a value? Using ${LINENUMBER} just means that the variable name is $LINENUMBER and not $LINENUMBERp, so the result will be something like 1p and not 1 p.

Put the command "set -x" in the beginning of your script, and you will get debug information and can see what is happening.

Hko · 09-12-2018, 07:31 AM

Quote:

Originally Posted by Entropy1024

Unfortunately that fails also. What should the {} do?

For sed, the line number and the 'p' cannot be seperated by space. And this would not work:

Code:

LINE=$(sed -n $LINENUMBERp myfile.txt)   #### WRONG

...because the shell would read the variable as LINENUMBERp, including the 'p'.

So, in bash, this can be solved to put {} around LINENUMBER to indicate what exactly the variable name is.
Are you using bash to run the script?

Anyway, don't iterate over lines in a file this way. See my other post above.

Entropy1024 · 09-12-2018, 08:04 AM

Quote:

Originally Posted by Hko

For sed, the line number and the 'p' cannot be seperated by space. And this would not work:

Code:

LINE=$(sed -n $LINENUMBERp myfile.txt)   #### WRONG

...because the shell would read the variable as LINENUMBERp, including the 'p'.

So, in bash, this can be solved to put {} around LINENUMBER to indicate what exactly the variable name is.
Are you using bash to run the script?

Anyway, don't iterate over lines in a file this way. See my other post above.

Yes I'm using BASH.

If I use:

Code:

LINE=$(sed -n {$LINENUMBER}p myfile.txt)
sed: -e expression #1, char 0: unmatched `{'

If I use:

Code:

LINE=$(sed -n ${$LINENUMBER}p myfile.txt)
line 57: ${$LINENUMBER}p: bad substitution

If I use:

Code:

LINE=$(sed -n {LINENUMBER}p myfile.txt)
sed: -e expression #1, char 3: unmatched `{'

What is thew correct syntax please?

Hko · 09-12-2018, 08:11 AM

Quote:

Originally Posted by Entropy1024

What is thew correct syntax please?

The correct syntax is the one Guttorm posted.

Entropy1024 · 09-12-2018, 08:18 AM

Quote:

Originally Posted by Hko

The correct syntax is the one Guttorm posted.

If you are refering to:

Code:

LINE=$(sed -n ${LINENUMBER}p myfile.txt)

It returns with:
sh: 1: myfile.txtp: not found

Hko · 09-12-2018, 08:20 AM

Quote:

Originally Posted by Entropy1024

If you are refering to:

Code:

LINE=$(sed -n ${LINENUMBER}p myfile.txt)

It returns with:
sh: 1: myfile.txtp: not found

It shouldn't.
It is correct.

Guessing from the error message you posted, renaming your file to myfile.txtp might make it work though.

If not, what does this output on your system?

Code:

echo ${LINENUMBER}

Entropy1024 · 09-12-2018, 09:08 AM

Quote:

Originally Posted by Hko

It shouldn't.
It is correct.

Guessing from the error message you posted, renaming your file to myfile.txtp might make it work though.

If not, what does this output on your system?

Code:

echo ${LINENUMBER}

I can see where you are going with concatenating a p to the filename. However would like to know why the correct solution is not working.

Code:

echo ${LINENUMBER}

returns a line within the myfile.txt

Hko · 09-12-2018, 09:11 AM

Quote:

Originally Posted by Entropy1024

Code:

echo ${LINENUMBER}

returns a line within the myfile.txt

What exactly?
A number?

Hko · 09-12-2018, 09:21 AM

Here's an entire script as a Proof Of Concept.

Copy-Paste into your editor.
Save as poc.sh.
Make executable: chmod +x poc.sh
Run it: ./poc.sh

Code:

#!/bin/bash

cat <<EOF >myfile.txt
line one
line two
line three
EOF

LINENUMBER=2
LINE=$(sed -n ${LINENUMBER}p myfile.txt)

echo "LINENUMBER = $LINENUMBER"
echo "LINE = $LINE"

Output is:

Code:

LINENUMBER = 2
LINE = line two

If it is not, check your operating system, and bash shell etc...

Entropy1024 · 09-12-2018, 09:23 AM

Quote:

Originally Posted by Hko

What exactly?
A number?

No. I understand whats going wrong now.
Originally I had a loop from 1-64, as the file always has 64 lines in it. Then was trying to get each one of the lines extracted using sed.
So I was passing this number to SED in the "sed -n $line p" which did not work. However I think at that point "sed -n ${line}p" would have worked. However I followed an earlier suggestion from Hko to use "while read line ; do" command.
Now this command was sending the actual line to that value and so sed was receiving a line of text and not a number.

Makes sense now.
Thanks for your help.

MadeInGermany · 09-12-2018, 10:10 PM

Each call to sed must open the file.
The smart alternative was stated in post#4:
The file is opened once for the while-do-done block, and each call to read reads a new line into a variable.

Code:

while read line

reads a line into the variable $line, tests the exit status, and ends the loop if there is no more line (eof).
The whole loop is redirected from the file.
If you do not want a loop, you can redirect a { code block } like this

Code:

{
for demo in 1 2 3
do
  if read line
  then
    echo "line#$demo is $line"
  else
    echo "eof"
  fi
done
} < myfile.txt

You can take an explicit file descriptor 3 or 4 or ... to avoid a conflict with stdin

Code:

{
...
  read line <&3
...
} 3< myfile.txt

If you do not want to have a code block then you can associate a descriptor with exec

Code:

exec 3< myfile.txt
...
for demo in 1 2 3
do
  if read line <&3
  then
    echo "line#$demo is $line"
  else
    echo "eof"
  fi
done