LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 09-03-2019, 02:10 AM   #1
royrsb
LQ Newbie
 
Registered: Feb 2009
Posts: 3

Rep: Reputation: 0
'sed' with Positional Parameters


I'm new with 'sed' and for sure something still I don't understand yet with it. If you see my output on ">Output..." portion, the new directory still on "source_dir" instead of "dest_dir". You may disregard for the "tar" part, this is just a test script, just for me to understand 'sed' using the Positional Parameters ($1 $2 $3..). Could you shed some light for me please? I really appreciate any help.

File contents (*.txt) in source folder (source_dir):
Code:
/home/test/scripts/sed/source_dir/file1/file1.txt
/home/test/scripts/sed/source_dir/file2/file2/file2.txt
/home/test/scripts/sed/source_dir/file3/file3/file3/file3.txt
Objective:
I want to compress all those *.txt files on a new directory "dest_dir" but with their own sub-directories (according to source folder), like the output below:
Code:
/home/test/scripts/sed/dest_dir/file1/file1.txt
/home/test/scripts/sed/dest_dir/file2/file2/file2.txt
/home/test/scripts/sed/dest_dir/file3/file3/file3/file3.txt
My script:
Code:
#!/bin/sh
input_file_type="txt"

find $1 -name *.$input_file_type | sort | while read in_file
do
        echo ">Input "$in_file

        out_file=$(echo $in_file | sed 's/$1/$2/g')

        echo ">Output "$out_file

        tar -zcvf "$in_file" "$out_file"
done
My Output with the script:
Code:
[test@centoslab sed]$ ./zip_test ~/scripts/sed/source_dir/ ~/scripts/sed/dest_dir/
>Input /home/test/scripts/sed/source_dir/file1/file1.txt
>Output /home/test/scripts/sed/source_dir/file1/file1.txt  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
tar: Removing leading `/' from member names
/home/test/scripts/sed/source_dir/file1/file1.txt
>Input /home/test/scripts/sed/source_dir/file2/file2/file2.txt
>Output /home/test/scripts/sed/source_dir/file2/file2/file2.txt  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
tar: Removing leading `/' from member names
/home/test/scripts/sed/source_dir/file2/file2/file2.txt
>Input /home/test/scripts/sed/source_dir/file3/file3/file3/file3.txt
>Output /home/test/scripts/sed/source_dir/file3/file3/file3/file3.txt  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
tar: Removing leading `/' from member names
/home/test/scripts/sed/source_dir/file3/file3/file3/file3.txt
[test@centoslab sed]$
 
Old 09-03-2019, 03:09 AM   #2
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 1,992

Rep: Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549
lets some vars
Code:
foo1="/path/to/src_dir"
foo2="/path/to/dst_dir"
input="/path/to/src_dir/dir1/dir2/file1.txt"
Code:
<<<$input sed 's/$foo1/$foo2/'
/path/to/src_dir/dir1/dir2/file1.txt
# nothing? why? .. the '' is stopping the expansion of $foo{1,2}
Code:
<<<$input sed s/$foo1/$foo2/
sed: -e expression #1, char 9: unknown option to `s'
# an error, it expanded BUT, look at this
Code:
echo sed s/$foo1/$foo2/
sed s//path/to/src_dir//path/to/dst_dir/
# do you see why sed was confused?

# you don't need to use /
Code:
<<<$input sed s,$foo1,$foo2,
/path/to/dst_dir/dir1/dir2/file1.txt
# I usually opt for [ ( unless I expect [ to be involved )

Code:
#going back to ''
<<<$input sed 's['$foo1'['$foo2'['
# is easier than
# this one, which is ugly
Code:
<<<$input sed s/${foo1//\//\\\/}/${foo2//\//\\\/}/
this is better
Code:
<<<$input sed s/${foo1//'/'/'\/'}/${foo2//'/'/'\/'}/
# https://www.tldp.org/LDP/abs/html/pa...stitution.html
# But normally not ugly, I use it alot
# Infact, I would use it instead of sed here
Code:
echo ${input/$foo1/$foo2}




sed is for files, it can be used on strings, but bash para.Sub. is usually better


also note I didn't use echo | pipe

http://tldp.org/LDP/abs/html/here-docs.html




hope that helps

Last edited by Firerat; 09-03-2019 at 03:19 AM.
 
Old 09-03-2019, 04:03 AM   #3
royrsb
LQ Newbie
 
Registered: Feb 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks Firerat for giving a hand. Seems I made it worked!

Below is the change I made:
Code:
#!/bin/sh
input_file_type="txt"

find $1 -name *.$input_file_type | sort | while read in_file
do
        echo ">Input "$in_file

        out_file=$(echo $in_file | sed 's,'$1','$2',g')

        echo ">Output "$out_file

        tar -zcvf "$in_file" "$out_file"
done
And my output, the destination folder is now come out correctly at last!:
Code:
[test@centoslab sed]$ ./zip_test ~/scripts/sed/source_dir/ ~/scripts/sed/dest_dir/
>Input /home/test/scripts/sed/source_dir/file1/file1.txt
>Output /home/test/scripts/sed/dest_dir/file1/file1.txt
tar: Removing leading `/' from member names
tar: /home/test/scripts/sed/dest_dir/file1/file1.txt: Cannot stat: No such file or directory
------------ snipped ------------
 
Old 09-03-2019, 04:26 AM   #4
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 1,992

Rep: Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549


I assume that is just education tool
normally you wouldn't tar a single file

tar ( Tape ARchive ) is to "ball up" multiple files in to one big file which could be put a a tape ( ask someone really old what tape is in this context )

Code:
tar -zcvf "$in_file" "$out_file"
tar will assume the file after the "f" flag to be the output
so this looks like you have overwritten the in_file with the out_file
( you must have had some files in dst from previous experiments )
# Edit: Nope, tar failed as it could not find
# tar: /home/test/scripts/sed/dest_dir/file1/file1.txt: Cannot stat: No such file or directory
Code:
tar -zcvf "${out_file}.tar.gz"  "$in_file" 
tar -zcv "$in_file" -f "${out_file}.tar.gz"
notice I added the extensions to the outfile

Last edited by Firerat; 09-03-2019 at 04:32 AM.
 
Old 09-03-2019, 08:16 AM   #5
ehartman
Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 898

Rep: Reputation: 470Reputation: 470Reputation: 470Reputation: 470Reputation: 470
Quote:
Originally Posted by royrsb View Post
out_file=$(echo $in_file | sed 's/$1/$2/g')
As you're using single quotes here, the $1 and $2 will not get replaced with their respective script arguments. Use double quotes instead.
 
Old 09-03-2019, 08:36 AM   #6
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 1,992

Rep: Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549
Quote:
Originally Posted by ehartman View Post
As you're using single quotes here, the $1 and $2 will not get replaced with their respective script arguments. Use double quotes instead.
"" will still fail





use "" to stop the shell from splitting due to space
Code:
sed s/this_wants_spaces/this has spaces/
sed s/this_wants_spaces/this\ has\ spaces/
sed "s/this_wants_spaces/this has spaces/"
sed 's/this_wants_spaces/this has spaces/'
both work
'' stops the shell from both splitting and expanding, we already know how to break out of that 's/foo'$foo'/foo'$bar'/'

and escape with \ also works
 
Old 09-04-2019, 03:11 AM   #7
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 1,179

Rep: Reputation: 533Reputation: 533Reputation: 533Reputation: 533Reputation: 533Reputation: 533
sed "s/$1/$2/"
has the further disadvantage that neither $1 nor $2 may contain a / character because it is used as the separator.
Output from find should be processed with a different separator that is unusual in pathnames.
Code:
out_file=$(echo "$in_file" | sed s\`"$1"\`"$2"\`)
or
Code:
out_file=$(echo "$in_file" | sed 's`'"$1"'`'"$2"'`')
More robust is the bash-internal
Code:
out_file=${in_file/$1/$2}
Here(!) the shell parses the / separator before it substitutes $1 and $2.
 
Old 09-04-2019, 03:27 AM   #8
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 1,992

Rep: Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549Reputation: 549
yeah, I used , or [ in my example

a common one @

Code:
sed 's@'$foo'@'$bar'@'
I've also seen | ~
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
HELP! Positional Parameters are giving me a headache! Scott_82 Linux - Newbie 7 05-06-2007 03:32 PM
shell: positional parameters... neidias Programming 1 04-04-2005 02:45 PM
Positional Parameters (arguments) desbyleo Solaris / OpenSolaris 2 10-25-2004 11:51 AM
positional parameters $1 $2 in sed command Warmduvet Programming 12 09-22-2004 03:56 PM
Help with positional parameters Read_Icculus Linux - General 7 11-02-2003 04:23 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration