LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-15-2017, 04:36 PM   #1
atjurhs
Member
 
Registered: Aug 2012
Posts: 220

Rep: Reputation: Disabled
sed for relative path to a literal path starting at the top of the directory structure in a text file


hi guys,

i have a text file that contains several lines of paths to several files located on my system, with each path on a newline. the paths are relative to my current directory, and i need to make them literal so each of the paths begin at the top level in the directory structure, like this:

Code:
./ABC/more_text.txt
and i want to come out as:
Code:
/topdir/some_text/ABC/more_text.txt
i've been trying to do this with sed commands, and have had some success with the string replacement if the string of the paths do not contain a ./ like this

Code:
ABC/more_text.txt
and it comes out like this
Code:
topdir/some_text/ABC/more_text.txt
but again the relative strings of paths on each new line really look like this:

Code:
./ABC/more_text.txt
and i want to come out as:
Code:
/topdir/some_text/ABC/more_text.txt
so i've been trying without success:

Code:
sed -i 's/\.\/ABC/\/topdir\/some_text/g' file.txt
and i get from standard i/o
Code:
unknown option to 's
what am i missing?

i've also tried this with awk using the sub command without success.

surely this is a relatively common occurance with file string manipulation

(unfortunately, the file is on a different machine and i'm not able to transfer it to this machijne, but i think i've copied over the exact text)

anyways, thanks for whatever help!

Todd

Last edited by atjurhs; 02-15-2017 at 04:59 PM.
 
Old 02-15-2017, 05:05 PM   #2
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,346
Blog Entries: 11

Rep: Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310Reputation: 3310
If the file paths.txt contains this...
Code:
./ABC/more_text.txt
./DEF/more_text.txt
GHI/more_text.txt
/JKL/more_text.txt
And you want to change any path that does not already begin at the filesystem root, then...

Code:
$ sed '/^[^\/]/s/^\(\.\/\)\?/\/topdir\/some_text\//' paths.txt
/topdir/some_text/ABC/more_text.txt
/topdir/some_text/DEF/more_text.txt
/topdir/some_text/GHI/more_text.txt
/JKL/more_text.txt
I added the address to skip lines already beginning at root and change anything beginning with ./ or a directory name.

I would also suggest never changing files in-place, at least until you know your sed expression works!

Last edited by astrogeek; 02-15-2017 at 05:08 PM. Reason: updated expression - redundant char class
 
Old 02-15-2017, 05:53 PM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,484

Rep: Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098Reputation: 3098
Quote:
Originally Posted by atjurhs View Post
and i get from standard i/o
Code:
unknown option to 's
what am i missing?
Try using a different separator character - it may be more obvious what you have omitted.

As for the broader question, I prefer a KISS approach - have separate stanzas (-e or separate sed commands) for each element of the problem. Makes it easier to nut out, and usually easier to spot what is happening.
What happens if a name such as "../../some/file" appears in the list ?.
 
Old 02-16-2017, 03:47 AM   #4
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 4,444
Blog Entries: 3

Rep: Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211
Also, there is an existing utility, realpath, which can resolve the absolute file name when given a relative name. It takes several options so as to make different priorities when working out the absolute path.

See also the utilities basename and dirname, if you need to strip off either just the path or just the filename.
 
Old 02-16-2017, 10:57 AM   #5
atjurhs
Member
 
Registered: Aug 2012
Posts: 220

Original Poster
Rep: Reputation: Disabled
hi astrogeek,

your sed command work perfectly for the text example that i provided, namely a file with paths that look like this:
Code:
./ABC/more_text.txt
./DEF/more_text.txt
GHI/more_text.txt
/JKL/more_text.txt
however the file has several hundred lines of paths and it turns out that some of the paths have a whitespace in the path (no doubt caused by a dirty rotten PC user), which i didn't find out until attempting to use a paths half way down the output file, and of course things went sideways, so some of the paths will look more like:

Code:
./ABC/more_text (more_texts)-even_more_text.txt
and using your sed command i do get

Code:
/topdir/some_text/ABC/more_text.txt
knowing to get a linux box to read a path with a white space i need to precede the whitespace with a \ so linux ignores the white space, and so i wrote a "tag on" command using a pipe to your command, namely:

Code:
sed '/^[^\/]/s/^\(\.\/\)\?/\/topdir\/some_text\//' pathfile.txt | sed 's/ /\\ /g' > new_pathfile.txt
but the new_pathfile did not contain a \whitespace as part of the result, the results were the same as had i just run your command, namely my output was:

/topdir/some_text/PDQ/more_text(more_texts)-even_more_text.txt[/code]

when i needed to have

/topdir/some_text/PDQ/more_text\ (more_texts)-even_more_text.txt[/code]

so the paths could be read and used by a linux box down the road.

i'm also thinking the maybe a should be using some sort of literal string pattern matching, for example, if i read anywhere in my path string:

./ABC then replace it with
Code:
/topdir/some_text/ABC
this should also work for a string that contains a whitespace, for example in the string

./ABC/more text.txt

pattern matching would find the whitespace between more and text and replace it with

./ABC/more\ text.txt

thank you so very much for your help! note, i save off info in threads like to to ues under similar circumstancse down the road..

Todd
 
Old 02-16-2017, 11:19 AM   #6
masinick
Member
 
Registered: Apr 2002
Location: Clinton Township, MI
Distribution: Debian, antiX, Kubuntu, Xubuntu, and many others
Posts: 611
Blog Entries: 15

Rep: Reputation: 89
I think that the suggestions made by Turbocapitalist are likely to help you in the long run.

In the scripts that I've written, I break down various sections of a pathname and when I do so, I'm able to make effective use of the basename and dirname utilities, though I don't recall taking advantage of the realpath - that would also be a useful tool to utilize. The suggestions from syg00 are also good - keeping variables and code paths simple and easy to debug makes the code easier to maintain years down the road, whether it's your change or someone who maintains or copies your work.

When I use one of my old scripts or grab some public code, when simple methods are used, it is so much easier to integrate the work of others into what I'm doing, so keeping your code simple and well documented benefits both you and others.

So some easy things to get your file and path parsing algorithms rock solid:

1. Use common utilities - basename, dirname, realpath to extract specific file elements
2. Use variables to store subsets of the full path
3. Keep the overall code simple and well-documented. If it's understandable, documenting it is easier, so that's yet another benefit of simplicity.
 
Old 02-16-2017, 12:54 PM   #7
atjurhs
Member
 
Registered: Aug 2012
Posts: 220

Original Poster
Rep: Reputation: Disabled
thanks masinick, but i'm not at all that good of a coder/script writer.

i do agree with you on making it well documented and keeping code public so that others can make use if it and as a backup copy for yourself, and i do reuse my previous codes, embed or hack on them as necessary.

Todd

Last edited by atjurhs; 02-16-2017 at 12:56 PM.
 
Old 02-16-2017, 02:36 PM   #8
Luridis
Member
 
Registered: Mar 2014
Location: Texas
Distribution: LFS 9.0 Custom, Merged Usr, Linux 4.19.x
Posts: 616

Rep: Reputation: 167Reputation: 167
You could go for getting the real path based on the relative path you're already generating. Assuming you have an array of relative paths, including the file name, you can regenerate the text file for each target in whole. With $files as your array...

Code:
for file in $files
do
    targetpath=$( realpath $( dirname $file ) )
    <call function to write out to files, or save to an array until all are ready>
done
dirname gets the path off the file, realpath gets it's absolute position in the filesystem. Note that if you're using links you should read the man pages for those, I've not had them follow links before.

If you're willing to put something unique in the file names, you can use find. Let's say all of these files will have "-MARKER-" in the name. You could use find to generate an array.

Code:
for file in $( find -name *-MARKER-* )
do
    call_update_function($file)
done
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Absloute path vs relative path in Linux/Unix LXer Syndicated Linux News 0 03-15-2016 07:01 PM
Permission Denied when using absolute path, relative path works OK eleventyfour Linux - Newbie 6 07-31-2015 04:56 PM
sed attempting to replace text with directory path emgey Programming 5 04-03-2008 08:05 PM
how to find a relative path to another directory bahadur Programming 9 04-04-2005 08:52 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:33 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration