LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 11-04-2011, 07:42 AM   #1
MMaddoxx
LQ Newbie
 
Registered: Nov 2011
Location: London, UK
Distribution: ubuntu
Posts: 10

Rep: Reputation: Disabled
queries about [:digit:] and cp --verbose effect: GNU shell


Hi,

I'm new to shell scripting.

I have written the shell below to enable a set of otherwise identically-named .png files to share a single directory.
The script extracts two sample identifiers (numeric) from a parent directory name and adds them to the front of the file.png names which reside in a daughter 'Images' folder.

Code:
fldrs=*fastqc

for sdir in $fldrs
do
  echo "looking at $sdir"
  prepend=`expr $sdir : 'SA\([0123456789]*\)'`
  #prepend=`expr $sdir : 'SA\([:digit:]*\)'`
  echo $prepend
  append=`expr $sdir : '.*ETC\(_[0123456789]*\.\)'`
  #append=`expr $sdir : '.*ETC\(_[:digit:]*\.\)'`
  echo $append
  
  for f in `ls $sdir/Images`
  do
    echo "considering $f"
    `cp -v $sdir/Images/$f $sdir/Images/$prepend$append$f`
  done
  
done

It all works but ...

1. the commented-out lines for the [:digit:] pattern match do not work (at all). I expected them to to be equivalent to the preceding lines with the less compact [0123456789] form. What have I missed??

2. despite the fact that the script does what I want, the -v flag gives me verbose messages suggesting otherwise. These occur just after each
Code:
echo "considering $f"
and are of the form
scriptname : line number : $f : not found. Which seems odd, since the script does what I want so the file must have been found. The line number is 24, which is the very last line of the script
Code:
done
. What have I missed?

I'm running Ubuntu 10.04 LTS & Gnome terminal 2.30.2

I have had a good look around this site (and elsewhere) but have been unable to figure out what's wrong. I'd appreciate some help.

Thanks

m
 
Old 11-04-2011, 10:08 AM   #2
berbae
Member
 
Registered: Jul 2005
Location: France
Distribution: Arch Linux
Posts: 540

Rep: Reputation: Disabled
Quote:
Originally Posted by MMaddoxx View Post
1. the commented-out lines for the [:digit:] pattern match do not work (at all). I expected them to to be equivalent to the preceding lines with the less compact [0123456789] form. What have I missed??
You have to use [[:digit:]]

Quote:
Originally Posted by MMaddoxx View Post
2. despite the fact that the script does what I want, the -v flag gives me verbose messages suggesting otherwise. These occur just after each
Code:
echo "considering $f"
and are of the form
scriptname : line number : $f : not found. Which seems odd, since the script does what I want so the file must have been found. The line number is 24, which is the very last line of the script
Code:
done
. What have I missed
You don't need the backquotes around the cp command line, because with them, the shell will want to execute the command output and fails, which results in the errors you see.

So here is the modified code:
Code:
fldrs=*fastqc

for sdir in $fldrs
do
  echo "looking at $sdir"
  prepend=$(expr $sdir : 'SA\([[:digit:]]*\)')
  echo $prepend
  append=$(expr $sdir : '.*ETC\(_[[:digit:]]*\.\)')
  echo $append
  
  for f in $sdir/Images/*
  do
    echo "considering $f"
    cp -v "$sdir/Images/$f" "$sdir/Images/$prepend$append$f"
  done
  
done
I replaced the backquotes with the preferable $(...) syntax.

Last edited by berbae; 11-04-2011 at 10:18 AM.
 
1 members found this post helpful.
Old 11-04-2011, 10:30 AM   #3
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,428

Rep: Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876
Well my first question back is ... which shell? You appear to not define an interpreter at the beginning and the default in Ubuntu is dash (so this may complicate some of the features you are trying to
use).

As for the script:

1. How are you guaranteeing that all 'sdir' found are directories?

# We will assume number 1 is true, ie all dirs at top level

2. [0123456789], [0-9] and [[:digit:]] are all equivalent character lists

3. Parsing ls is generally a bad idea as pointed out here

4. Most probably reason for error is spacing in file names which the for loop will perform word splitting.

5. As per point 1, there is no guarantee you are processing files in the inner for loop.

6. Not sure why you are using process substitution (``) for the copy line when you are not returning the data to anything

7. Assuming you do have unusual characters in file and / or directory names then you should quote your variables being used in the copy
 
0 members found this post helpful.
Old 11-05-2011, 02:12 PM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Just to make it clear, the named character class as enclosed by [::] is equal to a predefined set of characters. This is separate from the [] character range expression. You generally use the first one inside the second one.

So [:digit:] is equal to 0-9, and [[:digit:]] is equal to [0-9].

http://mywiki.wooledge.org/RegularExpression

By the way, the use of expr is generally unnecessary in most modern shells. Almost everything that it can do is now built-in in some form or other.

Here are a few ways to extract a string of digits using only shell built-ins (with a guess about your directory name format).

Code:
sdir='SA1234FOOBAR'

prepend="${sdir#SA}"
prepend="${prepend%%[^[:digit:]]*}"

echo "$prepend"
This uses standard parameter expansions supported by all posix shells (see link below).

In fact, if there's only a single string of digits in the name, you could even use this:
Code:
prepend="${sdir//[^[:digit:]]}"
If you have a fairly modern version of bash (v3+), you can also use this:
Code:
re='SA([[:digit:]]*)'
[[ $sdir =~ $re ]] && echo prepend="${BASH_REMATCH[1]}"
The [[ test can be used to apply a regex to the string, with the matched substring and any captures being held in the BASH_REMATCH array. Note that it's generally advisable to store the regex in a separate variable, to avoid having to backslash reserved characters on the right-hand-side of the expression (the regex as a whole needs to be unquoted in order to work).

They may take an extra line or two of code, but since everything is done internally, they should be more efficient than calling an external tool like expr.

parameter expansion
string manipulation
 
1 members found this post helpful.
Old 11-09-2011, 02:57 PM   #5
MMaddoxx
LQ Newbie
 
Registered: Nov 2011
Location: London, UK
Distribution: ubuntu
Posts: 10

Original Poster
Rep: Reputation: Disabled
thanks to all responders!

@David the H. Thank you; your explanations were very clear, and the links inserted very useful
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Substitue single-digit, two-digit, and 3-digit numbers with text using sed dmason165 Programming 13 08-07-2009 10:38 AM
Does setting of BASH as the default shell has any effect? msgforsunil Linux - Newbie 7 10-22-2006 10:41 PM
want perl to output shell commands verbose selgan Programming 2 02-19-2006 03:44 PM
Effect of splash=verbose in menu.lst tdonoughue Suse/Novell 6 03-03-2005 09:00 PM
MySQL queries from Shell - unexpected result philipz Programming 5 05-04-2004 05:38 PM


All times are GMT -5. The time now is 11:23 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration