LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Bash, LS, For loops, and whitespaces in directories (https://www.linuxquestions.org/questions/programming-9/bash-ls-for-loops-and-whitespaces-in-directories-126388/)

jhrbek 12-16-2003 02:45 PM

Bash, LS, For loops, and whitespaces in directories
 
Hi everyone. I have a problem that I have been unable to solve. I need to write a bash script that will recurse through a directory structure and write whatever directories names it finds to a file. Below is an example of how I am doing it. I have an embedded for loop structure that recurses to a depth that I need. This example is just a simple version of what I wrote.

What I know:

I know that "for" uses whitespaces as a delimeter and automatically splits up directories that have spaces in them. This is bad. I also know that spaces in directories is bad, but I can't change that. They are apart of an imap mail spool and spaces in the directory names was deemed "necessary" by my boss. :( I have been told that I should be able to accomplish what I need with a well constructed use of the "find" command and a while statement, but I have been unable to find/create a working model.

The code below works for directories that do not have a whitespace, but will not include those that do.

If I had a directory like:

trash/
sent items/
save/

it would print the results as such:

I found trash and it is a directory
I found save and it is a directory

If I "echo" all of the results it will print:

trash/
sent
items/
save/


Any help would be appreciated, thanks!


Code:

#!/bin/bash

# Build our directory list

for i in `ls`
 do
 if [ -d $i ]
  then
  echo "I found $i and it is a directory"
 fi
done

please help :confused:

Bebo 12-16-2003 02:59 PM

Hi,

find <dir> -type d

will give you a list of the directories in <dir>, recursively. Then you can just redirect the output to a file.

HTH

jhrbek 12-16-2003 03:10 PM

Thanks for the reply. Here is what I have so far, the complicated one. If I change directory to one of the sub directories, and issue:

find . -type d -name '* *' -o -type d -name '*'

It works great, returning both normal directories and directories with spaces. However, because the for loop chops things up based on white space, I'm getting hosed there. I don't know how else to do this without a for loop and still preserve the usernames that are associated with the directory structure (I obtain the usernames from the directory names).

Here is an example of the output from my script:

message folder found, saving: ./j/user/jjacobsen^foo^com/Web
message folder found, saving: Site
message folder found, saving: Stuff
message folder found, saving: ./j/user/jkelch^foo^com/Mail
message folder found, saving: ./j/user/jkeller^foo^com/News
message folder found, saving: ./j/user/jlancaster^foo^com/News

Note that the "jjacobsen^foo^com/Web" should really be "jjacobsen^foo^com/Web Site Stuff"

Code:

for i in `ls`
 do
 if [ -d $i/user ]
  then
  for j in `ls $i/user`
    do
    echo -e "user.$j\tdefault\t${j//^/.}\tlrswipcda\t$acluser\tlrswipcda" >>/tmp/newmboxlist.txt
      : $[users++]
      for k in `find . -type d -name '* *' -o -type d -name '*'`
      do
          echo "message folder found, saving: $k"
          echo -e "user.$j.$k\tdefault\t${j//^/.}\tlrswipcda\t$acluser\tlrswipcda" >>/tmp/newmboxlist.txt
      done
    done
 fi
done


Bebo 12-16-2003 03:20 PM

Aha... If you could somehow exchange the spaces in the dirnames to the regexp of space (backslash-space), so that the parsing gets right? Use something like tr ' ' '\ ' or maybe even tr '\ ' '\\\ '?

Bebo 12-16-2003 03:22 PM

Or... Since you know that the proper dirname should start with ./ then you can just append the strings that don't start with that to the former dirname?

Edit: a bit ugly, but... ;)


jhrbek 12-16-2003 03:35 PM

Well, getting a list and writing it to a file isn't a problem. But I need to be able to iterate through each result and insert that result into a larger string:

Here is what the final output must be:

user.bhartline^foo^com default bhartline.foo.com lrswipcda cyrus lrswipcda
user.bhartline^foo^com.Drafts default bhartline.foo.com lrswipcda cyrus lrswipcda
user.bhartline^foo^com.Sent Items default bhartline.foo.com lrswipcda cyrus lrswipcda

I would just normally rename the directories and be done with it but because it is for email, I can't do that w/o making people mad at me. :D

The echo string that creates the above output is:

echo -e "user.$j.$k\tdefault\t${j//^/.}\tlrswipcda\t$acluser\tlrswipcda" >>/tmp/newmboxlist.txt

where:

$j is an iteration of a letter of the alphabet, a, b, c, etc. It gets it's info from the output of an LS command.

\t is a special character I need to delinate the data.

$j is the name of the drectory holding the user's mail data

${j//^/.} is a regexp rewrite of the user^foo^com to user.foo.com, which is also an iteration of the output from another LS command

$k is any subdirectories found in $j

Hope that makes sense. :D

Will your regexp pattern prevent the for loop from cutting up the spaces in the directory names?

Would it be easier to write this info to a file, then open it again and use a while loop, eg while not EOF? I think this would be complicated because I would have to do this for every letter of the alphabet and every username under that letter. Gerr.

By the way, the ./ you mentioned is a product of my find command (i think). I'll try to implement your suggestion and see if it helps. :)

jhrbek 12-16-2003 03:39 PM

Quote:

Or... Since you know that the proper dirname should start with ./ then you can just append the strings that don't start with that to the former dirname?
This actually might work. I would have to be able to access the for loop's previous iteration though and I don't know how to do that. Hmm. I'll have to see if I can store them in an array or something. I'm new to BASH so i'll have to check into the array structure.

Bebo 12-16-2003 03:44 PM

Yeah, that's some horrible echo! :D

The ./ is, as you say, the result of the find command. BTW, you don't have to use that very long find: find . -type d -name '* *' -o -type d -name '*'. It should be enough with find . -type d, but then again, it might just be a result of the contents in the dir you're looking in :)

I was just trying to get some use of my tr commands, but to no avail. sed should be better: find . -type d | sed s/' '/'\\ '/g should be better.

jhrbek 12-16-2003 03:51 PM

Yeah, the echo statement is evil, but necessary.

Even if I changed my find command, how would I iterate through it though? The iteration is splitting the directories, not the find. I've only used sed once, and it was an example to learn bash, so I didn't really understand what I was doing. :) I'll keep hacking at it though.

Bebo 12-16-2003 04:18 PM

Yeah, you're right about that the iteration splits the directories. I see that now - sorry :) And my sed pipe didn't work either. Nothing I do today works... bwaaah! :cry: ;)

One last try to help you before I go and stand in the corner... To implement the ./ check, you can append the dirnames to one another until the next dirname starts with ./. Well, that's not helping very much, I guess...

(Well, what do you know, this is my 200th post! :))

jhrbek 12-16-2003 04:22 PM

Bebo, I appreciate your help!

Do you know anything about arrays with bash? As I read the bash documentation I find it lacking, especially in the area of examples.

I tried:

Code:

declare -a folders
read -a folders | find . -type d -name '* *' -o -type d -name '*'
#find . -type d -name '* *' -o -type d -name '*' | read -a folders # doesn't work either
echo ${folders[@]} #prints array

and that didn't work. It only seems to work if i manually type something in. The docs say read separates array values by whitespace, so if I could get this to work, I would be able to do the append operation with relative ease. Regardless, you would think that I could get at least 1 array element with all of the returned data.

this works, but does not address my need:

Code:

declare -a folders
read -a folders
echo ${folders[@]} #prints array

Ideas?
:scratch:

jhrbek 12-16-2003 05:41 PM

Almost there! All I need to do now is append the offending values and i've got it. Thanks for your help!

-j

Code:

declare -a folders

folders=(`find . -type d -name '* *' -o -type d -name '*'`)

arrSize=${#folders[@]} #array size of "folders"

for i in `seq 0 $arrSize`
 do
  if [ ! -d ${folders[$i]} ]
  then
    echo ${folders[$i]}
  fi
done


Bebo 12-16-2003 05:43 PM

I don't know anything about arrays, I'm afraid.

The last hour I've tried to solve the splitting problem with the continue statement, but I'm apparantly too stupid today, 'cause I can't even get the logic tests to work.

Bebo 12-16-2003 05:52 PM

Aha, great! I didn't see your last post before I wrote my previous one :) Now I can use your solution too :D

Bebo 12-16-2003 06:19 PM

OK, now I have a solution that can be used if one uses ls instead of find:

Code:

#!/bin/bash

unset name

for i in `ls -1F | grep \/$` ; do
    if test ! $name ; then
        name=$i
    else
        name=`echo $name $i`
    fi

    test `echo $name | rev | cut -c1` != \/ && continue

    echo "Here is your directory: $name"

    unset name
done



All times are GMT -5. The time now is 10:21 AM.