Loop through all files in a directory (as preliminary step in larger coding project)

EVL · 01-16-2017, 11:34 PM

Hello.

Here's my code so far to loop through all files in a directory structure, including subdirectories.

#!/bin/bash
FILES=$(find home/eric/doxDUP -type f -name '/*')
for f in $FILES
do
echo "Testing"
done

I've found related questions elsewhere on this forum and others, but for some reason it's not working.

It just returns me to the command prompt -- even though there are files in the subdirectories. And for each such file, it should write "Testing" to the standard output.

This is just a step toward a larger coding objective, which will involve changing file dates.

Any assistance greatly appreciated.

Thanks.

EVL

Turbocapitalist · 01-17-2017, 01:08 AM

Take a close look at what you have in the -name option and determine what it will really give you, as opposed to what you want it to give you.

Also, there seems to be a slash missing from the start of the path given.

TheEzekielProject · 01-17-2017, 01:10 AM

Try using code tags to preserve formatting. Try this:

Code:

 #!/bin/bash 
FILES=$(find /home/eric/doxDUP -type f -wholename '/*')
for f in $FILES 
do 
echo "Testing" 
done

Notice the forward slash before "home", and "-wholename" rather than "-name"

aragorn2101 · 01-17-2017, 03:12 AM

Hi,

I tried the following

Code:

#!/bin/bash

FILES=$( find /home/aragorn/Documents -type f -name "*" )

for f in $FILES; do
  echo "Testing $f"
done

exit 0

A few things you should note. The place where find is looking for the files should be verified. You used "home/eric/doxDUP", which will work only if you are in directory / because directory home/ is under /. Then, if you are using the option "-name", the expression following must match a name only, not a path. So, if you use "/*", find will try to match all files having names starting with a slash. If you read the man pages for find, you can learn about options "-wholename" and "-path". Maybe this is what you want. But the above code adapted to your filesystem should work. One last thing, I put a "$f" in the argument to echo so that it prints the file names from FILES.

rtmistler · 01-17-2017, 06:33 AM

As others have suggested, you should debug your find command details first. A great way to do this also is to just perform that find command on the command line to see the results. Then once you have it correct, you can code it within the script.

As others have mentioned, please use [code][/code] tags around code to make it easier to read and retain formatting.

When you say that this is part of a step towards a larger project, it is a good idea to share the intentions of that larger project. Reason for that is because you may be looking to perform operations which are far easier to do in another language, or another means of execution.

There are a number of tools you can use to debug bash scripts, some of them are described here.

You also may wish to check a highly similar question in this thread where the OP was also looking to perform a loop to find files and perform an operation on those files.

grail · 01-17-2017, 06:49 AM

Firstly, if you want ALL files, why both with -name or -wholename at all?

I would also add that all presented solutions will fail if there are any directories or files with white space in them. For this reason, you should use a while loop:

Code:

while read -r filename
do
...
done< <(find /home/aragorn/Documents -type f)

nodir · 01-17-2017, 08:32 AM

http://mywiki.wooledge.org/BashPitfa...8ls_.2A.mp3.29
and
http://mywiki.wooledge.org/UsingFind

So something like the last examples at the UsingFind page (one already given in the previous answer, the other round the lines of:

Code:

# POSIX
find ~/audio_archive/wip -type f -name '*.mp3' -exec sh -c '
  for f; do
    echo "current filename: ${f##*/}"
    (...lots of operations...)
  done
' _ {} +

EVL · 01-17-2017, 12:39 PM

I am working on these suggestions today .. & looking forward to reporting results ..

EVL · 01-17-2017, 02:44 PM

Quote:

Originally Posted by rtmistler

As others have suggested, you should debug your find command details first. A great way to do this also is to just perform that find command on the command line to see the results. Then once you have it correct, you can code it within the script.

There are a number of tools you can use to debug bash scripts, some of them are described here.

Thanks for the suggestions. As regards debugging the for loop,
I have tried many versions of a for loop --
all at the command-line, as you suggested.

For instance, using examples at https://www.cyberciti.biz/faq/bash-for-loop/
I tried

Code:

 
for f in {1 .. 3} do echo "eg $f" done

I tried enumerating 1 2 3 -- & several other variants, including a while loop, omitting $f, etc.
It just hangs, & I press CNTRL-C.

In order to not use a script, I type the above at the command line, without #!/bin/bash. (I hope that's correct.)

Your link to bash scripting for dummies is extremely informative.

Any suggestions much appreciated.

EVL

suicidaleggroll · 01-17-2017, 03:36 PM

Quote:

Originally Posted by EVL

For instance, using examples at https://www.cyberciti.biz/faq/bash-for-loop/
I tried

Code:

 
for f in {1 .. 3} do echo "eg $f" done

Get rid of the spaces between the "1", "..", and "3". You're also missing a couple of semicolons

Code:

for f in {1..3}; do echo "eg $f"; done

Quote:

Originally Posted by EVL

In order to not use a script, I type the above at the command line, without #!/bin/bash. (I hope that's correct.)

That's fine, as long as your default shell is bash.

nodir · 01-17-2017, 03:49 PM

To separate you either use a newline or a semi-colon, just like:

Code:

user$ pwd
/home/user/Programming/Tmp
user$ ls
getopts_rankmirrors  tester

or

Code:

user$ pwd; ls
/home/user/Programming/Tmp
getopts_rankmirrors  tester
user$

Hence the example from the link cyperciti works (it uses no semi-colons, as it uses newlines).

If you want to avoid quite some grief, go straight to mywiki.wooledge.org (or bashhackers). There you will find common problems, just like what grail mentioned in his post. Imho it is better to not get used to it at all, instead of getting rid of it later.

allend · 01-17-2017, 04:05 PM

It is also possible to use the native bash shell functionality provided by setting the globstar shell option rather than invoking 'find'.

Code:

#!/bin/bash

shopt -s globstar
for f in /home/eric/doxDUP/**
do
  [[ -f "$f" ]] && echo "$f is a file"
  [[ -d "$f" ]] && echo "$f is a directory"
done
shopt -u globstar

EVL · 01-30-2017, 11:59 PM

Thanks I got this .. when I have a second, I'll post the code