LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-16-2017, 11:34 PM   #1
EVL
LQ Newbie
 
Registered: Jan 2017
Posts: 12

Rep: Reputation: Disabled
Loop through all files in a directory (as preliminary step in larger coding project)


Hello.

Here's my code so far to loop through all files in a directory structure, including subdirectories.

#!/bin/bash
FILES=$(find home/eric/doxDUP -type f -name '/*')
for f in $FILES
do
echo "Testing"
done

I've found related questions elsewhere on this forum and others, but for some reason it's not working.

It just returns me to the command prompt -- even though there are files in the subdirectories. And for each such file, it should write "Testing" to the standard output.

This is just a step toward a larger coding objective, which will involve changing file dates.

Any assistance greatly appreciated.

Thanks.

EVL

Last edited by EVL; 01-16-2017 at 11:40 PM. Reason: to clarify
 
Old 01-17-2017, 01:08 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
Take a close look at what you have in the -name option and determine what it will really give you, as opposed to what you want it to give you.

Also, there seems to be a slash missing from the start of the path given.
 
Old 01-17-2017, 01:10 AM   #3
TheEzekielProject
Member
 
Registered: Dec 2016
Distribution: arch
Posts: 664

Rep: Reputation: 190Reputation: 190
Try using code tags to preserve formatting. Try this:
Code:
 #!/bin/bash 
FILES=$(find /home/eric/doxDUP -type f -wholename '/*')
for f in $FILES 
do 
echo "Testing" 
done
Notice the forward slash before "home", and "-wholename" rather than "-name"

Last edited by TheEzekielProject; 01-17-2017 at 01:11 AM.
 
Old 01-17-2017, 03:12 AM   #4
aragorn2101
Member
 
Registered: Dec 2012
Location: Mauritius
Distribution: Slackware
Posts: 553

Rep: Reputation: 277Reputation: 277Reputation: 277
Hi,

I tried the following

Code:
#!/bin/bash

FILES=$( find /home/aragorn/Documents -type f -name "*" )

for f in $FILES; do
  echo "Testing $f"
done

exit 0
A few things you should note. The place where find is looking for the files should be verified. You used "home/eric/doxDUP", which will work only if you are in directory / because directory home/ is under /. Then, if you are using the option "-name", the expression following must match a name only, not a path. So, if you use "/*", find will try to match all files having names starting with a slash. If you read the man pages for find, you can learn about options "-wholename" and "-path". Maybe this is what you want. But the above code adapted to your filesystem should work. One last thing, I put a "$f" in the argument to echo so that it prints the file names from FILES.
 
Old 01-17-2017, 06:33 AM   #5
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,354
Blog Entries: 13

Rep: Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411Reputation: 4411
As others have suggested, you should debug your find command details first. A great way to do this also is to just perform that find command on the command line to see the results. Then once you have it correct, you can code it within the script.

As others have mentioned, please use [code][/code] tags around code to make it easier to read and retain formatting.

When you say that this is part of a step towards a larger project, it is a good idea to share the intentions of that larger project. Reason for that is because you may be looking to perform operations which are far easier to do in another language, or another means of execution.

There are a number of tools you can use to debug bash scripts, some of them are described here.

You also may wish to check a highly similar question in this thread where the OP was also looking to perform a loop to find files and perform an operation on those files.

Last edited by rtmistler; 01-17-2017 at 07:13 AM.
 
Old 01-17-2017, 06:49 AM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,821

Rep: Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084Reputation: 3084
Firstly, if you want ALL files, why both with -name or -wholename at all?

I would also add that all presented solutions will fail if there are any directories or files with white space in them. For this reason, you should use a while loop:
Code:
while read -r filename
do
...
done< <(find /home/aragorn/Documents -type f)
 
Old 01-17-2017, 08:32 AM   #7
nodir
Member
 
Registered: May 2016
Posts: 222

Rep: Reputation: Disabled
http://mywiki.wooledge.org/BashPitfa...8ls_.2A.mp3.29
and
http://mywiki.wooledge.org/UsingFind

So something like the last examples at the UsingFind page (one already given in the previous answer, the other round the lines of:
Code:
# POSIX
find ~/audio_archive/wip -type f -name '*.mp3' -exec sh -c '
  for f; do
    echo "current filename: ${f##*/}"
    (...lots of operations...)
  done
' _ {} +

Last edited by nodir; 01-17-2017 at 08:35 AM.
 
Old 01-17-2017, 12:39 PM   #8
EVL
LQ Newbie
 
Registered: Jan 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
Thanks to all who replied ..

I am working on these suggestions today .. & looking forward to reporting results ..
 
Old 01-17-2017, 02:44 PM   #9
EVL
LQ Newbie
 
Registered: Jan 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
Tried debugging for-loop at command-line, as suggested

Quote:
Originally Posted by rtmistler View Post
As others have suggested, you should debug your find command details first. A great way to do this also is to just perform that find command on the command line to see the results. Then once you have it correct, you can code it within the script.

There are a number of tools you can use to debug bash scripts, some of them are described here.
Thanks for the suggestions. As regards debugging the for loop,
I have tried many versions of a for loop --
all at the command-line, as you suggested.

For instance, using examples at https://www.cyberciti.biz/faq/bash-for-loop/
I tried
Code:
 
for f in {1 .. 3} do echo "eg $f" done
I tried enumerating 1 2 3 -- & several other variants, including a while loop, omitting $f, etc.
It just hangs, & I press CNTRL-C.

In order to not use a script, I type the above at the command line, without #!/bin/bash. (I hope that's correct.)

Your link to bash scripting for dummies is extremely informative.

Any suggestions much appreciated.

EVL
 
Old 01-17-2017, 03:36 PM   #10
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138Reputation: 2138
Quote:
Originally Posted by EVL View Post
For instance, using examples at https://www.cyberciti.biz/faq/bash-for-loop/
I tried
Code:
 
for f in {1 .. 3} do echo "eg $f" done
Get rid of the spaces between the "1", "..", and "3". You're also missing a couple of semicolons
Code:
for f in {1..3}; do echo "eg $f"; done
Quote:
Originally Posted by EVL View Post
In order to not use a script, I type the above at the command line, without #!/bin/bash. (I hope that's correct.)
That's fine, as long as your default shell is bash.
 
Old 01-17-2017, 03:49 PM   #11
nodir
Member
 
Registered: May 2016
Posts: 222

Rep: Reputation: Disabled
To separate you either use a newline or a semi-colon, just like:
Code:
user$ pwd
/home/user/Programming/Tmp
user$ ls
getopts_rankmirrors  tester
or
Code:
user$ pwd; ls
/home/user/Programming/Tmp
getopts_rankmirrors  tester
user$
Hence the example from the link cyperciti works (it uses no semi-colons, as it uses newlines).

If you want to avoid quite some grief, go straight to mywiki.wooledge.org (or bashhackers). There you will find common problems, just like what grail mentioned in his post. Imho it is better to not get used to it at all, instead of getting rid of it later.
 
Old 01-17-2017, 04:05 PM   #12
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 5,656

Rep: Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213Reputation: 2213
It is also possible to use the native bash shell functionality provided by setting the globstar shell option rather than invoking 'find'.
Code:
#!/bin/bash

shopt -s globstar
for f in /home/eric/doxDUP/**
do
  [[ -f "$f" ]] && echo "$f is a file"
  [[ -d "$f" ]] && echo "$f is a directory"
done
shopt -u globstar

Last edited by allend; 01-17-2017 at 04:09 PM.
 
Old 01-30-2017, 11:59 PM   #13
EVL
LQ Newbie
 
Registered: Jan 2017
Posts: 12

Original Poster
Rep: Reputation: Disabled
Thanks I got this .. when I have a second, I'll post the code
 
  


Reply

Tags
for loop, recursion


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to apply 'for loop' recursively to files in a directory? mangya Linux - Newbie 9 10-24-2015 02:46 AM
[SOLVED] Script count files in the directory but keep loop not exit untill see 10 files. dotran Linux - Newbie 9 12-22-2014 04:34 AM
Putting user files in larger directory adocholiday Linux - Newbie 1 02-22-2012 08:25 PM
[SOLVED] PHP File open larger files or fopen files larger than 2gigs Nemus Linux - Server 6 02-18-2011 01:03 AM
loop move files up one directory level Melsync Programming 8 12-19-2005 10:51 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration