Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
08-22-2022, 08:32 PM
|
#1
|
Member
Registered: Jun 2017
Posts: 196
Rep: 
|
How to Conditionally Execute Code If File in OS-Walk is Empty in Python MacOS
Code:
import os
for root, dirs, files in os.walk("/Volumes/TM3gbBu/miscx4now/fr6tb24jul22/miscxDox-ddupt-8gb/sort/BUs/doxx"):
for name in files:
print('filename is ', name)
fileSize = os.path.getsize(name)
print('file size is ', fileSize)
. . produces this result . .
Code:
Traceback (most recent call last):
File "testOsWalk.py", line 5, in <module>
fileSize = os.path.getsize(name)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '._.DS_Store'
If I try to determine fileSize with
Code:
fileSize = os.path.getsize(name)
it tells me "No such file or directory"
But it's referring to a file or directory that's THERE.
If I omit the two lines about file size and just include the line before that to print file name,
it prints out all files in the directory, as expected.
In another version of this script, it would print out file size of hidden file but not visible file.
Confused. Help much appreciated.
Last edited by ericlindellnyc; 08-22-2022 at 10:20 PM.
Reason: to clarify
|
|
|
08-22-2022, 09:03 PM
|
#2
|
LQ Guru
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,352
|
As is typical, I'm going to start by being that guy. You've confirmed that it's not a broken link? What do os.path.exists(), os.path.isfile(), and similar things return?
Last edited by dugan; 08-22-2022 at 09:07 PM.
|
|
|
08-22-2022, 09:21 PM
|
#3
|
Moderator
Registered: Aug 2002
Posts: 26,508
|
Code:
for root, dirs, files in os.walk("/Volumes/TM3gbBu/miscx4now/fr6tb24jul22/miscxDox-ddupt-8gb/sort/BUs/doxx", topdown=True):
for name in files:
print(os.path.join(root, name))
filename=os.path.join(root,name)
fileSize = os.path.getsize(filename)
print('file size is ', fileSize)
files does not include the full path.
Last edited by michaelk; 08-22-2022 at 09:46 PM.
|
|
|
08-23-2022, 12:20 AM
|
#4
|
Member
Registered: Jun 2017
Posts: 196
Original Poster
Rep: 
|
Quote:
Originally Posted by michaelk
Code:
for root, dirs, files in os.walk("/Volumes/TM3gbBu/miscx4now/fr6tb24jul22/miscxDox-ddupt-8gb/sort/BUs/doxx", topdown=True):
for name in files:
print(os.path.join(root, name))
filename=os.path.join(root,name)
fileSize = os.path.getsize(filename)
print('file size is ', fileSize)
files does not include the full path.
|
Is
Code:
os.path.join(root,name)
considered the full path?
If so, I could write
Code:
fileSize = os.path.getsize(os.path.join(root,name))
but that produces the same error.
I'm not sure how else I can specify the full path when I'm doing a traversal of a big directory structure.
CORRECTION . .
Code:
fileSize = os.path.getsize(os.path.join(root,name))
DOES NOT PRODUCE THE SAME ERROR.
CODE AND OUTPUT GIVEN BELOW.
THANKS AGAIN
Last edited by ericlindellnyc; 08-23-2022 at 01:32 AM.
Reason: to clarify
|
|
|
08-23-2022, 12:52 AM
|
#5
|
LQ Guru
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,352
|
I assumed that the code sample was completely outdated and I just ignored it.  The updates (including in michaelk's edited post) look right to me.
That said, the error message you posted doesn't have a full path:
Quote:
Code:
FileNotFoundError: [Errno 2] No such file or directory: '._.DS_Store'
|
If you were building the paths correctly, then there would be a full path in the crash message:
Code:
>>> import os
>>> os.path.getsize('/path/does/not/exist.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.10/genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '/path/does/not/exist.txt'
Last edited by dugan; 08-23-2022 at 01:03 AM.
|
|
|
08-23-2022, 01:01 AM
|
#6
|
LQ Guru
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,352
|
Here's a complete demonstration that gets the sizes of two zero-length files. One hidden and one not:
Code:
❯ tree -a
.
└── root
├── .hidden
└── nothidden
1 directory, 2 files
❯ python3
Python 3.10.6 (main, Aug 2 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> for root, _, files in os.walk('./root'):
... for name in files:
... path = os.path.join(root, name)
... print(path, os.path.getsize(path))
...
./root/.hidden 0
./root/nothidden 0
Last edited by dugan; 08-23-2022 at 01:02 AM.
|
|
|
08-23-2022, 01:27 AM
|
#7
|
Member
Registered: Jun 2017
Posts: 196
Original Poster
Rep: 
|
Quote:
Originally Posted by dugan
As is typical, I'm going to start by being that guy. You've confirmed that it's not a broken link? What do os.path.exists(), os.path.isfile(), and similar things return?
|
It wasn't a broken link. It was throwing the error for a file that definitely wasn't a link.
That said, there were broken links in the traversed folder, which I have removed.
Your suggestion was very fruitful, implemented as follows.
import os
for root, dirs, files in os.walk("/Volumes/TM3gbBu/miscx4now/fr6tb24jul22/miscxDox-ddupt-8gb/sort/BUs/doxx"):
for name in files:
print('filename is ', name)
if os.path.exists(name):
print('it exists')
else:
print('it doesnt exist')
if os.path.isfile(name):
print('its a file')
else:
print('its not a file')
fileSize = os.path.getsize(os.path.join(root,name))
# fileSize = os.path.getsize(name)
print('file size is ', fileSize)
producing this result for three files (1st hidden; 2nd this script; 3rd dat file)
Code:
filename is .DS_Store
it exists
its a file
file size is 6148
filename is testOsWalk.py
it exists
its a file
file size is 559
filename is 1393748937451_AUTO.dat
it doesnt exist
its not a file
file size is 27072
There's another "dat" file that produces the same nonsensical result (saying it doesn't exist and printing its size).
But this works on the other two.
Hooray !! thanks much.
I will continue troubleshooting
Last edited by ericlindellnyc; 08-23-2022 at 01:29 AM.
Reason: to clarify
|
|
|
08-23-2022, 01:44 AM
|
#8
|
LQ Guru
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,352
|
Quote:
On some platforms, this function [os.path.exists] may return False if permission is not granted to execute os.stat() on the requested file, even if the path physically exists.
|
https://docs.python.org/dev/library/...os.path.exists
Can you run "stat" on the file in a terminal?
Last edited by dugan; 08-23-2022 at 01:45 AM.
|
|
|
08-23-2022, 09:55 AM
|
#9
|
Moderator
Registered: Aug 2002
Posts: 26,508
|
Code:
if os.path.exists(name):
print('it exists')
else:
print('it doesnt exist')
if os.path.isfile(name):
print('its a file')
else:
print('its not a file')
fileSize = os.path.getsize(os.path.join(root,name))
For simplicity lets call path1=/Volumes/TM3gbBu/miscx4now/fr6tb24jul22/miscxDox-ddupt-8gb/sort/BUs/doxx
Without knowing the exact file structure of path1 or where 1393748937451_AUTO.dat is located I will assume it is located in a subdirectory of path1.
os.path.exists(name) will only be true if testOsWalk.py is in the same directory as the file because name does not include the full path.
if 1393748937451_AUTO.dat is actually located in /path1/dir/1393748937451_AUTO.dat the above test fails.
fileSize = os.path.getsize(os.path.join(root,name)) works regardless because it is using the full path to 1393748937451_AUTO.dat i.e /path1/dir1/1393748937451_AUTO.dat.
|
|
|
All times are GMT -5. The time now is 10:35 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|