LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-18-2014, 01:55 AM   #1
Jaman42
LQ Newbie
 
Registered: Apr 2014
Posts: 9

Rep: Reputation: Disabled
ls -lh returns wrong output on large files


Hi everyone!
I have tried to find the answer on this question all over the internet but no one seem to know why this is happening. I run a Android 4.4.2 on my device, since it's built on Linux I thought maybe someone on this forum would know?

I want to run os.path.getsize() in Python but that gives me incorrect return on large files, when I tried it on a 700mb file the output was correct. But on the 4GB + file it returns 394 005 664 when in fact the file size is 4 688 972 960. To make sure nothing was wrong with my Python distribution I ran ls -lh on the large file and get in return: 394 005 664.

Only workaround I found so far is to use stat -c %s that returns the correct value.

Can anyone explain why this is happening or have any ideas on what to try?

Thank you for reading
 
Old 04-18-2014, 02:20 AM   #2
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,400

Rep: Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584Reputation: 2584
have a look at GiB verse GB or MiB VS. MB
1000 =1000
1024 =1000
 
Old 04-18-2014, 02:32 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 17,177

Rep: Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629Reputation: 2629
I'd be betting on 32-bit counters.

Seems Python does (or should do) automatic promotion of int to long. Ignore me.

Last edited by syg00; 04-18-2014 at 03:24 AM.
 
Old 04-18-2014, 02:43 AM   #4
Jaman42
LQ Newbie
 
Registered: Apr 2014
Posts: 9

Original Poster
Rep: Reputation: Disabled
I knew I should have posted in the newbie forum part Figured it was more of a programming type question thou.

I don't quite get what you mean, does your suggestions explain this behavior? Can I do something to make it work as intended?

Thank you for the quick replies
 
Old 04-18-2014, 10:18 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,629

Rep: Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944
I do not have a solution, but do not think John's idea would be the one as the values are more than a factor of 10 different.
As an example:
Code:
4688972960 / 1024 * 1000 = 4579075000
Which you can see is nowhere near 394005664
 
Old 04-18-2014, 11:17 AM   #6
Jaman42
LQ Newbie
 
Registered: Apr 2014
Posts: 9

Original Poster
Rep: Reputation: Disabled
Thank you for clearing that thought up.

@syg00 I got another suggestion on a swedish forum that it was a 32-bit overflow. This is a rough translation: "A unsigned 32 bit int can only contain maximum number of 4294967295. In the number 4 688 972 960 the 33rd bit is set and when it is reset it gives 394 005 664"

The calculation makes sense 4688972960-4294967295=394005665. Is there any way to fix it? Or is it just the way my system works with my current kernel?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
cp command failing for large files - Input/Output error googs Linux - General 5 08-09-2012 01:28 PM
Read large text files (~10GB), parse for columns, output vache Programming 57 04-07-2009 11:51 PM
Split a large file and get the names of output files using Perl Sherlock Programming 25 02-02-2007 01:43 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration