LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 09-22-2020, 07:10 PM   #1
bagpussnz
Member
 
Registered: Aug 2003
Posts: 54

Rep: Reputation: 15
ls quoting non existing files


Hi,
This came in post ls version 8.25.

If you do an ls of a file that doesn't exist, it puts quotes around the file in the error message.

E.g. if foo doesnt exist

# ls foo
ls: cannot access 'foo': No such file or directory

On older (pre 8.25 inclusive), you get,

# ls foo
ls: cannot access foo: No such file or directory

None of the quoting-style options change this.

Any ideas? I can potentially put alias and functions in place, but seems overkill.
It's for an automated regression system with expected output.

Cheers,
Ian
 
Old 09-22-2020, 07:57 PM   #2
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,223

Rep: Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320
Would building or otherwise installing ls 8.25 and setting the automated regression system to use it (perhaps by putting it in a prepended PATH entry) be an option?

Is updating the automated regression system an option?

Last edited by dugan; 09-22-2020 at 08:11 PM.
 
Old 09-22-2020, 08:23 PM   #3
bagpussnz
Member
 
Registered: Aug 2003
Posts: 54

Original Poster
Rep: Reputation: 15
Yeah - that was one of my options.. just seems a bit overkill for a minor functionality change (I'm sure the change makes sense to someone).

Thanks.
 
Old 09-22-2020, 08:27 PM   #4
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
I don't think you can rely on specific output formats, especially when it comes to error messages. Your program has to perform the test whether the file exists or not.

Other example: There have been heated discussions how ls should treat nonprintable or otherwise special characters in filenames. A recent revision of ls changed the behaviour. Again, the bottom line is that the output of ls is not standardized, and that you should not make assumptions.

It's not too hard to test for the existence of a file in Bash:
Code:
[ -e foo ] && echo foo exists

Last edited by berndbausch; 09-22-2020 at 08:28 PM.
 
Old 09-22-2020, 08:51 PM   #5
bagpussnz
Member
 
Registered: Aug 2003
Posts: 54

Original Poster
Rep: Reputation: 15
Thanks - I fully agree. If there is no backdoor to previous functionality, I'll implement something like this.
 
Old 09-23-2020, 07:43 AM   #6
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,599

Rep: Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546Reputation: 2546
Quote:
Originally Posted by bagpussnz View Post
If there is no backdoor to previous functionality, I'll implement something like this.
ls is part of GNU coreutils, the source code of which is linked from here: http://www.gnu.org/software/coreutil...ls.html#source

It's a simple search to identify the relevant commit and answer that question.

Your simplest fix is probably to quote the filenames in your expected output, but the better fix is to use a mechanism specifically intended to identify if a file exists (as per berndbausch's post), and which doesn't rely on a brittle error message.

 
Old 09-23-2020, 08:53 AM   #7
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,841

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
look for nullglob or failglob
https://superuser.com/questions/5193...are-no-matches
https://mywiki.wooledge.org/glob
 
Old 09-23-2020, 02:56 PM   #8
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,980

Rep: Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624
I'd guess your distro has an older version that can be used. Just replace it and watch for any updates.

I'm shocked it was updated at all.
 
Old 09-24-2020, 02:25 AM   #9
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,841

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
Quote:
Originally Posted by bagpussnz View Post
It's for an automated regression system with expected output.
You must not rely on the output of ls, but use nullglob or similar or other tool (written in perl/python/awk/shell/whatever).
from the other hand you may try to remove ' from the output with sed or tr or something else.
 
1 members found this post helpful.
Old 09-24-2020, 03:17 AM   #10
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.9.2009
Posts: 5,727

Rep: Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211Reputation: 2211
Quote:
Originally Posted by pan64 View Post
You must not rely on the output of ls, but use nullglob or similar or other tool (written in perl/python/awk/shell/whatever).
from the other hand you may try to remove ' from the output with sed or tr or something else.
An outstanding suggestion! Removing the quotes with sed is an excellent idea!
 
Old 09-24-2020, 08:08 AM   #11
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,223

Rep: Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320
I assume the "automated regression system" shells out to ls and parses its output, and that for bad reasons that are unfortunately common in the industry, that cannot be changed.
 
Old 09-24-2020, 08:14 AM   #12
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,841

Rep: Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308Reputation: 7308
in that case ls should be replaced by a script which will return the result using the expected form/style
 
Old 09-24-2020, 08:29 AM   #13
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,223

Rep: Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320
It is unfortunately that someone is stuck with a system that parses ls output, as that should never have been implemented in the first place.

bagpussnz: The "correct" solution would obviously be to update the automated regression system. I assume that there are reasons (technical, bureaucratic, or otherwise) why you can't do that?

Last edited by dugan; 09-24-2020 at 01:05 PM.
 
Old 09-24-2020, 10:08 AM   #14
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware64-15.0
Posts: 6,371

Rep: Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749
Quote:
It's for an automated regression system with expected output.
So the automated regression system has done it's job and detected a change.
Despite what @dugan suggests "for bad reasons that are unfortunately common in the industry, that cannot be changed", I fail to understand why it is needed to bend the new 'ls' output to match the old behaviour.

I consider the new behaviour to be better (e.g. differentiating a file name containing a space).

If that is not acceptable, stick with the old version.
 
Old 09-24-2020, 11:53 AM   #15
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,223

Rep: Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320
http://www.catb.org/jargon/html/B/bug-compatible.html
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Bash syntax - question regarding quoting: quoting doesn't work as I expect. c.stadegaart Linux - General 5 04-02-2012 11:40 PM
how to add existing files to existing imported kdevelop project?? santana Programming 1 05-22-2007 09:53 AM
quoting web contents in the 'member area' of company homepage -- legal or not? baikonur General 11 06-14-2006 07:42 AM
Quick question: quoting text in sed overbored Linux - Software 0 06-24-2004 01:23 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:13 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration