[SOLVED] Translating Standard Scripts to Linux

des_a · 02-05-2013, 06:50 PM

It still doesn't work for me for some reason.

Code:

#! /bin/sh


pushd .


echo Making directories for each operating system...
mkdir -p OS
cd OS


while read -a line
do
 echo ${line[0]}
 echo ${line[1]}
 while read -a line2
 do
  echo Creating ${line[0]}...
#  if [ -d $line2[0] ]; then   
#   echo There\'s already a directory named $line2[0]
#  else
#   mkdir $line2[0]

#   cd $line2[0]
#   cd ..
#  fi
 done < ${line[1]}
done < /etc/settings/Operating_Systems/osnames.sdr


popd

Output:

Code:

[root@c-des-main1-rec standard_scripts]# . ./mknasosnames2.sh
/standard_scripts /standard_scripts
Making directories for each operating system...
DOS
/etc/settings/Operating_Systems/dos.sdr
: No such file or directoryng_Systems/dos.sdr
LINUX
/etc/settings/Operating_Systems/linux/types.sdr
: No such file or directoryng_Systems/linux/types.sdr
WINDOWS
/etc/settings/Operating_Systems/windows/types.sdr
: No such file or directoryng_Systems/windows/types.sdr
/standard_scripts

Why would it work there and not here?

dive · 02-05-2013, 07:57 PM

It seems to be cutting off the name:

No such file or directoryng_Systems/linux/types.sdr

Which OS is this? Maybe you need to use a newer shell like bash for the script?

ntubski · 02-05-2013, 09:32 PM

Quote:

Originally Posted by dive

It seems to be cutting off the name:

No such file or directoryng_Systems/linux/types.sdr

That usually indicates carriage returns, try running dos2unix on both the script and input (.sdr) file.

sunnydrake · 02-06-2013, 01:02 AM

Quote:

Originally Posted by David the H.

@Sunnydrake

Please Do not Read Lines With For. Always use a while+read loop instead.

How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
http://mywiki.wooledge.org/BashFAQ/001

readarray/mapfile is another safe option, but probably not suitable if the input file is very large. It's also a bash-specific extension (available since 4.0), and not at all portable.

And do please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability.

regarding article
$(<afile); is improper use of <

+ !!!FOR IN LOOP!!! problem lies in IN instruction that read array separated by spaces!!!
remember FOR is just cycle construct not some evil overbuggy lib

you can do stupid mistakes with while loop too

most efficient / right path is place for competition.. but i don't want to write in bash sub program to read bytes in cache then determine end-line characters.
PS: thanks for read -r

my man is a little bit messed up with C header functions.. readarray all that i quick found upto job specs working good. hmm strange it's not part of coreutils .. also lines in bash man page help make doubts about using it on large files.

Quote:

If no names are supplied, the line read is assigned to the variable REPLY. The return code is zero, unless end-of-file is encountered, read times out (in which case the return code is greater than 128), or an invalid file descriptor is supplied as the argument to -u.

des_a · 02-06-2013, 03:48 PM

I ran dos2unix on those files and now it works! Now to finish the rest of the code.

Sergei Steshenko · 02-07-2013, 02:01 AM

Quote:

Originally Posted by sundialsvcs

Interesting thought ...

... has no one in Linux-land ever tackled the ("if I on-ly had a brain...") problem of dealing with DOS batch-files? Is it really true that no one ever wrote a command that could "gracefully and graciously" (of course...) accept DOS batch-files as input, and "have a good college try" at running them?

Casually googling, I didn't find one (yet), but ... it seems so obvious. Really?

You mean, I can't do: #!/bin/ihavenobrain ??

Wine ( http://www.winehq.org/ ) exist for ages, doesn't it ? Also there is DOSEMU: http://www.dosemu.org/ .

sunnydrake · 02-07-2013, 11:44 PM

1)wineconsole cmd
2)DOSBOX

David the H. · 02-09-2013, 07:24 AM

Quote:

Originally Posted by sunnydrake

regarding article
$(<afile); is improper use of < :) + !!!FOR IN LOOP!!! problem lies in IN instruction that read array separated by spaces!!!
remember FOR is just cycle construct not some evil overbuggy lib :) you can do stupid mistakes with while loop too :)

most efficient / right path is place for competition.. but i don't want to write in bash sub program to read bytes in cache then determine end-line characters.

Just because you can do something one way doesn't mean you should do it that way. I've had this debate before, and it has always been my position (and of most experienced scripters, as represented by the link I gave) that the for loop is the wrong way to go about it.

The main issue I have with it is that proper use of it relies on shell word splitting, which means that the coder must know in advance that the file is in the proper format, and will generate exactly the list of word tokens necessary for the loop. One small mistake there and you have errors.

The second issue is, indeed, with inefficiency and the possibility of hitting the ARG_MAX limit of your system. I consider this a secondary problem, as this unlikely to be a hard limit in most scripts. Still, there is potential for it.

In any case, it's much better in the long run to just do it the right way all the time, and use a proper while+read loop. With practice and experience it's no more difficult to write one of those than a for loop; and it's much safer, flexible, and efficient in all situations. In any case, calling it a "sub program" is a serious misnomer. It's just another type of loop, not any more complex to set up than any other.

Finally, and most importantly here, while you are certainly free to do whatever you want in your own private coding, when you're giving advice in a help forum like this it's your responsibility to always give the best advice you are capable of. That means attempting to show others correct practice, and not your lazy shortcuts (although certainly ignorance of correct practice is forgivable).

That's why I harp on such things so often. I consider it a duty to educate and to weed out poor scripting practice whenever I can. There are generally very good reasons why experienced coders always tend to advise the use of or avoidance of certain structures, and you ignore their hard-earned experience at your peril.

PS: "$( <file )" is simply a bash built-in convenience feature that behaves in exactly the same manner as "$( cat <file )". It only works inside command substitution brackets, and is certainly not portable.

readarray (a.k.a. mapfile, apparently the proper name) is also a bash built-in keyword, as it has to be in order to set environment values directly. So no, you'll never find it in the coreutils. It's only function is to safely load an array with lines from a file, so as with any other variable usage, the limit is in the amount of RAM you have available in which to store the text. As long as you know you can stay inside that limit, it's certainly a viable alternative to a while loop.

sunnydrake · 02-10-2013, 09:51 AM

hey i just posted mistakes that i found in that article.
< is stdio input redirection which is not cat

so construct $(< ) is illogical(call/fork shell with just input from file? try using just <filename in shell not like cat result?

).(and tend to have some hidden caveats(like double input,special chars,etc....) found some when i searched correct instructions for file read).
I can't and I won't do all work for poster... i shared exp and some quick search data that in my tests(yes i wrote mini tests for this specific scenario) made usable expected result).
Each specific task have not so many best approaches and big a number of still useful. I made working suggestion in given(!) task params.
You frame me that it's my lazy shortcuts? you make me smile

Also don't push on while loop you sound like religion priest, let's not start flame wars. In bare bones this is just same loops with almost identical code(for with read -a ? not a problem

).
However i can understand you if you say read -a is a good way to read files... despite time out's i mentioned(and yet untested memory usage) it's a challenge to find another std useful file read line tool/command usable in shell... my search ended with readarray (but i tested shells stdio redirection,for in,cat's keys scenarios which not produced useful results.. ).. i hope we not hijacking this thread too much.

PTrenholme · 02-10-2013, 03:57 PM

What happens if you use a mkdir -p instead of a simple mkdir?

David the H. · 02-11-2013, 10:34 AM

Quote:

Originally Posted by sunnydrake

hey i just posted mistakes that i found in that article.
< is stdio input redirection which is not cat

so construct $(< ) is illogical(call/fork shell with just input from file? try using just <filename in shell not like cat result?

).

It's not a mistake. As I (and the link) said, "$(<file)" is a minor bash-specific convenience feature. It only works in bash, only works inside command substitution brackets, and it behaves exactly as if there's an invisible cat command in front of it.

Any other attempt at raw redirection (unassociated with a command) will fail, just as you expect.

Quote:

You frame me that it's my lazy shortcuts? you make me smile

Also don't push on while loop you sound like religion priest, let's not start flame wars. In bare bones this is just same loops with almost identical code(for with read -a ? not a problem

).

I believe I explained my reasoning clearly. I will always, to the best of my ability, oppose poor coding practices. And I will continue to correct them any time anyone posts them here. If that marks me as a "religion priest", then so be it.

Whatever you use yourself in private is your own business, and you will accept the consequences of those choices until such time as you learn better. But I will do what I can to ensure that the new, inexperienced scripters who come here to learn do not get taught to use and perpetuate those same errors.

Quote:

it's a challenge to find another std useful file read line tool/command usable in shell... my search ended with readarray (but i tested shells stdio redirection,for in,cat's keys scenarios which not produced useful results.. ).

If it's not directly built into the shell, then there's no way you'll ever be able to directly add anything to the shell environment with it. All external commands run as separate processes, which by definition are isolated from the original environment and unable to affect it. Therefore the only options you have are variations of read, redirection, and command substitution.

theNbomr · 02-12-2013, 12:21 PM

Quote:

Originally Posted by sundialsvcs

Interesting thought ...

... has no one in Linux-land ever tackled the ("if I on-ly had a brain...") problem of dealing with DOS batch-files? Is it really true that no one ever wrote a command that could "gracefully and graciously" (of course...) accept DOS batch-files as input, and "have a good college try" at running them?

Casually googling, I didn't find one (yet), but ... it seems so obvious. Really?

You mean, I can't do: #!/bin/ihavenobrain ??

Sounds like a perfect suggestion for one of those "I just learned <language_of_the_day>. Now what problem can I solve?" questions.
--- rod.

des_a · 02-15-2013, 02:40 PM

Sorry that took so long, but I needed then to finish coding the rest of it, and it was not possible to entirely test everything, without first finishing a draft of the scripts so that everything is coded.

Code:

#! /bin/sh


pushd .


echo Making directories for each operating system...
mkdir OS
cd OS


while read -a line
do
 echo Creating ${line[0]}...
 if [ -d ${line[0]} ]; then
  echo There\'s already a directory named ${line[0]}
 else
  mkdir ${line[0]}
 fi

 cd ${line[0]} 

 while read -a line2
 do
  echo ${line2[0]}
  echo ${line2[1]}

  echo Creating ${line2[0]}...
  if [ -d ${line2[0]} ]; then   
   echo There\'s already a directory named ${line2[0]}
  else
   mkdir ${line2[0]}

   cd ${line2[0]}

   if [ ${line2[1]} == "(none)" ]; then
    cd ..
    break
   fi

   pushd .
   . ./mknastypes3.sh ${line2[1]}
   popd

   cd ..
  fi
 done < ${line[1]}

 cd ..
done < /etc/settings/Operating_Systems/osnames.sdr


popd

My focus now turns to the logical test of the scripts, and one script is giving me trouble again. This is a closely related cousin of the script above, but there are some differences.

From here on out, I can assume that the standard scripts will always be stored in /standard_scripts. Here is the code that's giving me trouble now:

mknasosnames:

Code:

#! /bin/sh


pushd . > /dev/null


echo Making directories for each operating system...


mkdir ALL
cd ALL
mkdir STANDARD
mkdir OPTIONAL
cd STANDARD
mkdir NORMAL
mkdir SHAREWARE
cd ..
cd OPTIONAL
mkdir NORMAL
mkdir SHAREWARE
cd ..
cd ..


while read -a line
do
 echo Creating ${line[0]}...
 if [ -d ${line[0]} ]; then
  echo There\'s already a directory named ${line[0]}
 else
  mkdir ${line[0]}
 fi

 cd ${line[0]}

 while read -a line2
 do
 if [ -d ${line2[0]} ]; then
  echo There\'s already a directory names ${line2[0]}
 else
  mkdir ${line2[0]}

  cd ${line2[0]}

  if [ ${line2[1]} == "(none)" ]; then
   pushd . > /dev/null
    . /standard_scripts/mknastypes2.sh
   popd > /dev/null
   cd ..
   break
  else
   pushd . > /dev/null
   . /standard_scripts/mknastypes.sh ${line2[1]}
   popd > /dev/null

   cd ..
  fi
 fi
 done < ${line[1]}

 cd ..
done < /etc/settings/Operating_Systems/osnames.sdr 



popd > /dev/null

Output:

Code:

[root@c-des-main1-rec standard_scripts]# . ./mknasosnames.sh
Making directories for each operating system...
Creating DOS...
Making directories for types of software...
Creating LINUX...
Creating 2008.0...
bash: ${line[1]}: ambiguous redirect
Creating WINDOWS...
Making directories for types of software...
[root@c-des-main1-rec standard_scripts]#

PTrenholme · 02-15-2013, 04:36 PM

In post #12, above, you state that /etc/settings/Operating_Systems/linux/Mandriva.sdr contains

Quote:

2008.0
2009.1

Thus, when processing the 2008.0, line[0]=2008.0 and line[1] is null, so the redirection done < ${line[1]} is somewhat ambiguous, don't you think?

By the way, I notice that your code often uses commands (like, e.g., mkdir) but hardly ever follows the command with a [ $? -ne 0 ] && echo "Could not ..." assert check. You might find the information about the bash built in trap function of interest.

Also, I find your use of the pushd and popd built in functions somewhat inconsistent. Why not use them everywhere you now use a cd command? Or, if you're simply using the pushd command as a placeholder for the current directory, you could just do a variable_name="$(cwd)" and, to return there, a cd "${variable_name}". Of course, you might need the quotes if you directory names sometimes include blanks, etc., but that not much of a hardship.

des_a · 02-15-2013, 05:00 PM

Thanks for catching that. I'll work on fixing it. That means I may not be able to use the exact same syntax.