[SOLVED] A question about file manipulation from the command-line

marcusrex · 03-22-2012, 10:39 PM

Hello!

My name is Mark and this is my second post to Linux Questions. I am not an utter newbie to the Unix command-line, however I've never really done anything more sophisticated than moving and copying files, and moving and copying folders. Recently at work, I wanted to perform batch conversion of a long list of files from one file extension type to another.

I searched the web and found this link (http://lab.artlung.com/unix-batch-file-rename/). I had to go about halfway down the page to find information that finally helped me; much of what was posted there didn't work on my system. Specifically, here is the text from the link that ultimately helped me.

>In general, if you want to add a suffix to your files, do this (.txt in this example):
>
>ls
>file1 file2 file3
>
>for i in *; do mv $i $i.txt; done
>
>ls
>file1.txt file2.txt file3.txt
>
>If you want to take it back off (.txt in this example again)
>
>for i in *.txt; do mv $i ${i%.*}; done
>
>ls
>file1 file2 file3

I would like to learn more about this but have no idea how to search for more information. Here is my question. What is this text (for i in *; do mv $i $i.txt; done) and this text (for i in *.txt; do mv $i ${i%.*}; done) called? In other words, when I type that text into the command-line, what is the name of the activity I am performing? Is it called shell scripting? Programming? Mucking about? The sequence of commands in those two texts reduced a job that would have taken hours manually into a nearly instantaneous process and I would really like to learn more about that.

Thank you for your help!

Mark

P.S. It would be great if you could recommend books or other reference materials I should seek out. In other words I would like to RTFM but I don't know what FM to R.

Dark_Helmet · 03-22-2012, 10:48 PM

Quote:

Originally Posted by marcusrex

What is this text (for i in *; do mv $i $i.txt; done) and this text (for i in *.txt; do mv $i ${i%.*}; done) called?

It's "called" entering a command. I'm not trying to sound like an arse, but that's really what it is. Some might call it shell scripting, but to call it shell scripting would also require that same person to call a simple "ls" shell scripting as well.

Here's the thing, a shell script is just a sequence of commands. The stuff you quoted are commands. They are "loops" but loops are commands just as well as anything else.

Anyway, I don't want to get distracted on a philosophical and ultimately impractical discussion.

While those are commands, they are first steps toward shell scripting. The usual link for shell scripting is:
The Advanced Bash Scripting Guide

Don't let the "advanced" discourage you. Just dive into it.

chrism01 · 03-22-2012, 11:24 PM

See also
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html

I'd say 'shell scripting' if you put those cmds into a shell file ie script (file) called eg myscript.sh, something like

Code:

#!/bin/bash

for i in *; do mv $i $i.txt; done

#  OR, for more complex work, put each cmd on a separate
# line so you can add extra lines in between (no line nums for shell )
for i in *
do 
   mv $i $i.txt
done

Everything between 'do' & 'done' is repeated as often as dictated by the 'for ...' line.
Note also that end-of-line is sufficient to be end-of-cmd; you only need ';' to separate
multiple cmds on the same line to make the parser's life easier

David the H. · 03-23-2012, 10:39 AM

Please use [code][/code] tags around your code and data, to preserve formatting and to improve readability. Please do not use quote tags, colors, or other fancy formatting.

Congratulations on starting out with scripting!

Of course the specific action you've decided to start with has been covered many times, but we all have to start somewhere.

To start with, here are a few useful bash scripting references:
http://mywiki.wooledge.org/BashGuide
http://www.linuxcommand.org/index.php
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashPitfalls
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html
http://wiki.bash-hackers.org/start
http://ss64.com/bash/

The first link in particular should cover all the basic concepts you need to know to understand your script.

The specific structure you asked about is simply called a "for loop" by the way, and one of the most commonly-used bits of shell syntax.

However, there is one very important issue with the above commands that needs to be addressed right now:

QUOTE ALL OF YOUR VARIABLE SUBSTITUTIONS. You should never leave the quotes off a parameter expansion unless you explicitly want the resulting string to be word-split by the shell. This is a vitally important concept in scripting, so train yourself to do it correctly now. You can learn about the exceptions later.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

You'll experience failures with the above loops as soon as you hit a file name with spaces in it. So double-quote the "$variables" to keep the parts of the names together.

Have fun scripting!

marcusrex · 03-23-2012, 11:07 AM

Hello,

I have learned that proper LQ ettiquette is to post a follow up message if your post was adequately addressed. My post was more than adequately addressed.
Thank you Dark Helmet, chrism01, and David the H., you've provided me with exactly the information I sought.

Sincerely,

Mark