what can i do with shell?
 

i have learned how to program in shell. but i do not know what should i do with it? how can i program with it? how to practice?
thanks in advance!:)

if you want to learn shell scripting, check out the bash scripting howto at tldp.org

i mean i already read a classic book called "advanced bash-scripting"and i know how to write shell script. but i do not know what can i do with it? how to practice it? how to practice programming with shell?
thanks!!!!!

I would start by making some utilities for monitoring your system, parsing log files, automating backups with rsync, or tar, and anything else. That should be a good start. Oh and you might also want to try and create install scripts for files.

oh yeah! that's it! but i do not know how to write it? i donot konw much about my system. could you give me yours to learn? shall we be friends? do you have msn or icq or AIM? or email? maybe you can guide me. thanks

Don't do msn, or icq or aim. But I am always glad to help. So lets start out by doing a simple backup.
1) I would decide what you want to backup.
2) Decide when to backup.
3) Where are you going to backup to. (My preference has always been to get a 500gb hard drive.)

4) Lets break our backups into folders. Say Mon, Tue, Wed, Thr, Fri. Week1, Week2. I use week1 and week2 so that I can go back as far as two weeks in my full backups.

5) Lets do some sanity checks. First part of your script will be to see if rysnc is installed. so you will want to check for that and if it fails then stop and create a log with the error.

This should get you started. What I would do is then use your book for reference. As for a system to start out with I would use debian because to me it is simple. Someother good ones are Slackware, Fedora, Ubuntu, CentOS, my favorite right now is arch. If all you are trying to do is a little bash scripting I would say Fedora, Ubuntu, CentOS. Debian, Slackware, and Arch normally have some burn in time where you have to setup some things.

You know, people don't usually start with "I want to do some scripting" and then search for something to script. They usually begin with a problem or need; something like, "I want to be able to take a folder full of mp3 files, convert them to .ogg format with metadata intact, rename them according to the track metadata, then sort them all into separate folders based on the artist and filename". They then go out and script a series of commands to do just that.

All a script really is is a sequence of commands designed to automate some process. You want practice? Take any cli command you know how to use (using ffmpeg to convert audio files to ogg vorbis, for example) and simply see if you can automate it to process a whole directory of files at once. Then you can start replacing the hard-coded commands with user-input choices (such as selecting ogg vs mp3 output). You can get gradually more and more complicated as you go on (e.g. how do I vary the bitrate? How can I have user-selected filenames?, having an option to move/remove the original file, etc.). The sky's the limit. It all comes down to your imagination and scripting experience.

So really, you just need to get down and dirty. You can't learn scripting by reading a book. Just decide on something you want to accomplish and start writing. It'll be slow going at first, but that's how you learn.

The first script I ever created was a "remove directory" script that asked me if I wanted to either remove the whole thing, including contents, at once, or to confirm the deletion of each file separately. I made it because I was paranoid about deleting a directory with "rm -r" and losing important stuff inside. But you know something? Even though I made my script ask me some questions first, at the heart of it there's still just the plain old "rm -r" command. I still use it today, but I probably should go back and rewrite it soon, because I've learned so much more about scripting since then that I'm almost embarrassed to look at the code for it. :)

sorry, i did not exactly understand this. this is a five step of make a back up script?
thanks
what should i do now?

i want to write a script to change all the file names of one folder. but i faild. could you guide me this?
thanks

if you don't know why you're doing something, don't do it! This approach makes no sense at all...

oh thanks very much for telling me this!:) once i want to write a script to change all the file names of one folder, but i faild.and i do not know how to wirte a right one. could you guide me?
thanks in advance.

i know why i am doing this, i just do not know how to make it right at first.

First get used to using the console. Cd to different directories; list files; use less & cat to examine text files. Get comfortable in the shell environment. Use the * and ? wildcards with ls. Become familiar with using a text editor. There are several to choose from.

The most common commands are provided by the coreutils package. It contains commands such as cp, mv, ls, cat. Scan through the info manual for coreutils. It may be easier reading it in the konqueror browser. You can do this by entering info:coreutils in the address bar.
You can read manpages the same way, e.g. man:mv

That's what the texts and tutorials are for. You don't just jump into big scripts, you start small and learn the basics.

You first have to know how to use individual commands on their own. Only when you're familiar with the tools can you start using scripting functions to combine them. For your "rename files" script, you need a few things: First you need a renaming program. mv will do nicely. If you don't know how to use mv to rename a file, then you have to stop right there and learn that first. Second, you need a way to run the command multiple times on different data input. The "for loop" scripting process is probably the one you'll want to use. Third, you need to be able to input the name changes you want, and for that you'll need to use variables, particularly the command positional parameters ($1, $2) and perhaps the read command if you want interactive input. You may also wish to do some arithmetical processing if you want to generate sequentially-numbered files, the output of which will be stored in a variable. Finally you'll want to use the "echo" command to provide some user-readable feedback. Again, if you don't know what all of these things are, you have to do more studying.

Your final script will probably be something that takes an input string from the command line (when starting the script or interactively), stores it in a variable, then uses that variable in the output of the mv command. And the mv command will be sitting in a for loop that takes each filename in the directory as input and runs the command on it.

I often recommend LinuxCommand.org as a good place to start learning the shell and scripting. Don't just read it, go through each page step-by-step and actually do the things he shows you in the examples. Try the commands out yourself in a console of your own. Play with them until you really understand the concepts behind them. After a while you should start to naturally recognize how you can combine them to do things that are not in the tutorials.

That is a pretty good site. I really like the flow and layout. Course I started out with this book and reference it a lot. http://www.amazon.com/Linux-Shell-Sc...4541634&sr=8-6 but then moved to perl. Only problem with perl is that when you stop using it, you seem to forget alot of the things it can do.

thanks so much! i will take it seriously.

this is the script i have wrote, and do not know what should i do next:

#!/bin/bash

directory=/home/zsf/pictures
for i in $directory/*

thanks

Ok, that's a good start. You've got the idea of a variable down.

Now put a command or two in the for loop. A good test to start with is to use echo. The 'i' in the loop is a variable that will hold each line of input (in this case, the individual filenames in $directory) in turn. So try this:

This should output a list of all the files in the directory. After you get a correct list of files with echo, you can add other commands in the same way; "cp $i /path/to/newdir/", for example. Basically, you use $i anywhere you want to insert the filename in the command. The loop will then run the command on each file in turn.

Search this and related forums on LQ for subjects containing bash. Doing this will bring you to many questions that are practical in nature, and virtually all will have solutions to the problem; as often as not there will be several solutions from different people (some of the solutions will actually work, too!). Try to develop your own solutions for such problems. You will be able to use what others have done to solve practical problems, and learn from them in the process. Eventually, you will be able to contribute solutions of your own. You will also accumulate ideas about how your new-found knowledge can be applied in practical ways for your own benefit.
--- rod.

thanks very much! and i still can not get it right. i do really want to generate a sequentially-numbered files.but i still do not know how to do this! please~:)

Consider any thread asking for help with a shell (bash/sed/awk/tr/…) script as a test to your new knowledge. Answer and you'll get better.

Yves.

Ok, try this. Forget about writing a full script at first and just try it straight from the shell. Type these two commands:

The output will be the text string stored in the variable "i".


Now let's try something different:

Now "$i" contains a list of the directory contents, each separated by a space.


Next, let's create a "for" loop instead:

This should give you a list of all the files in the current directory. Only now they're all on separate lines (because a separate echo command was run for each filename).

What the command is doing is taking the input list "./*" and feeding only the first entry (as separated by spaces) into the variable "i". Then it runs echo "$i", which outputs the value stored in "i". Finally, it loops back to the beginning and repeats the process on the second entry in ./*, then the third, etc., until there are no more entries left, at which point it exits.

You can replace the echo command with any other program or series of programs. everything between "do" and "done" will be executed on "i" for each entry in the filelist. For example, try replacing "echo $i;" with "file $i;" and see what happens.

Then try stringing multiple commands together and adding more detail, such as:

.

Finally, here's an interesting thing to try. After the for loop finishes, run "echo $i" again. The output should be the last entry in the directory filelist, which is still there because that was the last thing that went through the loop. Variables don't automatically get cleared after they are used.


Once you have the commands working, you can create a script. Place the them in a text file, with #!/bin/bash as the first line. Note that you can put commands on separate lines instead of using semicolons to break them up. Make that file executable with "chmod u+x myscript.sh", and run it with a simple "./myscript.sh". It should then do exactly the same thing your commands did when you did them manually.

This is not really difficult stuff here. You just have to make sure you understand how the different pieces go together. And for that you have to try it and practice it (just make sure you practice your commands in a safe test directory with files you can afford to lose, until you know that what you're doing is safe :)).

oh thanks so much! i appreciate.
i know what your mean. but i am so stupid, i can not make it generate a sequental-numbered output. the only i can do is this:

#!/bin/bash

directory=/home/zsf/Pictures/h
for i in $directory/*
do
echo "what is $i?"
file $i
mv $i /home/zsf/Pictures/h/00
done

it can change one file name. i do not know how to change more than one file names. :( sorry!

Please show us exactly what your input is (i.e. "ls" the directory), and what the output is after you run the script (the error messages). Also tell us what you've tried so far to work around your problem and what the outcome was. You have tried working it out yourself, haven't you?

One thing I can think of that might be affecting it is if your files contain spaces. Spaces in filenames can be a big headache when it comes to scripting, as spaces are usually viewed as field separators. Try putting quotes around all of your $1 instances so the contents will be seen as a single unit; e.g. file "$i".

sorry for giving you that thought that i was lazy.but i do try it.

after i run ls

the output is

3.jpg 4.jpg 5.jpg

i have tried

#!/bin/bash

directory=/home/zsf/c
for i in $directory/*
do
echo "what is $i?"
file $i
mv $i /home/zsf/c/new-name
done

and the output is a file named new-name.

i add one command mv $i /home/zsf/new-name, but it only can change one file name!
and i also tried this ${i%[1-9].jpg}.i know this is not complete.but i do not know how to do next.is that clear? thanks.

I see. The problem is that your mv command has only one fixed output name, so every file you run through it gets the same name. You're overwriting the same file over and over. For a straight move from one directory to another, change your mv command to

mv "$i" "/home/zsf/c/$i"

Which will give the output file the same name as the input file.

The ${} parameter substitution you tried to use is a good way to change the output filename, but you have the wrong structure. What ${i%.jpg} does is remove the string ".jpg" from the end of the variable, if it exists, leaving you with just the basename. After that you'll probably want to add something else back to the name, which you can simply tack onto the end of the variable.

For example, if you wanted to change all ".jpg" extensions to ".jpeg", you'd use this:

mv "$i" "${i%.jpg}".jpeg

You probably don't want to use a [0-9] (match any number) in a situation like this. As it is you were matching, and stripping, the entire filename, leaving you with a blank output.

So, to summarize, use the "$i" string everywhere in the script you want the filename to appear. This will usually be in both the input and output parts of the commands. To modify a filename, you can use the ${} parameter substitution (among other techniques). You generally won't want to do this on the input filename, because the substitution happens before the command is run, but it's great for altering the output name.

does mv can use this way?? mv "$i" "${i%.jpg}".jpeg ? i only know it can use like this:
mv existing-filename new-filename

why i did not know that? where can i look up for?

The command is doing just what it's supposed to. "$i" is the existing filename, and "${i%.jpg}".jpeg is the new filename.

For example:
So "mv $i ${i%.jpg}.jpeg" becomes "mv img01.jpg img01.jpeg" after the variable is evaluated.

A good way to test what the final command will look like after the variable evaluations are made is to enclose it in an echo statement.

when i run this

mv "$i" "/home/zsf/Pictures/G/$i"


why it says "cannot move, no such file or directory?"
but this works:
mv "$i" "/home/zsf/Pictures/G/"

Does the /home/zsf/Pictures/G/ directory exist? You can't move files into a nonexistant directory. You have to create it first.

does this called stubbing? i just saw that from the website you give to me.

Not exactly. Stubbing means inserting a quick, neutral line as a temporary placeholder, with the intent of coming back later and replacing it with something more useful. That way you can build the basic structure of the script before concentrating on the details of the individual commands. A real stub would be something like:

echo "Replace this line with the mv command".


The suggestion I gave you is more of a debugging test. It helps you see clearly what's happening so you can catch possible mistakes before you use the script on anything important. Sticking in echo lines for variables and commands is a good way to see what your code is really doing.

oh thanks,yeah it is, i get it ,thanks.
sorry for my belated reply for my work.

sorry, i do not know what is positional parameters, actually, i do not completely understand. could you tell me ?thanks

THIS is why i love the internets!

The positional parameters are the special variables that let you pass arguments to the script from the command line. "$0" is the name of the script itself, "$1" is the first argument, "$2" is the second argument, and so on. "$@" will list all of the parameters together and "$#" will give you the total number of arguments (excluding $0).

For example:
And when I run it I get:

If you're wondering about "${2:-empty}", it's a way to set a default value for a variable: ${var:-string} = if $var doesn't exist, use "string" instead.

"echo -e" allows you to use characters such as quotes in the string by escaping them with backslashes.

oh thanks! but now, i still donot know how to rename more than one file names,could you please tell me more? thanks very much.

Well, I admit that renaming files is a bit more difficult than some operations, because each output filename needs to be unique. But it's not really that hard. The easiest way is just to take the input filename and make a minor change to it, like changing the ending, which I showed you how to do before, or adding a text string as a prefix or suffix.

So if the input files are

01.jpg
02.jpg
03.jpg
foo.jpg
bar.jpg

the output filenames will be

myphoto-01.jpg
myphoto-02.jpg
myphoto-03.jpg
myphoto-foo.jpg
myphoto-bar.jpg

Remember, the variable "filename" in this case simply holds the name of the file. Just imagine the real name appearing everywhere you see "$filename" in the command, input and output. That's all there is to it.

For more complex naming, it would help if you would make it clear just what you want the output filenames to look like. Different techniques may be required for different kinds of output.

how to generate sequential number?

could i ask you one other thing?
my major is not computer, i learn it by myself at spare time,but i wanna do it good and better, so i was thinking to take some classes. maybe like CCNA CCNP RHCA etc. so what you think? i thought you are good at this, so it will be great of you to give me some advices. thanks in advance! i appreciate!

do you know other good books or something about shell? it seems simple and easy,but why i can not accomplish it by myself? thanks

I'm not sure how much more I can help you. You've already seen LinuxCommand, which is the easiest beginner's tutorial I know about, and I've guided you through a dozen posts of simple scripts. I don't see how anything else I can give you would be of benefit.

You seem to be getting lost on something very basic and fundamental about scripting, perhaps about how variables and embedded commands work. If you don't understand the concept of variables and substituting one thing for another, then you're not going to progress very far. The examples I've given you so far are for the most part very simple scripts that require only the use of variables, for loops and basic system commands. You need to stop here and work on what's already been covered until you clearly understand how it works, before moving on to anything else.

You also appear to need to familiarize yourself more with how basic cli commands such as mv work. If you don't know how to use the tools, then you can't accomplish anything using them.

Finally, there's a wealth of scripting information out there, some of which has already been given to you. LinuxCommand.org, the scripting section of the Rute User's Tutorial, the tdlp Bash HOWTO, Bash Guide for Beginners, and Advanced Bash Scripting Guide, and more.

You just have to keep plugging away until you understand it.

thanks!:)