LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-11-2011, 01:13 PM   #1
anishkumarv
Member
 
Registered: Feb 2010
Location: chennai - India
Distribution: centos
Posts: 289

Rep: Reputation: 10
How to Separate this using Awk!!


Hi all,

I am Writing a Script, in that i struggling in this part alone,

This is two Rows of a file.

Quote:
anis.com smallbusiness, db_docupoint, db_urlforwarding, db_asgroupdemo, online

names.com namestravel, supnamestravel
this is the file content like this so many contents are there in a file,
and my requirement is like this.

Quote:
anis.com smallbusiness db_docupoint db_urlforwarding db_asgroupdemo online

name.com namestravel supnamestravel
Like separate separate columns, i need is it possible Please guide me to solve this thread.
 
Old 08-11-2011, 02:01 PM   #2
janhe
Member
 
Registered: Jul 2007
Location: Belgium
Distribution: slackware64 13.1, slackware 13.1
Posts: 369

Rep: Reputation: 49
If you are using awk, you could use the built-in variables FS and OFS.
See http://www.thegeekstuff.com/2010/01/...-filename-fnr/

It does seem like you need to trigger some re-evaluation if you just want to do a 'print $0':
Code:
janhe@hostname:~$ echo -e "1,2,3\n4,5,6" | awk 'BEGIN{FS=","; OFS=";";} {$1=$1; print $0}'
1;2;3
4;5;6
But depending on what you use for the rest, you can also use tr:
Code:
janhe@hostname:~$ echo -e "1,2,3\n4,5,6" | tr "," ";"
1;2;3
4;5;6

Last edited by janhe; 08-11-2011 at 02:08 PM.
 
Old 08-11-2011, 03:22 PM   #3
anishkumarv
Member
 
Registered: Feb 2010
Location: chennai - India
Distribution: centos
Posts: 289

Original Poster
Rep: Reputation: 10
Hi

Thanks for the Reply man.

This is for one line in a file, but n number of rows in a file like this then how i used this command man.
 
Old 08-11-2011, 04:09 PM   #4
janhe
Member
 
Registered: Jul 2007
Location: Belgium
Distribution: slackware64 13.1, slackware 13.1
Posts: 369

Rep: Reputation: 49
I demonstrated it with two lines.
It will work the same way with any number of lines.
 
Old 08-11-2011, 05:09 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Am I missing something, or is removing the commas the only thing you really need to do?
Code:
sed 's/,//g' file
Unfortunately, you put your example text inside of quote tags instead of the [code][/code] tags that you should've used, so any formatting you may have had was lost. If the entries are actually supposed to be separated by tabs, for example, then you can use the same command to replace the comma+space combination with a tab character instead.
Code:
sed 's/, /\t/g' file
 
Old 08-11-2011, 08:28 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
Or the awk equivalent:
Code:
awk -F, '$1=$1' OFS="\t" file
Just set OFS to whatever you want as new separator.
 
Old 08-12-2011, 05:15 AM   #7
anishkumarv
Member
 
Registered: Feb 2010
Location: chennai - India
Distribution: centos
Posts: 289

Original Poster
Rep: Reputation: 10
Hi all,

Thanks alot guys, for your immediate response and sorry for the late reply.

Actually using your command i remove the comma and get the output what i expect. but still the script working is going,
My requirement is like this.



Code:
anis.com smallbusiness db_docupoint db_urlforwarding db_asgroupdemo online
in this line first column contains client name and 2,3,4,5,6 column contains client databases names.

and my requirement is every time the script runs it creates on directory in the client name(column1) and start dumping the databases(column 2,3,4,5,6) using mysql dump, and move those dump databaes into (column1) directory.

like this we have to do for our every clients. is it possible to do ? guide me guys to solve this thread.

Last edited by anishkumarv; 08-12-2011 at 08:30 AM.
 
Old 08-12-2011, 07:54 AM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
So how about we start at the beginning and you tell us exactly what is required instead of spoon feeding us bits and pieces?

My first question to you would be, what is the manual process you currently use to do whatever it is you are trying to accomplish?
(ie if you type one command at a time on the command line how can you accomplish your task)
 
Old 08-12-2011, 08:01 AM   #9
janhe
Member
 
Registered: Jul 2007
Location: Belgium
Distribution: slackware64 13.1, slackware 13.1
Posts: 369

Rep: Reputation: 49
I can see that English isn't your first language. There's nothing you can do about it, as long as you are trying your best. I does make it hard for me to understand what you want exactly.

I understand that you want to execute some external programs. You can do that inside awk:

Code:
BEGIN { 
        FS=",";
}

{
        dir=$1;
        $1="";
        system("mkdir "dir);
        system("mysqldump --databases "$0" >> "dir"/backup.sql");
}
This will create a directory with the name based on the first field in the file. Then it will write a mysqldump of all the databases named on the rest of the line to a file in that directory.

You can adapt my example for the purpose you really need.

Last edited by janhe; 08-12-2011 at 08:03 AM.
 
Old 08-12-2011, 10:04 AM   #10
anishkumarv
Member
 
Registered: Feb 2010
Location: chennai - India
Distribution: centos
Posts: 289

Original Poster
Rep: Reputation: 10
Hi grail,

Thanks Man, Even Iam also not like Spoon feeding, and my exact requirement is am writing a script to take mysql backups in shared hosting, using plesk control panel we configured shared hosting in our server,


Code:
#!/bin/bash
date=`date +%Y-%m-%d_%H.%M.%S`
echo Backup Started $date  >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log 
###########################################################
#create directory to store mysql database backup
mkdir /var/tmp/mysqldatabasebackup$date
if [ $? -ne 0 ]
then echo "Backup failed with errors" for $date >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
exit
else
echo “directory mysqldatabasebackup$date created”  >> /var/tmp/mysqlbackuplog/mmysqldatabasebackup$date.log
fi

# Back up MySQL databases
###########################################################

 
# Do not backup these databases
IGNORE="information_schema mysql test argali mechcicadd_rlite information_schema"

# Get list of all databases
DB_LIST=`mysql -pPASSWORD -Bse 'show databases'`
if [ $? -ne 0 ]
then
echo backup failed with errors >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
exit
fi
for db in $DB_LIST; do

        # set skip variable
        skip=0

        if [ "$IGNORE" != "" ]; then
                for i in $IGNORE; do
                        [ "$db" == "$i" ] && skip=1 || :
                done
        fi

        if [ "$skip" == "0" ]; then
                mysqldump   $db >> /var/tmp/mysqldatabasebackup$date/$db.sql  
       echo DUMPING DATABASE FOR CLIENT ""$db"" >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log 
       echo backup copy of $db is moved to /var/tmp/mysqldatabasebackup$date	>> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
        cd  /var/tmp/mysqldatabasebackup$date
        a=`du -sh $db.sql | cut -f1`
        echo THE SIZE OF $db.sql FILE IS $a >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
        fi
done
echo "Backup completed successfully $date" >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log

cat /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log | mail -s "mysqlbackuplog" dineshkumar.baskar@cevalsoft.com,anish.valsalam@cevalsoft.com
Using this script i can able to take all the databases backup,

but my requirement is need to take all the databases backup with owner details ( owner details in the sense need to create directory in the name of client name.) so first i follow these steps to customize the above script.

1. need to get the client name and databases name.

so using the mysql query i get the client name and along with the corresponding databases.

if i execute a mysql query means i got this output.

Quote:
anis.com smallbusiness, db_docupoint, db_urlforwarding, db_asgroupdemo, online

names.com namestravel, supnamestravel
in this first column specifies the client name and other corresponding specifies the databases names.


Sorry my English is very poor i know..i hope will improve.. kindly read the above all the posts i hope you understand my requirement
 
Old 08-12-2011, 11:56 AM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
No problems on not having great English as those that supposedly can speak it are often not very eloquent anyway

I think I am following better now, but just want to know at what point in the script you have shown would you now run the mysql query to retrieve the
data in the format you mention?

I only ask as the script appears to already be making backups. Is this new requirement to happen after all of what we currently see?
 
Old 08-12-2011, 12:44 PM   #12
anishkumarv
Member
 
Registered: Feb 2010
Location: chennai - India
Distribution: centos
Posts: 289

Original Poster
Rep: Reputation: 10
Hi grail,

Thanks for the patience To Read my English :-)

ya dude this is my new requirement, i need to add this in script. but still i didn't get any idea for this requirement struggling,


Code:
mysql -u admin  -pPASSWD -D psa -Bse "select a.name, group_concat(b.name separator ', ') from domains a, data_bases b where a.id=b.dom_id group by a.name;"
Using this command i am getting the client name and client databases alone.

Quote:
anis.com smallbusiness, db_docupoint, db_urlforwarding, db_asgroupdemo, online

names.com namestravel, supnamestravel
i need the output like this directory structure.

Example:

Code:
ls anis.com/
db_docupoint.sql  db_urlforwarding.sql  online.sql  smallbusiness.sql

Last edited by anishkumarv; 08-12-2011 at 12:50 PM.
 
Old 08-13-2011, 02:35 AM   #13
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
Well I would say the easiest thing to do would be simply change the separator to a space (not sure if that is the default as not familiar with mysql).

So I would probably do some thing like:
Code:
while read -r client dbs
do
    dir=/path/to/clients/$client
    mkdir "$dir"

    mysqldump --databases "$dbs" "$dir/"
done< <(mysql -u admin  -pPASSWD -D psa -Bse "select a.name, group_concat(b.name separator ' ') from domains a, data_bases b where a.id=b.dom_id group by a.name;")
Again I do not know how mysqldump works (ie if it generates the names for each database) so if it doesn't create the names you could easily implement a for loop to do each one.
 
Old 08-15-2011, 09:22 AM   #14
anishkumarv
Member
 
Registered: Feb 2010
Location: chennai - India
Distribution: centos
Posts: 289

Original Poster
Rep: Reputation: 10
Hi grail,

Thanks For the Reply. Sorry for the late reply we are in celebration mood (Independence day here :-)) but still My Script Problem is continuing :-( :-(

Code:
bbkup.sh: line 9: syntax error near unexpected token `<'
dbbkup.sh: line 9: `done < <(mysql -u admin  -pPASSWD -D psa -Bse "select a.name, group_concat(b.name separator ' ') from domains a, data_bases b where a.id=b.dom_id group by a.name;");'
 
Old 08-15-2011, 09:49 AM   #15
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
Why is there a semi-colon at the end of the second errored line?

Why are there 2 scripts? What is at line 9 in the first script?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
AWK won't separate fields slinx Programming 2 03-10-2009 05:11 PM
Encoding separate audio channels to separate files omnio Linux - Software 0 06-01-2007 08:46 AM
awk command to merge columns from two separate files into single file? johnpaulodonnell Linux - Newbie 4 01-23-2007 11:10 AM
Looking for a tool to auto crop and separate images in to separate files.. mlsfit Linux - Software 2 08-06-2006 04:13 PM


All times are GMT -5. The time now is 04:02 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration