How to Separate this using Awk!!

anishkumarv · 08-11-2011, 12:13 PM

Hi all,

I am Writing a Script, in that i struggling in this part alone,

This is two Rows of a file.

Quote:

anis.com smallbusiness, db_docupoint, db_urlforwarding, db_asgroupdemo, online

names.com namestravel, supnamestravel

this is the file content like this so many contents are there in a file,
and my requirement is like this.

Quote:

anis.com smallbusiness db_docupoint db_urlforwarding db_asgroupdemo online

name.com namestravel supnamestravel

Like separate separate columns, i need is it possible Please guide me to solve this thread.

janhe · 08-11-2011, 01:01 PM

If you are using awk, you could use the built-in variables FS and OFS.
See http://www.thegeekstuff.com/2010/01/...-filename-fnr/

It does seem like you need to trigger some re-evaluation if you just want to do a 'print $0':

Code:

janhe@hostname:~$ echo -e "1,2,3\n4,5,6" | awk 'BEGIN{FS=","; OFS=";";} {$1=$1; print $0}'
1;2;3
4;5;6

But depending on what you use for the rest, you can also use tr:

Code:

janhe@hostname:~$ echo -e "1,2,3\n4,5,6" | tr "," ";"
1;2;3
4;5;6

anishkumarv · 08-11-2011, 02:22 PM

Hi

Thanks for the Reply man.

This is for one line in a file, but n number of rows in a file like this then how i used this command man.

janhe · 08-11-2011, 03:09 PM

I demonstrated it with two lines.
It will work the same way with any number of lines.

David the H. · 08-11-2011, 04:09 PM

Am I missing something, or is removing the commas the only thing you really need to do?

Code:

sed 's/,//g' file

Unfortunately, you put your example text inside of quote tags instead of the [code][/code] tags that you should've used, so any formatting you may have had was lost. If the entries are actually supposed to be separated by tabs, for example, then you can use the same command to replace the comma+space combination with a tab character instead.

Code:

sed 's/, /\t/g' file

grail · 08-11-2011, 07:28 PM

Or the awk equivalent:

Code:

awk -F, '$1=$1' OFS="\t" file

Just set OFS to whatever you want as new separator.

anishkumarv · 08-12-2011, 04:15 AM

Hi all,

Thanks alot guys, for your immediate response and sorry for the late reply.

Actually using your command i remove the comma and get the output what i expect. but still the script working is going,
My requirement is like this.

Code:

anis.com smallbusiness db_docupoint db_urlforwarding db_asgroupdemo online

in this line first column contains client name and 2,3,4,5,6 column contains client databases names.

and my requirement is every time the script runs it creates on directory in the client name(column1) and start dumping the databases(column 2,3,4,5,6) using mysql dump, and move those dump databaes into (column1) directory.

like this we have to do for our every clients. is it possible to do ? guide me guys to solve this thread.

grail · 08-12-2011, 06:54 AM

So how about we start at the beginning and you tell us exactly what is required instead of spoon feeding us bits and pieces?

My first question to you would be, what is the manual process you currently use to do whatever it is you are trying to accomplish?
(ie if you type one command at a time on the command line how can you accomplish your task)

janhe · 08-12-2011, 07:01 AM

I can see that English isn't your first language. There's nothing you can do about it, as long as you are trying your best. I does make it hard for me to understand what you want exactly.

I understand that you want to execute some external programs. You can do that inside awk:

Code:

BEGIN { 
        FS=",";
}

{
        dir=$1;
        $1="";
        system("mkdir "dir);
        system("mysqldump --databases "$0" >> "dir"/backup.sql");
}

This will create a directory with the name based on the first field in the file. Then it will write a mysqldump of all the databases named on the rest of the line to a file in that directory.

You can adapt my example for the purpose you really need.

anishkumarv · 08-12-2011, 09:04 AM

Hi grail,

Thanks Man, Even Iam also not like Spoon feeding, and my exact requirement is am writing a script to take mysql backups in shared hosting, using plesk control panel we configured shared hosting in our server,

Code:

#!/bin/bash
date=`date +%Y-%m-%d_%H.%M.%S`
echo Backup Started $date  >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log 
###########################################################
#create directory to store mysql database backup
mkdir /var/tmp/mysqldatabasebackup$date
if [ $? -ne 0 ]
then echo "Backup failed with errors" for $date >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
exit
else
echo “directory mysqldatabasebackup$date created”  >> /var/tmp/mysqlbackuplog/mmysqldatabasebackup$date.log
fi

# Back up MySQL databases
###########################################################

 
# Do not backup these databases
IGNORE="information_schema mysql test argali mechcicadd_rlite information_schema"

# Get list of all databases
DB_LIST=`mysql -pPASSWORD -Bse 'show databases'`
if [ $? -ne 0 ]
then
echo backup failed with errors >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
exit
fi
for db in $DB_LIST; do

        # set skip variable
        skip=0

        if [ "$IGNORE" != "" ]; then
                for i in $IGNORE; do
                        [ "$db" == "$i" ] && skip=1 || :
                done
        fi

        if [ "$skip" == "0" ]; then
                mysqldump   $db >> /var/tmp/mysqldatabasebackup$date/$db.sql  
       echo DUMPING DATABASE FOR CLIENT ""$db"" >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log 
       echo backup copy of $db is moved to /var/tmp/mysqldatabasebackup$date	>> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
        cd  /var/tmp/mysqldatabasebackup$date
        a=`du -sh $db.sql | cut -f1`
        echo THE SIZE OF $db.sql FILE IS $a >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log
        fi
done
echo "Backup completed successfully $date" >> /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log

cat /var/tmp/mysqlbackuplog/mysqldatabasebackup$date.log | mail -s "mysqlbackuplog" dineshkumar.baskar@cevalsoft.com,anish.valsalam@cevalsoft.com

Using this script i can able to take all the databases backup,

but my requirement is need to take all the databases backup with owner details ( owner details in the sense need to create directory in the name of client name.) so first i follow these steps to customize the above script.

1. need to get the client name and databases name.

so using the mysql query i get the client name and along with the corresponding databases.

if i execute a mysql query means i got this output.

Quote:

anis.com smallbusiness, db_docupoint, db_urlforwarding, db_asgroupdemo, online

names.com namestravel, supnamestravel

in this first column specifies the client name and other corresponding specifies the databases names.

Sorry my English is very poor i know..i hope will improve.. kindly read the above all the posts i hope you understand my requirement

grail · 08-12-2011, 10:56 AM

No problems on not having great English as those that supposedly can speak it are often not very eloquent anyway

I think I am following better now, but just want to know at what point in the script you have shown would you now run the mysql query to retrieve the
data in the format you mention?

I only ask as the script appears to already be making backups. Is this new requirement to happen after all of what we currently see?

anishkumarv · 08-12-2011, 11:44 AM

Hi grail,

Thanks for the patience To Read my English :-)

ya dude this is my new requirement, i need to add this in script. but still i didn't get any idea for this requirement struggling,

Code:

mysql -u admin  -pPASSWD -D psa -Bse "select a.name, group_concat(b.name separator ', ') from domains a, data_bases b where a.id=b.dom_id group by a.name;"

Using this command i am getting the client name and client databases alone.

Quote:

anis.com smallbusiness, db_docupoint, db_urlforwarding, db_asgroupdemo, online

names.com namestravel, supnamestravel

i need the output like this directory structure.

Example:

Code:

ls anis.com/
db_docupoint.sql  db_urlforwarding.sql  online.sql  smallbusiness.sql

grail · 08-13-2011, 01:35 AM

Well I would say the easiest thing to do would be simply change the separator to a space (not sure if that is the default as not familiar with mysql).

So I would probably do some thing like:

Code:

while read -r client dbs
do
    dir=/path/to/clients/$client
    mkdir "$dir"

    mysqldump --databases "$dbs" "$dir/"
done< <(mysql -u admin  -pPASSWD -D psa -Bse "select a.name, group_concat(b.name separator ' ') from domains a, data_bases b where a.id=b.dom_id group by a.name;")

Again I do not know how mysqldump works (ie if it generates the names for each database) so if it doesn't create the names you could easily implement a for loop to do each one.

anishkumarv · 08-15-2011, 08:22 AM

Hi grail,

Thanks For the Reply. Sorry for the late reply we are in celebration mood (Independence day here :-)) but still My Script Problem is continuing :-( :-(

Code:

bbkup.sh: line 9: syntax error near unexpected token `<'
dbbkup.sh: line 9: `done < <(mysql -u admin  -pPASSWD -D psa -Bse "select a.name, group_concat(b.name separator ' ') from domains a, data_bases b where a.id=b.dom_id group by a.name;");'

grail · 08-15-2011, 08:49 AM

Why is there a semi-colon at the end of the second errored line?

Why are there 2 scripts? What is at line 9 in the first script?