LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 06-08-2008, 09:40 PM   #1
sthompson
LQ Newbie
 
Registered: Jun 2008
Posts: 11

Rep: Reputation: 0
parsing script


ok i am trying to parse a .xml file every couple mins i would need to be able to do it constantly and have the parse be very specific to mabey a couple of lines it is for a live audio stream and i'm compeletly lost at this i don't know where to begin any help suggesttions i can run with or anything would be helpful.
 
Old 06-09-2008, 05:05 AM   #2
emi_ramo
Member
 
Registered: Apr 2007
Location: Barcelona, Spain
Distribution: Debian, KUbuntu
Posts: 213

Rep: Reputation: 36
What do you mean with 'constantly'? What is exactly the service you're trying to serve?

Last edited by emi_ramo; 06-09-2008 at 05:06 AM. Reason: correction
 
Old 06-09-2008, 07:14 AM   #3
sthompson
LQ Newbie
 
Registered: Jun 2008
Posts: 11

Original Poster
Rep: Reputation: 0
the script would have to run constanly i would want it to run 24/7 and if possible have it start a new log file every day or once a week or something like that but the script would never be stopped. and i am trying to parse a list of ip adresses for stats on a mount point from an icecast stream. there is an xml page that comes free but it does not give past history of clients only current clients so i want to parse it to make a log.

Last edited by sthompson; 06-09-2008 at 07:17 AM.
 
Old 06-09-2008, 07:28 AM   #4
emi_ramo
Member
 
Registered: Apr 2007
Location: Barcelona, Spain
Distribution: Debian, KUbuntu
Posts: 213

Rep: Reputation: 36
What about using crontab?

as a user with read access to logs and write access to stats, run:

Code:
crontab -e
For a complete description, see manual page. With that, you will be able to run commands periodically, with a user specified time interval.
 
Old 06-09-2008, 08:19 AM   #5
sthompson
LQ Newbie
 
Registered: Jun 2008
Posts: 11

Original Poster
Rep: Reputation: 0
the crontab would allow me to run the command every set time is that right? what about the parsing itself?
 
Old 06-09-2008, 08:31 AM   #6
emi_ramo
Member
 
Registered: Apr 2007
Location: Barcelona, Spain
Distribution: Debian, KUbuntu
Posts: 213

Rep: Reputation: 36
crontab is an ancient app from UNIX systems. It runs commands periodically; user or sysadmin sets its intervals. Each user has it's own crontab table. You tell crontab at which time/times to run an specified program (your script, for example), which days, months, years, ...

Syntax is a little bit complicated because it lets you a very high range of possibilities.

Just google a little bit and you'll find it out, sure.

For the parsing, you can do a every-minute parsing. Or, if it's a better idea, parse the XML every time it changes. Think, you cannot parse it constantly. At least, you must wait until a parsing process to begin another one. It's very CPU expensive to parse anything constantly, even when XML file is little, because it ends fast, and fast it rebegins...

Think if you REALLY need to change stats every microsecond or not...
 
Old 06-09-2008, 08:40 AM   #7
sthompson
LQ Newbie
 
Registered: Jun 2008
Posts: 11

Original Poster
Rep: Reputation: 0
no not that offten i said constently cause i can't think of what it would be called but i want it at max every min but what would a parse script look like for an xml file.
 
Old 06-09-2008, 10:42 AM   #8
p_s_shah
Member
 
Registered: Mar 2005
Location: India
Distribution: RHEL 3/4, Solaris 8/9/10, Fedora 4/8, Redhat Linux 9
Posts: 228
Blog Entries: 1

Rep: Reputation: 34
I am not an expert, but you can try to parse XML using perl module XML::Rules.
Link: http://search.cpan.org/~jenda/XML-Ru...b/XML/Rules.pm

Prepare your script and schedule it using Cronjob.
 
Old 06-09-2008, 12:45 PM   #9
sthompson
LQ Newbie
 
Registered: Jun 2008
Posts: 11

Original Poster
Rep: Reputation: 0
what...? that just lost me
 
Old 06-09-2008, 01:01 PM   #10
emi_ramo
Member
 
Registered: Apr 2007
Location: Barcelona, Spain
Distribution: Debian, KUbuntu
Posts: 213

Rep: Reputation: 36
I think p_s_shah told is that if you need to parse an XML file to do your stats, you can use an already done XML parser, written on perl and named XML::Rules, downloadable from http://search.cpan.org/~jenda/XML-Ru...b/XML/Rules.pm . Once you've got your program/script working, you put it into your crontab every minute.

Your script crontab entry should look something as:
* * * * * your_xml_parser.pl

And your stats rotater (at 3:10 morning, every day):
10 3 * * * your_rotater

And your stats rotater (at 3:10 morning, every sunday):
10 3 * * 7 your_rotater
 
Old 06-09-2008, 06:27 PM   #11
archtoad6
Senior Member
 
Registered: Oct 2004
Location: Houston, TX (usa)
Distribution: MEPIS, Debian, Knoppix,
Posts: 4,727
Blog Entries: 15

Rep: Reputation: 231Reputation: 231Reputation: 231
What are you parsing for?
 
Old 06-09-2008, 07:15 PM   #12
sthompson
LQ Newbie
 
Registered: Jun 2008
Posts: 11

Original Poster
Rep: Reputation: 0
i'm am parsing for a list of clients on a streaming mount point that does a live update and i want to be able to see stats on it for like 2 in the morning but i don't want to pay for a service that is stupid and give way to much i just want what i get for free an ip so i can get a location of listener and how many people conected and a rough ideal of how long. thats it.
 
Old 06-09-2008, 09:01 PM   #13
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
If you want to parse it that often, don't use cron, its a waste of resources to keep starting a new process.
Write a Perl daemon and use http://search.cpan.org/~mgrabnar/Fil...0.99.3/Tail.pm instead.
 
Old 06-10-2008, 09:46 AM   #14
archtoad6
Senior Member
 
Registered: Oct 2004
Location: Houston, TX (usa)
Distribution: MEPIS, Debian, Knoppix,
Posts: 4,727
Blog Entries: 15

Rep: Reputation: 231Reputation: 231Reputation: 231
I meant what is the format of the string(s) you are parsing for, as in what does the search regex need to look like.
Sorry for the ambiguity.

Would a background bash script w/ a sleep command be as light as a Perl Daemon?

I'm thinking something as simple as:
Code:
while :
do
   grep -o <regex> <infile>  >> <outfile>
   sleep 5m
done
Since I don't know what you're looking for, I don't know if this approach would work.

Last edited by archtoad6; 06-10-2008 at 09:51 AM. Reason: Add a thought
 
Old 06-10-2008, 10:39 AM   #15
emi_ramo
Member
 
Registered: Apr 2007
Location: Barcelona, Spain
Distribution: Debian, KUbuntu
Posts: 213

Rep: Reputation: 36
That will be worst than a crontab entry, for sure. If the problem is to launch new processes, bash scripting is far away from the solution. A daemon is a program that stays "doing nothing" until it needs to do something. In C, it should be something as:

Code:
first = t = time(NULL);
while(1)
{
   if((t-first) >= interval)
   {
      first = t = time(NULL);
      do_stats();
   }
   else
      t = time(NULL);
}
or just (for a minimal CPU usage but less accurate)

Code:
while(1)
{
   do_stats();
   sleep(interval * 1000);
}
and run this program backgrounded (with '&' at the end of the execution line, or some other fork solution).
 
  


Reply

Tags
parsing


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Help with Text Parsing Script NiallC Programming 6 06-24-2006 04:28 PM
Parsing a File in a Bash Script TGWDNGHN Programming 4 12-02-2005 03:38 PM
(shell script) string parsing kuru Programming 4 09-12-2005 08:59 PM
bash script, parsing email addresses kepler Programming 6 01-26-2004 07:47 AM
File Parsing using a Shell Script yasir15 Programming 5 08-22-2003 01:17 PM


All times are GMT -5. The time now is 11:24 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration