LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-27-2017, 03:35 PM   #1
mermelmadness
LQ Newbie
 
Registered: Jan 2017
Location: Brooklyn, NY
Posts: 3

Rep: Reputation: Disabled
Scripting Help (Complex)


So, here's what I have and what I need.

I have approximately 800 pairs of files. A .log and a .xml:

5559_27589_TVE_HULU_AE5559_AEN_BSST_158395_TVE_000_2398_60_20150721_000.log
5559_27589_nmr.xml

I also have an Excel spreadsheet of metadata

The XML has specific fields that are empty and need the metadata inserted:

<NMRVodMetadata>
<ADIFileName/>
<DsrcId></DsrcId>
<DsrcName></DsrcName>
<AssetId></AssetId>
<AssetName></AssetName>
<EpisodeId></EpisodeId>

<WatermarkedTime>01/06/17 14:22:35</WatermarkedTime>
<SID>5559</SID>
<StartingTIC>27589</StartingTIC>
<EndingTIC>30179</EndingTIC>
<AllocatedStartTIC>27589</AllocatedStartTIC>
<AllocatedEndTIC>30206</AllocatedEndTIC>
<TicsRemaining>268405248</TicsRemaining>
<HDContent></HDContent>
<FileSizeBytes>1490688000</FileSizeBytes>
<FileSizeSeconds>2588</FileSizeSeconds>
<ApplicationName></ApplicationName>
<ApplicationVersion></ApplicationVersion>

<EncoderEngineName>Nielsen Watermark Engine</EncoderEngineName>
<EncoderEngineVersion>1.9.15</EncoderEngineVersion>
</NMRVodMetadata>

The orange fields are static values. The yellow fields are information that exists in the Excel doc.

Basically, for all the XMLs, we need to add the static values into the orange fields.

AND

- We need to correlate the ID# in the first column (158395) with the .log file, which has the value in the file name.
- Then, we need to correlate that specific .log file with the corresponding .xml file that shares the same prefix (5559_27589).
- Then we need to take the information in the columns next to the ID# and populate the blank fields in the XML with that information.

Any help in automating this process or giving me some CLI commands would be greatly appreciated.

Thanks.
 
Old 01-31-2017, 03:51 AM   #2
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6 / 7
Posts: 3,185

Rep: Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377
Ok, so what have you done so far?
 
Old 01-31-2017, 06:38 AM   #3
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,320
Blog Entries: 13

Rep: Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372Reputation: 4372
Hi mermelmadness and welcome to LQ,

Please read the link in TenTenth's post about how to ask a question, and here is a repeat of that link: How to ask a Question.

The point here is that while you have described the input and your desired output, you have not discussed what language you intend to do your project in, and whether or not you have tried to solve any part of this yet.

LQ members are volunteers and not here to provide "on demand" code. Instead we volunteer to help you to learn the tools you need to accomplish your goals. Along the way, when you show some effort, people will gladly offer suggested improvements.

What are your ideas for automation? Were you planning to write a script or a program? What CLI commands are you considering and have you checked the manual pages?
 
Old 01-31-2017, 06:41 AM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,818

Rep: Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083Reputation: 3083
As above, but also, please use [code][/code] tags around your code and data as this will help maintain the formatting.

Probably a suggestion would be to convert the Excel docs to csv text files so they can be easily read.
 
Old 01-31-2017, 06:42 AM   #5
MrMeeSeeks
Member
 
Registered: Jan 2017
Posts: 32

Rep: Reputation: Disabled
Well, I'm a newb myself, so by now I'd probably export the Excel-file to CSV, and then go crazy with grep, head/tail/cut and paste.
The correlations you need should be not that big of a deal with string-matching-conditionals generally available in bash.

Last edited by MrMeeSeeks; 01-31-2017 at 06:45 AM.
 
Old 01-31-2017, 06:47 AM   #6
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,617
Blog Entries: 3

Rep: Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871Reputation: 2871
Quote:
Originally Posted by mermelmadness View Post
The XML has specific fields that are empty and need the metadata inserted:
Perl does XML parsing rather easily. Try looking at the XML::TreeBuider::XPath module or one of the other XML parsers. There are also modules which can read CSV.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Shell Scripting Part I: Getting started with bash scripting LXer Syndicated Linux News 0 04-29-2015 08:03 AM
Bash scripting: a very complex question about......... zemir Programming 1 08-09-2011 05:27 AM
LXer: Scripting the Linux desktop, Part 2: Scripting Nautilus LXer Syndicated Linux News 0 02-17-2011 04:02 AM
teaching shell scripting: cool scripting examples? fax8 Linux - General 1 04-20-2006 04:29 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:21 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration