LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 09-27-2008, 08:42 AM   #1
donnied
Member
 
Registered: Oct 2006
Distribution: Debian x64
Posts: 198

Rep: Reputation: 30
python how to work with fake csv


I'd like to parse some data for use in an sql database (or work with it directly in Python). The data is currently in a pseudo-csv format (from xls); it only exports to pdf or xls. I'm trying to think of how I should go about parsing/ editing/ working with the data.

I am not asking for explicit code as much as thought processes and modules. I don't know how to approach the problem; I'm imagining readline , a lot of 'if, thens, fors, whiles', and globs. This does not seem appropriate.

The data comes from a file with columns (but not true columns or I think it would have exported to cvs better).

Basically, I'm interested in keeping student name as an entry with all the information for that particular student have each test 'math, reading, writing, science, social studies' have the fields associated with it 'objective, question, answer'
,,,,,,,,,,,,,,,,,,,,,,,,,.
,,,,,,,,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,
9/26/08,,,,,,,,,,,,,,,,,,,,,,,,,
"Student Name:",,,"DOE, JOHN",,,,,,"TAKS Assessment Year:",,,,,"2007-2008 0408",,,,,"Current Grade Level: 16th Grade",,,,,,
"Gender:",,"M",,,,"Eco. Dis.:",,,"Y",,,"Migrant:",,"N",,,"Bilingual:",,,"N",,,"SpecialEd:",,"N"
"Ethnicity:",,"Hispanic",,,,,"Title 1:",,"N",,,"LEP:",,"N",,,"ESL:",,,"Y",,,"Gifted:",,"N"
"Test Grade:",,9,,,,"At Risk:",,,"Y",,,"Campus:",,"My HS",,,,,,,,,,,
"Mathematics",,,,,"Reading",,,,,,"Writing",,,,,"Science",,,,,,"Social Studies",,,
"Score Code",,,"S",,"Score Code",,,,"S",,"Score Code",,,,,"Score Code",,,,,,"Score Code",,,
"Nbr of MC Qst",,,52,,"Nbr of MC Qst",,,,33,,"Nbr of MC Qst",,,,,"Nbr of MC Qst",,,,,,"Nbr of MC Qst",,,
"MC Raw Correct",,,26,,"MC Raw Correct",,,,22,,"MC Raw Correct",,,,,"MC Raw Correct",,,,,,"MC Raw Correct",,,
"MC Pct Correct",,,"50.00",,"MC Pct Correct",,,,"66.67",,"MC Pct Correct",,,,,"MC Pct Correct",,,,,,"MC Pct Correct",,,
"Scale Score",,,"2,009",,"Scale Score",,,,"2,059",,"Scale Score",,,,,"Scale Score",,,,,,"Scale Score",,,
"Met Standard",,,"N",,"Met Standard",,,,"N",,"Met Standard",,,,,"Met Standard",,,,,,"Met Standard",,,
"Commended",,,"N",,"Commended",,,,"N",,"Commended",,,,,"Commended",,,,,,"Commended",,,
"Version",,,"E",,"Version",,,,"E",,"Version",,,,,"Version",,,,,,"Version",,,
"Obj","Qst",,"Ans",,"Obj","Qst",,,"Ans",,"Obj",,"Qst","Ans",,"Obj",,"Qst",,"Ans",,"Obj",,"Qst"," Ans"
1,15,,"+",,1,1,,,"+",,,,,,,,,,,,,,,,
1,18,,"+",,1,2,,,"+",,,,,,,,,,,,,,,,
1,25,,"B",,1,3,,,"D",,,,,,,,,,,,,,,,
1,29,,"B",,1,4,,,"+",,,,,,,,,,,,,,,,
1,38,,"F",,1,15,,,"+",,,,,,,,,,,,,,,,
2,5,,"+",,1,16,,,"+",,,,,,,,,,,,,,,,
2,10,,"+",,1,17,,,"B",,,,,,,,,,,,,,,,
2,22,,"+",,1,18,,,"J",,,,,,,,,,,,,,,,
2,28,,"G",,1,19,,,"+",,,,,,,,,,,,,,,,
2,50,,"+",,2,5,,,"D",,,,,,,,,,,,,,,,
3,7,,"+",,2,6,,,"+",,,,,,,,,,,,,,,,
3,11,,"C",,2,7,,,"D",,,,,,,,,,,,,,,,
3,31,,"+",,2,8,,,"G",,,,,,,,,,,,,,,,
3,42,,"+",,2,9,,,"+",,,,,,,,,,,,,,,,
3,48,,"F",,2,10,,,"+",,,,,,,,,,,,,,,,
4,17,,"+",,2,11,,,"D",,,,,,,,,,,,,,,,
4,21,,"+",,2,12,,,"+",,,,,,,,,,,,,,,,
4,36,,"+",,2,20,,,"+",,,,,,,,,,,,,,,,
4,45,,"+",,2,21,,,"+",,,,,,,,,,,,,,,,
4,52,,"J",,2,22,,,"+",,,,,,,,,,,,,,,,
5,1,,"+",,2,28,,,"J",,,,,,,,,,,,,,,,
5,12,,"F",,3,13,,,"+",,,,,,,,,,,,,,,,
5,41,,"B",,3,14,,,"F",,,,,,,,,,,,,,,,
5,47,,"B",,3,23,,,"+",,,,,,,,,,,,,,,,
6,13,,"+",,3,24,,,"J",,,,,,,,,,,,,,,,
6,20,,"H",,3,25,,,"+",,,,,,,,,,,,,,,,
6,33,,"C",,3,26,,,"+",,,,,,,,,,,,,,,,
6,39,,"C",,3,27,,,"+",,,,,,,,,,,,,,,,
7,2,,"+",,3,29,,,"+",,,,,,,,,,,,,,,,
7,8,,"+",,3,30,,,"+",,,,,,,,,,,,,,,,
7,37,,"C",,3,31,,,"+",,,,,,,,,,,,,,,,
7,46,,"+",,3,32,,,"+",,,,,,,,,,,,,,,,
8,4,,"H",,3,33,,,"B",,,,,,,,,,,,,,,,
8,19,,"+",,,,,,,,,,,,,,,,,,,,,,
8,23,,"D",,,,,,,,,,,,,,,,,,,,,,
8,26,,"F",,,,,,,,,,,,,,,,,,,,,,
8,34,,"F",,,,,,,,,,,,,,,,,,,,,,
8,43,,"C",,,,,,,,,,,,,,,,,,,,,,
9,6,,"J",,,,,,,,,,,,,,,,,,,,,,
9,9,,"B",,,,,,,,,,,,,,,,,,,,,,
9,24,,"+",,,,,,,,,,,,,,,,,,,,,,
9,49,,"+",,,,,,,,,,,,,,,,,,,,,,
9,51,,"A",,,,,,,,,,,,,,,,,,,,,,
10,3,,"+",,,,,,,,,,,,,,,,,,,,,,
10,14,,"+",,,,,,,,,,,,,,,,,,,,,,
10,16,,"+",,,,,,,,,,,,,,,,,,,,,,
10,27,,"D",,,,,,,,,,,,,,,,,,,,,,
10,30,,"H",,,,,,,,,,,,,,,,,,,,,,
10,32,,"+",,,,,,,,,,,,,,,,,,,,,,
10,35,,"+",,,,,,,,,,,,,,,,,,,,,,
10,40,,"G",,,,,,,,,,,,,,,,,,,,,,
10,44,,"F",,,,,,,,,,,,,,,,,,,,,,
 
Old 09-28-2008, 06:13 PM   #2
amani
Senior Member
 
Registered: Jul 2006
Location: Kolkata, India
Distribution: 64-bit GNU/Linux, Kubuntu64, Fedora QA, Slackware,
Posts: 2,758

Rep: Reputation: Disabled
open it in gnumeric and save it properly
 
Old 09-29-2008, 06:05 AM   #3
donnied
Member
 
Registered: Oct 2006
Distribution: Debian x64
Posts: 198

Original Poster
Rep: Reputation: 30
It looks the same.

I opened and saved with Gnumeric and it looked the same. A problem is that the columns aren't clearly defined because each student starts with a lot of demographic info. I would like to be able to fish the data out for each student and import it into sql (using the sqlite module). The xlrd module looks like the way to go.
I am not sure however how I will get the data I want organized by student name. I would like to stay in python but I grep, sed, and awk look like the (albeit inefficient) backup plan.
 
  


Reply

Tags
csv, data, python, sql


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Comparing two csv files and write different record in third CSV file irfanb146 Linux - Newbie 3 06-30-2008 09:15 PM
Can't get Python to work. Semoset Linux - Software 10 08-19-2007 02:45 AM
Tk for Python doesn't work datbenik Linux - Software 0 12-30-2005 01:42 PM
Python doesn't work zirtik Programming 9 10-17-2005 05:28 AM
Python XML won't work. Travis86 Programming 0 09-14-2004 05:13 PM


All times are GMT -5. The time now is 06:25 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration