LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 05-11-2010, 01:28 PM   #1
action_owl
Member
 
Registered: Jan 2009
Location: 127.0.0.1
Distribution: Fedora, CentOS, NetBSD
Posts: 115

Rep: Reputation: 17
Python, find unique values in a tuple or dictionary


Is there a python equivalent for the "array_unique" function in php?

I have to run a query on a very large table (1.5 million records, with no index!) and php can't beat the max execution time. So I am converting my php script to python, which I have never used before.

my php script basically does this:

gets results from database
gets the number of unique values in column "X"
counts how many times each unique value in column "X" occurs
echos the results

My php script was originally running on a smaller version of the table, I would have started in python if I would have known the "real" table was so large, python is the only language I have access to in my current environment (besides php).

Thanks
 
Old 05-11-2010, 02:27 PM   #2
urban.yoga.journeys
Member
 
Registered: Jun 2008
Posts: 31

Rep: Reputation: 16
from a spamfilter i wrote...
Code:
def word_count(s):
    """
    take a string and count the number of occurences of each word
    """
    word_rec = {}
    for word in s.split():
        word_rec[word.lower()] = 1 + word_rec.get(word.lower(), 0)
    return word_rec
the input is assumed to be a string, so if it's a list(read: array) or dictionary (hash table) values, you'll have to modify the code a bit.

the result is placed in a dictionary, the word acts as the key and the number of times it appears is the value.

hope this helps
 
Old 05-11-2010, 07:16 PM   #3
bgeddy
Senior Member
 
Registered: Sep 2006
Location: Liverpool - England
Distribution: slackware64 13.37 and -current, Dragonfly BSD
Posts: 1,810

Rep: Reputation: 227Reputation: 227Reputation: 227
A python set may be useful here. This little program counts unique occurrences in a tuple and puts those unique members and their corresponding counts into a dictionary.
Code:
mytuple=(10,4,2,3,10,6,8,5,2,5,4,4,10,1,9,2) #Create a sample tuple
mycntdict=dict([(i,mytuple.count(i)) for i in set(mytuple)]) #Fill dict
print "Tuple:",
print mytuple
print "Count dict:",
print mycntdict
Sets are useful for a simple way to get unique values. The list comprehension goes through the set members and counts their occurrrences in the original tuple and passes the resulting list as an argument to dict.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
python noob: arg1 "must be string not tuple"? agrestic Programming 5 01-07-2009 06:03 PM
Show unique values with related data in one line cgcamal Programming 1 11-21-2008 02:48 AM
HOWTO access Processor ID / other unique values?? kartheekpn Linux - Security 5 03-20-2005 04:10 AM
MySQL - UNIQUE values across two columns - non-standard constraint issue benr77 Linux - General 2 11-06-2004 10:48 AM
python; retrieving dictionary of ley=value pairs passed into method TheLinuxDuck Programming 5 08-21-2003 11:56 AM


All times are GMT -5. The time now is 10:28 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration