LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 09-01-2005, 09:02 AM   #1
nodger
Member
 
Registered: Oct 2003
Location: Ireland
Distribution: Slackware 9.1, Ubuntu
Posts: 192

Rep: Reputation: 30
Data Storage Question


I need to implement a database system, either using an existing library, or with my bare hands, that does the following:

Can store records of arbitrary length
Can increase/decrease the length of these records
Can insert/delete/retrieve records given a user-defined key

Whats my best option for fastest performance? At the moment all i can think of is just storing each record as a file on a ReiserFS file system. This is a quick and dirty solution, my database will (hopefully) run into the millions so I need something thats fast and reliable.

Edit: Don't be afraid to point me in the direction of some really complicated theoretical webpage thats full of equations ant the like, as long as its rlevant
All suggestions welcome

Last edited by nodger; 09-01-2005 at 09:07 AM.
 
Old 09-01-2005, 10:19 AM   #2
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Of course, a full featured SQL-database would be the most flexible. The best-known open-source options are MySQL and PostgreSQL. Arguably, MySQL is more widely used, has more information available on the internet and is easier to use, than PostgreSQL. On the other side, PostgreSQL is probably more reliable/robust.

For local-disk database-libraries (no direct networking support, very fast, but more difficult to use), Berkeley DB and CDB come to mind. CDB has a 4Gb limit per database, and only source-distribution is allowed, but can be expected to be robust and fast.

Just files on you ReiserFS, would be a bad idea, if you're serious about lots of data and still having good performance.

From the information you posted, and assuming you're not afraid of some bare-metal programming, I'd think my choice would be Berkeley DB.
 
Old 09-01-2005, 10:44 AM   #3
nodger
Member
 
Registered: Oct 2003
Location: Ireland
Distribution: Slackware 9.1, Ubuntu
Posts: 192

Original Poster
Rep: Reputation: 30
I had a look at the Berkeley DB API, and the only database type that allows for arbritrary-length records is RECNO. However this is not suited to my needs because the keys with RECNO are generated by Berkeley, whereas I want to use a 64-bit hash as a key.

No offence, but I want to steer clear of connecting to MySQL or Postgre Sql because theres a major overhead involved there. also to the best of my knowledge, MySQL does not allow arbritrary record lengths.
 
Old 09-01-2005, 04:08 PM   #4
nodger
Member
 
Registered: Oct 2003
Location: Ireland
Distribution: Slackware 9.1, Ubuntu
Posts: 192

Original Poster
Rep: Reputation: 30
come on!!! help me out
 
Old 09-01-2005, 04:18 PM   #5
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,000
Blog Entries: 11

Rep: Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893
Please do not bump your posts before 24 hours have
passed. Your problem is no more important than any-
one elses, and you're not paying for support here,
either ...


Cheers,
Tink
 
Old 09-01-2005, 06:42 PM   #6
carl.waldbieser
Member
 
Registered: Jun 2005
Location: Pennsylvania
Distribution: Kubuntu
Posts: 197

Rep: Reputation: 32
What about SQLLite?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Common Data Storage Woodsman Slackware 1 10-30-2005 11:00 PM
Location for script data storage Skazi Linux - General 1 10-15-2005 10:55 AM
data storage in applications dave bean Programming 3 01-18-2005 06:39 PM
Data storage of LDAP? grubjo Linux - General 1 09-03-2004 11:03 PM
Data Storage and Recovery Docs shahul75 Linux - General 3 09-10-2002 03:28 AM


All times are GMT -5. The time now is 03:13 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration