replace into is extreme slow, does it occur in slack-current?
SlackwareThis Forum is for the discussion of Slackware Linux.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
The solution is either to use myisam or upgrade to 5.7. (insert ...duplicate command isn't right for my situation.) I don't know what's different between innodb and myisam, but I'm told to use innodb because something technical. But if myisam is faster, shouldn't I use that one? Why ride a snail when you have a horse?
Anyway, I'm wondering if slackware-current's mariadb has this issue? I can't tell by the file name, since it doesn't use Oracle's versioning.
I don't know what MySQL/MariaDB is good for other than wasting precious time on it, pretending to use a relational DB, as I myself did a long time ago, by the time PostgreSQL was a little difficult to build, to tune and a lot of people called it "way too complex", BullS..t.
MySQL became popular as a small DB for storing webpages data and simple tables, part of the LAMP bundle.
As far as I remember, MyISAM was the default engine MySQL started with, not ACID compliant (thus cannot be called a database, but a trivial form of organizing data), didn't support transactions and very fast because of these "handicaps". Then, recently, the slower innoDB was introduced as an alternative engine, ACID compliant and supporting transactions, helping MySQL to become and be called a DB.
You can start with this article (and the links contained) if you'd like to learn more about these two engines: https://stackoverflow.com/questions/...sam-and-innodb
MariaDB has some more improvements over these two engines, but I cannot tell you if these improvements are faster or not, as I'm not even installing it in Slackware: https://mariadb.com/kb/en/library/ch...torage-engine/
Depending on your use case of DB, or if you'd like to get serious about databases, learn (standard) SQL and use a true&professional open source DB, then I suggest moving to PostgreSQL. You'll thank me later. https://slackbuilds.org/repository/1...em/postgresql/
Last edited by abga; 09-06-2018 at 01:58 PM.
Reason: typo
The solution is either to use myisam or upgrade to 5.7. (insert ...duplicate command isn't right for my situation.) I don't know what's different between innodb and myisam, but I'm told to use innodb because something technical. But if myisam is faster, shouldn't I use that one? Why ride a snail when you have a horse?
Anyway, I'm wondering if slackware-current's mariadb has this issue? I can't tell by the file name, since it doesn't use Oracle's versioning.
EDIT: I haven't used PostgreSQL for ~3 years for anything special, but you will have to run a job to purge deleted records from your tables every so often. Even so, I'll agree with @abga's assessment of the two databases; PostgreSQL was the DB we migrated to at work after deciding the Oracle was just too expensive for our use case. It was not possible to migrate to MySQL since it did not support database triggers at that time (assuming that it does so now).
Last edited by Richard Cranium; 09-07-2018 at 10:54 PM.
The solution is either to use myisam or upgrade to 5.7. (insert ...duplicate command isn't right for my situation.) I don't know what's different between innodb and myisam, but I'm told to use innodb because something technical. But if myisam is faster, shouldn't I use that one? Why ride a snail when you have a horse?
Anyway, I'm wondering if slackware-current's mariadb has this issue? I can't tell by the file name, since it doesn't use Oracle's versioning.
The REPLACE INTO is (in)famous slow since Ice Age in MySQL.
AND generally speaking I seen its usage being considered a quite bad habit which slowdowns considerably the sites.
The real solution is to use general compatible queries like "INSERT INTO <blah blah> ON DUPLICATE KEY UPDATE <blah blag>" and call a day.
Last edited by Darth Vader; 09-08-2018 at 02:02 AM.
EDIT: I haven't used PostgreSQL for ~3 years for anything special, but you will have to run a job to purge deleted records from your tables every so often. Even so, I'll agree with @abga's assessment of the two databases; PostgreSQL was the DB we migrated to at work after deciding the Oracle was just too expensive for our use case. It was not possible to migrate to MySQL since it did not support database triggers at that time (assuming that it does so now).
With PostgreSQL 8.1 (2005) the AUTOVACUUM daemon was introduced, it's automatically started and you can tune its vacuuming frequency/schedule.
In addition to the lack of some functionalities, MySQL/MariaDB doesn't adhere to standard SQL and that's a big PITA if you need to export its code / import other standard SQL code in it.
With PostgreSQL 8.1 (2005) the AUTOVACUUM daemon was introduced, it's automatically started and you can tune its vacuuming frequency/schedule.
Ah, that's good to know!
Quote:
In addition to the lack of some functionalities, MySQL/MariaDB doesn't adhere to standard SQL and that's a big PITA if you need to export its code / import other standard SQL code in it.
Well, there's MySQL unique stuff that people can (and do) use, which leads to the PITA you mention. (Or "vendor lock-in" as others call it.)
As for choosing the right DB engine, or storage engine, with MySQL/MariaDB, based on my previous post, I can advise you to:
- use MyISAM for when you just want to load some data and run some ad-hoc analysis on it, as fast as MySQL/MariaDB can do it.
- use InnoDB for when you need an actual database and care about your data, be ACID compliant, like to support transactions and some other extensions. This is definitely slower. https://en.wikipedia.org/wiki/ACID
& again, you might want to consider PostgreSQL
@Richard Cranium https://mariadb.com/kb/en/library/replace/
"REPLACE is a MariaDB/MySQL extension to the SQL standard." ... useless as it turns out, or as you say, just for the purpose of "vendor lock-in"
Last edited by abga; 09-10-2018 at 04:58 PM.
Reason: eating letters
I did not consider that. "REPLACE INTO" is very convenient, I didn't realize it was "unique" to MySQL/MariaDB. I will consider postegreSQL and try out "INSERT...DUPLICATE" even if it feels really awkward.
I'm really an SQL newb, and was actually considering SQLite, since what I'm doing isn't really the most demanding of tasks, and almost could be done with a spreadsheet, but the files would just be too big.
I did not consider that. "REPLACE INTO" is very convenient, I didn't realize it was "unique" to MySQL/MariaDB. I will consider postegreSQL and try out "INSERT...DUPLICATE" even if it feels really awkward.
I'm really an SQL newb, and was actually considering SQLite, since what I'm doing isn't really the most demanding of tasks, and almost could be done with a spreadsheet, but the files would just be too big.
Nope, is not something very convenient (appearances excluded), but a crap extremely prone for dubious errors.
Because it removes the old record if exists and insert always a new one. That will mess with the primary keys and will eventually produce many dubious errors.
For example, imagine a site where the user "admin" has the primary key "id" "1", and a field "realname"
Usually it is supposed that the IDs stay constant, and many sites use things like
Code:
if ($user->id === 1) {
// This is the super-user, permit him a full access
}
BUT, with that goddamned "REPLACE INTO" when you update the "realname" as "John Doe", the record will be deleted and be inserted one with "id" 666 or whatever. And the site go nuts.
It is just like in Linux, as if when you update the root password, then its UID suddenly become 1755. That will be pretty pretty bad. It will not be root anymore, but some user with elevated access.
Long story short, the "REPLACE INTO" is not "a convenience" but right on another name for "big troubles".
Last edited by Darth Vader; 09-15-2018 at 02:24 AM.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.