Attempting to determine reason for slowdown of database software between CentOS 6.2 and CentOS 7.2

EnderX · 11-09-2016, 12:48 PM

I am not that familiar with Linux - if this belongs in the 'Newbies' section, or anywhere else on this site, please move it there. I chose to post here because I am dealing with an issue between two versions of CentOS.

Recently, my boss has had me trying to build a new machine to use as a replacement device for our main system. I haven't had any problems installing anything, but on attempting to run an update against our database, we ran into a major slowdown relative to our other machine. (As in: A nightly update file, which normally takes ~20-30 minutes, takes between six and seven hours on the new machine when using the same upload process.) I am looking into the differences in the two versions of the database software, but given that I've tuned their config files as close to identically as possible I'm also trying to figure out where else the slowdown might be coming from, and am hoping that those here might be able to assist me in figuring out at least where to look next.

Old server:
Actual desktop device within our office.
Being loaded from Windows machine within our office.
'cat /etc/*-release' shows 'CentOS release 6.2 (Final)
Database software is Postgresql 8.4.9.
free results:

Code:

             total       used       free     shared    buffers     cached
Mem:      32586484   13285288   19301196          0     312600   11874580
-/+ buffers/cache:    1098108   31488376
Swap:     10010616          0   10010616

New server:
Virtual machine located in an offsite office.
Being loaded from Windows machine sitting in same offsite office.
'cat /etc/*-release' shows 'CentOS Linux release 7.2.1511 (Core)'
Database software is Postgresql 9.2.15
free results:

Code:

              total        used        free      shared  buff/cache   available
Mem:       32781448      377436     5725172       71660    26678840    31935344
Swap:      16515068         480    16514588

Again, I am not familiar with the inner workings of Linux. What are the best places to start looking for the source of this slowdown, and what do I need to be looking for when I get there to track the source of this problem down?

smallpond · 11-09-2016, 05:06 PM

Your three areas to look at are network speed, storage speed and CPU speed. You will want to compare what you are seeing on the VM to your old system.

Network: upload a large file from Windows to /tmp using WinSCP. Since tmpfs is in memory, this eliminates disk speed issues.

Storage speed: Same test but go to disk and include the time to do 'sync' at the end.

CPU: dmesg |grep Bogo

Doug G · 11-09-2016, 06:32 PM

Verify DNS resolving works properly on the db server.

chrism01 · 11-10-2016, 10:29 PM

Are you absolutely sure the DB structures etc are identical eg if someone added/removed an index from the old DB, but that change never made it into the official schema version that you used to construct the new one, that could make a difference.
Did you define the initial disk space (inside the DB) to be the same or bigger when originally building the new one? Maybe its having to do space extensions on the fly?