LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-29-2014, 05:21 AM   #1
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Rep: Reputation: Disabled
Mysql not starting


mysql stopped working over the weekend and I can't get it to restart on my vps
I'm getting this error.

141229 10:00:36 [Note] Plugin 'FEDERATED' is disabled.
141229 10:00:36 InnoDB: The InnoDB memory heap is disabled
141229 10:00:36 InnoDB: Mutexes and rw_locks use GCC atomic builtins
141229 10:00:36 InnoDB: Compressed tables use zlib 1.2.3.4
^G/usr/sbin/mysqld: Can't create/write to file '/tmp/ibHw49jH' (Errcode: 28)
141229 10:00:36 InnoDB: Error: unable to create temporary file; errno: 28
141229 10:00:36 [ERROR] Plugin 'InnoDB' init function returned error.
141229 10:00:36 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
141229 10:00:36 [ERROR] Unknown/unsupported storage engine: InnoDB
141229 10:00:36 [ERROR] Aborting


I've been trying every post i can find but am not getting any further so any help would be amazing
 
Old 12-29-2014, 05:46 AM   #2
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
Code:
^G/usr/sbin/mysqld: Can't create/write to file '/tmp/ibHw49jH' (Errcode: 28)
I think it's fairly basic. Error number 28 is "No space left on device."

/tmp is full. Go figure.
 
Old 12-29-2014, 06:53 AM   #3
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
My understanding is limited and that's what I thought but when I run df -h

Filesystem Size Used Avail Use% Mounted on
/dev/vda3 49G 19G 27G 42% /
udev 490M 4.0K 490M 1% /dev
tmpfs 200M 224K 199M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 498M 0 498M 0% /run/shm
/dev/vda1 461M 37M 401M 9% /boot


which seems to show lots of space
 
Old 12-29-2014, 02:35 PM   #4
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
tmpfs is not /tmp - That's a newbie banana skin, I'm afraid. Your check is interesting, though.

Code:
tmpfs 200M 224K 199M 1% /run
My box shows tmpfs the size of half the total ram fitted in tmpfs. I have 6G; tmpfs is 2.9G. With mysqld loading up it maybe ran out of memory. You can check ram with free and top, (each with man pages). A command like

free -ms 1

will print a free memory report every 1 second (ctrl_c to quit). So

free -ms 1 && <command to start mysql> would tell you how the ram survives as mysql starts. I suspect you have not enough ram in your vm.

Last edited by business_kid; 12-29-2014 at 02:38 PM.
 
Old 12-30-2014, 04:03 AM   #5
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
free -ms 1 brings back

total used free shared buffers cached
Mem: 995 650 345 0 163 97
-/+ buffers/cache: 388 606
Swap: 476 0 476

and after deleting some log files yesterday mysql seems to be running

Code:
 sudo netstat -tap | grep mysql
tcp        0      0 localhost:mysql         *:*                     LISTEN      1340/mysqld
but I'm still getting the same error on my site

SQLSTATE[HY000]: General error: 1 Can't create/write to file '/tmp/#sql_53c_0.MYI' (Errcode: 28)

which again points to low disk space or inability to write to /tmp

(by the way you have no idea how much I appreciate your help on this)
 
Old 12-30-2014, 04:49 AM   #6
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
why not have a look around in /tmp?
Code:
ls -a /tmp
The directory can fill up with crap - I get reams of hidden files with names like
..xfsm-ICE-<random_crap> session manager files
Symlinks 5<random_crap> - pages printed via cups
lilo<random_crap> lilo sessions I presume
dhcp<random_crap> dhcp logons

also .tmp and .part files in abundance. I have this in rc.local to keep the crap down
Code:
rm -rf /tmp/{*.part,*.tmp,gpg*,5*}
that cleans them out every time I boot. Do not do a global rm -rf on /tmp ever. Some of that stuff is/may be needed.
 
Old 12-30-2014, 06:11 AM   #7
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
There doesn't seem to be very much there at all

Code:
ls -a /tmp
.  ..  .ICE-unix  .X11-unix
 
Old 12-30-2014, 07:41 AM   #8
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
ok, cd there as root. Check what the mysql user is and try

Code:
su <mysql_user>
touch /tmp/somefile
How does that go?
 
Old 12-30-2014, 09:04 AM   #9
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
touch /tmp/somefile
touch: cannot touch `/tmp/somefile': No space left on device
 
Old 12-30-2014, 10:12 AM   #10
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
That looks to be your problem, which I told you about in post #2. You have the box. Go figure.

If you exit from mysql-user and use root, is there still no space? It could be write permissions (Unlikely)

As a workaround, you may be able to set $TMPDIR to ~/ where they will have write permissions.

Code:
export TMPDIR=~/
 
Old 12-30-2014, 10:29 AM   #11
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
Nope same answer with root

This is where my I'm getting lost, The file system shows I have 27g available but there is no space in the /tmp directory, looking in the /tmp directory there doesn't seem to be many files there either.

If I Move the temp directory would that solve the space problem or is that just if there were permission issues.

Again thanks for your patients
 
Old 12-30-2014, 12:09 PM   #12
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
It could be a directory issue, or a filesystem problem. Mioving it sounds good

Code:
cd /
du -sh /tmp  #tells the size
mv tmp crap
install -dm 0777 tmp
 
Old 12-31-2014, 03:54 AM   #13
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
moved the directory when I got to
Code:
install -dm 0777 tmp
install: cannot change permissions of `tmp': No such file or directory
Tried
Quote:
sudo install -dm 0777 tmp
install: cannot change permissions of `tmp': No such file or directory
Tried restarting mysql

Code:
sudo service mysql restart
mysql stop/waiting
start: Job failed to start
 
Old 12-31-2014, 08:12 AM   #14
dfeg
LQ Newbie
 
Registered: Dec 2014
Location: Ireland
Posts: 13

Original Poster
Rep: Reputation: Disabled
I've just noticed the the error message on the site has now changed to
SQLSTATE[HY000] [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

when i checked the sql erro file if found this

Code:
141229  9:00:02 [Note] /usr/sbin/mysqld: Normal shutdown

141229  9:00:02 [Note] Event Scheduler: Purging the queue. 0 events
141229  9:00:04  InnoDB: Starting shutdown...
141229  9:00:08  InnoDB: Shutdown completed; log sequence number 4760563155
141229  9:00:08 [Note] /usr/sbin/mysqld: Shutdown complete

141229  9:00:09 [Note] Plugin 'FEDERATED' is disabled.
141229  9:00:09 InnoDB: The InnoDB memory heap is disabled
141229  9:00:09 InnoDB: Mutexes and rw_locks use GCC atomic builtins
141229  9:00:09 InnoDB: Compressed tables use zlib 1.2.3.4
141229  9:00:09 InnoDB: Initializing buffer pool, size = 128.0M
141229  9:00:09 InnoDB: Completed initialization of buffer pool
141229  9:00:09 InnoDB: highest supported file format is Barracuda.
^G/usr/sbin/mysqld: Can't create/write to file '/tmp/ibKuG8ir' (Errcode: 28)
141229  9:00:09  InnoDB: Error: unable to create temporary file; errno: 28
141229  9:00:09  InnoDB: Assertion failure in thread 139994462189376 in file dict0dict.c line 732
InnoDB: Failing assertion: dict_foreign_err_file
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
09:00:09 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 346681 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x29)[0x7f5300869039]
 
Old 12-31-2014, 08:15 AM   #15
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 7,576

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
for information, post the output of
Code:
ls -a /
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Can't find file mysql/plugin.frm” when starting up MySQL byran cheung Linux - Software 1 01-11-2014 06:06 AM
Mysql not starting after changing mysql admin password in fedora prastal Linux - Newbie 1 02-20-2010 08:26 PM
Problem Starting MySQL on Fresh Fedora 8 Install w/ Yum'd MySQL blong4life Linux - Server 2 07-04-2008 08:44 PM
How to overcome the RPM "mysql-5.0.27-1.fc6" conflict with mysql database starting kamakshiganesh Linux - General 0 10-22-2007 08:07 AM
Starting MySQL OLD Error : imeout error occurred trying to start MySQL Daemon Fr33B5D Linux - Software 9 10-28-2006 07:23 AM


All times are GMT -5. The time now is 10:45 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration