LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-27-2014, 08:36 AM   #1
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Rep: Reputation: Disabled
Mysql2::Error: Lost connection to MySQL server during query


I am at my wits end! I am receiving the following error:
Code:
[root@michaels gitlab]# sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production
This will create the necessary database tables and seed the database.
You will lose any previous data stored in the database.
Do you want to continue (yes/no)? yes

gitlabhq_production already exists
-- enable_extension("plpgsql")
   -> 0.0013s
bla bla bla
-- add_index("namespaces", ["type"], {:name=>"index_namespaces_on_type", :using=>:btree})
   -> 0.2003s
-- create_table("notes", {:force=>true})
rake aborted!
Mysql2::Error: Lost connection to MySQL server during query: CREATE TABLE `notes` (`id` int(11) DEFAULT NULL auto_increment PRIMARY KEY, `note` text, `noteable_type` varchar(255), `author_id` int(11), `created_at` datetime, `updated_at` datetime, `project_id` int(11), `attachment` varchar(255), `line_code` varchar(255), `commit_id` varchar(255), `noteable_id` int(11), `system` tinyint(1) DEFAULT 0 NOT NULL, `st_diff` text) ENGINE=InnoDB
/home/git/gitlab/vendor/bundle/ruby/2.0.0
bla bla bla
/gems/activerecord-4.0.3/lib/active_record/railties/databases.rake:260:in `block (3 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/setup.rake:17:in `setup_db'
/home/git/gitlab/lib/tasks/gitlab/setup.rake:4:in `block (2 levels) in <top (required)>'
Tasks: TOP => db:schema:load
(See full trace by running task with --trace)
[root@michaels gitlab]#

I am using mysql55.x86_64 , 5.5.36-1.ius.centos6.

I've tried modifing /etc/my.cnf as follows, but it produces no change:
  • Uncomment innodb_log_file_size and change from 8M to 1G
  • Uncomment innodb-buffer-pool-size and change from 128M to 4G
  • Change max-allowed-packet from 16M to 32M

What could be the cause, and how can I troubleshoot it?

Thank you
 
Old 03-27-2014, 08:41 AM   #2
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,204

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
First of all,.. Gitlab reccomends using postgres. Not sure how useful this program will be without it.

Quote:
We do not recommend using MySQL due to various issues. For example, case (in)sensitivity and problems that suggested fixes have.
But anyway, it looks like that rake task is trying to use postgres connections? (pgsql) instead of mysql.


https://github.com/gitlabhq/gitlabhq...nstallation.md
https://github.com/gitlabhq/gitlabhq...abase_mysql.md

Have you followed all the steps in those guides? Including the part that says 'skip this if you dont want to use postgres, and use mysql instead'?
 
Old 03-27-2014, 09:25 AM   #3
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by szboardstretcher View Post
First of all,.. Gitlab reccomends using postgres. Not sure how useful this program will be without it.



But anyway, it looks like that rake task is trying to use postgres connections? (pgsql) instead of mysql.


https://github.com/gitlabhq/gitlabhq...nstallation.md
https://github.com/gitlabhq/gitlabhq...abase_mysql.md

Have you followed all the steps in those guides? Including the part that says 'skip this if you dont want to use postgres, and use mysql instead'?
I successfully did this same install on another server, so it should work with MySQL. To the best of my knowledge, I've followed https://github.com/gitlabhq/gitlab-r...install/centos exactly, and skipped the steps regarding postgres.

I see what you are talking about regarding '-- enable_extension("plpgsql")'. I don't recall whether this came up on the other server successful install. Before executing the command which results in an error, I executed the following which should not use postgres.
Code:
sudo -u git -H /usr/local/bin/bundle install --deployment --without development test postgres aws
I've tried starting the install process over, but had the same results. Could there be any configuration that I need to manually remove? I would rather not rebuild the entire server just really start over.

What I've read about the "Lost connection to MySQL server during query" error is it might be memory related.

Any suggestions would be very appreciated.

Thank you
 
Old 03-27-2014, 09:44 AM   #4
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
Could this be related?

Code:
[root@michaels git]# sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-shell.git -b v1.9.1
Initialized empty Git repository in /home/git/gitlab-shell/.git/
remote: Counting objects: 1150, done.
remote: Compressing objects: 100% (703/703), done.
remote: Total 1150 (delta 677), reused 725 (delta 400)
Receiving objects: 100% (1150/1150), 160.40 KiB, done.
Resolving deltas: 100% (677/677), done.
warning: Remote branch v1.9.1 not found in upstream origin, using HEAD instead
[root@michaels git]# cd gitlab-shell
 
Old 03-27-2014, 10:41 AM   #5
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,204

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
That just means you are in a detached HEAD mode... explanation in the warning:

Quote:
git clone https://gitlab.com/gitlab-org/gitlab-shell.git -b v1.9.1
Cloning into 'gitlab-shell'...
:blah:
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
Do you have a moment to show the versions that you are using here? of git, mysql, ruby,.. any other requirements? Or are you certain that you are meeting the requirements?

Last edited by szboardstretcher; 03-27-2014 at 10:43 AM.
 
Old 03-27-2014, 11:00 AM   #6
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by szboardstretcher View Post
Do you have a moment to show the versions that you are using here? of git, mysql, ruby,.. any other requirements? Or are you certain that you are meeting the requirements?
I think you are on to something!!!! git is only 1.7.1. I am going to post a new question about this. Hopefully, this is the problem. Thank you!

On a side note, per https://dev.mysql.com/doc/refman/5.0...onnection.html
Quote:
C.5.2.3 Lost connection to MySQL server

There are three likely causes for this error message.

Usually it indicates network connectivity trouble and you should check the condition of your network if this error occurs frequently. If the error message includes “during query,” this is probably the case you are experiencing.
I will look into the git version before proceeding with this.
 
Old 03-27-2014, 11:12 AM   #7
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,204

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
Np. If you are using an old ruby version with an old gem, that might explain the mysql stuff. I would suggest following the guide and upgrading Ruby to 2.0.x and Git to 1.7.10+ where needed.
 
Old 03-27-2014, 12:07 PM   #8
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
Ugg!

I deleted the git user (and its home directory) and the database and started over.

Still the same errors. Sometimes I get "rake aborted! Errno::EBADF: Bad file descriptor: SHOW TABLES LIKE 'projects'", but mostly it is the "Lost connection to MySQL server during query" error

In regards to versions of MySQL, Ruby, and git, I should be fine on all of them.

Could it be a network issue as the link I posted to mysql suggested? How could I troubleshoot it.

Thank you

Code:
[root@michaels gitlab]# mysql -u root -p                                        Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 30
Server version: 5.5.36 Distributed by The IUS Community Project

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye
[root@michaels gitlab]# git --version
git version 1.8.3.1
[root@michaels gitlab]# ruby -v
ruby 2.0.0p451 (2014-02-24 revision 45167) [x86_64-linux]
[root@michaels gitlab]#
 
Old 03-28-2014, 07:42 AM   #9
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
http://omegadelta.net/2014/03/15/the-rails-grim-reaper/

I ended up solving the problem by commenting out "reaping_frequency: 10" in /home/git/config/database.yml.

Why this solved the issue, I am still not 100% certain.

Thanks for your help
 
2 members found this post helpful.
Old 03-28-2014, 11:09 AM   #10
divzero
LQ Newbie
 
Registered: Nov 2002
Location: Brisbane
Distribution: Fedora
Posts: 12

Rep: Reputation: 1
Hey, I'm glad my blog post was helpful

The "reaper" is meant to remove "dead" connections but it seems to be very buggy and removes live ones too.

The reaper setting is default OFF in rails now (the rails team made that change due to bugs like the ones we're having), but it looks like a bunch of sites like Heroku copied some config from when it was default ON.

Really unless you are having problems with dead/leaking connections, you don't need the reaper – it's a really bad setting. And if you *are* having issues, then there must be a better way to solve them than enabling the reaper.
 
1 members found this post helpful.
Old 03-28-2014, 12:45 PM   #11
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
Thanks divzero,

Yes your blog was very helpful! Amazing how difficult it was to find this solution. I was changing buffer sizes, etc, etc in mysql configuration.

The buggyness (if that is a word) appears to even be dependent upon hardware.
 
Old 03-28-2014, 10:39 PM   #12
divzero
LQ Newbie
 
Registered: Nov 2002
Location: Brisbane
Distribution: Fedora
Posts: 12

Rep: Reputation: 1
NotionCommotion, you're welcome Yeah I know the feeling, because most of the debugging advice for that error message are generic MySQL tips, I ended up tweaking at least a dozen config variables for MySQL with no luck. Only when I found a rails-specific reference to this error message did the penny drop.

I rolled out the fix a while ago, and the server has been running fine since so I'm confident that was the issue.

Out of curiosity, why did you have the reaper enabled? For me, it was due to following some Heroku docs.
 
Old 03-29-2014, 06:05 AM   #13
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 691

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by divzero View Post
Out of curiosity, why did you have the reaper enabled? For me, it was due to following some Heroku docs.
I did not explicitly enable the reaper. I was installing gitlab per https://github.com/gitlabhq/gitlab-r...install/centos, and it was preset in /home/git/gitlab/config/database.yml.

I will not soon forgot about the reaper.

Thanks again.
 
Old 03-30-2014, 06:49 AM   #14
divzero
LQ Newbie
 
Registered: Nov 2002
Location: Brisbane
Distribution: Fedora
Posts: 12

Rep: Reputation: 1
Quote:
Originally Posted by NotionCommotion View Post
I will not soon forgot about the reaper.
Neither will I.

I'm going to contact Heroku and petition them to change their recommendations, I suggest you do the same with gitlab. I'm actually surprised more people are not encountering this issue…
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Error Code : 2013 Lost connection to MySQL server during query jayakumar01 Linux - Server 1 01-03-2012 09:10 AM
MySQL Server Linux, CentOS VPS - Lost connection to MySQL server during query snooze1 Linux - Newbie 3 10-29-2011 03:01 AM
Error: Lost connection to MySQL server during query the expert Linux - Newbie 6 09-04-2008 12:40 PM
ERROR 2013: Lost connection to MySQL server during query kicap Linux - Software 0 11-23-2004 08:30 PM
Error: lost connection to MySQL server during query patpawlowski Programming 2 01-23-2004 07:56 AM


All times are GMT -5. The time now is 10:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration