LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud
User Name
Password
Linux - Virtualization and Cloud This forum is for the discussion of all topics relating to Linux Virtualization and Linux Cloud platforms. Xen, KVM, OpenVZ, VirtualBox, VMware, Linux-VServer and all other Linux Virtualization platforms are welcome. OpenStack, CloudStack, ownCloud, Cloud Foundry, Eucalyptus, Nimbus, OpenNebula and all other Linux Cloud platforms are welcome. Note that questions relating solely to non-Linux OS's should be asked in the General forum.

Notices


Reply
  Search this Thread
Old 10-21-2016, 06:30 AM   #1
af7567
Member
 
Registered: Nov 2012
Posts: 295

Rep: Reputation: 106Reputation: 106
Can't communicate with libvirt anymore


Hi
I originally posted this here: http://www.linuxquestions.org/questi...gs-4175591815/

But this might be a better forum to ask on.
Slackware64-current, libvirt-2.2.0, Linux kernel 4.7.4 and 4.4.22 tested.

I have been using libvirt with virt-manager for a while, and yesterday after turning on my PC virt-manager wasn't able to connect to libvirt anymore. I tried "virsh connect" but that also just freezes. No packages have been installed or upgraded on my system and no changes made to any configuration.

It seems as if libvirtd is getting stuck when reading from its socket.

After some playing around I found that if I remove libvirt_driver_network and libvirt_driver_nwfilter from /usr/lib64/libvirt/connection-driver then libvirtd will start up and I can communicate with it with virsh and virt-manager - but it is lacking functionality.

If I put libvirt_driver_network back into /usr/lib64/libvirt/connection-driver but leave nwfilter out of there then libvirtd will start up and be usable the SECOND time I run it after a system restart. The first time running it, it will still get stuck on the socket so I have to kill it.

If I put nwfilter back at all then it will always get stuck no matter how many times I kill and restart it.

This is the output I get when running libvirtd -v with nwfilter driver enabled.
Code:
...
2016-10-21 10:45:31.400+0000: 6873: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fc8040ffd10
2016-10-21 10:45:31.400+0000: 6873: info : virObjectUnref:261 : OBJECT_DISPOSE: obj=0x7fc8040ffd10
2016-10-21 10:45:31.400+0000: 6873: info : virObjectNew:202 : OBJECT_NEW: obj=0x7fc8040fba50 classname=virNodeDeviceEventLifecycle
2016-10-21 10:45:31.400+0000: 6873: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fc8040fba50
2016-10-21 10:45:31.400+0000: 6873: info : virObjectUnref:261 : OBJECT_DISPOSE: obj=0x7fc8040fba50
2016-10-21 10:45:31.411+0000: 6873: info : virObjectNew:202 : OBJECT_NEW: obj=0x7fc8040f6740 classname=virNodeDeviceEventLifecycle
2016-10-21 10:45:31.411+0000: 6873: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fc8040f674
0
2016-10-21 10:45:31.411+0000: 6873: info : virObjectUnref:261 : OBJECT_DISPOSE: obj=0x7fc8040f6740
2016-10-21 10:45:31.417+0000: 6873: info : virObjectNew:202 : OBJECT_NEW: obj=0x7fc8040f6740 classname=virNodeDeviceEventLifecycle
2016-10-21 10:45:31.417+0000: 6873: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fc8040f6740
2016-10-21 10:45:31.417+0000: 6873: info : virObjectUnref:261 : OBJECT_DISPOSE: obj=0x7fc8040f6740
2016-10-21 10:45:31.419+0000: 6873: info : virObjectNew:202 : OBJECT_NEW: obj=0x7fc8040282f0 classname=virSecretObjList
2016-10-21 10:45:31.425+0000: 6873: info : virFirewallCheckUpdateLock:127 : using locking for /usr/sbin/iptables
2016-10-21 10:45:31.430+0000: 6873: info : virFirewallCheckUpdateLock:127 : using locking for /usr/sbin/ip6tables

....  This is where it stops so <CTRL-C>  ....

2016-10-21 10:45:33.700+0000: 6857: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=5 timeout=-1
2016-10-21 10:45:33.700+0000: 6857: info : virEventPollDispatchHandles:506 : EVENT_POLL_DISPATCH_HANDLE: watch=2 events=1
2016-10-21 10:45:33.700+0000: 6857: info : virNetlinkEventServiceStopAll:715 : stopping all netlink event services
2016-10-21 10:45:33.701+0000: 6857: info : virEventPollRemoveHandle:186 : EVENT_POLL_REMOVE_HANDLE: watch=6
2016-10-21 10:45:33.701+0000: 6857: info : virEventPollRemoveHandle:186 : EVENT_POLL_REMOVE_HANDLE: watch=7
2016-10-21 10:45:33.701+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c2670
2016-10-21 10:45:33.701+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c27f0
2016-10-21 10:45:33.701+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c2900
2016-10-21 10:45:33.701+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c3250
2016-10-21 10:45:33.701+0000: 6857: info : virEventPollRemoveHandle:186 : EVENT_POLL_REMOVE_HANDLE: watch=5
2016-10-21 10:45:33.701+0000: 6873: info : virFirewallCheckUpdateLock:125 : locking not supported by /usr/sbin/ebtables
2016-10-21 10:45:33.701+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c5340
2016-10-21 10:45:33.701+0000: 6857: info : virEventPollRemoveHandle:186 : EVENT_POLL_REMOVE_HANDLE: watch=3
2016-10-21 10:45:33.701+0000: 6873: info : virFirewallApplyGroup:895 : Starting transaction for firewall=0x7fc8040fd170 group=0x7fc8040ef010 flags=0
2016-10-21 10:45:33.701+0000: 6873: info : virFirewallApplyRule:838 : Applying rule '/usr/sbin/iptables -w --version'
2016-10-21 10:45:33.701+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c2980
2016-10-21 10:45:33.701+0000: 6857: info : virEventPollRemoveHandle:186 : EVENT_POLL_REMOVE_HANDLE: watch=4
2016-10-21 10:45:33.702+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c3c50
2016-10-21 10:45:33.702+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2bab50
2016-10-21 10:45:33.702+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2b9290
2016-10-21 10:45:33.702+0000: 6857: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x55adac2c2a40
Without nwfilter it would continue past the virFirewallCheckUpdateLock lines and start loading my virtual machine configurations.

If I try running it with gdb and doing a backtrace when it is stuck I get
Code:
(gdb) bt
#0  0x00007ffff267fcdd in poll () from /lib64/libc.so.6
#1  0x00007ffff72ae3c4 in virEventPollRunOnce () from /usr/lib64/../lib64/libvirt.so.0
#2  0x00007ffff72ad0c1 in virEventRunDefaultImpl () from /usr/lib64/../lib64/libvirt.so.0
#3  0x00007ffff73fc1cd in virNetDaemonRun () from /usr/lib64/../lib64/libvirt.so.0
#4  0x000055555556db90 in main ()

Last edited by af7567; 10-21-2016 at 06:32 AM.
 
Old 10-25-2016, 04:08 PM   #2
af7567
Member
 
Registered: Nov 2012
Posts: 295

Original Poster
Rep: Reputation: 106Reputation: 106
After 5 days I finally found the problem. When libvirtd is starting it executes ebtables as

Code:
/usr/sbin/ebtables --concurrent -L
and this creates a lock file in /var/lib/ebtables/lock

The lockfile was still there on my PC from the morning of the 20th and it wasn't removed after restarts. This means that libvirt couldn't start properly because it was always stuck waiting for the ebtables lock to go away. I only noticed tonight that libvirtd was running ebtables and when running the ebtables command manually I saw the warning message that it was waiting for the lock.

(The reason for the lockfile being left behind was because the nvidia driver crashed at system startup that morning so I had to force the power off)
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to Communicate Virtual Serial Port on Host(windows) to Communicate with Virtual Serial Port on Guest System(Linux) aquamarine Linux - Newbie 2 09-16-2016 02:48 PM
libvirt VMs will not communicate with each other lahfordie Linux - Newbie 1 09-08-2016 07:46 PM
[SOLVED] Playing with libvirt anon190 Slackware 7 07-04-2016 08:18 PM
libvirt -installation palanism Linux - Virtualization and Cloud 0 01-01-2013 05:45 PM
NIC not installed anymore, KDE not installed anymore, LILO duplicate Volume ID error scottad Slackware 2 03-31-2012 12:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud

All times are GMT -5. The time now is 09:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration