LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud
User Name
Password
Linux - Virtualization and Cloud This forum is for the discussion of all topics relating to Linux Virtualization and Linux Cloud platforms. Xen, KVM, OpenVZ, VirtualBox, VMware, Linux-VServer and all other Linux Virtualization platforms are welcome. OpenStack, CloudStack, ownCloud, Cloud Foundry, Eucalyptus, Nimbus, OpenNebula and all other Linux Cloud platforms are welcome. Note that questions relating solely to non-Linux OS's should be asked in the General forum.

Notices


Reply
  Search this Thread
Old 11-17-2019, 11:19 AM   #1
rainman715
LQ Newbie
 
Registered: Sep 2017
Posts: 10

Rep: Reputation: 0
Fedora 31 Upgrade "broke" libvirtd


Hello,

I just upgraded from Fedora 30 to 31, and now the libvirtd service will not stay active; it dies exactly two minutes after starting. The service does not have an error message. I enabled debug for libvirtd, but I do not understand the information coming back. Unfortunately, the file is too big to upload. Happy to break it up if it helps me resolve this.

Below is the status from libvirtd after it dies.

Code:
systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Sun 2019-11-17 10:37:00 EST; 4min 3s ago
     Docs: man:libvirtd(8)
           https://libvirt.org
  Process: 1984 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 1984 (code=exited, status=0/SUCCESS)
      CPU: 195ms

Nov 17 10:35:00 ########## systemd[1]: Starting Virtualization daemon...
Nov 17 10:35:00 ########## libvirtd[1984]: 2019-11-17 15:35:00.695+0000: 1984: info : libvirt version: 5.6.0, package: 5.fc31 (Fedora Project, 2019-11-11-2>
Nov 17 10:35:00 ########## libvirtd[1984]: 2019-11-17 15:35:00.695+0000: 1984: info : hostname: ##########
Nov 17 10:35:00 ########## libvirtd[1984]: 2019-11-17 15:35:00.695+0000: 1984: debug : virLogParseOutputs:1765 : outputs=1:file:/var/log/libvirt/libvirtd.l>
Nov 17 10:35:00 ########## libvirtd[1984]: 2019-11-17 15:35:00.695+0000: 1984: debug : virLogParseOutput:1593 : output=1:file:/var/log/libvirt/libvirtd.log
Nov 17 10:35:00 ########## systemd[1]: Started Virtualization daemon.
Nov 17 10:35:00 ########## dnsmasq[1311]: read /etc/hosts - 2 addresses
Nov 17 10:35:00 ########## dnsmasq[1311]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Nov 17 10:35:00 ########## dnsmasq-dhcp[1311]: read /var/lib/libvirt/dnsmasq/default.hostsfile
Nov 17 10:37:00 ########## systemd[1]: libvirtd.service: Succeeded.
I'm not sure what else to check, please help.

Thanks
Ray

Last edited by rainman715; 11-17-2019 at 12:01 PM.
 
Old 11-17-2019, 12:08 PM   #2
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,679

Rep: Reputation: Disabled
You could try running libvirtd manually in a terminal (sudo libvirtd) and see what messages appear at the console.
 
Old 11-17-2019, 12:15 PM   #3
rainman715
LQ Newbie
 
Registered: Sep 2017
Posts: 10

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Ser Olmy View Post
You could try running libvirtd manually in a terminal (sudo libvirtd) and see what messages appear at the console.
Thanks Ser Olmy, however this doesn't seem to work either. All I get is the following:

Code:
2019-11-17 17:13:46.224+0000: 2935: info : libvirt version: 5.6.0, package: 5.fc31 (Fedora Project, 2019-11-11-20:24:00, )
2019-11-17 17:13:46.224+0000: 2935: info : hostname: shenron.DBZ
2019-11-17 17:13:46.224+0000: 2935: debug : virLogParseOutputs:1765 : outputs=1:file:/var/log/libvirt/libvirtd.log
2019-11-17 17:13:46.224+0000: 2935: debug : virLogParseOutput:1593 : output=1:file:/var/log/libvirt/libvirtd.log
It stays past the two minutes, but I cannot connect to the hypervisor. Are there any other logs etc. I can check?
 
Old 11-17-2019, 12:25 PM   #4
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,679

Rep: Reputation: Disabled
Quote:
Originally Posted by rainman715 View Post
It stays past the two minutes, but I cannot connect to the hypervisor.
What do you mean by that? Does virsh list fail, and if so, what's the exact error message?
Quote:
Originally Posted by rainman715 View Post
Are there any other logs etc. I can check?
Well, there's:
Quote:
Originally Posted by rainman715 View Post
Code:
2019-11-17 17:13:46.224+0000: 2935: info : libvirt version: 5.6.0, package: 5.fc31 (Fedora Project, 2019-11-11-20:24:00, )
2019-11-17 17:13:46.224+0000: 2935: info : hostname: shenron.DBZ
2019-11-17 17:13:46.224+0000: 2935: debug : virLogParseOutputs:1765 : outputs=1:file:/var/log/libvirt/libvirtd.log
2019-11-17 17:13:46.224+0000: 2935: debug : virLogParseOutput:1593 : output=1:file:/var/log/libvirt/libvirtd.log
Delete/empty the log before running libvirtd and post the results. If the log is too big to post, use pastebin or something similar and post a link.
 
Old 11-17-2019, 01:17 PM   #5
rainman715
LQ Newbie
 
Registered: Sep 2017
Posts: 10

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Ser Olmy View Post
What do you mean by that? Does virsh list fail, and if so, what's the exact error message?
virsh list returns an empty list. I also tried to create a new domain to see if I just lost configs, but receive the following:

Code:
sudo virt-install --name guest1-rhel7 --memory 2048 --vcpus 2 --disk path=/vms/os//myRHELVM1.img,bus=virtio,size=8 --network=bridge:br0 --pxe --os-variant rhel7.0 
WARNING  Unable to connect to graphical console: virt-viewer not installed. Please install the 'virt-viewer' package.
WARNING  No console to launch for the guest, defaulting to --wait -1

Starting install...
Allocating 'myRHELVM1.img'                                                                                                                     | 8.0 GB  00:00:00     
ERROR    can't connect to virtlogd: Cannot recv data: Connection reset by peer
Removing disk 'myRHELVM1.img'                                                                                                                  |    0 B  00:00:00     
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start guest1-rhel7
otherwise, please restart your installation.
virtlogd was not running, but even after starting it I received the same message as above. Also, the console generated the following:
Code:
libvirtd -v
2019-11-17 17:49:10.331+0000: 3283: info : libvirt version: 5.6.0, package: 5.fc31 (Fedora Project, 2019-11-11-20:24:00, )
2019-11-17 17:49:10.331+0000: 3283: info : hostname: #########
2019-11-17 17:49:10.331+0000: 3283: debug : virLogParseOutputs:1765 : outputs=1:file:/var/log/libvirt/libvirtd.log
2019-11-17 17:49:10.331+0000: 3283: debug : virLogParseOutput:1593 : output=1:file:/var/log/libvirt/libvirtd.log
2019-11-17 17:49:17.445+0000: 3370: info : virSecuritySELinuxSetFileconImpl:1280 : Setting SELinux context on '/var/lib/libvirt/qemu/domain-1-guest1-rhel7' to 'unconfined_u:object_r:svirt_image_t:s0:c654,c834'
2019-11-17 17:49:17.446+0000: 3371: info : virSecurityDACSetOwnership:799 : Setting DAC user and group on '/var/lib/libvirt/qemu/domain-1-guest1-rhel7' to '107:107'
2019-11-17 17:49:17.447+0000: 3372: info : virSecuritySELinuxSetFileconImpl:1280 : Setting SELinux context on '/var/lib/libvirt/qemu/channel/target/domain-1-guest1-rhel7' to 'unconfined_u:object_r:svirt_image_t:s0:c654,c834'
2019-11-17 17:49:17.448+0000: 3373: info : virSecurityDACSetOwnership:799 : Setting DAC user and group on '/var/lib/libvirt/qemu/channel/target/domain-1-guest1-rhel7' to '107:107'
2019-11-17 17:49:17.449+0000: 3374: info : virSecuritySELinuxSetFileconImpl:1280 : Setting SELinux context on '/var/lib/libvirt/qemu/domain-1-guest1-rhel7/master-key.aes' to 'unconfined_u:object_r:svirt_image_t:s0:c654,c834'
2019-11-17 17:49:17.450+0000: 3375: info : virSecurityDACSetOwnership:799 : Setting DAC user and group on '/var/lib/libvirt/qemu/domain-1-guest1-rhel7/master-key.aes' to '107:107'

Quote:
Originally Posted by Ser Olmy View Post
Well, there's:
Delete/empty the log before running libvirtd and post the results. If the log is too big to post, use pastebin or something similar and post a link.
I wasn't sure if you wanted the log from trying to start the service or from running it manually, so I included both. Both are broken into two postings because they exceeded the limit on pastebin.

Attempting to start the service:
https://pastebin.com/SG3sGVLA
https://pastebin.com/72KXFVe1

Running libvirtd manuall and trying to create a new domain:
https://pastebin.com/iedPhWa4
https://pastebin.com/kqsyPRDr

Really appreciate your help with this.

Thanks
Ray
 
Old 11-17-2019, 01:59 PM   #6
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,679

Rep: Reputation: Disabled
This certainly doesn't look right:
Code:
2019-11-17 18:03:29.291+0000: 3608: info : virNetSocketNew:290 : RPC_SOCKET_NEW: sock=0x7f3cdc026670 fd=34 errfd=-1 pid=0 localAddr=127.0.0.1;0, remoteAddr=127.0.0.1;0
2019-11-17 18:03:29.291+0000: 3608: info : virNetClientNew:322 : RPC_CLIENT_NEW: client=0x7f3cdc027bf0 sock=0x7f3cdc026670
2019-11-17 18:03:29.291+0000: 3608: info : virNetClientSendInternal:2124 : RPC_CLIENT_MSG_TX_QUEUE: client=0x7f3cdc027bf0 len=116 prog=2270401305 vers=1 proc=1 type=0 status=0 serial=0
2019-11-17 18:03:29.291+0000: 3608: error : virNetSocketReadWire:1835 : Cannot recv data: Connection reset by peer
2019-11-17 18:03:29.291+0000: 3608: info : virNetSocketDispose:1357 : RPC_SOCKET_DISPOSE: sock=0x7f3cdc026670
A shot in the dark: Do you have libvirtd-tcp.socket on your system? I'm not too familiar with systemd, but I believe it should reside in /usr/lib/systemd/system.
 
Old 11-17-2019, 02:05 PM   #7
rainman715
LQ Newbie
 
Registered: Sep 2017
Posts: 10

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Ser Olmy View Post
A shot in the dark: Do you have libvirtd-tcp.socket on your system? I'm not too familiar with systemd, but I believe it should reside in /usr/lib/systemd/system.
I do, and it indeed is located in /usr/lib/systemd/system/libvirtd-tcp.socket

If it helps......

Code:
[Unit]
Description=Libvirt non-TLS IP socket
Before=libvirtd.service
BindsTo=libvirtd.socket
After=libvirtd.socket

[Socket]
# This must match the /etc/libvirt/libvirtd.conf tcp_port setting
# when using systemd version < 227
ListenStream=16509
Service=libvirtd.service

[Install]
WantedBy=sockets.target
 
Old 11-17-2019, 02:52 PM   #8
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,679

Rep: Reputation: Disabled
That looks about right.

However, there seems be a version conflict between libvirt and its dependencies. From the service startup log:
Code:
2019-11-17 15:44:34.218+0000: 2089: debug : virCommandRun:2506 : Result status 0, stdout: 'Usage: dnsmasq [options]

Valid options are:
<snip>
As you can see, dnsmasq is being called with parameters it doesn't recognize, so it spews out the "usage" help text.

It seems the upgrade process has bungled some packages. I'd try uninstalling and then reinstalling libvirt, dnsmasq and any other dependencies, perhaps even qemu.
 
Old 11-18-2019, 08:58 PM   #9
rainman715
LQ Newbie
 
Registered: Sep 2017
Posts: 10

Original Poster
Rep: Reputation: 0
I ended up rebuilding the hypervisor and now all is working as expected.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Libvirtd keeps blowing away my IPTABLEs andrew44 Linux - Security 4 05-26-2014 11:30 PM
libvirtd and hal on -current rpedrica Slackware 14 08-08-2012 01:07 PM
libvirtd overwrites iptables phatrik Linux - Virtualization and Cloud 3 06-11-2012 09:40 AM
Virt-Manager issue "please make sure libvirtd is running" chrisportela Linux - Server 2 07-08-2011 10:54 AM
libvirtd errors mickeyboa Linux - Virtualization and Cloud 1 07-04-2010 11:15 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud

All times are GMT -5. The time now is 06:46 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration