CentOS-7: curl & wget only working infrequently, but yum works fine
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
CentOS-7: curl & wget only working infrequently, but yum works fine
I originally posted this in the networking section a couple of weeks back, but I've not had any suggestions for solutions, so I've reposted here.
My problem is that curl and wget usually fail to retrieve more than just the first part of any given file, and often nothing at all. Both have worked previously on the same server, and nothing else on my network has changed - and I'm not really sure what's changed on my server to now make them not work, although very intermittently it will work just fine.
I'm not exactly quite sure when this started happening, but it's been like it for a while (a few months), and I just put off doing anything about it - and also because I have no idea how to resolve, other than what I've already tried.
So, my server is running CentOS Linux release 7.6.1810
The issue was first apparent to me when I was trying to do WordPress updates (and it's been an issue for a while), when updates will fail with, for example:
Code:
Download failed. cURL error 28: Operation timed out after 300000 milliseconds with 1355444 out of 5007255 bytes received
However, I'm certain this is not a WP issue, as I've also tried from the command line using both curl and wget, and have the same issue.
Occasionally the downloads will work, but I think that tends to be with smaller files. However, even so, every now and then it will work just fine with bigger files.
Code:
e.g. $ curl -O -v https://downloads.wordpress.org/plugin/nimble-builder.1.5.0.zip
....
curl: (56) I/O operation timed out
I've tried several things, including:
a) resolv.conf
- previously had added: options single-request-reopen, which resolved a conflict with IPv4/IPv6 many months ago
- also used several different DNS server setups
b) tried fetching stuff from both http and https sites
c) tried fetching using IP addresses instead of domain names, and tried downloading different stuff from different sites.
d) firewall - temporarily disabled iptables and tried more curl requests, but made no difference. Re-enabled iptables.
e) rebooted
f) all OS updates are installed
I can use yum with no problems.
Sample strace output is given below:
Code:
Near beginning, a lot like this:
open("/lib64/libk5crypto.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220H\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=210824, ...}) = 0
mmap(NULL, 2302664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff0cbf8c000
mprotect(0x7ff0cbfbd000, 2093056, PROT_NONE) = 0
mmap(0x7ff0cc1bc000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x30000) = 0x7ff0cc1bc000
close(3) = 0
Part way through, stuff like this:
access("/home/xyz/.pki/nssdb/.57765795_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765796_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765797_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765798_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765799_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
access("/home/xyz/.pki/nssdb/.57765800_dOeSnotExist_.db", F_OK) = -1 ENOENT (No such file or directory)
...
and quite a few of these:
recvfrom(3, 0x184e0d5, 12115, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}], 1, 1000) = 1 ([{fd=3, revents=POLLIN}])
poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 1 ([{fd=3, revents=POLLIN|POLLRDNORM}])
recvfrom(3, "\322U\200l9\332+\31w\235*\322dt\233\273\37]\244~\264{\24g\2\301\320\276\271\207\337["..., 12115, 0, NULL, NULL) = 1448
recvfrom(3, 0x184e67d, 10667, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
Close to end, a lot like this:
write(2, "k", 1) = 1
poll([{fd=3, events=POLLIN}], 1, 1000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
write(2, "\r", 1)
Should be no issues with memory:
Memory:
Code:
total used free shared buff/cache available
Mem: 8175404 1734268 310528 20832 6130608 5936760
Swap: 8257532 85472 8172060
However, I'm certain this is not a WP issue, as I've also tried from the command line using both curl and wget, and have the same issue.
Occasionally the downloads will work, but I think that tends to be with smaller files. However, even so, every now and then it will work just fine with bigger files.
Have you tried to download anything else besides WP files?
Thanks for the comments. Yeah, I already tried lots of different stuff besides downloading WP files. (among them the test files from https://www.thinkbroadband.com/download).
Your other idea with the BIOS clock was interesting: I hadn't checked that, but subsequently I've now learned about hwclock and set BIOS to system time. I've now tried a couple of downloads from different places (WP and non-WP), but it's seemed to have made no difference.
As noted in the first line of the post, "I originally posted this in the networking section a couple of weeks back, but I've not had any suggestions for solutions, so I've reposted here." ... where at least I've had one useful response so far, even if it didn't pan out.
As noted in the first line of the post, "I originally posted this in the networking section a couple of weeks back, but I've not had any suggestions for solutions, so I've reposted here." ... where at least I've had one useful response so far, even if it didn't pan out.
Normally, if the above is the case, then you can report the opening post in your original thread and ask a mod/admin to move the thread to a different subforum, thus avoiding thread duplication.
Ah, okay; I didn't know that. I did deliberately mention that I'd previously posted in the other section, to avoid any annoyance on anyone's part, but I'll know for future reference; thanks.
Anyway, if anyone is able to offer some sparks of genius to help resolve the problem then I'll be hugely grateful; it seems I've tried just about everything, but to no avail.
Thanks for the comments. Yeah, I already tried lots of different stuff besides downloading WP files. (among them the test files from https://www.thinkbroadband.com/download).
Your other idea with the BIOS clock was interesting: I hadn't checked that, but subsequently I've now learned about hwclock and set BIOS to system time. I've now tried a couple of downloads from different places (WP and non-WP), but it's seemed to have made no difference.
Any other ideas welcome. Thx!
I'm curious, have you tried with "--no-check-certificate"
Code:
wget --no-check-certificate <url>
While you get the error with wget and curl, is there anything else going into messages / syslog?
I think I had already checked messages in /var/log/messages before, but just cleared it out and started trying downloads again, with both curl and wget, but nothing new appeared in the log when I did try.
I hadn't previously tried wget with
Code:
--no-check-certificate
, but just did so; same problem. Also tried curl with -k:
- which I think is curl's equivalent to wget's --no-check-certificate, but it made no difference. Tried several different downloads, and each of them begins to download, but after maybe three or four seconds it just stalls as almost always.
Last edited by cheddarcheese; 03-22-2019 at 06:20 PM.
Well, being no expert in such things I can't really rule anything out unless I can explicitly test for something. Still, with regards to physical layer stuff then I don't really think it would be (but welcome to be proven wrong), as all other network functionality seems normal from the same machine (yum works, and a web-server works on there, etc, etc), and the same firewall/router is also used by another machine (Windows) where I have no problems, and I'm not sure what else I would really check.
I can do a switcheroo with cables and ports, but not possible with NICs. I'll play around with that tomorrow when I'm back at the machine.
However, with due respect, I don't see how that could make any difference when it's already established that no other network services on the same machine have any problems with the same setup.
I can do a switcheroo with cables and ports, but not possible with NICs. I'll play around with that tomorrow when I'm back at the machine.
However, with due respect, I don't see how that could make any difference when it's already established that no other network services on the same machine have any problems with the same setup.
To be honest, I'm pretty baffled myself. I suggested hardware since hardware failures can do some weird things and are easy to test. I remember the old story about some engineers that couldn't figure out why a complicated machine wouldn't work. Some less knowledgeable person asked an obvious question about a cable being plugged in. It turns out they were right, somebody had tripped over it.
Well, a month later, but still no progress. I cannot think of anything else to try. Would be eternally grateful if anyone ever gets any ideas on what the problem could be.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.