LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (http://www.linuxquestions.org/questions/slackware-14/)
-   -   [BUG] lftp 4.4.10 too slow (http://www.linuxquestions.org/questions/slackware-14/%5Bbug%5D-lftp-4-4-10-too-slow-4175481856/)

zerouno 10-23-2013 08:34 AM

[BUG] lftp 4.4.10 too slow
 
There is some bug on lftp 4.4.10 (slackware64-current).

The transfer rate is too slow when I use that version (sometime < 1kb), specially in function 'mirror' during the retrieve file listing (the single file transfer somtime obtains a good speed).

Is not the adsl problem.
I tried to roll-back to 4.4.9 and I obtain a fastest connection.

We (users of slacky.eu community) found the problem during download alienbob's multilib
# lftp -c 'open http://slackware.com/~alien/multilib/ ; mirror current'
about 45min to give directory content

the same command with lftp 4.4.9: about 2 seconds to get directory content.

drmozes 10-23-2013 09:34 AM

Quote:

Originally Posted by zerouno (Post 5050878)
There is some bug on lftp 4.4.10 (slackware64-current).

The transfer rate is too slow when I use that version (sometime < 1kb), specially in function 'mirror' during the retrieve file listing (the single file transfer somtime obtains a good speed).

Is not the adsl problem.
I tried to roll-back to 4.4.9 and I obtain a fastest connection.

We (users of slacky.eu community) found the problem during download alienbob's multilib
# lftp -c 'open http://slackware.com/~alien/multilib/ ; mirror current'
about 45min to give directory content

the same command with lftp 4.4.9: about 2 seconds to get directory content.

Not that I know if there is a bug here, but what I'd do is first issue 'deb 9' inside lftp to turn on max debugging.
It could be that the functionality has changed in the newer version, such that it's issuing different FTP commands than the previous one which is adding delay. I would be more descriptive, but it's been ages since I had to debug FTP so I can't recall - but if you do deb 9 on both versions and see a difference in the FTP commands being issued, it might allow you to revert to the previous behaviour.

zerouno 10-23-2013 09:40 AM

I will try to debug. However I tested lftp with http protocol only.

lftp -c 'open http://slackware.com/~alien/multilib/ ; mirror current'

TommyC7 10-23-2013 01:08 PM

Have you tried using lftp for anything else other than alien's multilib packages?

If you could, can you also do:

Code:

echo "debug" >> ~/.lftprc
And then give us the output when you attempt to use lftp? It should help out a bit.

zerouno 10-23-2013 04:11 PM

es, I tried with other sites, with the same result.

The ftp protocol have not problems.

In debug mode I can notice the high latency (but the debug mode does not show time of request) and a large number of disconnections for timeout.
Now I've not the pc, so I can't post output, but that output isn't really significant becouse lftp 4.4.9 generate exactly the same (that is the http header of all requests)

willysr 10-23-2013 06:24 PM

i can confirm that this bug is reproduceable, even though i can't find any git commits that are relevant to this regression. I have tested to revert some of their commits, but still it has the same effect.

chrisretusn 10-23-2013 06:33 PM

Here is a debug output from lftp 4.4.10, while I was trying to mirror Alien Bob's openjdk build. Note the "**** Hit EOF while fetching headers". I've reverted back to 4.4.9.

Code:

---- Resolving host address...
---- 1 address found: 98.158.153.254                 
---- Connecting to taper.alienbase.nl (98.158.153.254) port 80
---- Sending request...                                                                 
---> HEAD /mirrors/people/alien/slackbuilds/openjdk/ HTTP/1.1
---> Host: taper.alienbase.nl
---> User-Agent: lftp/4.4.10
---> Accept: */*
---> Connection: keep-alive
--->
<--- HTTP/1.1 200 OK                                                                               
<--- Date: Wed, 23 Oct 2013 01:37:34 GMT
<--- Server: Apache/2.2.24 (Unix) DAV/2 PHP/5.3.23
<--- Keep-Alive: timeout=5, max=100
<--- Connection: Keep-Alive
<--- Content-Type: text/html;charset=ISO-8859-1
<---
---- Sending request...
---> HEAD /mirrors/people/alien/slackbuilds/openjdk/build/ HTTP/1.1
---> Host: taper.alienbase.nl
---> User-Agent: lftp/4.4.10
---> Accept: */*
---> Connection: keep-alive
--->
<--- HTTP/1.1 200 OK                 
<--- Date: Wed, 23 Oct 2013 01:37:34 GMT
<--- Server: Apache/2.2.24 (Unix) DAV/2 PHP/5.3.23
<--- Keep-Alive: timeout=5, max=99
<--- Connection: Keep-Alive
<--- Content-Type: text/html;charset=ISO-8859-1
<---
---- Closing HTTP connection
---- Connecting to taper.alienbase.nl (98.158.153.254) port 80
---- Sending request...                       
---> GET /mirrors/people/alien/slackbuilds/openjdk/build/ HTTP/1.1
---> Host: taper.alienbase.nl
---> User-Agent: lftp/4.4.10
---> Accept: */*
---> Pragma: no-cache
---> Cache-Control: no-cache
---> Connection: keep-alive
--->
<--- HTTP/1.1 200 OK                                     
<--- Date: Wed, 23 Oct 2013 01:37:34 GMT
<--- Server: Apache/2.2.24 (Unix) DAV/2 PHP/5.3.23
<--- Content-Length: 3290
<--- Keep-Alive: timeout=5, max=100
<--- Connection: Keep-Alive
<--- Content-Type: text/html;charset=ISO-8859-1
<---
---- Receiving body...
---- Received all
---> HEAD /mirrors/people/alien/slackbuilds/openjdk/build/../ HTTP/1.1
---> Host: taper.alienbase.nl
---> User-Agent: lftp/4.4.10
---> Accept: */*
---> Connection: keep-alive
--->
<--- HTTP/1.1 200 OK                                     
<--- Date: Wed, 23 Oct 2013 01:37:34 GMT
<--- Server: Apache/2.2.24 (Unix) DAV/2 PHP/5.3.23
<--- Keep-Alive: timeout=5, max=99
<--- Connection: Keep-Alive
<--- Content-Type: text/html;charset=ISO-8859-1
<---
**** Hit EOF while fetching headers                     
---- Closing HTTP connection
Getting files information (7%) [Delaying before reconnect: 22]


chess 10-23-2013 10:10 PM

Looks like Pat has reverted back to 4.4.9 as well in the ChangeLog.txt.

hba 10-24-2013 01:48 AM

I was getting the same message "Delaying before reconnect", but i solved it with echoing 'set net:reconnect-interval-base 0' to ~/.lftprc

https://github.com/nihilismus/bob-in...bob-sources.sh


All times are GMT -5. The time now is 11:42 PM.