LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 10-25-2005, 02:49 PM   #1
hgb
Member
 
Registered: Jun 2004
Distribution: Mandrake 10, SUSE 10.x, DEbian
Posts: 125

Rep: Reputation: 15
FTP client multi upload.


Hi there, I need one program that can handle that, I have tryied from FTP to konqueror gftp(-kde), KBEAR, but none of them can handle multiple uploads like I see. I come from Windows FileZilla FTP program.

The FTP host allow 4 connections at a time.



For example gftp claim to have it, but like I see FTP only let upload 1 file at a time, then for upload multiple things, you need open more connections. I have look a little in the source code, and aparently there is a thread mutex (thing that I dont know), but it handle multiple uploads for 1 session/connection), then I guess is by that gftp can handle multiple connections but cant handle multiple connections/sessions... (like I understand from source, can handle multiple uploads from the same connection [dont know wich protocol let you do that], but I never see that it try to open more connections to the server). Yes, I have clicked the option in settings that say something like "one file at a time" and it work the same way (I dont see that try open other connections to the site.. in the log by instance...).



ftp mget put multiple but dont multiple at the same time, the same behaviour with KBEAR and konqueror.
 
Old 10-27-2005, 01:51 AM   #2
caal
LQ Newbie
 
Registered: Oct 2005
Location: Doylestown, PA
Posts: 28

Rep: Reputation: 15
Hi,

Not should about uploading, being kind of new, but here is a guess a downloading.

wget -c URL1 &
wget -c URL2 &
wget -c URL3 &
wget -c URL4 &

Even throw in a nohup, and you don't have to be there.
nice nohup wget -c URL5 &

I am sure uploading has a simular program.

Hope it helps a bit,
Grae
 
Old 10-27-2005, 10:44 AM   #3
hgb
Member
 
Registered: Jun 2004
Distribution: Mandrake 10, SUSE 10.x, DEbian
Posts: 125

Original Poster
Rep: Reputation: 15
I gues I will really mis fileZilla :S...


I have tryied to send the next to the gftp-users, but I have not been able to use the major dome thing well (If some of you can send it, feel free to do it... [I only need that this feature is reached])... I only put here the way that I see posible to solve it... (peraphs) because you already know the problem, I can't do it... because I really dont know the internals or much about this topic in programming, and the thread and mutext things... for the original developers with this posible hint should be somewhat easy, becaus they have almost all done for support it...






======================================================
===========================================================

Like I see now, in FTP is only possible upload one file per session at a time, now I think that perhaps such options "one transfer at a time" is for other protocols?? or Im wrong?, I say this, because I have tryied to find where to modify this...

I have see that the the function from /src/gtk/transfer.c the that do the handle job is "transfer_window_files" is composed in 3 parts:

(1) it first check for the selected items (in window1 or window 2 [left, right])) and is not recursive?, ie: if there is a directory selected, it will not get the files inside.

(2) will check if there are files to upload(/download) and if there are, then will be called "gftp_gtk_get_subdirs" that mainly get the files attributes? (name, directory, size, ...)if Im not wrong, it get subfolders and content of them(in some way.. I guess... Im unsure what is the function of call "gftpui_common_run_callback_function" inside such function) somewhere here will be called the popup dialog of actions taked (for discard, resume, overwrite) that will change the files to upload "transfer->files".

(3) after have selected now really what to do with the files that are selected, is time to upload(/download) them "gftpui_common_add_file_transfer" I guess is the function that do the real job of upload then, then inside this function is where the problem reside????


I have taked a surface look at those last functions, but like I see, there is a mutex (I dont have knowledge about them, I only know that are related to multi-thread or some like that), but in fact, from what I understand from connecting manually to the FTP server, is that for transfer in this case 4 files, I must open the session 4 times.

My actuall way of think, for open multiple sessions will be in step (3) before call the function that I guess do the work of upload...:

a) see how many sessions can be opened (a input popup should be ok for the moment?)

b) We have now 1 session open, then connect nSessions-1 times a again

c) copy the direction that is poiting the actual list of files to really upload(/download) to the new sessions.

d) Modify "gftpui_common_add_file_transfer" for accep an array of connections for upload?
d.1) I guess the actual code should work ok (with minime changes???), you will need only loop the sessions opened, because I guess there is a flag for file transfer by each one? (is printed in the list of upload), then there will be no problem if two sessions try upload the same file.

c) Now we have uploaded in parallel with nSessions, time to close the nSessions-1 and let the "original" session continue.


Like you see I have passed the open of multiple sections to the part (3) upload(/download), because there no exist the necessity of open them before??, perhaps only for the step (2) when getting the comparations of the 2 locations.







Is that the correct way??, or Im missing something about the option "one transfer at a time", yes I have unckecked it, and I dont see in the log more messages about the welcome of the FTP server (mean that there is opening more sessions).

======================================================
===========================================================

Last edited by hgb; 10-27-2005 at 10:53 AM.
 
Old 11-02-2005, 12:16 AM   #4
caal
LQ Newbie
 
Registered: Oct 2005
Location: Doylestown, PA
Posts: 28

Rep: Reputation: 15
Hi,

At first I didn't want to post this, since it might be a lot simply then you are looking for, however, since no one in answering, maybe I can help.

Unix allows many processes at once, as you probably know already. I say this, because i think you can let the operating system do the multicasking part for you rather than the program.

So maybe you could try something like this.

nice nohup curl -c -t filename -u userName:password -url ftp.server.com/theDirectory &
nice nohup curl -c -t filename1 -u userName:password -url ftp.server.com/theDirectory &
nice nohup curl -c -t filename2 -u userName:password -url ftp.server.com/theDirectory &
nice nohup curl -c -t filename3 -u userName:password -url ftp.server.com/theDirectory &
nice nohup curl -c -t filename4 -u userName:password -url ftp.server.com/theDirectory &

This isn't mult-connection per file, but I believe it would be a seperate connection for each file. This would make sense, two different user can download two different file at the same time. In linux, processed are treated simularly; meaning two downloads can happen at once.

The nice nohup, is not really necessary. However nohup is nice incase you forget they are downloading and log off. If it is your computer, and you shut it down regularly, nohup probably won't help. Maybe a guru can help us, I don't klnow if nohup restarts a program after a shutdown. Nice just lowers the processes important, "nice," pun inteneded, if you are on a system with other users.

God Bless You,
Grae

Last edited by caal; 11-02-2005 at 12:20 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
FTP upload hinetvenkat Linux - Software 3 06-02-2005 09:34 PM
ftp client with support for fxp, multi-threading and... NonSumPisces Linux - Networking 1 08-28-2004 01:18 PM
client can`t upload file, can reply email verchrom Linux - Newbie 8 06-17-2003 01:56 PM
Cant upload to another FTP brandog Linux - Software 1 05-03-2003 02:47 PM
PHP: upload file to client... ugenn Programming 3 08-26-2002 10:59 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration