LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-25-2015, 09:28 AM   #1
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Rep: Reputation: Disabled
Exec and ftp script in memory only.


All the FTP scripts I've seen need to be written out and redirect back into the FTP command.

But has anyone ever successfully pushed the script into the command from memory, as a data string??
 
Old 08-25-2015, 09:37 AM   #2
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fedora-35
Posts: 5,313

Rep: Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918Reputation: 918
not sure what you are trying to achieve but it seems like scp with public-key-encryption seems worth checking out ?
 
Old 08-25-2015, 10:01 AM   #3
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
Reads and writes to the disk array in a few programs are intense in a few report distribution programs we have. These slow the programs to a crawl. I am trying look for ways to avoid as many as possible and there are enough ftp scripts in the program to make a difference. The more I can avoid them, the faster the programs will run.
 
Old 08-25-2015, 12:04 PM   #4
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,864
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
I'm almost sure you wish to achieve something interesting, but I cannot fathom out what it is.

Perhaps you want to implement FTP-client functionality in your program? Try components like curl or libftp.
 
Old 08-25-2015, 12:16 PM   #5
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
These are existing programs in a body of code we are porting in October. It has to work now, be vetted and stable before we port the DB and ERP to Linux. Curl won't work as these are dedicated ftp servers, there's no html or even web access. All internal. We do not have libftp.

What I am trying to do is consolidate about 120 program so the port will go smoothly and not crash anywhere. ftp is the latest in a series of command line functions I write consolidation code. We has say 50 calls to ftp throughout our system. I am write a one-call-for-all subroutine. so if we have issues, we only need to fix the subroutine, not all 50 calls. But I am also working on performance improvement. That's why I posted about a non-script, straight feed into ftp. It would about a 100k reads and write, read and deletes actions a day.

Last edited by Scott Johnston; 08-25-2015 at 12:27 PM.
 
Old 08-25-2015, 12:31 PM   #6
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,864
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
Are you using some programming language or script language or any programming environment? (I could use clairvoyance/legilimency if it weren't illegal in my country.)
 
Old 08-25-2015, 12:34 PM   #7
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
The problem is the disk access for the gets and puts by FTP and not FTP being invoked itself?

Why not establish a RAM disk for the locations where you perform the get and put commands and then resolve that RAM disk at a fixed time?

And if accessing the FTP binary seems to be a problem, then copy the FTP binary also to the RAM disk and likewise run it from that location.
 
Old 08-25-2015, 12:37 PM   #8
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by NevemTeve View Post
Are you using some programming language or script language or any programming environment? (I could use clairvoyance/legilimency if it weren't illegal in my country.)
We with with something you have never heard of. Universe Basic as part the Rocket Software's US package. It's support it's Multi-Value DB. Something else I'm sure you haven't heard of.
 
Old 08-25-2015, 12:39 PM   #9
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by rtmistler View Post
The problem is the disk access for the gets and puts by FTP and not FTP being invoked itself?

Why not establish a RAM disk for the locations where you perform the get and put commands and then resolve that RAM disk at a fixed time?

And if accessing the FTP binary seems to be a problem, then copy the FTP binary also to the RAM disk and likewise run it from that location.
That's an interest idea. A ram disk would do the trick. I'm just not sure we even have the capabilities or if our DB coding system could handle one.

I'll ask the admin.
 
Old 08-25-2015, 12:47 PM   #10
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,864
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
@OP: Cool. Then how do you expect anyone to give you advice? Nonetheless, my memories come back: you're the guy migrating from AIX to RedHat Linux.

Anyways, if you call (fork+exec) /usr/bin/ftp to transfer a file, it will be slow. Not because the commands to the program come from a file, but because fork and exec are slow. Also network traffic is slow. Reading a few lines (the commands) from a text-file isn't slow.
 
1 members found this post helpful.
Old 08-25-2015, 12:51 PM   #11
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by NevemTeve View Post
@OP: Cool. Then how do you expect anyone to give you advice? Nonetheless, my memories come back: you're the guy migrating from AIX to RedHat Linux.

Anyways, if you call (fork+exec) /usr/bin/ftp to transfer a file, it will be slow. Not because the commands to the program come from a file, but because fork and exec are slow. Also network traffic is slow. Reading a few lines (the commands) from a text-file isn't slow.
Just asking the question......
 
Old 08-25-2015, 12:56 PM   #12
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Quote:
Originally Posted by Scott Johnston View Post
That's an interest idea. A ram disk would do the trick. I'm just not sure we even have the capabilities or if our DB coding system could handle one.

I'll ask the admin.
My impression is that the DB would require a path or a similar structure. When you create a RAM disk, it becomes a path, and you can make a link to it if you require a certain structure or to need it to appear as if it were within the hierarchy of an existing physical disk tree.

I guess you'll have to evaluate that with your team.
 
Old 08-25-2015, 01:00 PM   #13
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
If we could link to a ram drive that might a be a much more efficient way to do this.
 
Old 08-25-2015, 01:33 PM   #14
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,864
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
If you have unused RAM, give it to the kernel so that it could use it as disk-cache. Now that does improve performance.
 
Old 08-25-2015, 03:27 PM   #15
Scott Johnston
LQ Newbie
 
Registered: Oct 2014
Posts: 26

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by NevemTeve View Post
If you have unused RAM, give it to the kernel so that it could use it as disk-cache. Now that does improve performance.

The Kernel has enough... with his fried chicken!
 
  


Reply

Tags
ftp



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how exec <script> works? pan64 Programming 3 02-01-2013 06:21 AM
[SOLVED] calling a shell script from exec. ankitm Programming 4 04-09-2012 06:05 AM
how to use exec in shell script sagar.gunjal Programming 16 12-19-2011 11:10 PM
Using exec in a shell script opus-outlaw Linux - Desktop 1 01-19-2008 11:47 AM
Programming: fork, exec and system memory atienza Programming 2 08-24-2005 07:26 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration