feed lynx options from another program

tinieprotonjam · 12-25-2006, 06:59 PM

Actually I have difficulty phrasing what my problem is, but i'm trying to be as descriptive as possible.
I am doing a program in C and right now I can call lynx to execute in the program that I made, by using a fork() and execvp(). Is there a programming solution so that emulated typing the options on lynx? To illustrate what I mean, here's what an example of what I want to happen:

if I am actually using lynx and I want to print the page I am viewing to a file, I press P for the printing options, and then select the print to file option, and then I enter the filename. Can a make a program that does this so that the following sequence of options can be fed to lynx from my C program? I would greatly appreciate any help. Thanks in advance.

demon_vox · 12-25-2006, 07:33 PM

Hi,
what you want to do is to search for the command line options of lynx to achieve what you want. I think that any of the features you have can be used in the command line.

What you want to do it is teoricaly possible, but I am not sure how to do it. And it is too ofuscated for my taste

(the first solution is much more elegant).

Cheers!

tinieprotonjam · 12-26-2006, 07:22 AM

What I mean is that I want a certain webpage to be saved using lynx without the user manually entering the commands on how to save the text or html file of a certain webpage, as viewed in lynx. Is there a way to feed the sequence of commands to lynx by an external program, and how can I do it? It is assumed that the user is not familiar with lynx and does not have the time to learn using it. It is like automatically feeding the necessary commands to lynx without the user doing it all, the user just needs to select a certain option from the program that I made. I hope this clears out what I wanted to happen. I really need some suggestions on how to do this. I'd be really greatful in any help that can be given to me.

demon_vox · 12-26-2006, 09:26 AM

Here its the complete manual:
http://lynx.isc.org/current/lynx2-8-....html#Invoking

You can use

Code:

lynx -dump www.yoururl.blah > yourCrazyFile.html

And reading the above guide I notice there was -cmd_script option that reads a file a sequence of commands (letters) and executes them the way you wanted before. Weird, but its a built in function hehe

Cheers!

tinieprotonjam · 12-29-2006, 12:03 AM

Quote:

Originally Posted by demon_vox

Here its the complete manual:
http://lynx.isc.org/current/lynx2-8-....html#Invoking

You can use

Code:

lynx -dump www.yoururl.blah > yourCrazyFile.html

And reading the above guide I notice there was -cmd_script option that reads a file a sequence of commands (letters) and executes them the way you wanted before. Weird, but its a built in function hehe

Cheers!

I really have to read this, and I really appreciate your help.

tinieprotonjam · 12-31-2006, 09:02 PM

Quote:

Originally Posted by demon_vox

You can use

Code:

lynx -dump www.yoururl.blah > yourCrazyFile.html

if I type in the terminal

Code:

lynx -dump http://myurlhere.com > sample.txt

I am able to save the contents of myurl into a file called sample.txt

So I wanted to do this in my C program, so I use:

Code:

execlp("lynx","lynx","-dump","http://myurlhere.com",">","sample1.txt", NULL);

where "sample.txt" is the filename where I want to save the dumped webpage, and is automatically saved in the my root folder.

But what I get when I run my program using the piece of code above:

Can't Access `file://localhost/root/sample1.txt'
Alert!: Unable to access document.

and no sample1.txt is generated.

So I thought,maybe there is something wrong in my previous arguments, so I tried:

Code:

execlp("lynx","lynx","-dump","file://localhost/root/code_cpy/error.html", NULL);

where "file://localhost/root/code_cpy/error.html" refers to my url or to a local html file,
whatI get is the display of the my html document on the screen

which is just like typing on the terminal window the following:

Code:

lynx -dump file://localhost/root/code_cpy/error.html

and it displays the output to the screen.

Is there anyone who would be able to help me out save the dumped contents into the file? I thought I would be able to figure out once I knew how it is done in lynx, since I can use the execlp function, but I have but it seems that I have problems with the arguments ">" and "filename". Can someone enlighten me about the problem?

tuxdev · 12-31-2006, 09:44 PM

You need to set up the file descipters for stdout using dup2 before calling exec.

Code:

int fd = open("sample1.txt", O_WRONLY);
dup2(fd, STDOUT_FILENO);
execlp(...);

tinieprotonjam · 01-01-2007, 06:50 PM

Quote:

Originally Posted by tuxdev

You need to set up the file descipters for stdout using dup2 before calling exec.

Code:

int fd = open("sample1.txt", O_WRONLY);
dup2(fd, STDOUT_FILENO);
execlp(...);

I have problems with the O_WRONLY variable, the compiler says that the variable is undefined. I think I am lacking header files. I have already included termios.h, stdio.h, unistd.h and sys/types.h. Have I forgotten some other header files? I am not familiar with most of the header files in Linux. Thank you so much for the help. I hope this fixes my problem.

tuxdev · 01-01-2007, 08:07 PM

try including fcntl.h, as stated on the man page. You can also pull a file descripter out of a FILE*, but make sure everything is synced up right.

tinieprotonjam · 01-02-2007, 05:37 AM

Quote:

Originally Posted by tuxdev

try including fcntl.h, as stated on the man page. You can also pull a file descripter out of a FILE*, but make sure everything is synced up right.

i've already tried including fcntl.h, but it did not work. i have read somewhere that the open() function works only in dos, so i tried using an alternative, creat() and it seemed to compile, but nothing was written to the text file that i created, the code is:

Code:

  int fd;
  fd = creat("sample1.txt",S_IWRITE|S_IREAD);
  dup2(fd, STDOUT_FILENO);
  execlp("lynx","lynx","-dump","file://localhost/root/code_cpy/error.html",">","/root/sample1.txt", NULL);

where did I go wrong?

tuxdev · 01-02-2007, 09:08 AM

Would have been good to know you're trying to do Windows programming. Lots of stuff doesn't work quite as you would expect, and you have to do the nasty fix below a lot.

fcntl.h is still what you want, but you have to do

Code:

#ifdef _MSCVER
#define _open open
#define _O_WRONLY O_WRONLY
#endif

Blech, not-quite-POSIX-ness of Windows.

theNbomr · 01-02-2007, 11:48 AM

If you are merely trying to copy webpages/sites, the definitive tool for the purpose would be wget. More flexible, and made specifically for the purpose. You can specify an output file as a commandline argument (via execlp(...) ), and the whole business can be concealed from your users, if you wish to run it that way. I think there are Windows binary packages of wget, but it should be quite portable should you need to build it yourself.

--- rod.

tinieprotonjam · 01-02-2007, 06:28 PM

Quote:

Originally Posted by tuxdev

Would have been good to know you're trying to do Windows programming. Lots of stuff doesn't work quite as you would expect, and you have to do the nasty fix below a lot.

fcntl.h is still what you want, but you have to do

Code:

#ifdef _MSCVER
#define _open open
#define _O_WRONLY O_WRONLY
#endif

Blech, not-quite-POSIX-ness of Windows.

Did I forgot to mention that I am doing my program in linux? Sorry about that. I hope I can find a fix to this.

tinieprotonjam · 01-02-2007, 06:31 PM

Quote:

Originally Posted by theNbomr

If you are merely trying to copy webpages/sites, the definitive tool for the purpose would be wget. More flexible, and made specifically for the purpose. You can specify an output file as a commandline argument (via execlp(...) ), and the whole business can be concealed from your users, if you wish to run it that way. I think there are Windows binary packages of wget, but it should be quite portable should you need to build it yourself.

--- rod.

Thanks for the suggestion, what I really wanted is the stripped off version of the webpages/sites, the way it is displayed on lynx, without the html formatting, yet I will be able to distinguish which ones have hyperlinks to other sites, and with the intention of using the program in Linux.

theNbomr · 01-02-2007, 07:09 PM

Sure. You do understand that the output of lynx will be littered with screen formatting escape codes that are used to position the text on the screen, create colors, create underlines, and various other formatting? I would think that it would be easier to get rid of the HTML formatting, but that's just me, perhaps.

Here's a sample, formatted with od:

Code:

0000000 esc   [   ?   1   0   4   9   h esc   [   1   ;   5   0   r esc
0000020   [   m esc   (   B esc   [   4   l esc   [   ?   7   h esc   [
0000040   ?   1   h esc   = esc   [   3   9   ;   4   9   m esc   [   3
0000060   9   ;   4   9   m esc   [   m esc   (   B esc   [   H esc   [
0000100   2   J esc   [   4   8   d esc   [   0   ;   1   m esc   (   B
0000120 esc   [   3   3   m esc   [   4   4   m   G   e   t   t   i   n
0000140   g  sp   h   t   t   p   :   /   /   s   0   1   0   6   0   0
0000160   4   0   f   4   8   4   0   1   8   a   .   v   f   .   s   h
0000200   a   w   c   a   b   l   e   .   n   e   t   /   c   g   i   -
0000220   b   i   n   /   h   i   t   S   o   u   r   c   e   s   .   p
0000240   l esc   [   3   9   ;   4   9   m esc   [   m esc   (   B  cr
0000260 esc   [   0   ;   1   m esc   (   B esc   [   3   3   m esc   [
0000300   4   4   m   L   o   o   k   i   n   g  sp   u   p  sp   s   0
0000320   1   0   6   0   0   4   0   f   4   8   4   0   1   8   a   .
0000340   v   f   .   s   h   a   w   c   a   b   l   e   .   n   e   t
0000360 esc   [   3   9   ;   4   9   m esc   [   m esc   (   B esc   [
0000400   K  cr esc   [   0   ;   1   m esc   (   B esc   [   3   3   m
0000420 esc   [   4   4   m   M   a   k   i   n   g  sp   H   T   T esc
0000440   [   1   5   @   P  sp   c   o   n   n   e   c   t   i   o   n
0000460  sp   t   o esc   [   4   8   ;   6   1   H esc   [   3   9   ;
0000500   4   9   m esc   [   m esc   (   B  cr esc   [   0   ;   1   m
0000520 esc   (   B esc   [   3   3   m esc   [   4   4   m   S   e   n
0000540   d   i   n   g  sp   H   T   T   P  sp   r   e   q   u   e   s
0000560   t   . esc   [   3   9   ;   4   9   m esc   [   m esc   (   B
0000600 esc   [   K  cr esc   [   0   ;   1   m esc   (   B esc   [   3

--- rod