LinuxQuestions.org - cheap http server problems...

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - cheap http server problems... (https://www.linuxquestions.org/questions/programming-9/cheap-http-server-problems-168100/)

cheap http server problems...

Hi,

Im studying C and C Socket network programming, and I decided to try to write a generic http server and improve on it as i learn. Well the problem i have is the http server below works (although barely), it will work for a pure html file, because it reads in chars from .html file, then sends them to client(web browser). But as soon as it import images or any other objects into .html file, it doesnt work, because objects not a char (i believe). im sure there are 1000 better ways to do this. I need a way to send the whole web page, images and all.. Any other comments on this cheap server are also appreciated!!! :)

THX!!

Heres the code---------------------------------------------------------------

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/wait.h>
#include <signal.h>

#define MYPORT 8080 // The port users will be connecting to
#define BACKLOG 10 // How many pending connections queue will hold
#define MAXLINE 4096 // Max text line length

int hit = 0; //Keeps track of number of client connects.

void sigchld_handler(int s)
{
while(wait(NULL) > 0);
}

int main(int argc, char *argv[])
{
if(argc!=2) { //checks to see if filename was inputed in command line.
printf("Usage: http <file.html>\n");
exit(1);
}

//=== READ IN HTTP FILE ========================================================================

int count = 0;
char data[MAXLINE];
char ch;
static char* pidfile;

/* open source file ---*/
FILE *html;
if((html = fopen(argv[1], "rb"))==NULL) {
printf("Cannot open source file.\n");
exit(1);
}

while ((ch = fgetc(html)) != EOF) {
data[count] = ch;
count++;
}

fclose(html);

/* Write PID number to pid file in /var/run/http_pid --IGNORE WORK IN PROGESS==
int count2 = 0;
char data2[6];
char pd, pid2;
int pid;
char http_pid[6];
FILE *html_pid;
fopen(http_pid, "a+");
pid = getpid();
printf("PID: %d\n", pid);
/*pid2 = atoi(pid);
while ((pid2 = fputc(data2[count2] ,html_pid)) != EOF) {
data2[count2] = pd;
count2++;
}

//fputs(pid, html_pid);
fclose(html_pid); -------------WORK IN PROGRESS --- IGNORE*/

//=== NETWORKING PORTION =======================================================================

int sockfd, new_fd; // listen on sock_fd, new connection on new_fd
struct sockaddr_in my_addr; // my address information
struct sockaddr_in their_addr; // connector's address information
int sin_size;
struct sigaction sa;
int yes=1;

if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) == -1)
{
perror("socket");
exit(1);
}

if (setsockopt(sockfd,SOL_SOCKET,SO_REUSEADDR,&yes,sizeof(int)) == -1)
{perror("setsockopt");
exit(1);
}

my_addr.sin_family = AF_INET; // host byte order
my_addr.sin_port = htons(MYPORT); // short, network byte order
my_addr.sin_addr.s_addr = INADDR_ANY; // automatically fill with my IP
memset(&(my_addr.sin_zero), '\0', 8); // zero the rest of the struct

if (bind(sockfd, (struct sockaddr *)&my_addr, sizeof(struct sockaddr)) == -1)
{
perror("bind");
exit(1);
}

if (listen(sockfd, BACKLOG) == -1) {
perror("listen");
exit(1);
}

sa.sa_handler = sigchld_handler; // reap all dead processes
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_RESTART;
if (sigaction(SIGCHLD, &sa, NULL) == -1) {
perror("sigaction");
exit(1);
}

while(1) { // main accept() loop
sin_size = sizeof(struct sockaddr_in);
if ((new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size)) == -1)
{
perror("accept");
continue;
}

hit++; //increment number of client connects or "hits".
printf("HIT: %d CLIENT IP: %s\n", hit, inet_ntoa(their_addr.sin_addr));
if (!fork()) { // this is the child process
close(sockfd); // child doesn't need the listener
send(new_fd, data, count, 0); //This line sends http data to client
close(new_fd);
exit(0);
}
close(new_fd); // parent doesn't need this
}

return 0;
}

If I understand HTTP correctly then the client will send you

GET / HTTP 1.1\n
\n

You then reply by sending

HTTP/1.1 200 OK
Connection: close
Content-Type: text/html
// Contents of default html page e.g. index.html

and close the connection. The server then reconnects and requests

GET /image1.png HTTP 1.1\n
\n

to which you reply

HTTP/1.1 200 OK
Connection: close
Content-Type: image/png
// Contents of image1.png

after which you close the connection again. There can be other data between the GET line and the blank line which you can simply ignore and you can also add extra fields in your response (e.g. Content-Length: File length in bytes). A good way of finding extra information out is to telnet into your local web server and issue your own GET requests.

3 Things...... and correct me if im wrong.

1) (QUESTION) I will need to have program read through html, when it finds a .png, .jpg, etc, then copy this file separately, after the html?

2) (OBSERVATION & QUESTION) I used a network sniffer on my eth0 and captured the data when I typed in www.yahoo.com using my mozilla browser. It gave me little text, losts of raw data. No html, at least in format readable by humans. When I did the same thnig on my crappy web server, I actually saw the html text. Do I need to somehow send the html data or file as binary data? Or maybe read EVERYTHING in, send it as binary? I dont know. :\

1. As NHS said, the client will first send a request for the HTML page, you send that back, then the client parses the HTML and determines if it needs to make requests for images, etc. From the servers perspective, you simply need to handle the individual requests for files.

2. The basic HTTP protocol is text-based, and the only time you will need to send back binary data, is for binary context-types like images.

As with most standard protocols, there is an RFC that explains the protocol. In this case, it appears that you want to look at RFC 2616

Also, as NHS also said, you can use telnet as a simple text-based HTTP client in order to see what is going on.

For example, try this.

$ telnet www.opengl.org 80<return>
Trying 192.48.159.181...
Connected to www.opengl.org.
Escape character is '^]'.
GET / HTTP/1.1<return>
Host: www.opengl.org<return>
<return>

Where the stuff in bold is what you type, and the <return> indicates that you should hit the enter/return key.

Thanks for the reply. Definetly gives me some ideas and some info to work with. Guess I need to study some http as well, since i obviously dont knnow nothing. :)

Thanks again for he help.

I know this is an old thread, but I need to hijack it for at bit. And mayby you can help me, with what you learned since 03-04

I'm too programming a server (well actually a proxy s. so I'm doing both client/server responses). I'm at the point, where I need to transfer images, and I'm a little bit puzzled about this. Do you actually need to write a CGi script to handle this, you can you do it, with C only? If I try to read the data, and save in char buffer, the data is not usable (well, at least that's what the client browser says). Can anyone send me in the right direction with advice, link ect. Any information is greatly appriciated.

/Jnusa

PS. Your data, you tried to read, was most likely gzip'ed :)

Transferring images is no different to transferring any other data. The Content-Type header will most likely be image/jpeg or image/png, etc. and this is the hint to the browser to treat the data differently. As you are acting as a proxy you should already have this information as the site you forward the request to will give you the Content-Type. I think trying to save the image in a large buffer is asking for trouble so I would read a bit write a bit same as when sending an HTML file. Provided you don't change the data (slight possibility if you somehow have network order/host order byte swapping problem but I can't think how) then the client should display it fine providing the server you are proxying for sends valid data.

Initially when a started to code the server basics, I did read about 1024 bytes and write them to the client at once. When I think of it, I did actually transfer pictures also...hmm. I need to manipulate with the pictures (everything actually), so reading /writing a bit to the client is not an option.
It's very strange then. I can transfer everything except images... If the client request an gif image, I read the whole image (say 30k) into a buffer, but when I send it to the client, the data is corrupted. But only images. Guess I need to check my char buffer. Thanks anyway. At least I'm in the right direction.

Well, here's my first version of the http server I wrote. It isn't pretty, but it works ;) The function that might help you out is the 'void ReadFile(char *name, int FD)' function. It accepts the file descriptor of the socket and the name of the path that is extracted from the http GET that comes out of the 'void parse_requests(char *buffer, int FD, char *HttpFileName)' function. Hope this helps, and dont laugh!

Code:

#include <stdio.h>

#include <time.h>

#include <stdlib.h>

#include <dirent.h>

#include <sys/dir.h>

#include <sys/param.h>

#include <unistd.h>

#include <errno.h>

#include <string.h>

#include <sys/types.h>

#include <sys/socket.h>

#include <netinet/in.h>

#include <arpa/inet.h>

#include <sys/wait.h>

#include <signal.h>



#define BACKLOG 10  // How many pending connections queue will hold



char pathname[MAXPATHLEN];

char Client_Addr[15];  // Global holds client IP address



void sigchld_handler(int s) {

        while(wait(NULL) > 0);

}



#define AUTH\

        "HTTP/1.1 401 Authorization Required\r\n"\

        "WWW-Authenticate: Basic realm=\"Public\r\n"\

        



// === TIME() ========================================================================================|



char *TIME(void)

{

        struct tm *ptr;  //Get Time & Date.

        time_t lt;

        lt = time(NULL);

        ptr = localtime(&lt);

        return asctime(ptr);

}



// === httpHEAD() ====================================================================================|



void httpHEAD(int FD)

{

        char Head[150];

        char *time;

        time = TIME();

        sprintf(Head, "HTTP/1.1 200 OK\r\n"

                      "Server: Scrag Server 0.7\r\n"

                      "Date: %s"

                      "Cache-control: private,max-age=60\r\n"

                      "Content-type: text/html\r\n"

                      "Connection: close\r\n\r\n", 

                      time);

        send(FD, Head, strlen(Head), 0);

        //printf("%s\n", Head);

        return;

}



// === httpAUTH() ====================================================================================|



void httpAUTH(int FD)

{

        char buf[1000];

        send(FD, AUTH, sizeof(AUTH), 0);

        //recv(FD, buf, 1000, 0);

        //printf("%s\n", buf);

        return;

}



// === httpERROR() ===================================================================================|



void httpERROR(int FD)

{

        FILE *file;

        char *buffer;

        unsigned long fileLen;



        //If error.html doesnt exist, send text msg error.

        file = fopen("html/error.html", "rb");

        if (!file) {

                send(FD, "ERROR 404 -Object not found.", 28, 0);

                return;

        }

        

        //Get file length

        fseek(file, 0, SEEK_END);

        fileLen=ftell(file);

        fseek(file, 0, SEEK_SET);



        //Allocate memory

        buffer=(char *)malloc(fileLen+1);

        if (!buffer) {

                fprintf(stderr, "Memory error!");

                fclose(file);

                return;

        }



        //Read file contents into buffer

        fread(buffer, fileLen, 1, file);

        fclose(file);

        

        int cnt=0;

        send(FD, buffer, fileLen, 0);

        free(buffer);



}



// === ReadFile () ===================================================================================|



void ReadFile(char *name, int FD)

{

        FILE *file, *file2;

        char *buffer;

        unsigned long fileLen;



        //Open file, if doesnt exist, httpERROR()

        file = fopen(name, "rb");

        if (!file) {

                httpERROR(FD);

                return;

        }

        

        //Get file length

        fseek(file, 0, SEEK_END);

        fileLen=ftell(file);

        fseek(file, 0, SEEK_SET);



        //Allocate memory

        buffer=(char *)malloc(fileLen+1);

        if (!buffer) {

                fprintf(stderr, "Memory error!");

          fclose(file);

                return;

        }



        //Read file contents into buffer

        fread(buffer, fileLen, 1, file);

        fclose(file);

        

        httpHEAD(FD);

        send(FD, buffer, fileLen, 0);

        free(buffer);

}



// === LOGGING =======================================================================================|



void LOG(void)

{

        FILE *logfile;

        if((logfile = fopen("log", "a+")) == NULL) {

                printf("Error opening log file.\n");

                exit (1);

        }



        fprintf(logfile, "HIT- CLIENT IP: %s  TIME: %s", Client_Addr, TIME());

        printf("HIT- CLIENT IP: %s  TIME: %s", Client_Addr, TIME());

        fclose(logfile);

}



// === Parse client recieve buffer, get requested file name ==========================================|



void parse_requests(char *buffer, int FD, char *HttpFileName)

{

        char str[4096], *p = buffer, *q;

            char str2[4096];

        

        while((p = strstr(p, "GET "))) {

                p += 4;  // 4 comes from strlen("GET ");

                if(!(q = strchr(p, ' ')))

                q = p + strlen(p);

                    strncpy(str, p, q-p); 

                    str[q-p] = '\0';

                int cnt, cnt2 = 0;

                for (cnt = 1; cnt <= strlen(str); cnt++) {

                        str2[cnt2] = str[cnt];

                        cnt2++;

                }                        



                //This IF checks if GET client request is for html.  

                if (!strcmp(str2, "")) {                                                        

                        int count_ch = 0;

                        char ch, html_text[10000];

                        FILE *html;

                        if((html = fopen(HttpFileName, "rb"))==NULL) {

                                printf("Cannot open source file.\n");

                                exit(1);

                        }

                        while ((ch = fgetc(html)) != EOF) {  

                                html_text[count_ch] = ch;

                                  count_ch++;

                          }

                        fclose(html);

                        //httpAUTH(FD);

                        httpHEAD(FD);

                        send(FD, html_text, count_ch, 0);

                        LOG();  

                }        

                else { 

                        ReadFile(str2, FD);

                }                

        }

}



// MAIN() ============================================================================================|



int main(int argc, char *argv[])

{

          if(argc!=3) {                //checks to see if filename & port were inputed in command line.

                printf("Usage: http <file.html> <Port>\n");    

                exit(1);

        }

        

        FILE *file;

        file = fopen(argv[1], "rb");

        if (!file) {

                printf("Error: Html file not found!\n");

                printf("Usage: http <file.html> <Port>\n");

                return;

        }

        fclose(file);



        //Set MYPORT variable to port entered at command line & convert from asci to int.

        int MYPORT;

        MYPORT = atoi(argv[2]);  

        

        int pid;

        pid = getpid();  //Get process ID number.

        

        printf("\033[1;36m--------------------------------------------------------------\n");

        printf("\033[1;36m HTTP SERVER RUNNING:\n");

        printf("\033[1;36m PID: %d  PORT: %d \n", pid, MYPORT);

        printf("\033[1;36m DOCUMENT ROOT: %s/%s\n", pathname, argv[1]);

        printf("\033[1;36m--------------------------------------------------------------\033[0m\n");

        

            // NETWORKING PORTION ===========================================================================|



        int sockfd, new_fd;  // listen on sock_fd, new connection on new_fd

          struct sockaddr_in my_addr;    // my address information

          struct sockaddr_in their_addr; // connector's address information

          int sin_size;

          struct sigaction sa;

          int yes=1;



            if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {

                          perror("socket");

                          exit(1);

                  }



          if (setsockopt(sockfd,SOL_SOCKET,SO_REUSEADDR,&yes,sizeof(int)) == -1) 

                {perror("setsockopt");

                      exit(1);

          }

              

          my_addr.sin_family = AF_INET;                // host byte order

            my_addr.sin_port = htons(MYPORT);            // short, network byte order

            my_addr.sin_addr.s_addr = INADDR_ANY;        // automatically fill with my IP

          memset(&(my_addr.sin_zero), '\0', 8);        // zero the rest of the struct



            if (bind(sockfd, (struct sockaddr *)&my_addr, sizeof(struct sockaddr)) == -1) {

                perror("bind");

                exit(1);

            }



            if (listen(sockfd, BACKLOG) == -1) {

                perror("listen");

                exit(1);

            }



            sa.sa_handler = sigchld_handler; // reap all dead processes

            sigemptyset(&sa.sa_mask);

            sa.sa_flags = SA_RESTART;

            if (sigaction(SIGCHLD, &sa, NULL) == -1) {

                perror("sigaction");

                exit(1);

            }



        // CLIENT (re)connect LOOP (re)starts HERE =====================================================|

        while(1) {                                  // main accept() loop

                sin_size = sizeof(struct sockaddr_in);

                if ((new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size)) == -1) {

                        perror("accept");

                        continue;

                }

                strcpy(Client_Addr, inet_ntoa(their_addr.sin_addr));



            // SPAWN CHILD & PROCESS CLIENT REQUEST ========================================================|

                char buf[2000];

                if (!fork()) {                        // This is the child process.

                        close(sockfd);                // Child doesn't need the listener.

                        recv(new_fd, buf, 1000, 0);

                        parse_requests(buf, new_fd, argv[1]);                        

                        close(new_fd);

                        exit(0);

                }

        close(new_fd);  //Parent doesn't need this.

    }

    return 0;

}

I think the problem is with the httpHEAD() function. It outputs a Content-Type header of text/html regardless of the input type. This will cause the web browser to treat as HTML a perfectly valid png/gif. What you need it a way of specifying the Content-Type correctly or if needs be leave it out when unknown. If missing (I think) the browser will attempt to guess by itself however if told that some data is HTML then it will attempt to render it no matter how wierd the HTML looks. I haven't checked your code 100% but it looks like you call httpHEAD() regardless of whether it's an image or HTML.

As an aside there is a point in your code where you have two counters in a loop (cnt and cnt2) where as far as I can see only one is needed given that cnt2 = cnt - 1. Hope this solves the problem.

I originally posted this thinking that the previous posting was by jnusa (as opposed to Scrag) however I think my commentary still holds assuming this server is also giving trouble with images. This may also help jnusa and if not then posting (or emailing me) a copy of the source code might well help pin down the problem.

OK, i've been debuggin the code, and it seems that I only recieve a fraktion of the data in the filedescriptor (also with text/html, just had not recognized. I've only been transfering small html files). It reads about 2000b-8000b from a image of about 400kb (within one read cycle). On the second read cycle, it return -1. Anyone encountered the same problem?

[EDIT]
I've used memset, to make sure that my buffer is allocated right, så that shoudln't be the problem.
[/EDIT]

Oka, here's my code. Nothing special, and I would think it is pretty straight forward. (not very optimized, but I just want functionality first)

Code:

int read_body(int fd, char *ptr, int length)

{        

        int        res = 0;

        int temp=1;

        int total = 0;



        if(ptr != NULL)

        {

                while(total < length )

                {        

                        res = (read(fd, ptr+total, length));

                        if(res != -1)

                                total += res;

                        if(res == 0)

                        {        puts("\n\n\nBREAK 0!!!!\n\n\n");

                                break;

                        }

                        if(res < 0)

                        {        puts("\n\n\nBREAK -1!!!\n\n\n");

                                break;

                        }

                        printf("\n Read: %d\tTotal read: %d\t Total length: %d\tfd: %d", res-temp, total, length, fd);

                }

                logit("\n", "content.log");

                logit(ptr, "content.log");

        }

        else 

                logit("\nEvt. malloc error -> no error handeling!", "system.log");        

        return res;

}

Here's the output of the terminal:

Code:

Status-code: 200

strlen(obj->body_content): 5548

Read: 2381    Total read: 2382        Total length: 5548    fd: 5



BREAK -1!!!

 

strlen(obj->body_content): 5548

Request end

Just a little additional error info. I've tried to see what the errno is, when I recieve the -1. It is "Connection reset by peer"
Do I have to maintain some kind of ACK / NAK confirmation, while reading data from the server? Most sites give me this error message, while a few work fine (very small sites - sub 3000b). Any information or prior experience will be greatly appriciated (Yes I'm pretty much stuck :|)

Regards Jnusa

I'm starting a new thread. Think people are scared away, with all the source code :)

Please post in thread read socket problem (proxy server related)

Thanks. Jnusa