LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-07-2021, 07:50 PM   #1
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Rep: Reputation: Disabled
GET requests knock out my server


Hello. I decided to scrap my original server. I managed to compile ncat. I am using -k option to keep server open. Instead of accepting data to STDOUT, my goal is to write to files instead. So far I was able to write to a file instead of STDOUT but my goal is to loop through new files on each new connection. Right now it is appending to the same filename_0 and f++ is not incrementing. Here is what I have so far. The original code will be below.

Code:
MODIFIED CODE:
/* Read from a client socket and write to stdout. Return the number of bytes
   read from the socket, or -1 on error. */
int read_socket(int recv_fd)
{
    char buf[DEFAULT_TCP_BUF_LEN];
    struct fdinfo *fdn;
    int nbytes, pending;
    int f = 0;    

    fdn = get_fdinfo(&client_fdlist, recv_fd);
    ncat_assert(fdn != NULL);
    nbytes = 0;
    do {
        int n, s;
  
        n = ncat_recv(fdn, buf, 512, &pending);
        
        if (n <= 0) {
            if (o.debug)
                logdebug("Closing fd %d.\n", recv_fd);
#ifdef HAVE_OPENSSL
            if (o.ssl && fdn->ssl) {
                if (nbytes == 0)
                    SSL_shutdown(fdn->ssl);
                SSL_free(fdn->ssl);
            }
#endif
            close(recv_fd);
            checked_fd_clr(recv_fd, &master_readfds);
            rm_fd(&client_fdlist, recv_fd);
            checked_fd_clr(recv_fd, &master_broadcastfds);
            rm_fd(&broadcast_fdlist, recv_fd);

            conn_inc--;
            if (get_conn_count() == 0)
                checked_fd_clr(STDIN_FILENO, &master_readfds);

            return n;
        }
        else {
            char filename[20];
            snprintf(filename, sizeof(char) * 20, "filename_%i", f);
            FILE *fp = fopen(filename, "a");
            if (fp == NULL)
            {
                printf("Could not open file");
                return 0;
            }
            //Write(STDOUT_FILENO, buf, n);
            s = fwrite(buf, 1, n, fp);
        }
            fclose(fp);
            f++;
            nbytes += n;
    } while (pending);

    return nbytes;
}

ORIGINAL CODE:
Code:
int read_socket(int recv_fd)
{
    char buf[DEFAULT_TCP_BUF_LEN];
    struct fdinfo *fdn;
    int nbytes, pending;

    fdn = get_fdinfo(&client_fdlist, recv_fd);
    ncat_assert(fdn != NULL);

    nbytes = 0;
    do {
        int n;

        n = ncat_recv(fdn, buf, sizeof(buf), &pending);
        if (n <= 0) {
            if (o.debug)
                logdebug("Closing fd %d.\n", recv_fd);
#ifdef HAVE_OPENSSL
            if (o.ssl && fdn->ssl) {
                if (nbytes == 0)
                    SSL_shutdown(fdn->ssl);
                SSL_free(fdn->ssl);
            }
#endif
            close(recv_fd);
            checked_fd_clr(recv_fd, &master_readfds);
            rm_fd(&client_fdlist, recv_fd);
            checked_fd_clr(recv_fd, &master_broadcastfds);
            rm_fd(&broadcast_fdlist, recv_fd);

            conn_inc--;
            if (get_conn_count() == 0)
                checked_fd_clr(STDIN_FILENO, &master_readfds);

            return n;
        }
        else {
            Write(STDOUT_FILENO, buf, n);
            nbytes += n;
        }
    } while (pending);

    return nbytes;
}

Last edited by rootaccess; 11-09-2021 at 12:48 AM.
 
Old 11-07-2021, 08:19 PM   #2
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
Error code 104 most likely means "Client reset by peer", which means the client closed the connection.

My guess would be that your server does not respond properly to the HTTP GET request sent by the client, so the client ends the connection and your server bails out.

An HTTP GET request requires a proper response code from the server. HTTP requests are two way communications, not just dumps from the client end.

Last edited by astrogeek; 11-07-2021 at 08:22 PM.
 
Old 11-07-2021, 08:25 PM   #3
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
yes that is exactly what is happening. i guess the question is..how should i handle this? do I need a handler for this? I basically want to prevent any requests that would require me as the server to respond to those.
 
Old 11-07-2021, 08:39 PM   #4
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
You either need to handle the HTTP protocol, at least a subset of requests/responses sufficient to your needs, or you should set up the clients to simply stream the data instead of wrapping it with HTTP protocol headers.

Have a look at netcat (ncat on many Linux distros) as a way of doing that, or you could write a corresponding client to simply open the connection and stream the data. Writing your own client would also answer the part about using a specific header string - you would be in complete control of what you send and in what order, but you would also be responsible for detecting and handling transmission errors.

** UPDATE ** - When writing the above I had in mind that the HTTP requests were from your own client - but after reading again I think you mean they are just drive by requests... If that is the case then you might try detecting the HTTP headers and simply close the connection yourself without exiting the server, or have your real requests send a unique identifier as the first set of characters, then close the connection for anything else (including the HTTP headers).

Last edited by astrogeek; 11-07-2021 at 08:50 PM.
 
Old 11-07-2021, 09:12 PM   #5
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
yes these are drive by requests. what you mentioned is exactly what i want to do. yes i do have clients that i wrote myself but i have not programmed them with a unique header but that was what i had planned. can you point me to where i would need to for handling other requests (detecting these HTTP requests or anything other than what my clients will be sending) and basically just dropping their packets? which functions would i be using to do that? thanks a lot.
 
Old 11-07-2021, 11:16 PM   #6
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
Well, I guess you would need to detect the closed connection, clean up the loop environment and return to the top of the loop instead of exiting.

Using your existing code, perhaps something like this...

Code:
...
if (errno == EAGAIN){
   printf("recv() timed out.\n");
   }
else if(errno == ECONNRESET ){
   printf("Connection closed by peer (HTTP REQUEST?)...\n");
   /* Clean up and continue here if needed, else fall through */
   }
else{
   fprintf(stderr, "recv() failed due to errno = %d\n", errno);
   exit(1);
   }
...
But if it were me and this were a known problem I think I would provide a means of detecting whether the connection were valid or not before entering the write block at least.
 
Old 11-07-2021, 11:51 PM   #7
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
i think you just helped me fing the problem right there. its the `exit(1)` that is happening right after the errno message. i didnt write this server, i wrote half of it. but that block was left by the original author and i believe that's the issue. He only address EAGAIN and someone else even mentioned short writes dont have errors, but there is an if and else, which exits. meaning any errors that don't time out, will just exit which isn't something i want to do.

that errno message is exactly what was displayed on my terminal (and in server_logs.txt where I log the same exact output) just before it exited. However, the GET / HTTP 1.1 was also logged, so fr_block_sz definitely had at least the 9 or so bytes it needed to output that line into one of the next files. Which means that the data is coming through. You mentioned I could in control of all the data coming through but where (before the write block) could I add something to deny all packets unless they start with some unique string like "Hello World!" ?

As for ECONNRESET, not sure how to handle that because I am a little confused if my code is actually correct since I am not threading. The file descriptor is the same one for each client. It picks one from the beginning and rolls with it the entire process.
 
Old 11-08-2021, 01:07 AM   #8
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
well i have a big problem now. when i run my client or a python script to send, it works fine, but if i use nc without < file.txt it waits, and here i can just ctrl-z and stop the job, then the server hangs indefinitely until i ctrl-c from the client. i tried non-blocking with accept4() but then the client and python have broken pipes. its really strange.
 
Old 11-08-2021, 06:42 AM   #9
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 1,780

Rep: Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456

Why are you trying to use that C script instead of simply using an already existent HTTP-compliant server?

 
Old 11-08-2021, 10:14 AM   #10
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
This is a custom script because a custom port/service will be utilized here. the client will be communicating directly over the custom port. i tried compiling rcat from nmap but even that does not compile. that would have solved a lot of the multithreading/handling for me since all I would have needed to do was insert my file creation (its keep-open option stays listening continuously), but they've had an issue for 10 years and haven't resolved it. Truthfully, I do have the source code, it does not compile for other reasons I'm not sure of but I can follow that code and figure out what they're doing but that is pretty painful.
 
Old 11-08-2021, 10:33 AM   #11
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 1,780

Rep: Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456
Quote:
Originally Posted by rootaccess View Post
This is a custom script because a custom port/service will be utilized here. the client will be communicating directly over the custom port.
I don't understand - you're using an incomplete/buggy script because you want to change the port number it listens on!?

I can't name a single server that doesn't allow you to do that.

 
Old 11-08-2021, 10:58 AM   #12
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
I know that I can do that but the point is the client sends files automatically every couple of hours directly and seamlessly. Plus the server should only be designed to accept a unique string as the initial buffer, and should drop all other packets, and it should only accept files, it will basically be my FTP server. That way no one else can even make a connection to it. The problem with FTP is i cannot modify it to ONLY accept files with certain unique strings as the initial header/buffer before the rest of the data. Also, when i was trying out FTP, I was not able to anonymous upload directly, there always needs to be some login required which I don't want to deal with. The solution I have works, it's just that I need to handle multi-threading somehow so that a client or some other entity can't just hang up/lock up my server. NONBLOCKING seems to create other issues, not sure why.

BTW, I do understand that files are just streams of bytes, no different than if a buffer would accept manually typed data.

Last edited by rootaccess; 11-08-2021 at 11:02 AM. Reason: files are just byte streams
 
Old 11-08-2021, 12:08 PM   #13
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 1,780

Rep: Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456Reputation: 1456

You want to transfer files between servers, but without dealing with a login process or allowing unauthorized access?

Use ssh keys + rsync.

(Also read this)

 
1 members found this post helpful.
Old 11-08-2021, 12:22 PM   #14
rootaccess
Member
 
Registered: Mar 2012
Posts: 227

Original Poster
Rep: Reputation: Disabled
Thanks for the tips but I know all that. Not what I'm looking for. This is for something specific.

I'll read the ncat code myself and figure it out since it seems a bit too complicated for others. I've mentioned this is for something specific. The clients are windows machines and they will be sending data that is being generated on the system automatically. They aren't going to be initiating any connections themselves....

Last edited by rootaccess; 11-08-2021 at 12:25 PM.
 
Old 11-08-2021, 12:36 PM   #15
michaelk
Moderator
 
Registered: Aug 2002
Posts: 22,081

Rep: Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430Reputation: 4430
Just for a different perspective I do something similar with xinetd which uses stdin to pipe the stream to your program. Since this was used on a limited LAN no encryption/login is necessary.

Basically xinetd manages the connection and the program handles the data. You can run your program as any user on any port. I question your beliefs that no one else can make a connection. Since your not using encryption technically anyone can monitor the traffic and figure out your initial buffer. One question would be why are you not using sftp or rsync with keys.

I also use PuTTy's command line tools to automatically send data.

Last edited by michaelk; 11-08-2021 at 12:41 PM.
 
1 members found this post helpful.
  


Reply

Tags
c++, linux, networking, programming


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Large botnet of CCTV devices knock the snot out of jewelry website LXer Syndicated Linux News 0 06-27-2016 10:24 PM
LXer: Microsoft: Knock, Knock, Knocking on Nokia’s Door LXer Syndicated Linux News 0 09-14-2013 01:52 AM
Knock knock kandiru LinuxQuestions.org Member Intro 1 03-31-2009 04:40 PM
LXer: Knock, Knock, Knockin' on EnGarde's Door (with FWKNP) LXer Syndicated Linux News 0 12-05-2007 01:50 AM
LXer: Knock Knock. Who's There? Mandriva LXer Syndicated Linux News 0 12-16-2005 03:16 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:58 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration