Open and use a port for data streaming.

malcmcmul · 05-14-2012, 06:05 AM

Assume that I'd like to test the performance between two servers (S1 and S2) by sending multiple gigs worth of files via the DD command and a configured file share. Now, I'm trying to mimic this with streaming data via ports.

The firewall on S1 and S2 have been configured to accept all connections from port 1x.
What else is required to fully open port 1x?
Are there any commands available to create a random bunch of data (much like dd) to pass to this port?
How do I go about monitoring the port at the other side - a daemon - to then, say, convert this stream into an ever-growing file e.g. 1x-output.txt?

fukawi1 · 05-14-2012, 06:23 AM

2) Something needs to be listening for connections/data.
3) the (u)random device..
4) This can be done fairly easily with netcat.

Listenning side:

Code:

nc -l 1234 -v
x��8�]���<i������=\�~/́h�RQ%��}i��5���ɮ4B�J��{H�\	\g������E�sLB|o�b
�5$�0�ƿ3�6\z�F���ʂ�o���N'��4R�7�芧�&��F��������_;�~W�Ҷ�ɐ1J�4'�ڭ�e,XAW�,����B<��R���C<�=��Ӷ�\��.�eOV�:`lm�;��r��;�*��ͤȱ
{b���ye���;����?=�ĺ�qL��4f�RiA4��Z��OI*��C?x�7�H�P��F�]sC��E	b�m���*fA��ď�_�Q���="���dwh����k�j'@�TF�I��ώ�!�h	U�g��-�]��ǋ�����E�e��s����[I;c62;9;c^C���z�A��kU��)�cޮ�.#=�#�Y���5�0��U�� ���f4�?�62;9;c62;9;c62;9;c62;9

The gibberish here, is what is being sent from the command below.
You can use normal BASH redirection to output the data to a file.

Sending side:

Code:

dd if=/dev/urandom | nc localhost 1234

See http://www.manpagez.com/man/1/nc/ for more info on using nc.

malcmcmul · 05-14-2012, 06:38 AM

Quote:

Originally Posted by fukawi1

2) Something needs to be listening for connections/data.
3) the (u)random device..
4) This can be done fairly easily with netcat.

Listenning side:

Code:

nc -l 1234 -v
x��8�]���<i������=\�~/́h�RQ%��}i��5���ɮ4B�J��{H�\	\g������E�sLB|o�b
�5$�0�ƿ3�6\z�F���ʂ�o���N'��4R�7�芧�&��F��������_;�~W�Ҷ�ɐ1J�4'�ڭ�e,XAW�,����B<��R���C<�=��Ӷ�\��.�eOV�:`lm�;��r��;�*��ͤȱ
{b���ye���;����?=�ĺ�qL��4f�RiA4��Z��OI*��C?x�7�H�P��F�]sC��E	b�m���*fA��ď�_�Q���="���dwh����k�j'@�TF�I��ώ�!�h	U�g��-�]��ǋ�����E�e��s����[I;c62;9;c^C���z�A��kU��)�cޮ�.#=�#�Y���5�0��U�� ���f4�?�62;9;c62;9;c62;9;c62;9

The gibberish here, is what is being sent from the command below.
You can use normal BASH redirection to output the data to a file.

Sending side:

Code:

dd if=/dev/urandom | nc localhost 1234

See http://www.manpagez.com/man/1/nc/ for more info on using nc.

Thanks for the response, you've provided an excellent answer!

With regards to [2], the fact we're using netcat to send the files at source and then using netcat to receive the files and redirect to a file, this has in respect set up the "Something [that] needs to be listening for connections/data. ", correct? Or is there an additional initialisation process we still haven't done? I was expecting to set up a daemon or does the "nc -l 1234 -v" accomplish this?

I've used the dd command in the past. Obviously if the file size needed to be limited, you'd just add 'bs=[size] count=1', right ?

fukawi1 · 05-14-2012, 06:51 AM

Quote:

With regards to [2], the fact we're using netcat to send the files at source and then using netcat to receive the files and redirect to a file, this has in respect set up the "Something [that] needs to be listening for connections/data. ", correct?

This is correct, nc is the listener.
You can see this using "netstat -tnlp".

Code:

~ $ netstat -tnlp 
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:28764               0.0.0.0:*                   LISTEN      16600/skype         
tcp        0      0 0.0.0.0:45001               0.0.0.0:*                   LISTEN      22636/transmission- 
~ $ nc -l 1234 -v &
[1] 3197
~ $ netstat -tnlp
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:28764               0.0.0.0:*                   LISTEN      16600/skype         
tcp        0      0 0.0.0.0:45001               0.0.0.0:*                   LISTEN      22636/transmission- 
tcp        0      0 0.0.0.0:1234                0.0.0.0:*                   LISTEN      3197/nc             
~ $

(I have cut out some irrelevant crap from this example, but the important bits are still there)

malcmcmul · 05-15-2012, 08:26 AM

Quote:

Originally Posted by fukawi1

This is correct, nc is the listener.
You can see this using "netstat -tnlp".

(I have cut out some irrelevant crap from this example, but the important bits are still there)

Excellent, thanks for that. I've been doing a little bit of a testing with this and noticed it stops listening for the file once a stream stops. Is there any reason for this - I can't logically understand how it would even know a data stream has stopped unless it's keeping a timer once it starts receiving data (unless the file being sent states it's the end of the file and the listening process sees this as a stop?)? Is there a way to force it to continually listen (and output to file) until told to stop? At present I'll pass through a 1MB or 1GB file and once this ends, the listening ends. I'd like to periodically pass through data without closing the connection at the destination side.

Hopefully that question made sense. I'll gladly clarify further...

fukawi1 · 05-16-2012, 04:07 AM

Quote:

(unless the file being sent states it's the end of the file and the listening process sees this as a stop?)?

As I understand it, this is pretty much the case, except that rather than the server (listener) detecting the end of the stream, the

Quote:

Is there a way to force it to continually listen (and output to file) until told to stop?

Yes, with the "-k" option. I would suggest reading the man pages for detailed info on using nc. I don't believe you can tell it to stop from the client (sending) side with nc itself, you would have to terminate the process manually (ctrl+c, or ssh the server(listener) and kill the process).

That said, nc wont be able to distinguish the end of one stream from the beginning of the next, as far as redirecting to a file goes..

For transferring files though, scp may be a better option, (encryption, error checking, blah blah).