I'd like to examine network programming at the packet level, never done this before, so not really sure what to ask. I'd like to examine the tcp/ip stack, down to the contents of a packet, and make decisions/actions based on that content (do some routing, blocking, data manipulation, etc). Is there a "best" language that I should use for this type of low-level control? I lean toward a interpretive rather than compiled language, as this would speed up my interaction time with writing code and seeing the results, but I don't want to limit my control too extensively. Any recommendations for tutorials/books that an entry person would find useful? I will need to set up a box or two as my lab. Any hints on a suggested test only architecture? I assume that to help move things along a server/client and monitor software would be needed in each box, what should I consider there? Any other software tools? What did I miss?

This is a lab type of exercise, not aimed at a practical solution to any specific problem. I just want to really learn this area well.

for book, i found UNIX Network Programing 2nd Edition very helpful. i used that book for UDP programming it was very easy to follow. Safari E-books has a copy of it and if your lab has access to Safari, you can read it there. for Kernel level, Understanding Linux Network Internals looks good and it is available in Safari.

for programming language, when you dive into low level, there's only C, and that's it. however Python can do C and you can use it for userspace programming.

and there's some network sniffers for reading raw packages and inserting data into them. SBo lists dsniff but i never used one.

You can use Perl, eg http://search.cpan.org/~yanick/NetPa...tPacket/TCP.pm.
Basically, look at search.cpan.org and use the search box and/or the Networking IPC menu.

Perl docs/tutorials:
http://perldoc.perl.org/
http://www.perlmonks.org/?node=Tutorials
Highly recommend the Perl Cookbook as well.
It feels like a scripting lang, but it's compile don the fly, so its actually pretty fast (http://www.perl.com/doc/FMTEYEWTK/comp-vs-interp.html)

The classic TCP/IP books are W. Richard Stevens TCP Illustrated series. Vol 1 is prob enough for you.
http://www.kohala.com/start/tcpipiv1.html