Im working on a project which I need to take security to the next level.

The core of what I need is this:

When a host requests a IP address from the DHCP server it needs to be given a temperary IP.
Then a script Im working on needs to be triggered to check the netbios name ( hosts are windows primary )
If the netbios names has not changed since last ( checking in a table of valid netbios names ) or if the name is allowed by a list, it will be all fine with the DHCP server.
If the netbios name is NOT allowed I need a way of the DHCP to retract the lease of the IP ( making it invalid somhow ) not just to the firewall, but to the network itself.

So in short: I need a way for the DHCP server to issue a temporary IP or somhow retract the IP given to a client making it unable to use the network untill that client reboots for another check.

Anyone ever done anything like that ?

Not done it,
Why not just set static addresses and set the dhcp server to only give out addresses to those with a static address (the mac addresses you know)

Well i basicly need a dhcp which can be altered to run a script every time it offers a IP.
The project itself might sound stupid i know but its a part of a bigger plan here.

I need it to run a netbios namecheck every time a computer gets on the network as well as verify the mac and such.

why don't you just forget the temporary ip thing and instead just tell the dhcp server to give the clients the IP you want them to have depending on their mac address (the common method)??

as for the netbios, your dhcp server can also give them the address of the netbios name server, which could be on the same box/IP the dhcp server is running on... i assume the software one uses to serve netbios names to windows boxes is something samba-related but i'm not sure cuz i've never used netbios...

as for the making the IP unusable, i think you're beter off scripting something with iptables to handle that... like, the script could check the lease file, then cross-check with the netbios name server, and if things don't seem right then it would just add a DROP/REJECT rule to the FORWARD chain on the gateway for that source IP... so the client would need to get a new/proper lease in order to be able to connect to the outside world...

anyways, it's just a thought...

Not exactly sure if this is what your trying to accomplish, but check out NetReg

http://www.netreg.org/

Supposing you get it to work the way you want.

What happens AFTER the IP has been rejected?
The client can request a new IP
and the whole process starts all over again... sounds like a good way to attack and DoS your dhcp server.

You could assign new/unknown MAC addresses to a subnet that has limited network access (ie access only to a jumpstart or patch server).