I cant understand the role of tcp flag option. Manuel says first list is mask second is flag must be set. Does it examine packets including RST ??

I know -m limit meaning but how does tcp-flags prevent port scanning?

I can't see how that would be effective .. port scans can be customised pretty heavily, for example running nmap with a syn scan would not be stopped by these rules, you could also alter the timing to avoid these limits.

It doesn't; that's a mis-statement of the problem. Essentially, what we have here is 'if we are getting a lot of new stuff from one source, it can't be constructive, we'll just drop it'. Look at the flags that the traffic that it catches doesn't have.

That's true, and until you can just look for the 'packets that you probably just want to drop' flag, and I don't think that'll be along in a hurry, will always be true of this kind of thing. OTOH, if you can make scanning pretty tedious for a (presumed) evil-doer, then they'll probably go and make a mess on someone else's lawn first.

As with many of these things, there is an extent to which these things can do good, but there is also an extent to which these things can be gamed, so you've got to asses whether they are really doing any good. (Or, in this case, more likely cause the legitimate users of the network inconvenience by stopping them from scanning.) But the point is that there is a cost, and it may not be apparent what that is until you sit back and think about it in context.

If, for example, you are only trying to hide where you have put your ssh port, you might be better advised to ensure that ssh was otherwise secured. Potentially.

Actually "stealth" is one of those problems that has been created and hyped by a person or persons in the previous millennium. Best filed under "/IntarWEB/misconceptions/security by obscurity" and left to rot there w/o implementing it. Better search for: hardening, intrusion detection, port scan detection.