I have been working xvid encoding of 720p & 1080i transport steams via mencoder.

I get excellent results with very little options using the following.

mencoder *.mpg -ovc xvid -vf crop=1248:712:0:0,scale=624:352 -ofps 24000/1001 -sws 2 -xvidencopts pass=1 -nosound -o /dev/null

mencoder *.mpg -ovc xvid -vf crop=1248:712:0:0,scale=624:352 -ofps 24000/1001 -sws 2 -xvidencopts bitrate=1020:pass=2 -nosound -o sample-mencoder.avi

I am encoding the audio and video separately and then merging them together when both are complete; this seems to help keep the audio in sync.

The problem I am having is although the picture is crisp movement is very jittery, meaning if there is very little movement the video looks as good as the source but if there is a quick adjustment it seems as if frames are being missed. From what I read with the standard setting (not really optimized) this should be compensate for.

Can any give me any input on options to use to help this?

You can download the sample output here

Obviously, the -ofps ("output framerate") option is the problem.

I assume your source is interlaced? If so, try just deinterlacing rather than altering the frame rate.

If your source was originally shot on film and then converted to video, like anime or a feature film, then instead try using an "ivtc" filter rather than altering the frame rate.