As always, you will find the answer in the man-page of the command:
In this case, if you want to remove a user from a group you have to use the -G option without -a and specify in which groups the user should remain a member. The user will be removed from any group that exists but is not specified in the command.