Your primary group is the group that gets used when you create files/directories. Any file/directory with a group id that you are a member of (primary or otherwise) you will have access to according to the group permissions.
You can modify the behaviour by setting the group id bit on directories, but that's the simple overview