As you propose, I tried to play with different combination of permissions and I finally find a way of treating my probleme.
I have created a second directory in which only allowed users could read and then I linked all the content of the first directory in the second one.
I gave rwx to the group 1 on the first directory and only --x to the group2
I then gave rwx to the group 2 on the second directory and none to the other groups
Finally I changed the rights of the content of the first directory to 775 so that the links of the second directory could work right for members of group2
Thanks for your help to a