Yes, I've taken that into account. All shared memory is opened 0600.
The main point of that aspect of security is to preclude targeting of particular processes. All pipes connected to the server are available via /proc, also.
When I mentioned that the supposed detriment caused by not being able to attach to flagged memory segments was actually a useful feature, I meant that once the memory is flagged, nothing else can access it. By flagging it immediately after the client accesses it, nothing further may attach to it, thereby protecting it.
Really the only weakness of the shared memory over using pipes is possible corruption. I've got that covered, also, though. I've designed the flow so that clients can't force exceptions within the server by corrupting shared memory.
PS Here is the entire security policy if you'd like to see. It explains a little more about the server itself.