Sorry for the slow reply and thanks for your help.
Running passenger with or without site config didn't change anything (there used to have a site configured, but I disabled it for testing purposes when I discovered the problem), but in fact, the problem didn't come from there.
I suspect it came from the leap second on June 30th that wasn't handled correctly for some reason by Passenger/Ruby.
In fact I found this thread
describing the same problem I had.
The solution was "just" to reboot the server and indeed, it works now.
I still don't know where the real problem was though, but at least my server works.