Problem solved. I tried things like changing the order of module loading and giving pci arguments to the kernel, but this was not the problem at all. I started looking at the network setup and changed the channel of my wireless ad-hoc network to 9 instead of 1, and that
made all the difference. No more lock-ups. So after all the laptop was handling the devices fine even though they were using the same irq.
Why using channel 1 made the whole system freeze I just can't get my head around, but now it just works