I have a couple questions in this thread. I have a sneaking suspicion that "udev" may be involved in this somehow. A little bit of background. I'm trying to set up a system with NVIDIA devices. There is on board video and multiple Tesla devices. The system loads and operates properly. Now I'm just trying to fine tune the system to work exactly the way I want. The issues I am trying to resolve are:

1) The onboard video is enumerated first making it addressed at 2:0:0, where as the other devices that get loaded under the same driver are addressed to 5:0:0, 6:0:0, etc. Naturally, this makes logical sense in most configurations, but not in what I'm trying to do.

Because of this, the onboard video gets loaded into /dev/nvidia0, and therefore, becomes the first CUDA device that CUDA applications see. I would like to try and reassign the onboard device to a different /dev/nvidia* node so that CUDA queries the device last, and all other CUDA devices first.

2) I want to get all NVIDIA device drivers loaded and /dev/nvidia* populated BEFORE the X window interface tries to load up. Currently, this isn't the case, and when X does load up, it halts with no video response for a couple minutes as it tries to load in driver modules for all devices.

I have a feeling that if the drivers get loaded before hand, then loading X shouldn't have any issues since currently, when I get into the GUI (after X has queried all devices), I can init 3 out of X, and init 5 back in, and X loads instantly.

This issue is primarily a boot issue since at the moment, it adds about 2 minutes to the overall boot sequence, unless I boot in runlevel 3, in which case, there are no issues at all in boot times.

Any thoughts and pointers?

Look in /sys and see if there is information which allows you to uniquely identify each NVidia device. For example:

udevinfo -a -p /sys/module/nvidia

Or something like that. If there is no information to uniquely identify the various devices then you're in trouble. Looking at my own system (NVidia 6800) sysfs has no unique info. I guess you'll have to use the PCI Bus ID somehow to instruct udev to name the devices.

Thanks. I just tried that command and it is just spitting out some udev rules (I'm still very green with udev, so I don't fully understand the protocol). Nothing device unique.

Using 'lspci', the NVIDIA devices are listed as (assigned to):

02:00.0 (/dev/nvidia0)
05:00.0 (/dev/nvidia1)
06:00.0 (/dev/nvidia2)
07:00.0 (/dev/nvidia3)
08:00.0 (/dev/nvidia4)

While I actually want it to look more like:

05:00.0 (/dev/nvidia0)
06:00.0 (/dev/nvidia1)
07:00.0 (/dev/nvidia2)
08:00.0 (/dev/nvidia3)
02:00.0 (/dev/nvidia4)

Somehow though, I want to make sure that all of this gets done before X actually loads. I just don't know where to begin on either of these things.