Infiniband on Solaris 10 help needed

TronCarter · 02-01-2010, 07:54 AM

I need some help getting Infiniband to work on Solaris 10. I would like to be able to use IPoIB for ssh communications between nodes of a cluster. IB is new to me, all I have used before for clustering is ethernet connections. I don't really know where to begin and can't really seem to find any clear documentation online that talks about how to do it. So far I have installed:

Sun Studio 12.1
Sun ClusterTools 8.2.1
IB Updates 3.1 (so that I could update the firmware on the ConnectX cards)

cfgadm shows this:

Code:

# cfgadm
Ap_Id                          Type         Receptacle   Occupant     Condition
hca:3BA00010068A8              IB-HCA       connected    configured   ok
ib                             IB-Fabric    connected    configured   ok

If I do an ifconfig -a plumb and then ifconfig -a, all I see is my ethernet ports and loopback.

dladm also just shows my two ethernet ports.

I don't really know where to go from here.

Tron

TronCarter · 02-03-2010, 10:49 AM

I ended up figuring it out. I needed to run:

Code:

devfsadm -C

Then it showed up under ifconfig -a. I could then assign and IP and it works now.

linuxunix · 02-08-2012, 02:07 AM

I have two Solaris 10 Update 10 machine where I have by default Infiniband card driver installed but I am unable to see the ibd0 port even after running devfsadm -C command.Any idea why its not working?

Below are the few details:

Code:

Scanning PCI-e slot for Infiniband Card:-

#/usr/X11/bin/scanpci | grep Mellanox
Mellanox Technologies MT26428 VPI PCIe 2.0 5GT/s – IT QDR / 10GigE]


#ls –R /devices | grep 15b3
Pci15b3, 22@0
Pci15b3,22@0:devctl
/devices/pci@0,0/pci8086,3410@9/pci15b3,22@0:

#pkginfo | grep –I infini
System SUNWeoib  Solaris Ethernet over InfiBand
System SUNWib  Sun InfiniBand Framework
System SUNWibsdp   Sun InfiniBand layered Sockets Direct Protocol
System SUnWibsdpib
System SUnWibsdpu
System SUNWiopoib
System WUNWrpcib

#modinfo | grep infi
92 ffffffffefb0a000   b4d0  -  1 ibdm ( InfiniBand Device Manager)
250 fffffffffff08e2000  e9c8 170 I ibd (InfiniBand GLDv3 Driver 1.38)