According to the configuration under the "Jumbo Packets" advanced property, the MTU configured for device Mellanox ConnectX-3 IPoIB Adapter is 4092. The effective MTU is the supplied value + 4 bytes (for the IPoIB header). This configuration exceeds the MTU reported by OpenSM, which is 2048. This inconsistency may result in communication failures. Please change the MTU of IPoIB or OpenSM, and restart the driver.
First thing we want to adjust is partitions.conf file, set mtu to 5 (4092):
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Default=0x7fff,ipoib,mtu=5:ALL=full; |
By default, ancient OpenSM 3.3.11 for some reason will not read partitions.conf file specified in global opensm.conf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
# PARTITIONING OPTIONS | |
# | |
# Partition configuration file to be used | |
partition_config_file %ProgramFiles%\OFED\OpenSM\partitions.conf |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
# Check if we have OpenSM service | |
# | |
Get-Service op* | |
# | |
# If it's present and running, stop and remove this service. | |
# There's Remove-Service available with PowerShell 6.0, which you likely don't have. | |
# Use sc.exe delete OpenSM or command below | |
# | |
Stop-Service 'OpenSM'; Get-CimInstance -ClassName Win32_Service -Filter "Name='OpenSM'" | Remove-CimInstance | |
# | |
# Then, create a new OpenSM service | |
# | |
New-Service -Name "OpenSM" -BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`" --service -P `"C:\Program Files\OFED\OpenSM\partitions.conf`" -L 128" -DisplayName "OpenSM" -Description "OpenSM for IB subnet" -StartupType Automatic | |
# | |
# Start service | |
# | |
Start-Service OpenSM | |
# | |
# Check if the service is running: | |
# | |
Get-Service OpenSM |
Done, check System log in Event Viewer.