Sunday, June 9, 2019

Fixing "This configuration exceeds the MTU reported by OpenSM, which is 2048"

I got tired of the following EventLog message:

According to the configuration under the "Jumbo Packets" advanced property, the MTU configured for device Mellanox ConnectX-3 IPoIB Adapter is 4092. The effective MTU is the supplied value + 4 bytes (for the IPoIB header). This configuration exceeds the MTU reported by OpenSM, which is 2048. This inconsistency may result in communication failures. Please change the MTU of IPoIB or OpenSM, and restart the driver.

First thing we want to adjust is partitions.conf file, set mtu to 5 (4092):
Default=0x7fff,ipoib,mtu=5:ALL=full;

By default, ancient OpenSM 3.3.11 for some reason will not read partitions.conf file specified in global opensm.conf
#
# PARTITIONING OPTIONS
#
# Partition configuration file to be used
partition_config_file %ProgramFiles%\OFED\OpenSM\partitions.conf
The easiest way to force it to read is to recreate a service with forced -P parameter. Here's how to do it in default PowerShell:
#
# Check if we have OpenSM service
#
Get-Service op*
#
# If it's present and running, stop and remove this service.
# There's Remove-Service available with PowerShell 6.0, which you likely don't have.
# Use sc.exe delete OpenSM or command below
#
Stop-Service 'OpenSM'; Get-CimInstance -ClassName Win32_Service -Filter "Name='OpenSM'" | Remove-CimInstance
#
# Then, create a new OpenSM service
#
New-Service -Name "OpenSM" -BinaryPathName "`"C:\Program Files\Mellanox\MLNX_VPI\IB\Tools\opensm.exe`" --service -P `"C:\Program Files\OFED\OpenSM\partitions.conf`" -L 128" -DisplayName "OpenSM" -Description "OpenSM for IB subnet" -StartupType Automatic
#
# Start service
#
Start-Service OpenSM
#
# Check if the service is running:
#
Get-Service OpenSM

Done, check System log in Event Viewer.

Fixing OpenSM service not running

So, we've set up OpenSM as a service in Windows, with Start Type = Automatic. https://wchukov.blogspot.com/2019/06/fixing-this-configura...