OpenSolaris

Heads Up: NFS/RDMA changes impact NFS mounts over IB

Date: Fri, 10 Apr 2009 23:17:10 -0700
From: Mahesh Siddheshwar <siddheshwar.mahesh at sun dot com>
To: onnv-gate at onnv dot eng dot sun dot com, on-all at eng dot sun dot com, on-discuss at opensolaris dot org
Subject: Heads Up: NFS/RDMA changes impact NFS mounts over IB

You can safely ignore this message if you do not use NFS over Infiniband.

The integration of changes for:

6760947 NFS/RDMA port should be changed to IANA assigned 20049
6762173 rdma panic on writes from linux client
6790590 readdir fails from Linux client against Solaris server
6790588 linux client fails to decode READ replies from Solaris server
6790586 Solaris server should better handle chunked RPC/RDMA messages
6826476 rpcib leaks memory registrations while handling multiple chunks

requires that both the NFS client and the NFS server run the
above changes to use NFS/RDMA. This is due to a change in the
NFS/RDMA port number that is used to the IANA assigned port
number of 20049, along with the enforcement of 4-byte aligned
XDR encodings, as per protocol requirements.

These changes enable interoperability with the Linux NFS/RDMA
implementation.

If the client and server are inconsistent with respect to
the above changes, then NFS/RDMA mounts will hang while mounting.
You can work around the limitation by doing the mount over
IP-over-IB, by using the "proto=tcp" mount option.

You can verify if a system is running the above changes by
checking for the presence of the "nfs_rdma_port" symbol, by
running:

# echo "nfs_rdma_port/D" | /usr/bin/mdb -k
nfs_rdma_port:
nfs_rdma_port:  20049

If the system does not have the changes, then it fails
to de-reference the symbol:

# echo "nfs_rdma_port/D" | /usr/bin/mdb -k
mdb: failed to dereference symbol: unknown symbol name

(requires root access or additional system privileges to run the above)

A planned backport of the NFS/RDMA Transport Version update
(PSARC 2007/347) to Solaris 10, will include these new changes and
will enable NFS/RDMA compatibility with Solaris 10 systems, which
is currently non existent.

Mahesh