Using the mptt SAS target-mode driver in COMSTAR
Background
The COMSTAR mptt driver provides the capability of using the following Sun SAS HBAs in target-mode within COMSTAR:
Marketing P/N Sun P/N Chipset Form factor ~---------------- ~-------- ~------- ~----------- SG-XPCIE8SAS-E-Z 375-3487 1068e PCIe SG-XPCI8SAS-E-Z 375-3488 1068 PCI-X
Driver
The mptt driver is a COMSTAR target-mode driver loosely based on the "mpt" initiator driver that is currently part of the ON consolidation. It has been extensively modified to fit in the COMSTAR framework and to provide target-mode capability for SAS connectivity.
This driver is not part of any Solaris release and is provided on an as-is basis. However, most of the necessary functionality is in place. This driver should operate well in standard SAS environments. Development was more or less complete in August, 2008, although a quick sanity check on build 107 indicates that the driver is still functional.
Caveats
The mptt driver does not support DDI suspend/resume, nor does it implement power management. This driver is designed to operate in wide- port configurations only.
General Usage
Usage of the mptt driver is similar to that of the qlt driver. See the existing COMSTAR documentation for details on creating and maintaining view entries (for zoning) and for details on enabling or disabling the COMSTAR service.
The details provided here are intended for users that are already familiar with how to use COMSTAR and will aid in explaining how the various administrative actions (e.g. stmfadm) alter the behavior of the mptt driver.
Installation
The mptt driver was originally developed with the intent to have it installed as part of the SUNWstmf packages. Since it is not integrated into Solaris and the packaging has not been updated, there are no standalone packages for mptt.
- Download the "mptt.tar.gz" file from the Files link. It contains the following files:
./root_i386-debug/usr/lib/mdb/kvm/amd64/mptt.so ./root_i386-debug/usr/lib/mdb/kvm/mptt.so ./root_i386-debug/kernel/drv/mptt.conf ./root_i386-debug/kernel/drv/amd64/mptt ./root_i386-debug/kernel/drv/mptt ./root_sparc-debug/usr/lib/mdb/kvm/mptt.so ./root_sparc-debug/usr/lib/mdb/kvm/sparcv9/mptt.so ./root_sparc-debug/kernel/drv/mptt.conf ./root_sparc-debug/kernel/drv/sparcv9/mptt ./root_sparc-nondebug/kernel/drv/mptt.conf ./root_sparc-nondebug/kernel/drv/sparcv9/mptt ./root_sparc-nondebug/usr/lib/mdb/kvm/mptt.so ./root_sparc-nondebug/usr/lib/mdb/kvm/sparcv9/mptt.so ./root_i386-nondebug/kernel/drv/amd64/mptt ./root_i386-nondebug/kernel/drv/mptt ./root_i386-nondebug/kernel/drv/mptt.conf ./root_i386-nondebug/usr/lib/mdb/kvm/amd64/mptt.so ./root_i386-nondebug/usr/lib/mdb/kvm/mptt.so
Pick the set of files for your system type and whether or not you want DEBUG drivers. That is, for x64 non-DEBUG, you want the files in the root_i386-nondebug directory. Or, for SPARC DEBUG, you'd want the files in the root_sparc-debug directory.
The rest of the path indicates where the files need to be installed (you can just copy them with cp) relative to the root directory. The driver binaries and mdb library (mptt.so) should be "chown"ed to root:sys and "chmod"ed to 755 for the driver and 555 for the shared library. The mptt.conf file can also be "chown"ed similarly and "chmod"ed to 644.
- Modify driver bindings for a specific card type
If you wish to run mptt on the PCIe (375-3487) card, remove the existing binding (likely to the "mpt" driver):
update_drv -d -i '"pciex1000,58"' mpt
and add a binding for mptt
update_drv -a -i '"pciex1000,58"' mptt
NOTE: If another driver is currently running on the card, a reboot will be required.
- Another possibility with later builds is to add a path-based binding for the specific instance of the card you wish to use. The device path can be found either from 'dmesg' output, or from 'prtconf -v' output. For example, the PCI-X HBA (375-3488) has a device ID of 0x3160. The device path can be found by doing:
-bash-3.2# prtconf -v|egrep 'dev_path.*3160' dev_path=/pci@1d,0/pci1022,7450@1/pci1000,3160@1:mptt1
The above controller is already being managed by the mptt driver, and there may be multiple lines of output depending on your system configuration, but the device path should be consistent per-HBA.
You can add a binding for mptt to that specific instance by doing the following:
update_drv -a -i '"/pci@1d,0/pci1022,7450@1/pci1000,3160@1"' mptt
- Reboot the system if required.
The mptt driver will now be attached and ready to run.
Using mptt
The svcadm(1M) command is used, among other things, to disable or enable the stmf service. When the stmf service is disabled, the mptt driver is taken off-line by means of resetting the IOC and releasing many of the driver's resources. When the stmf service is enabled, the IOC is configured and communication is enabled.
Although these cards are single-function cards, there are actually two physical ports. Thus, the mptt driver will register two target ports with the COMSTAR framework; one for each connector.
For example, running "stmfadm list-target" will result in output similar to this:
-bash-3.2# stmfadm list-target Target: wwn.500605B0000D9324 Target: wwn.500605B0000D9320
Each target represents one physical connector on the HBA. This isn't strictly necessary, but does allow for ease of administration specifically with respect to zoning. Logical units can be assigned/zoned to one individual port if desired.
mptt.conf
The mptt.conf file currently contains three entries. These are:
ddi-forceattach
force-target-mode
force-initiator-mode
The ddi-forceattach entry is required and must not be modified.
The force-target-mode and force-initiator-mode entries are used to tell mptt how to program the HBA. These parameters allow the user to ensure that the HBA is capable of operating either as a target or an initiator. This is required because these HBAs, by default, act only as an initiator. There is no way to allow the HBA to operate in both modes simultaneously.
In order to allow the HBA to function as a target, the force-target-mode parameter must be set to "1". Similarly, to revert back to the original HBA behavior (i.e. "Initiator"), the force-initiator-mode value must be set to a "1".
If both of these parameters are set to 0, no changes will be made to the current operating parameters of the HBA. If both are set to "1", target mode will take precedence and the HBA will be programmed to act in target mode (if it isn't already doing so).
NOTE: The information below only pertains to versions of opensolaris prior to build 83. As of snv_83, this CR will allow the mpt driver to put an HBA back into initiator mode if it is currently in target mode.
If you wish to re-program an HBA that is currently being controlled by mptt in target mode back into initiator mode, you must perform the following steps:
1. svcadm disable stmf -- Offline the HBA port
- modunload -i -- Use modinfo to determine the module ID.
- Edit /kernel/drv/mptt.conf. Set force-target-mode to 0 and force-initiator-mode to 1.
- update_drv mptt -- This will reload mptt. The driver will re-program the HBA to operate as an initiator. After that occurs, mptt will no longer be able to use this HBA.
- rem_drv mptt -- This will uninstall the mptt driver (Optional)
- Use 'update_drv' to modify the driver bindings as required.
- Reboot the system to allow mpt to reclaim that HBA.