OpenSolaris

OSTEP - OpenSolaris Test Execution Platform

What is OSTEP?

**OSTEP** is the open source version of the Solaris Test Execution Project (STEP). 
OSTEP is a powerful automated test execution framework that allows users to select, 
configure and automatically run test suites on physical and virtual test machines. 

Currently OSTEP comprises the following:

**Command Line Interface** - Used to kick off tests.
**Framework Driver** - Used to run tests on test systems.
**Testsuites** - Selection of OpenSolaris functional tests.
**Results Webapp** - Used to display test results.

The goal of OSTEP is to make executing any of these testsuites as simple and as
painless as possible. It does this through standardisation of testsuite execution and
results reporting, the use of an intuitive command line interface, and the ability
to generate batch files that enable multiple test tasks to be run automatically.

Results from each test run will be uploaded to a central mysql database for easy
perusal at [[test.opensolaris.org/ostep>>http://test.opensolaris.org/ostep]].

OSTEP operates under the following [[terms and conditions>>attach:opensolarisosteptermsv1.pdf]].

How To Use OSTEP

**Testfarm/OSTEP Quick Guide**

**Note: You will have to have a valid opensolaris.org username.**

**1.** Reserve the number of systems you require for testing using the Testfarm 
   Reservation webtool - [[Here>>http://test.opensolaris.org/testfarm/]].

**2.** Register your username with the ostep server. Follow the link listed
   [[Here>>http://test.opensolaris.org/testfarm/]] and click on the "Servers" button

   Click "Add Account" at the "OSTEP Test Execution Server" column.

**3.** ssh onto your reserved OSTEP server system.

**4.** Run OSTEP

   Answer questions when you are prompted. SUT means 'System Under Test'. SUT1 would relate
   to the Client Test system you reserved via the Testfarm. SUT2 would relate to the Server.

   **Note: Please use the IP Address of Test System for SUT_HOSTNAME**

   Users may user 'step' or 'ostep' to run any of the following commands.

   **For a full list of options OSTEP can take, please type 'ostep'**
   [[image:ostep1.png||alt="ostep1.png"]]

   **To kick off a test, please type 'ostep start'**
   [[image:ostep2.png||alt="ostep2.png"]]

   **To monitor execution of your test, please type 'ostep log [stepid]'** 
   [[image:ostep3.png||alt="ostep3.png"]]

   **To view the ostep report for a testrun, type 'ostep summary [stepid]'**
   [[image:ostep4.png||alt="ostep4.png"]]

   **To List all previous test executions, please type 'ostep list'**
   [[image:ostep5.png||alt="ostep5.png"]]

OSTEP Tests

**Update (Apr 29th 2009)**
The smf testsuite is now available for Local Zones testing.

**Note (March 31st 2009)** 
Initially, in addition to the tools and infrastructure, we shall be making 
three open-sourced test-suites available. We plan to add 11 further test-suites
over the next few weeks as the required systems are added to the testfarm.
Expect email notification when this is complete.

There are currently 14 Solaris Testsuites available in OSTEP format. This list
will grow as more suites are ported to OpenSolaris. 

** Table listing testsuites verified to run on system types**
|**Testsuite**|**Traditional**|**Local Zone**|**Ldom**|**# Systems**
|findleaks|Yes|No||1
|smf|Yes|Yes||1
|nfsv4|Yes|No||2
|sharemnt|Yes|No||2
|sharemgr|Yes|No||1
|fnfs|Yes|No||2
|iSCSI|Yes|No||1
|zfstest|Yes|No||1
|lofi|Yes|Yes||1
|sdts|Yes|No||1
|megacli|Yes|No||1
|sbd|Needs console access|No||1
|kmf|Yes|Yes||1
|nc|Yes|Yes||1

The tests are listed by OpenSolaris category below. Number of test systems 
required per test is listed in '()'.

**Core OS - Core operating system, system calls etc**

    **findleaks** - Uses the mdb findleaks macro to look for memory leaks.        **(1)** 
    **smf**       - Tests functionality of the smf services in Opensolaris.       **(1)** 

**nfs - Network File System**

    **nfsv4**     - Tests NFSv4 server and client.                                **(2)**
    **sharemnt**  - Tests share_nfs(1M) and mount_nfs(1M).                        **(2)**
    **sharemgr**  - Tests the sharemgr file system share utility.                 **(1)**
    **fnfs**      - Tests the new automounter in both kernel and the daemon.      **(2)** 

**fs - File System**

    **iSCSI**     - Tests the functionality of the Solaris iscsi target software. **(1)**
    **zfstest**   - Tests the various different aspects of zfs functionality.     **(1)**
    **lofi**      - Tests compression functionality added to lofi.                **(1)**

**dev_io - Device IO Testing**

    **sdts**      - Tests areas of the storage HBA drive.                         **(1)**

**NWS - Network Storage**

    **megacli**   - mega_sas driver regression testing.                           **(1)**

**Security - Security Testing - including kerberos and TX**

    **sbd**       - Functional tests for Secure By Default.                       **(1)**
    **kmf**       - Tests the Key Management Framework within Solaris.            **(1)**  

**Network Testing**

    **nc**        - Tests netcat (nc) functionality in Opensolaris.               **(1)**

Viewing Results

You can view all your OSTEP test results [[Here>>http://test.opensolaris.org/ostep/index.jsp]]

Future

Phase 2 of OSTEP is to make the ostep tools and tests available for download
from opensolaris.org (in IPS format). We hope to have this complete by the end
of May 2009.