1.0 Overview of the Crypto Framework
[ Note that these are extracts from the original white paper, there are
sections omitted because they are no longer relevant but the original
numbering has been left alone ]
The (Open)Solaris Cryptographic Framework provides cryptographic services to users and applications through commands, a user-level programming interface, a kernel programming interface, and user-level and kernel-level frameworks. The Solaris Cryptographic Framework provides these cryptographic services to applications and kernel modules in a manner seamless to the end user, and brings direct cryptographic services, like encryption and decryption for files, to the end user.
The user-level framework is responsible for providing cryptographic services to consumer applications and the end user commands. The kernel-level framework provides cryptographic services to kernel modules and device drivers. Both frameworks give developers and users access to software optimized cryptographic algorithms.
The programming interfaces are a front end to each of the frameworks. A library or a kernel module that provides cryptographic services can be plugged into one of the frameworks by the system administrator, making the plugin's cryptographic services available to applications or kernel modules. This flexibility allows the system administrator to plugin different cryptographic algorithm implementations or hardware accelerated cryptographic providers.
2.0 Framework Architecture
This section provides an overview of the Solaris Cryptographic Framework's architecture. The following diagram illustrates this architecture and the interrelationships between its many components and the Solaris Operating System.
2.1 Cryptographic Applications
Components found in the application layer generally are not intended to provide generalized features to other applications, but rather perform a single class of function. An example application would be a Sendmail mail server whose role is to route the transfer of an email message towards its recipient. The mail server may make use of cryptography in order to protect the message as it traverses a hop to the next mail server. However the mail server does not provide protection for any other type of communication between the nodes.
In Solaris’s cryptographic architecture, applications will not directly contain cryptographic algorithms (e.g. DES). Instead they will just make use of features of the Cryptographic Security Services available from the Solaris Operating System. This reduces code redundancy and provides access to optimized algorithms for all applications.
2.2 Cryptographic Security Services
The Solaris Operating System contains a variety of security services that will make indirect use of cryptography. A high level library offers a generalized interface for authenticating entities communicating across the network and optionally message integrity and privacy. Similar to the application components, these security services will not contain cryptographic algorithms in them, but rather will leverage a common set of cryptography frameworks below.
2.3 Framework
The framework components provide an abstracted, consistent interface to the Cryptographic Security Services for use of cryptography. These interfaces intend to make it easier for operating system components to integrate strong security into their offerings. An example framework component is the User-Level Cryptographic Framework. This framework offers an RSA Security Inc. PKCS #11 Cryptographic Token Interface (Cryptoki) standards-based [1] programmatic interface to security services, and a pluggable interface where appropriately signed cryptographic providers can register and provide services via the framework.
Similar to the Cryptographic Security Service components, the framework components do not directly include cryptography, but do offer a generalized programmatic interface to other parts of the operating system.
2.4 Cryptography Providers
The components at this bottom layer offer the actual cryptographic algorithms. For example, this layer includes software implementations of 3DES, AES, RSA, and SHA-1 just to name a few in the Sun Software Crypto Plugin. These providers offer a PKCS #11 compliant user-level interface and PKCS #11 like kernel-level interface that is designed to plug into the Cryptographic Frameworks.
Since it is possible for 3rd parties to write new or replace existing algorithms used by the Solaris Cryptographic Framework, the framework validates that each plugin provider has an appropriate, embedded cryptographic signature. This signature limits who can write a provider that the framework will recognize and allow to operate and potentially limit the consumers who can make use of it.
3.2 User-Level Cryptographic Framework (uCF)
The first framework exists outside the kernel (at the user process level) and is intended for use by most consumers. This framework is referred to as the User-Level Cryptographic Framework, or uCF for short. The uCF offers a PKCS #11 v2.11 based API to callers wishing to have cryptographic operations performed. PKCS #11 was chosen for the User-Level Cryptographic Framework API because of the large volume of existing software that already supports this interface and the large number of developers that already understand this interface.
Below the API is mostly logic intending to glue the many cryptographic providers together and make it appear as single aggregate provider. The uCF exposes a list of cryptographic ?slots? (as they are known in PKCS #11) that enable the application to select the provider which best suits the application's particular needs.
At the bottom of the uCF is the pluggable cryptography provider interface. This interface is what cryptographic providers must implement in order to be usable with uCF. This provider interface also follows the PKCS #11 v2.11 interface, with the additional requirement that any provider that is configured to plug in must include a cryptographic signature. The signature is associated with an RSA key included in a certificate issued by Sun. This framework uses the Module Verification Daemon to verify that the provider is authorized for use, and to determine if there are usage restrictions on the provider. See section 3.3 for details on this daemon.
3.2.1.1 Convenience Functions
In addition to the standard PKCS #11 interfaces, Sun has provided two convenience functions that assist in the setup of PKCS #11 and with object creation. These functions do not exist in other implementations of PKCS #11, but rather are part of uCF.
CK_RV SUNW_C_GetMechSession(CK_MECHANISM_TYPE mech,
CK_SESSION_HANDLE_PTR hSession);
Calling SUNW_C_GetMechSession() initializes the framework and does all of the necessary PKCS #11 calls to create a session that is capable of providing operations on the requested mechanism. It is not necessary to call C_Initalize() or C_GetSlotList() before the first call to SUNW_C_GetMechSession().
If the function is called multiple times it will return a new session without re-initalizing the framework. If it is unable to return a new session, then CKR_SESSION_COUNT is returned.
C_CloseSession() should be used to release the session when it is no longer required.
CK_RV SUNW_C_KeyToObject(CK_SESSION_HANDLE hSession,
CK_MECHANISM_TYPE mech, const void *rawkey,
size_t rawkey_len, CK_OBJECT_PTR obj)
Calling SUNW_C_KeyToObject() creates a symmetric key object for the given mechanism from the rawkey data. The rawkey is a pointer to any buffer. The value of rawkey will be used to set the CKA_VALUE attribute in the resulting object. This function is useful if the programmer does not want to take all the steps necessary to setup a key template. The object should be destroyed with C_DestroyObject() when it is no longer required.
For more detailed information on the uCF API, please see the RSA PKCS #11 website [1].
3.2.2 Kernel-Level Cryptographic Framework (kCF)
The other framework associated with the Solaris Cryptographic Framework exists
within the kernel boundary and is known as the Kernel-level Cryptographic
Framework, or kCF. The kCF contains similar functionality to the uCF. The kCF
offers a consumer and provider interface that is more applicable for kernel
modules, such as cryptographic modules and device drivers from Sun and from
third parties. The kCF can be used for Sun and third party cryptographic modules
and device drivers. The kCF contains no cryptography itself, not even for
performing plugin provider verification. This is because verification of each
kernel-level provider is delegated to the Module Verification Daemon that
executes at the user level and uses the same verification code as is used by
uCF.
There are some differences between the frameworks. The kCF not only offers a
consumer and provider interface, but it also has two private device drivers,
/dev/crytpo and /dev/cryptoadm. uCF uses /dev/crypto (via a plugin provider) to
communicate with kCF. The module verification daemon and the cryptoadm(1M)
command use /dev/cryptoadm to communicate with kCF. Additionally, kCF provides
scheduling and load balancing for cryptographic operations for kernel consumer,
as well as offers an asynchronous mode for the consumer interface routines.
4.0 Kernel Programming Interface
The common data structures crypto_mechanism_t, crypto_data_t,
crypto_dual_data_t, crypto_key_t, crypto_context_t and
crypto_context_template_t are defined in the following man pages:
crypto~_data(9S)
crypto~_dual~_data (9S)
crypto~_mechanism(9S)
crypto~_mech~_info(9S)
crypto~_object~_attribute(9S)
crypto~_key(9S)
crypto~_call~_req(9S)
The routines that make up this programming interface are detailed in the
following man pages:
crypto~_bufcall(9F)
crypto~_bufcall~_alloc(9F)
crypto~_bufcall~_free(9F)
crypto~_cancel~_ctx(9F)
crypto~_cancel~_req(9F)
crypto~_create~_ctx~_template(9F)
crypto~_digest(9F)
crypto~_encrypt(9F)
crypto~_encrypt~_mac(9F)
crypto~_get~_all~_mech~_info(9F)
crypto~_get~_mech~_list(9F)
crypto~_key~_check(9F)
crypto~_mac(9F)
crypto~_mac~_decrypt(9F)
crypto~_mech2id(9F)
crypto~_notify~_events(9F)
crypto~_unbufcall(9F)
Draft text only copies of the man pages can be found here.
The programmer should be particularly aware of the following features emphasized
hereafter.
4.1.1 Call requests
Most of the functions of this programming interface take a pointer to a
crypto_call_req structure that specifies the conditions of an asynchronous call
and the expected behavior. A NULL crypto_call_req_t pointer indicates a
synchronous call.
The API/framework internally allocates memory for a request when it needs to
queue it. This allocation uses KM_SLEEP for synchronous calls and KM_NOSLEEP for
asynchronous calls. As a result, asynchronous calls can fail with a
CRYPTO_HOST_MEMORY error if memory is not available when the request is issued.
Some of the Cryptographic Service Provider Interface (CSPI) entry points, such
as digest_init(), allocate memory, and take a kmflag argument which can be
KM_SLEEP or KM_NOSLEEP. This kmflag is set to the appropriate value by the
framework when calling the provider's CSPI entry point.
4.1.2 Asynchronous calls
For asynchronous calls, a caller has to pass non-NULL crypto_call_req_t pointer.
When the caller passes in a non-NULL crypto_call_req_t pointer, there are two
cases:
- CRYPTO_ALWAYS_QUEUE cr_flag set:
The calling code does not want to spend any CPU cycles on costly cryptographic
operations, or cannot safely make any blocking call (blocking memory allocation
acquiring a mutex possibly held at a different PIL). This is typically the case
of a call coming from an interrupt context, or timeout handler.
The request associated to this call is always queued and the call returns
CRYPTO_QUEUED.
- CRYPTO_ALWAYS_QUEUE cr_flag clear:
The calling code can spend the CPU cycles on the cryptographic operations, and
can tolerate spinning on locks. However, the calling code doesn't want to be put
in cv_wait() internally.
A typical consumer of this case is a streams module running its service or write
put procedure. This is especially reasonable with message digest operations that
are fast enough in software and the overhead of extra queuing/interruptions and
context switching overcomes the potential performance gain from offloading these
operations to hardware accelerators.
The framework will return asynchronously only when it is necessary; that is,
when it needs to either queue the request or dispatch it to a hardware
provider.
When a request is submitted; that is, when the returned value from a crypto
function is CRYPTO_QUEUED, the cr_req is initialized to a handle for the
request. The consumer code will use that handle to cancel the request if it
needs to do so.
In all cases of a queued request, upon completion, the framework will call
(cr->cr_callback_f)(cr->cr_callback_arg, finish_status);
where finish_status contains an integer value corresponding to the execution
status of the request. Possible values are defined in <sys/crypto/common.h>,
see Appendix E.
Once a crypto context of a multipart crypto operation is initialized (an
xxx_init() completed), a consumer does not need to wait for the completion of
the previous part before submitting the next one. Several requests for the same
crypto_context can be pending.
The order of submission is guaranteed to be preserved for requests of the same
context. Thus, a notification of request completion implicitly notifies the
completion of ALL the preceding requests.
The consumer code can choose to disable the callback to be invoked after the
intermediate calls are complete with success, by leaving the cr_flag
CRYPTO_NOTIFY_OPDONE clear in the crypto_call_req. This flag is ignored for
final and single step operations, as well as for nonsuccessful termination of
the request.
The typical use of this feature is a consumer that initializes a crypto_context,
then submits the parts of the operation as it receives them asynchronously. The
consumer doesn't have a useful thing to do with the information of the
completion of a part of the operation. It is notified only after the final step
is done.
CRYPTO_SKIP_REQID is set in cr_flag by a client if it does not need a handle for
the request that is submitted. The framework will skip generating a request ID
and will skip the related routines.
Note that the callback routine for a request with the CRYPTO_SKIP_REQID set can
still be called by the framework, in case an error occurs during the
asynchronous processing of the request. A client using the CRYPTO_SKIP_REQID
flag must make sure the callback routine is valid, and take the necessary
measures to prevent getting modunloaded when it has any pending requests.
Notes:
- The callback routine can run in an interrupt context, and has to adhere to
the same restrictions as interrupt and timeout handlers (no sleeping, no door
upcalls, no blocking memory allocations, etc.) - The framework uses the information passed in the crypto_call_req_t structure
to internally build its own requests, but does not keep any reference to that
crypto_call_req_t structure. That structure does not need to persist until the
operation finishes and the call-back routine is invoked (it can be allocated on
the stack of the calling code). However, the cr_callback_f function, and the
cr_callback_arg have to stay valid until either the operation is complete and
the framework calls cr_callback_f, or until the request is canceled. - Although canceling a request from a multipart operation is likely to make the
whole operation useless, the consumer is responsible for explicitly canceling
all the other pending requests. The consumer is given the choice of canceling
all the requests submitted for one context (cancel_req() vs cancel_ctx()).
4.1.3 Synchronous calls
When the caller passes in a NULL crypto_call_req_t pointer, the function is
attempted and possibly blocks. This is typically the case of a consumer code
running in the context of a system call coming from a user process.
When a request needs to be queued or dispatched to a hardware provider, the
framework will internally schedule it asynchronously, and block until the
operation is complete, then wakeup and return, thus simulating a synchronous
behavior.
4.1.4 Flow control
When there is a temporary shortage of queue resources to handle a cryptographic
request, the synchronous calls will block until enough are released.
Asynchronous calls will fail with CRYPTO_BUSY.
The calling code can either tolerate this failure and adapt to it (e.g. dropping
incoming IP packets, retry after some timeout), or may register for a
notification when resources are made available.
Four routines are provided to this effect:
crypto_bc_t crypto_bufcall_alloc();
int crypto_bufcall(bc_handle, func, arg);
int crypto_unbufcall(crypto_bc_t bc_handle);
int crypto_bufcall_free(bc_handle)
The semantics of these routines are similar to the STREAMS bufcall(9F) routine,
with one difference: crypto_bufcall() takes a preallocated bufcall handle, so it
is guaranteed to succeed.
Note that the client should use bufcall(9F) when it gets a CRYPTO_HOST_MEMORY
error. It should use crypto_bufcall(), for the CRYPTO_BUSY case only.
4.1.5 Deadlock avoidance
Canceling asynchronous crypto requests and bufcalls follows the semantics of
timeout(9F) routine in Solaris. The code that cancels an asynchronous request
must not attempt to acquire a mutex lock that could be acquired by the callback
routine for that request. When the callback routine for a request already
started, the cancel call for that request is guaranteed to block until that
callback is finished.
4.2 Kernel Provider Interface
The CSPI (Cryptographic Service Provider Interface) is defined by routines and
constants exported by providers, routines imported by providers, and by data
structures. Routines exported by providers are those that perform cryptographic
operations.
The following two header files have the constants and data structure
definitions, as well as the prototype declarations of the CSPI routines. They
need to be included by kernel modules and device drivers that use this SPI. They
are attached in Appendix E:
<sys/crypto/common.h>
<sys/crypto/spi.h>
Providers import two routines that allow them to register and unregister with
the framework and two routines that notify the framework of changes in status.
These routines are defined in the following man pages:
crypto~_provider~_notification(9F)
crypto~_op~_notification(9F)
crypto~_register~_provider(9F)
crypto~_unregister~_provider(9F)
Providers export a set of function vectors for cryptographic operations,
session and cryptographic context management, and for object manipulation. The
function vectors are defined in the following man pages:
crypto~_cipher~_ops(9S)
crypto~_control~_ops(9S)
crypto~_ctx~_ops(9S)
crypto~_digest~_ops(9S)
crypto~_dual~_cipher~_mac~_ops(9S)
crypto~_dual~_ops(9S)
crypto~_key~_ops(9S)
crypto~_mac~_ops(9S)
crypto~_object~_ops(9S)
crypto~_ops(9S)
crypto~_provider~_management~_ops(9S)
crypto~_random~_number~_ops(9S)
crypto~_session~_ops(9S)
crypto~_sign~_ops(9S)
crypto~_verify~_ops(9S)
This CSPI introduces a new "crypto" module type for software cryptographic
modules and device drivers. Software modules in /kernel/crypto have a modlinkage
structure containing a single pointer to a modlcrypto structure. Device drivers
have a modlcrypto pointer as well as a modldrv pointer. Device drivers in
/kernel/crypto have a hard link to their entry in /kernel/drv. 'modlcrypto' is
defined in the modlcrypto(9S) man page.
The remainder of this section has some additional notes for programmers writing
to this SPI.
4.2.1 Locking Considerations
The following locking issues must be considered by providers when interacting
with the framework:
crypto_unregister_provider() can be called from user context only, for example
from the provider detach(9E) routine.
crypto_unregister_provider() may sleep while pending requests are being
processed by the Provider, and will cause any remaining sessions on the provider
unregistering to be closed. For these reasons, the provider must not hold any
mutex that it may need to acquire from any of the entry points it exports to the
framework.
The same restrictions apply to the crypto_provider_notification() entry point
exported by the framework.
Entry points exported by providers to the framework are never invoked from an
interrupt context.
The crypto_op_notification() can be called by a provider from its interrupt
context while it holds locks.
4.2.2 Multiple Registrations
Some cryptographic accelerators may consist of one device with multiple
cryptographic engines, for example one engine for asymmetric operations (RSA,
DSA, etc.) and another engine for symmetric ciphers (3DES, AES, etc.), or two
separate cryptographic engines on a single board. Device drivers for these
devices can register two providers for one device with the framework, which
allows separate scheduling queues to be created, and allows the separate engines
of a single device instance to be flow-controlled independently.
Providers that register more than once using the same crypto_provider_dev_t
appear as a single device to administrators and users.
Input and Output Data Buffers
Data buffers passed to cryptographic operations are specified by crypto_data_t
and crypto_dual_data_t structures.
Initialization information for block-chaining ciphers can be passed to providers
via the crypto_mechanism structure or the cd_miscdata field in the crypto_data
structure. If the cd_miscdata field is used for update operations, then it
should be set to NULL for all but the first update operation. When cd_miscdata
is used, the initialization information from the crypto_mechanism structure is
ignored.
When the result of a cryptographic operation is returned by a provider to the
framework, the provider entry point must return the length in bytes of the
output data. Depending on the data format, the provider uses the following
fields to return this value:
- crypto_data_t: cd_length.
- crypto_dual_data_t: dd_len1 if encryption or decryption is the first of the
- dual operation (e.g. encrypt_mac_atomic()), or dd_len2 if encryption or
- decryption is the second of the dual operations (e.g. mac_decrypt_atomic()).
4.2.4 Recoverable errors
We define the following error codes as recoverable. All of these error codes
require that the provider does not modify any of the input parameters.
The provider must return different error codes in cases where the input
parameters are modified. Further to this case, it must return CRYPTO_FAILED, if
no specific error code except one of the following is available:
- CRYPTO_BUFFER_TOO_BIG
Input data buffer too big for this mechanism of the provider. This error code
must not be returned by an update operation.
- CRYPTO_BUSY
This is an optional error code the provider may return if flow controls in
effect. The provider must notify the framework (or already must have notified)
with a crypto_provider_notification(CRYPTO_PROVIDER_BUSY) before returning this
error.
- CRYPTO_DEVICE_ERROR
The device has failed. This error code should be returned for all the pending
requests on the internal wait queue of the provider and any new requests sent to
the provider.
The provider should notify the framework with a
crypto_provider_notification(CRYPTO_PROVIDER_FAILED, etc.), so that new
requests will not be sent to the provider.
- CRYPTO_DEVICE_MEMORY
Operation failed due to lack of memory on the device. Note that this is
different from CRYPTO_HOST_MEMORY, which is a failure of kmem_alloc().
- CRYPTO_KEY_SIZE_RANGE
The key size is outside the range allowed or possible on this provider for the
mechanism. For example, a non-full strength version of a provider may return
this error for a key size outside its supported range.
- CRYPTO_NO_PERMISSION
The client has no permission to perform operation on the provider. The
permission check mechanism is private to the provider and the client. The
framework does not do any checks.
4.2.5 Multipart operations
The ctx_provider_private field in the context structure may be used by a
provider to store a pointer to provider-supplied storage. Such storage is
typically allocated by providers in initialization routines such as
digest_init(). If the ctx_provider_private field is NULL, then a provider is
free to use the field. If, however, the field is non-NULL, then pre-initialized
storage is being passed to a provider.
Note: A provider supporting init/update/final can not fail with
CRYPTO_BUFFER_TOO_BIG during an update. If it can not support this, it should
have only the atomic entry point.