OpenSolaris

Flag Day and Heads Up: Power Aware Dispatcher and Deep C-States

Date: Wed, 25 Feb 2009 21:31:24 -0800
From: Eric Saxe <Eric.Saxe at sun dot com>
To: on-all at sun dot com, onnv-gate at onnv dot eng dot sun dot com
Subject: Flag Day and Heads Up: Power Aware Dispatcher and Deep C-States

Greetings,

Our putback of:
PSARC 2008/777 cpupm keyword mode extensions
PSARC 2008/663 cpu deep idle keyword
6567156 bring CPU power awareness to the dispatcher
6700904 deeper C-State support required on follow-ons to Intel Penryn
processor generation microarchitecture
6805661 cmt_root may contain duplicates on UMA systems

...introduces into OpenSolaris the Power Aware Dispatcher project, which
includes support for Intel's Deep Power Down (a.k.a Deep C-states) power
management feature beginning with the Core i7 (Nehalem) processor
microarchitecture.

This integration also represents a flag-day for Install(1) users, as the
kernel, the pmconfig(1M) and powertop(1M) commands are all touched, and
should be upgraded (or downgraded) together. Failure to heed this flag
day may result in an assert being tripped (in the case of a mismatched
pmconfig(1M)). BFU will do do the right thing.

The Power Aware Dispatcher project extends the kernel's CMT scheduling
infrastructure with awareness of CPU power management domains (groups of
logical CPUs that may be power managed), which it uses to implement
thread placement strategies that deliver superior performance, while
facilitating (and driving) power management of underutilized CPU resources.

In the "tickless" spirit, PAD also introduces an event-based CPU power
management implementation, where CPU power state changes are driven by
utilization "events" delivered by the dispatcher that are triggered by
changes in power domain utilization.

Event based CPUPM is enabled by default, however polling based CPUPM may
still be used by explicitly specifying it via the optional cpupm keyword
"mode" argument in power.conf(4).

   cpupm enable poll-mode

Please note that support for Deep C-states is initially available only
on Intel's Nehalem based processors, which implement a Time Stamp
Counter (TSC) register (used by OpenSolaris as the high resolution time
source) that remains invariant across deep C-state transitions.

Some Nehalem based systems may use BIOSes that do not enable Deep
C-states by default. On such systems, you may need to ensure that the
ACPI C2 and C3 states are enabled before OpenSolaris can make use of
them. These states may also appear in BIOS as Intel defined C-states C3
through C7. In some cases, a BIOS upgrade may be required to utilize
Deep C-states.

Deep C-States can be disabled in BIOS, and may also be disabled by
including the following line in /etc/system:
   set idle_cpu_no_deep_c = 1;

They may also be enabled/disabled via the cpu_deep_idle power.conf(4)
keyword:
   cpu_deep_idle    disable
or
   cpu_deep_idle    enable

Bugs may be reported under bugster via solaris/kernel/sched, and may
also be reported to tesla-dev at opensolaris dot org

On behalf of the PAD team:
   Bill Holler <bill.holler at sun dot com>
   Mark Haywood <mark.haywood at sun dot com>
   Anup Pemmaiah <napanda.pemmaiah at sun dot com>
   Aubrey Li <aubrey.li at intel dot com>
   Pat Bredenberg <patrick.bredenberg at sun dot com>
   Darrin Johnson <darrin.johnson at sun dot com>
   ...and myself,

Thanks and enjoy.
-Eric