Improved Latency Discovery

Improved Latency Discovery

Description

This project intends to improve latency discovery on Opteron by using two static tables provided in ACPI 3.0 for determining the NUMA configuration of a machine.  The names and purpose of each table are the following:

• The System Resource Affinity Table (SRAT) specifies which CPUs and memory are in each node in a NUMA machine.

• The System Locality Information Table (SLIT) specifies the relative distance between nodes.

The SRAT and SLIT were not used in the initial implementation of the MPO platform support for Opteron because the SRAT was Microsoft encumbered and both the SRAT and SLIT were not part of the ACPI standard.  Currently, the MPO platform support for Opteron uses the DRAM address map registers in PCI configuration space to determine what physical addresses belong to which node and probes from each node to all other nodes during boot to determine the distance between nodes.

While these methods are sufficient, they are not ideal since the DRAM address map registers may not be supported on future Opteron processors and probing is inherently error prone.

Issues

• Not all machines (BIOS's) support the SRAT and SLIT (:-(

• Need to at least read SRAT to know how to split up physical page free lists used by the VM system before VM system is up during kernel boot, but need to map the ACPI tables (which are referred to and referred to each other by physical addresses) into kernel virtual memory to read them somehow.

Documents

• ACPI 3.0 specification

• BIOS and Kernel Developer's Guide for AMD Athlon 64(TM) and AMD Opteron Processors(TM)    specifies DRAM address map register format, location, etc..

Tools

• Intel ACPI Source Language (iASL) compiler can dump the ACPI tables into files on a given machine (using its -g option) and is very useful for discovering whether a given machine supports the SRAT and SLIT.

Status

• 11-29-06 Mostly on hold due to lack of resources....Need to file RFE for support needed to read SRAT and SLIT early during boot, but have filed RFEs to get support for SLIT on at least Sun x86/amd64 platforms