OpenSolaris
Collectives
Discussions
Documentation
Download
Source Browser
Free CD
Log-in
|
Community Group fm
:
Fault Management
Top Menu
Show
:
Comments
Attachments
History
Information
Print
:
Print
Print preview
Export as PDF
Export as RTF
Export as HTML
Export as XAR
Wiki code for
Fault Management
Hide Line numbers
1: =Fault Management Community 2: 3: {{toc start=2 depth=3 numbered=false scope=page /}} 4: 5: ==About Predictive Self-Healing 6: 7: Self-healing functionality for users and administrators of a modern operating system provides fine-grained fault isolation and restart where possible of any component — hardware or software — that experiences a problem. To do so, the system must include intelligent, automated, proactive diagnoses of errors that are observed on the system. The diagnosis system is used to trigger targeted automated responses or guided human intervention that mitigates a specific problem or at least prevents it from getting worse. Finally, these new system capabilities are connected to a new model for system administrators oriented around simpler, higher-level abstractions. Sun’s first Predictive Self-Healing features are part of the OpenSolaris OS and the [[Solaris 10 OS>>http://www.sun.com/software/solaris/10/]] and include the Fault Manager and the [[Service Manager>>Community Group smf.WebHome]]. 8: 9: ==About Fault Management 10: 11: The Solaris Fault Management effort (originally code-named **FMA** inside of Sun) provides a new architecture for building resilient error handlers, error telemetry, automated diagnosis software, response agents, and a consistent model of system failures for a management stack. Many parts of Solaris are already participating in FMA, including the CPU and Memory error handling for UltraSPARC III and IV, the UltraSPARC PCI HBAs, and more. And a variety of projects are underway, including full support for CPU, Memory, and I/O faults on Opteron, conversion of key device drivers, and integration with various management stacks. 12: 13: The legacy UNIX failure model was simply to leave error handling up to each subsystem author, and simply provide the ability to emit an error message for a human to the system log in a non-standard format. When a subsystem is converted to participate in Fault Management, error handling is made resilient so that the system can continue to operate despite some underlying failure, and telemetry events are produced that drive automated diagnosis and response. The Fault Management tools and architecture enable development of self-healing content for software //and// hardware failures, for both microscopic and macroscopic system resources, all with a unified, simple view for administrators and system management software. 14: 15: Some objectives for the Fault Management Community are to: 16: 17: * Convert hardware and software subsystems to participate in Fault Management 18: * Connect Solaris Fault Management to system management protocols and software 19: * Evolve and enrich the tools and common architecture for Fault Management 20: * Research underlying failure modes and design and develop automated diagnosis software that is able to build effective self-healing for particular resources 21: 22: ==Information Resources 23: 24: ===Discussion Forum 25: 26: Participate in Fault Management discussions. 27: 28: **fm-discuss** General discussion of the tools, technology, infrastructure, and content to permit Solaris and Sun products to recover from, diagnose, isolate, and explain faults 29: [[Jive forum interface>>http://www.opensolaris.org/jive/forum.jspa?forumID=49]] or [[Mailman interface>>http://mail.opensolaris.org/mailman/listinfo/fm-discuss]] 30: To subscribe, see "Subscribing to edu-discuss" on the [[mailman page>>http://mail.opensolaris.org/mailman/listinfo/fm-discuss]] or send an empty email to fm-discuss dash subscribe at opensolaris dot org 31: 32: ===Documentation 33: 34: * **FMA Events and Messages** 35: ** Diagnosis results obtained from the Fault Management software in Solaris contain links to the [[Knowledge Article Web>>http://www.sun.com/msg/]]. 36: ** The FMA [[Event Registry>>Project events-registry.WebHome]] is the central repository for all fault management events passed between error handlers, the fault manager and its agents. 37: * **Fault Management for System Administrators.** This [[Fault Management>>http://wikis.sun.com/display/OpenSolarisInfo/Fault+Management]] piece is pointed to from [[Working with the Fault Manager>>http://wikis.sun.com/display/OpenSolarisInfo/Working+with+the+Fault+Manager]] in the Troubleshooting section of the //[[OpenSolaris System Administration Guide>>http://wikis.sun.com/display/OpenSolarisInfo/OpenSolaris+System+Administration+Guide+Table+of+Contents]]//. 38: * **Man Pages** 39: ** [[fmd(1M)>>http://docs.sun.com/app/docs/doc/819-2240/fmd-1m]] 40: ** [[fmadm(1M)>>http://docs.sun.com/app/docs/doc/819-2240/fmadm-1m]] 41: In OpenSolaris 2008.11, the fmadm repair command was replaced by the following new commands: fmadm repaired (synonymous with fmadm repair), fmadm replaced, and fmadm acquit. 42: ** [[fmdump(1M)>>http://docs.sun.com/app/docs/doc/819-2240/fmdump-1m]] 43: In OpenSolaris 2008.11, the preferred method to display fault information and determine the FRUs involved is the fmadm faulty command, not the fmdump command. 44: ** [[fmstat(1M)>>http://docs.sun.com/app/docs/doc/819-2240/fmstat-1m]] 45: * **Writing Device Drivers for FMA.** The //[[Writing Device Drivers>>http://docs.sun.com/app/docs/doc/819-3196]]// guide contains a section "Sun Fault Management Architecture I/O Fault Services" in "Chapter 13, Hardening Solaris Drivers." This section describes the steps and techniques used to write an FMA-aware driver. 46: * **Fault Management Daemon Programmer’s Reference Manual.** The [[FMD PRM>>attach:Community Group fm.WebHome@FMDPRM.pdf]] [PDF] is a description of the internal architecture of the Sun Fault Management Daemon, fmd(1M), and the programming interfaces exported by the daemon. 47: ** FMDPRM 1.4 April 2008. Added -b option to the fmtopo command. Changed descriptions of TOPO_WALK_CHILD and TOPO_WALK_SIBLING. 48: ** FMDPRM 1.3 March 2008. Added repaircode to the table of Fault Management Configuration Properties. 49: ** FMDPRM 1.2 August 2007. Initial post of this document. 50: * **Eversholt Fault Tree Description Language.** The //[[Eversholt Fault Tree Description Language>>attach:Community Group fm.WebHome@EVERSHOLT.pdf]]// [PDF] explains how to use the eversholt language to describe fault trees in the Sun Fault Management Architecture. 51: ** //Eversholt Fault Tree Description Language,// 1.8, November 2008. Initial post of this document. 52: 53: === What’s New 54: 55: The OpenSolaris universe of [[new fault management>>Community Group fm.Features]] activities (projects, ARC cases, diagnosis engines, recovery agents and error handlers).
Search
Collectives
Community Group
Academic and Research
Accessibility
Advocacy
Appliances
Approachability
Architecture Process and Tools
BrandZ
Chinese Users
Community Advisory Board
Databases
Desktop
Device Drivers
Distribution
Documentation
DTrace
Emerging Platforms
Fault Management
Games on OpenSolaris
HA Clusters
HPC Developer
Installation and Packaging
Internationalization and Localization
Laptop
Logical Domains
Modular Debugger (MDB)
Networking
NFS
Observability
OpenSolaris Governing Board (OGB)
OpenSolaris Printing
OS/Net (ON)
Performance
Power Management
PowerPC
Security
Service Management Facility (smf(5))
Software Porters
Solaris Volume Manager
Storage
Systems Administration Community Group
Testing
Tools Home
Unix File Systems (UFS)
Website Community
X Window System
Xen
ZFS
Zones
Project
ADSL Modem Enhancement
ARC Process Definition
ARM Platform Port
Automatic Data Migration
BIND Update
Bluetooth Stack & Drivers
Brocade FC HBA - Initiator
Brocade FC HBA - Target
Brussels - unified network link configuration
Caiman, Solaris Install Revisited
Celeste
Český portál
Chime Visualization Tool for DTrace
CIFS client for Solaris
CIFS Server
Clearview: Network Interface Coherence
Cluster Agent: Informix Dynamic Server
Cluster Agent: OpenSolaris Container
Cluster Agent: OpenSolaris xVM
Cluster Agent: Oracle E-Business Suite
Cluster agent: PostgreSQL
Cluster Agent: Samba
Cluster Agent: Tomcat
CMT
Coarse Data Flow Parallelism
Colorado: Open HA Cluster on OpenSolaris
Command Assistant
Common Array Manager
Companion - /opt/sfw: Free and Open Source software
COMSTAR: Common Multiprotocol SCSI Target
Content
Contest
CPU Observability
Credentials Process Groups
Crossbow: Network Virtualization and Resource Control
Crypto KMS Agent Toolkit
Cryptographic Framework
Data Migration Manager
Data Tethers
Deutsches Portal
Device Detection Tool
Device Driver Utility
Device Manager
Device Mapper
Direct Rendering Infrastructure & 3D drivers
DTrace Guide
Duckwater: Simplified name services management
Easy Tools
Emancipation
Emulex Fibre Channel Device Driver
Emulex Advanced Ethernet Device Driver
Enable/Enhance Solaris support for Intel Platform
Enhance the support of USB webcams
Enhanced SMF Profiles
Enhancements for AMD-based Platforms
Erlang DTrace Integration
Ethernet bridge module for Solaris
Evaluate Conary
Events Registry
Ext3 file system support
F/OSS Package Base
Facilitation
Fibre Channel over Ethernet
Fine Grained Access Policy (FGAP)
Fingerprint Authentication
Flexible Mandatory Access Control
Forensic Tools
Fully Open X Project
Fuse on Solaris
gcore
Generic Machine Check Architecture Improvements
Google SOC
HA-JBoss
HA-MySQL
Hadoop Live CD
Hitachi
HoneyComb Fixed Content Storage
HPC Stack
Image Packaging System
Improved Performance MIB
Indiana
Innovation Awards
Input Method
Intel Graphics
Internet Key Exchange, version 2
Interrupt Resource Management
IP Datapath Refactoring
IP over Infiniband
IPsec Tunnel Reform
iSCSI Extensions for Remote DMA (iSER)
iSNS Server
JeOS - Just enough Operating System
JKstat - a java binding for libkstat
Journaled File System (JFS)
K Desktop Environment
Kerberos
Kernel Sockets
Kernel SSL Enhancements
Key Management Framework
Korn Shell 93 integration/migration project
Labeled IPsec
LatencyTOP
Layer 2 Filtering
LDoms Manager
Lending
libMicro - portable microbenchmarks
Link Layer Discovery
Live Media: Technologies for distributions running from CD and other media
Locale Data
lofi compression and cryptography support
lx64 brand
Media Management System
Mega_sas
Mexico
MilaX minimal Live Distribution
MIPS Platform Port
Mozilla DTrace
MRSL.NONsharedDevice
Multi-lingual Glossary
Multi-pathing software (MPxIO)
Multiple disk sector size support
Multiple DOI
Muskoka: An open repository for OpenSolaris technical content
Navigator
Nemo: A Framework for High-Performance Networking
Network Auto-Magic
Network Data Management Protocol
Network MIBs
Network Storage
Network Time Protocol (NTP)
Nevada Globalization
New Design of 4over6 Mechanism Based on OpenSolaris
NFS RDMA transport update and performance analysis
NFS Server in non-Global Zones
NFS version 4.1 pNFS
NFSv4 namespace extensions
Nightingale: Port Songbird to OpenSolaris
NPort ID Virtualization (NPIV)
NUMA
Object Storage Device (OSD) support for Solaris
OHACGE Script Based Plug-in
ON/Nevada (ONNV) Project
Open Development Infrastructure
Open HA Cluster Utilities
Open Sound System
OpenGrok
OpenPegasus CIM Server
OpenRTI
OpenSolaris Busybox
OpenSolaris Desktop
OpenSolaris Hispano
OpenSolaris Security Audit
OpenSolaris support for the QEMU processor emulator: host and guest
PEF: Packet Event Framework
Performance Wrappers
Pkgfactory
Polski Portal
Portail Francophone
Portal Brasil
Portals
Power Management Usability Interfaces
Presto: Automatic Printing Configuration
Printable Many Page Solaris Manuals
Promise SuperTrak RAID HBA Driver
QLogic Converged Network Adapter GLDv3 NIC Driver
Quagga Routing Protocol Suite Integration
RAID Configuration Utility
RBridge (IETF TRILL) support
RDMA Offload Framework
Reno: Login Process Enhancements for Interop
Resource Management
s10brand
SAM/QFS
SCM Migration Project
SCSI RDMA Protocol
SDcard Drivers
Sensor Abstraction Layer
Session Initiation Protocol
SFW
Shell: bourne shell, korn shell, C shell, etc.
Sierra: Intel WiFi Chipsets Support
Simple Panels
SM-HBA Based SAS HBA Management
SMF Documentation
Solaris iSCSI Target
Solaris PowerPC Port
SourceJuicer
Sparks: name service switch/nscd enhancements
Squashfs
Star integration/migration project
Starfish
Starter Kit
Storage Power Management
Sun Security Toolkit
Sun StorageTek Availability Suite
Support for OpenFabrics User Verbs / API on OpenSolaris OS
Support gcc4/GCCfss in Solaris
Suspend/Resume
SVR4 Packaging
Systemz
Tamarack: Removable Media Enhancements in Solaris
Tesla: OpenSolaris Enhanced Power Management
Test Development
Tickless Kernel Architecture
TIPC
Trademarks
Trusted networking interface policy database for Trusted Extensions
Trusted Platform Module support
Use Case
Validated Execution Project
Virtual Console
Virtual Network Machines
Visual Panels
Visualization for HPC
Volo
VRRP: Virtual Router Redundancy Protocol Implementation
VSCAN service
Web Stack
Website
Winchester: Schema mapping and ID mapping for AD Interoperability
Wireless USB Support
Wireless Wide Area Network
X Consolidation
x86 Generic FMA Topology Enumerator
Xen Gate
Xfce: A lightweight desktop environment
ZFS Boot and Install
ZFS on disk encryption support
Zone Manager
Zone Statistics
Русский портал
البوابة العربية
भारतीय पोर्टल
中国门户
日本ポータル
한국 포탈
User Group
Adelaide
Argentina
Arizona
Atlanta
Baltimore-Washington
Bangalore
Bangkok
Bangladesh
Beijing
Bélem
Berlin
Bhimavaram
Bloomington
Campus Ambassadors
Capital Region
Cardiff
Charlotte
Chengdu
Chennai
Chihuahua
Chile
Cleveland
Colombia
Columbus
Connecticut
Cracow
Czech
Dallas/Ft. Worth
Danish
Delaware
Edinburgh
Egypt
Finland
Florida
Front Range
FuZhou
Great Lakes
Greece
Hangzhou
Hawaii
HeFei
Houston
Hyderabad
Indonesia
Irish
Israel
Italian
Jinan
Kabul
Kansas City
Latvia
London
Madurai
Manchester
Mato Grosso
Melbourne
Minas Gerais
Minnesota
Montreal
Moscow
Mumbai
Munich
NEA
Netherlands
New England
New York City
New Zealand
NIT Hamirpur
Noroeste
Oklahoma City
Osnabrück
Peru
Philadelphia
Piaski
Pittsburgh
Porto Alegre
Puget Sound
Pune
Queensland
Research Triangle Park
Romania
Russia
San Antonio
San Diego
San Francisco
São Paulo
Scottish
Serbia
Shanghai
Shenzhen
Silicon Valley
Singapore
Slovak
South African
Southern Connecticut
St. Louis
Sweden
Switzerland
Sydney
Szczecin
Taiwan
Tecum
Thames Valley
Tokyo
Toronto
Trondheim
Tulsa
Turkey
Ukraine
University of Melbourne
Vale do Paraíba
Vancouver
Venezuela
Welsh - Cymru
Wisconsin
Xi'an
Subsites
Code Reviews
Code Repositories
Package Search
Bugster
Bugzilla
Test Machines
Planet
Mailing Lists
Elections & Polls
ARC Case Logs
Source Juicer
Package Factory
User Authentication
Community Group fm Pages
Fault Management
FMA Aware Device Drivers
New Fault Management Features
Solaris FMA Demo Kit
FM Blogs
FM News
SNMP Connector