Wiki code for Design Document

Hide Line numbers
1: = **Device Detection Tool Design**
2: 
3: : |
4: Date
5: |
6: Revision
7: |
8: Author
9: |
10: Comment
11: |
12: Mar. 26, 2008
13: |
14: 1.0
15: |
16: Ye.Li@Sun.Com
17: |
18: Design Doc of DDTool 2.0
19: |
20: Feb. 26, 2009
21: |
22: 1.1
23: |
24: Ye.Li@Sun.Com
25: |
26: Design Doc of DDTool 2.2
27: 
28: == 1. Overview
29: 
30: Device Detection Tool can tell you in just a couple of minutes whether the Solaris OS supports the devices that are detected in your x86/x64 system.
31: 
32: It produces a table that shows whether a Solaris driver is available for each device the tool detects. The table tells you whether the driver is built in to the Solaris OS or available in [[OpenSolaris.org>>http://www.opensolaris.org/]] or whether a third-party driver is available.
33: 
34: For its functionality and usage, refer to its [[official web site>>http://www.sun.com/bigadmin/hcl/hcts/device_detect.jsp]].
35: 
36: In a certain aspect, Device Detection Tool is a ’predict’ tool. It could predict whether those devices in you Windows, Linux, Mac or FreeBSD system work when a Solaris OS is installed on the machine. How does it do that? It is actually simple:
37: [1] The tool collects device data of PCI devices in users’ systems.
38: [2] The tool compares the detected device data with a Solaris driver database which is maintained by Sun to check the Solaris driver availability status for each device.
39: [3] The tool generates a Solaris driver availability report to show users.
40: 
41: == 2. Architecture
42: 
43: Following is the architecture diagram of Device Detection Tool.
44: 
45: [[image:sddt22archdiagram.jpg||alt="sddt22archdiagram.jpg"]]
46: 
47: From the architecture diagram you can see that there are three parts of the tool: (1) Driver Database; (2) PCI device data probe program for specific system; (3) and the function unit. In the later chapters, the parts are introduced one by one.
48: 
49: == 3. Driver Database
50: 
51: The driver database of Device Detection Tool are special format text files – configuration file, pci.ids and a driver.db specified to each target OS.
52: 
53: === 3.1 configuration file
54: 
55: DDTool reads arguments from a single configuration file, and this file will be put on a server. DDTool gets the configuration file remotely while launching. And then DDTool reads arguments from it. Those arguments are URL of pci.ids and driver.db, and
56: 
57: The configuration file is a XML file, and its format is as follows:
58: <?xml version="1.0" encoding="UTF-8"?>
59: <Configuation>
60:   <timeout></timeout>
61:   <conntimeout></conntimeout>
62:   <readtimeout></readtimeout>
63:   <pciids>
64:     <hostip></hostip>
65:     <dir></dir>
66:     <filename></filename>
67:   </pciids>
68:   <driverdb>
69:     <db>
70:         <number></number>
71:         <hostip></hostip>
72:         <dir></dir>
73:         <filename></filename>       
74:     </db>
75:     <db>
76:         .... ....
77:     </db>
78:   </driverdb>
79: </Configuation>
80: 
81: === 3.2 pci.ids
82: 
83: pci.ids is a public repository of all known ID’s used in PCI devices, and we can get full device names and vendor names instead of the numeric device id and vendor id by using it. It is maintained by developers outside Sun, and can be distributed under the 3-clause BSD License. We can adopt it as a part of DDTool Database without any license problem.
84: 
85: The following are examples of records in pci.ids:
86: 
87: ~--------------------------------------------------------------------------------
88: 8086  Intel Corporation
89:     ... ...
90:     0008  Extended Express System Support Controller
91:     0039  21145 Fast Ethernet
92:     ... ...
93: ~--------------------------------------------------------------------------------
94: In this case, two devices are listed. Their vendor id is identical ~-- 8086 which indicates that they are manufactured by Intel Corporation, and their device id are 0008 and 0039 respectively which indicate their name are ’Extended Express System Support Controller’ and ’21145 Fast Ethernet’.
95: 
96: === 3.3 driver.db
97: 
98: ==== 3.3.1 overview of driver.db
99: 
100: driver.db is created and maintained by our team. We can use it to get driver availability status of devices. At present, DDTool provides driver.db for the latest Solaris 10 and Solaris Express Developer Edition OS.
101: 
102: 1. At the beginning of driver.db, there is a line of Solaris Version Information used to indicate which Solaris version is supported by the current DDTool Database. This line is like following:
103:     ~------------------------------------------------------------------------------------------
104:     Solaris_Versions        Solaris 10 8/07   Solaris 10 8/07   S10
105:     ~------------------------------------------------------------------------------------------
106:     It means the current DDTool Database provide driver support status for Solaris 10 8/07, the short name of this Solaris release is Solaris 10 8/07, and the S10 is the sign to indicate this release.
107: 1. The driver.db consists kernel driver database, Xorg driver database, and drivername-classcode mapping table.
108: 
109: ==== 3.3.2 Kernel driver db
110: 
111: (1) every line in driver.db is a record which corresponds to a single device;
112: (2) each line consists of 5 elements: pci_string, driver_name, driver_type, 64-bit_capability, and driver_location;
113: 
114: * pci_string is a string consists of vendor id, device id and so on, and its the keyword used to find a device’s driver status
115: * driver_name is the name of driver corresponding to the device identified by pci_string
116: * driver_type should be one letter. It has totally two values: S and T.    
117: ** S means this device has a Solaris driver (which is provided by Solaris OS). 
118: ** T means this device has a third-party driver (which is provided by third-party vendor).
119: 
120: * 64-bit_capability should be one letter as well. It has totally three values: Y, N and Z.
121: ** Y means the driver is available on both of 32-bit and 64-bit Solaris OS. 
122: ** N means the driver is only available on 32-bit Solaris OS.  
123: ** Z means the driver is only available on 64-bit Solaris OS.
124: 
125: * driver_location
126: ** If driver_type is ’S’, it is a Solaris bundled driver. And driver_location is the specific version of Solaris OS.
127: ** If driver_type is ’T’, it is a third-party driver. And driver_location is the download address of this driver.
128: ** If driver_type is ’O’, it is a driver published on OpenSolaris community. And driver_location is the download address of this driver.
129: 
130: The following are examples of records in updated driver.db:
131: 
132: //[example 1]:
133: ~--------------------------------------------------------------------------------
134: # pci_string           driver_name    driver_type    64-bit_capability    driver_location
135: pci14e4,1644          bge                 S                   Y                       Solaris 10 8/07
136: ~--------------------------------------------------------------------------------//
137: This device ( vendor id = 14e4, device id = 1644 ) has a Solaris driver which is supported by 32-bit and 64-bit Solaris 10 8/07.
138: //[example 2]:
139: ~--------------------------------------------------------------------------------
140: # pci_string           driver_name    driver_type    64-bit_capability    driver_location
141: pci1274,5000          audio1371           T               Y                       http://www.tools.de/solaris/audio/beta/
142: ~--------------------------------------------------------------------------------//
143: This device ( vendor id = 1274, device id = 5000 ) has a third-party driver which can be downloaded from http://www.tools.de/solaris/audio/beta/, and is supported by 32-bit and 64-bit Solaris 10 8/07.
144: 
145: ==== 3.3.3 Xorg driver db
146: 
147: Device Detection Tool 2.0+ detects both the kernel video driver and the Xorg video driver for each video device. Not all video devices have both of the two types of video drivers. Some video devices have only the kernel driver. For example, the ATI RV280[Radeon 9200 PRO] video device has only a kernel video driver named vgatext.
148: 
149: Xorg driver db is a mapping table between pciids and Xorg driver name. Every line in Xorg Driver DB is a record which corresponds to a single device. The syntax is as follows:
150: 
151: ~----------------------------------------------------------
152: 
153: pic<vendor_id>,<device_id> Xorg_drivername
154: ~----------------------------------------------------------
155: 
156: ==== 3.3.4 drivername-classcode mapping table
157: 
158: The table is used to check whether a matched driver really supports devices in a specific type. It is created manually by our team members, and at the end of the driver.db file. The syntax is as follows:
159: 
160: ~------------------------------------
161: driver_name class_code
162: ~------------------------------------
163: The two columns are divided by a Tab. If there are more than one class code for the driver, all of them will be listed in the class_code column, and divided by “;”.
164: 
165: == 4. Collect PCI device data
166: 
167: === 4.1 Parsing PCI device data from output message of system commands
168: 
169: Device Detection Tool 1.0 gets the device information by running specific system commands respectively on Solaris, Linux and Windows OS as follows, and parses the output message of system commands to extract device information such as vendor id, device id, etc..
170: 
171: |
172: **OS**
173: |
174: **System Command**
175: |
176: **Utility**
177: |
178: Solaris OS
179: |
180: prtconf -pv
181: |
182: prtconf
183: |
184: Windows OS
185: |
186: reg query hklm\system\currentcontrolset\enum\pci /s
187: |
188: register.exe
189: |
190: Linux OS
191: |
192: lspci -vv -n
193: |
194: PCIUtilities
195: 
196: For this reason, Device Detection Tool 1.0 depends on the availability and robustness of above system commands, and their output message format.
197: 
198: === 4.2 Probe users’s system to get PCI device data directly
199: 
200: Device Detection Tool 2.2 enhances the device data collecting method. It imports following interfaces to collect device information:
201: 
202: |
203: **OS**
204: |
205: **Interface **
206: |
207: **Solution  **
208: |
209: Solaris
210: |
211: libdevinfo
212: |
213: Looking up PCI nodes in prom tree and reading device data from the nodes
214: |
215: Windows
216: |
217: Win32 API
218: |
219: Reading PCI device data from the registry through Win32 API
220: |
221: Linux
222: |
223: /proc/bus/pci
224: |
225: Scanning the files in /proc/bus/pci and reading PCI device data
226: |
227: Mac OS X
228: |
229: IOPCIDevice
230: |
231: Collecting PCI Device Information
232: |
233: FreeBSD
234: |
235: /dev/pci 
236: PCIOCREAD
237: |
238: Reading PCI device data by scanning /dev/pci
239: 
240: Device Detection Tool is a Java application, it builds a JNI binding for each of the interfaces to collect device information. The enhancement frees it from dependence of system commands.
241: 
242: The **interface between native probe programs and DDTool** is very simple. It is also output message porting to DDTool by the natvie programs. The message has a unique format on all platform as follows:
243: //**vendor_id(4b),device_id(4b),class_code(8b),sub-system_vendor_id(4b),sub-system_id(4b),revision_id(2b)**//
244: 
245: The ids are separated by a ’,’ without space. One line ending with a line break (“\n”) is corresponding to a single device.
246: 
247: Following is an example:
248: //8086,29a0,00060000,1028,01dc,02
249: 1002,7187,00030000,1028,0402,00
250: 1002,71a7,00038000,1028,0403,00
251: 8086,104c,00020000,1028,01dc,02
252: 8086,284b,00040300,1028,01dc,02
253: 8086,283f,00060400,ffff,ffff,02
254: 8086,2836,000c0320,1028,01dc,02
255: 104c,8023,000c0010,1028,01dc,00
256: 8086,2825,00010185,1028,01dc,02
257: ... ...//
258: 
259: === 4.3 PCI device data files importing function
260: 
261: Device Detection Tool 2.0+ also allows device information to be feed into it from plain text files that contain the output message of system commands listed in the first table. The received device information is scanned and parsed by Device Detection Tool 2.0+ to extract device data (as what Device Detection Tool 1.0 did). This function enables the users to check the Solaris driver availability status for remote machines or those without Internet access.
262: 
263: == 5. Retrieve Device Data
264: 
265: === 5.1 Overview of Retrieving
266: 
267: After the PCI device configuration data is collected, DDTool will retrieve the DDTool Database with these keywords(the values of device id, vendor id, class code, subsystem vendor id and subsystem id).
268: 
269: Retrieving a single device will look up pci.ids and driver.db sequentially ( please refer the table below):
270: 
271: |
272: Retrieved Text File
273: |
274: Keywords
275: |
276: Retrieval Result
277: |
278: Algorithm of Retrieval
279: |
280: comments
281: |
282: pci.ids
283: |
284: values of device id and vendor id
285: |
286: device name and vendor name
287: |
288: DDTool simply searches pci.ids (a text file) for the keywords  line by line.
289: |
290: Technically, any PCI device has a corresponding entry in pci.ids.
291: |
292: driver.db
293: |
294: values of device id vendor id, class code, subsystem vendor id, and subsystem id
295: |
296: driver support status for 32-bit and 64-bit  Solaris OS
297: |
298: DDTool simply searches driver.db (a text file) for the keywords  line by line.
299: |
300: While DDTool finds a entry in driver.db corresponding to the current device in processing, it means that this device has a driver. Otherwise, it mean the device has no driver supported by Solaris OS
301: 
302: DDTool puts those retrieval result into report data. After every device is processed, the report data is created completely.
303: 
304: === 5.2 Driver Searching Policy
305: 
306: ==== 5.2.1 Device Hiding
307: 
308: {{{
309: Bridges, SMBus, Memory controller, secondary video controllers are filtered out from the final report. Because (a) they don’t have impact on the installation of Solaris OS; and (b) Solaris OS generally don’t have a driver for them. Thus all of these devices are ignored. DDTool won’t search driver status for them, and they won’t be in the final report.
310: 
311: Therefore, DDTool ignores following devices and won’t display them in the final report: those devices whose class code is begin with one of "000600", "000601", "000602", "000603", "000604", "000680", "0005", "000c05", "0008", or "000380".
312: }}}
313: 
314: ==== 5.2.2 Searching Kernel Driver DB
315: 
316: {{{
317: When a device is detected, DDTool is aware of its class_code, vendor_id, device_id, subsystem_vendor_id, subsystem_id, and revision_id. With the 6 values, 7 keywords can be created for this device as follows:
318: 
319: (KW1): pci<vendor_id>,<device_id>.<subsystem_vendor_id>.<subsystem_id>.<revision_id>
320: (KW2): pci<vendor_id>,<device_id>.<subsystem_vendor_id>.<subsystem_id>
321: (KW3): pci<subsystem_vendor_id>,<subsystem_id>
322: (KW4): pci<vendor_id>,<device_id>.<revision_id>
323: (KW5): pci<vendor_id>,<device_id>
324: (KW6): pciclass<class_code> (6 bytes class code)
325: (KW7): pciclass<class_code> (4 bytes class code)
326: 
327: The 7 keywords are sort by degressive priority. The priority of KW1 is the highest.
328: 
329: DDTool searches Kernel Driver DB line by line (each line in Kernel Driver DB is a separated record)with the 7 keywords.
330: 
331: (step 1) First of all, DDTool go through Solaris bundled records in PCI Kernel Driver DB complying with following rules:
332: 
333: (a) It compares the key-value of each record with keywords in decreasing priority: KW1 -> KW2 -> ... -> KW7.
334: 
335: (b) If a record is matched, DDTool goes to check whether the class code of the device is listed corresponding record of the founded driver name in drivername- classcode mapping table. If so, the record is matched record. Otherwise, the record is ignored. 
336: 
337: (c) If any record matches the highest priority keyword (KW1), the record is returned.
338: 
339: (d) If any record matches one of non-highest priority keywords (KW2, KW3 ... KW7 ), this record is saved in buffer. And DDTool continues to search for other records. 
340: 
341: (e) In the searching process, if any record matches a keyword with another non-highest priority keyword which has a higher priority than the current buffered one, the new record is saved in buffer to replace the old one. 
342: 
343: (f) The searching process won’t stop until (c) or all driver records in the current category are compared.  In the latter situation, if DDTool only finds some record(s) matched with non-highest priority keyword(s), the buffered record is returned. If DDTool doesn’t find any record, null is returned.
344: 
345: (step 2) If a record is returned from searching Solaris bundled driver, and this driver is supported by both of 32-bit and 64-bit6 Solaris OS, this record is the result.
346:     Otherwise, if there is only 32-bit Solaris bundled driver available. This record is saved in buffer, and go through third-party records continuely following rule (a), (c) ~ (f) to search a 64-bit driver. 
347: 
348: (step 3) If a record is returned from searching third-party driver, and this driver is supported by both of 32-bit and 64-bit Solaris OS, or it is supported by 64-bit Solaris OS only, the 32-bit Solaris bundled driver and 64-bit thrid-party driver are displayed in the final report. 
349:     Otherwise, the tool continuely go through  OpenSolaris records following rule (a), (c) ~ (f). 
350: 
351: (step 4) If a record is returned from searching OpenSolaris driver, and this driver is supported by both of 32-bit and 64-bit Solaris OS, or it is supported by 64-bit Solaris OS only, the 32-bit Solaris bundled driver and 64-bit OpenSolaris driver are displayed in the final report. 
352:     Otherwise, there is only the 32-bit Solaris bundled driver displayed in the report.
353: 
354: Note: Since the classcode-drivername mapping table is only available for Solaris Bunlded driver now, the step of (b) is not useful for searching third-party of OpenSolaris driver.
355: }}}
356: 
357: ==== 5.2.3 Searching Xorg Driver DB
358: 
359: {{{
360: DDTool searches the corresponding Xorg driver status for a video device in the Xorg Driver DB.
361: 
362: (Step 1)
363:     (1.1) If a device’s kernel driver name is ’nvidia’, its Xorg driver name is ’nvidia’. In this case, DDTool returns and appends ’nvidia’ to its kernel driver name.
364:     (1.2) If a device’s kernel driver is a third party one, its Xorg driver name is the same as the third party kernel driver name. In this case, DDTool returns and appends the same name to its kernel driver name.
365:     (1.3) If a device has a Solaris kernel driver which is not ’nvidia’, DDTool begins to searche each record in Xorg Driver DB with a single keyword which is "pci<vendor_id>,<device_id>" (KW5).
366: 
367: (Step 2) DDTool reads a line from Xorg Driver DB, split it into 2 strings, and extracts the first string as the key-value of this record.
368: 
369: (Step 3) In any record matches KW5, the record is the result record, and DDTool returns and appends the Xorg driver name contained in this record to this device’s kernel driver name.
370: 
371: (Step 4) After DDTool searches all records in Xorg Driver DB, if there is no any record matching KW5 of this device, that means this video device has no Xorg driver. DDTool just returns without any other action.
372: }}}
373: 
374: == 6. Generate Report
375: 
376: The report data will be sorted and formatted again to create the final report which could be displayed by UI.
377: The field of ’device type’ in final report is filled while creating final report. For each device, DDTool judges its type with its class code. Currently, DDTool divides all PCI devices into 7 types: ’Storage’, ’Network’, ’Video’, ’Wireless’, ’USB’, ’Firewire’ and ’Other’, and build a map between class code and device type simply in the program.
378: 
379: == 7. Reference
380: 
381: [[Sun Device Detection Tool web site>>http://www.sun.com/bigadmin/hcl/hcts/device_detect.jsp]]
382: 
383: [[Device Detection Tool Open Project Page>>Project ddtool.WebHome]]
384: 
385: [[Chinese Blog for Sun Device Detection Tool>>http://developers.sun.com.cn/blog/sddtool/]]
386: 
387: [[Developer’s Blog>>http://blogs.sun.com/moonocean/]]
OpenSolaris

Top Menu

Wiki code for Design Document

Search

Collectives

Community Group

Project

User Group

Subsites

Project ddtool Pages
