Kernel Initialization
Status - 10/13/07
We are executing userland code now. Still a ways to go to get a functional libc and rtld but it is close at hand. libc builds but there is a bug in the ld with relocations that prevent a successful link but it's being looked at now. The same status applies to rtld. The only supported platform at the moment is the ODW workstation.
Status - 06/15/07
More detailed content will be posted after the source release. We are well past startup() and into main. Currently we are at the point of vfs_mountroot(). We have the 2 drivers we need for the ODW platform, the Rhine driver and the console driver. The other targets will quickly run into a problem with a lack of appropriate drivers. In vfs_mountroot() until trying to send/receive the first ethernet packets through the vfe (Via Fast Ethernet) driver.
Status - 10/02/06
Much progress has been made towards a functional KRTLD but as of the code drop static linking is still the only option for unix/genunix on the ODW target. We have run into limitations on OF. The 1:1 VM mapping has been modified to now allow a kernel location @ 0xE000.0000 but exceptions result when we go into the 1st OF call from the kernel. We are working with the Genesi folks on this. The kernel initialization flow through the file structure remains the same.
Research Notes from 7/17/06
We have 3 ways of booting the kernel, with OF directly, inetboot or with the Metrowerks tool. Either way you have to at least pass the CIF pointer correctly to have a chance to get past the 1st few instructions. KRTLD is not operational so we have been statically linking the kernel as a post-step to the make process and dealing with it that way. Look at your uts/chrp/conf/Mapfile to see where you are locating it (ie: 0x800000 was what I see it today but it will change). The code here builds but true functionality and actual debugging for much of this is pending. It is characterized at generally 3 levels - functional, coded but not tested, old code that just builds and is here mostly as a placeholder or reference awaiting the real development to start. The approach has been to engage code in the specific functions when we need to. The kernel init, exception table install and general level 0 exception handlers, and some timer calls have been directly worked on, other areas like interrupts, sys calls, level 1, 2 handlers still have the old 2.6 framework and need attention going forward. References to TRAPTRACE have been removed. KADB has been replaced by KMDB so much of the boot related KADB code has been removed, however some related items may be laying around.
--
Kernel Initialization
We begin here leveraging the PPC/2.6 release work as templates to start with the boot of the kernel. Much of the architecture specific code appears to be useful to look at, the platform specific code is less so. We have transitioned to the GNU assembler and revised the assembly source syntax accordingly. One of the main issues appears to be the fact that the PPC port was little endian. The main portion of the code that puts the PPC into little endian mode (it defaults on power up to big endian mode) was implemented in VOF (Virtual Open Firmware). The kernel inits with this assumption.
usr/src/uts/prep/ml/locore.s
_start()
- register loads for THREAD_REG, bopp, cif_handler
- init thread 0 stack
- clear bss (temporary for standalone since KRTLD really should do this but we don't have it yet)
- check for CPU type
- save PVR reg
- now to mlsetup lots of cpu/thread/proc/boot flags
usr/src/uts/prep/os/mlsetup.c
mlsetup()
- init thread 0
- cpu_list_init() inits CPU lists for the 1st CPU see -->> usr/src/uts/common/os/cpu.c
- prom_init() setup CIF see -->> usr/src/psm/promif/ieee1275/prep/prom_init.c
- setcputype() void function on PPC this is done in locore.s -->> function is in /usr/src/uts/prep/io/autoconf.c
- fiximp_obp() set the magic constants of the implementation ie: get dcache, icache, cache attributes, clock & timebase freq, (embedded obp calls)
- init_cpu_info() not implemented - mostly for MP, uni sets clock freq value
- map_wellknown_devices() not implemented
- copy_handlers() set up exception table; calls back to locore for this
now back to locore...
- check_OF_revision bypassed for now - checks the correct VOF version... but there is no VOF
- ppcmmu_param_init() inits kernel variables --> /usr/src/uts/ppc/vm/mach_ppcmmu.c
- va_to_pa() simple --> /usr/src/uts/ppc/vm/mach_ppcmmu.c
- main() we're here --> /usr/src/uts/common/os/main.c
leave locore not to return
/usr/src/uts/common/os/main.c
main() (kernel's main) / Borrowed from Shudongs x86 list, still need to sync to PPC 2.6 /
- * startup() See Guy's startup.c review below
- * segkmem_gc()
- * callb_init()
- * callout_init()
- * cbe_init()
- * clock_init()
- * call all init_tbl[]
- * vm_init()
- * physio_bufs_init()
- * spl0() allow interrupts
- * vfs_mountroot() can happen earlier?
- * errorq_init()
- * cpu_kstat_init()
- * post_startup()
- * strplumb()
- * init_mstate()
- * consconfig() XXX move to earlier...
- * forceattach_drivers() XXX still needed?
- * process init
- * exacct_init
- * sysevent_evc_thrinit
- * start_other_cpus() do it earlier?
- * lgrp_root_init()
- * kmem_mp_init()
- * vmem_update()
- * run everything in mp_init_tbl[]
- * create init process
- * create pageout
- * create fsflush
- * cluster hooks
- * ...
- * sched()
switch root in init?
STARTUP.C NOTES - Guy Shaw
Solaris/PPC 2.5.1 usr/src/uts/prep/os/startup.c was derived from usr/src/uts/i86pc/os/startup.c. Although there are many lines of differences, the basic structure is the same, and there are no big surprises. For example, things having to do with 4MByte large pages and with MTTRs just do not apply to PowerPC, and things having to do with serial number and some firmware builtin language support and PROM CRCs were added to prep/os/startup.c, all of which is pretty obvious and has no
counterpart in i86pc.
Where things really get interesting is when we look at the difference between 2.5.1 and 2.11. startup.c has been reorganized, radically.
For starters, 2.5.1 startup() was a sequence of straightline code, going on for page after page. I have attached a summary of the sequence with my own step numbers, which I use to track the new location of the same functionality in the 2.11 startup() scheme. 2.11 has a top down structure; it is broken down into
a few component functions. The body of the startup() function itself is so short and simple that it can be included here, in its entirety, without cluttering up this message with unwanted detail. Here it is:
{{{
void
startup(void)
{
extern void startup_bios_disk();
/// Make sure that nobody tries to use sekpm until we have initialized it properly. ///
#if defined(__amd64)
kpm_desired = kpm_enable;
#endif
kpm_enable = 0;
}}}
{{{
progressbar_init();
startup_init();
startup_memlist();
startup_modules();
startup_bios_disk();
startup_bop_gone();
startup_vm();
startup_end();
progressbar_start();
}
}}}
{{{
}}}
Now, all we have to do is descend down the 9 component functions.
There must be at least vaguely corresponding functionality, but
presented in a different order in the source code and possibly
some things being executed in a different order. But this
reorganization is a welcome change, even if it does mean a bit more
work to find the right places to put PowerPC startup code
where it belongs in the new order.
Each of the 9 startup() functions is, in turn, decomposed
into functions that do a logical chunk of work. Each
is instrumented with tracing at the function boundaries.
I am sure this helps to know where you are when bringing up
a new system and things die a horrible death.
The progress bar is new. I think we can skip it, at least at first.
The whole file i86pc/os/graphics.c is new. Boot didn't used to do
anything in graphics mode, if I recall correctly.
Here is a sort of top-level parts explosion of the remaining 7
Solaris 2.11 startup() support functions, along with a list of
step numbers from the list of 2.5.1.steps.
startup_init
--
startup_init()
cpuid_pass2() [[ new, 02 ]]
check_boot_version(BOP_GETVERSION()) [[ 01 ]]
Set prom_debug [[ new ]]
get_system_configuration() [[ new ]]
Note: In 2.11, all code related to identifying CPU type, etc
is kept together in i86pc/os/cpuid.c. Something similar can
be done for PPC.
Note: In 2.11, per cpu information is kept in a table of
cpuid_info structures. This is cleaner. It is worth while
to take a look at the big commont at the top of cpuid.c.
I would expect the situation for PowerPC to be much much MUCH
simpler, because OpenFirmware would provide this kind of
information in a standard way, instead of the 25 year
history of ad hoc tests needed for x86. Still, keeping
it separate is a good idea.
get_system_configuration() looks up all sorts of properties
such as "nodes" and "cpus_pernode". On x86 boxes, these
must come from the simulated PROM, which is just a file
with these settings. I'm sure it is better on PowerPC,
because it has real Open Frimware with real PROM, right?
startup_memlist
--
startup_memlist() 17
Note: vmems have superceded all sorts of ad hoc resource
allocators. It is cleaner. We want to adapt to the modern
Solaris way.
Note: page coloring is new. I don't know if we have any reason
to do page coloring on PPC.
Note: no kstat_init() in 2.11. Not needed?
startup_modules
--
startup_modules() 37, 42
Note: modload of sysinit is new.
startup_bios_disk
--
startup_bios_disk() is in separate file, i86pc/os/biosdisk.c new
startup_bop_gone
--
startup_bop_gone() 29
Note: call to hat_kern_alloc() is new.
startup_vm
--
startup_vm() 45, 46, 49, 50, 51
startup_end
--
startup_end() 52, 54, 55
Note: no reference to kmem_gc().
Note: kcpc_hw_init() is new.
Solaris 2.5.1 i86pc/os/startup.c
01 tstile_init()
02 setcputype()
03 Set valloc_base
04 Set pmeminstall
05 Get the list of physically available memory
06 Set total_memory
07 Set pagehash_sz
08 mt_lock_init()
09 Set memspace_sz
10 Set npages
11 Set pp_sz
12 Allocate memspace
13 Set page_hash
14 Set kernelmap
15 mapinit()
16 kphysm_init()
17 kmem_init()
18 kstat_init()
19 Display SunOS Release banner
20 Display Copyright
21 ? Display DEBUG enabled
22 ? Display TRACE enabled
23 ? Display GPROF enabled
24 microfind()
25 Read /etc/system
26 Read /etc/rtc_config
27 param_calc()
28 mod_setup()
29 setup_mttr()
30 param_init()
31 Allocate space for pagetables
32 hat_init()
33 seg_init()
34 modload specfs
35 modload swapgeneric
36 dispinit()
37 setup_ddi()
38 get_font_ptrs()
39 establish_console()
40 ddi_load_driver("kd")
41 loadrootmodules()
42 psm_modload()
43 BOP_QUIESCE_IO()
44 copy_memlist()
45 hat_kern_setup()
46 kvm_init()
47 lomem_init()
48 setup_kvpm()
49 Possibly shrink availrmem
50 Set segkp
51 Set segkmap
52 kern_setup1()
53 kmem_gc()
54 configure()
55 init_intr_threads(CPU)
56 Free low pages
locore.s is the one we have started to massage. The Metaware assembler has different assembly syntax then GCC. We are viewing code examples from DINK32, a debugger written to support the PowerPC arch from Freescale/Motorola to outline the mnemonic differences.
The original [[2.6 memory map>>http://www.blastwave.org/dclarke/opensolaris/power_pc_memory_map.html]]