Gast
2009-06-03, 10:19:24
Kann das mal jemand noch mal genauer erläutern, wie das mit dem 32 Bit Kernel und 64 Bit Userland funktioniert?
Fullquote:
One of the recurring bits of misinformation that I see floating about message forums is how OS X has supposedly better “64bit support” with comments stating that OS X Leopard is “more 64bit” than Vista. I find this assertion amusing because it is, contrary to Apple marketing, completely wrong.
First, let me state that, their Windows software notwithstanding, Apple makes terrific products and OS X is an excellent operating system. However, the question at hand is whether OS X Leopard is a 64bit OS, and the answer to that is an unequivocal no.
First, to make myself clear, I define the OS as the core kernel, drivers, system services, shell and primary UI libraries. An OS can support application code of different word lengths, via subsystems, without actually being natively said word length. For example, DOS, via the 4GW runtime environment, could run 32bit protected mode code, however this did not make DOS itself a 32bit OS (though some would claim that DOS4GW provided so many functions that it was itself an OS).
Some operating systems, like Windows 9x, are legitimately hybrid systems, as its core kernel was derived from a v86 VMM (virtual machine monitor that, yes, you could almost call a hypervisor). The Windows V86 VMM was a true 32bit preemptive multitasking, virtual memory, Ring 0 kernel that managed 1 “system VM”, where your 16bit windows apps and 16bit Windows OS code lived, and other DOS v86 vm’s. In fact, even as far back as Windows 2.11 /386, the system VM (where your 16bit windows apps ran using cooperative multitasking) was preemptively multitasked alongside all DOS applications running in their separate dos “box” v86 virtual machines.
The VMM was extended in Windows95 (not by much) to support preemptive thread scheduling and memory management for 32bit protected-mode Windows processes. This meant that Windows 9x was not a pure 32bit system, since much functionality was derived from the 16bit components in the system VM and, even in a few occurrences, DOS int21h and BIOS int13h 16bit real mode functions were invoked. Because of its 32 bit kernel, however, it was also not a 16bit OS in the strict sense and is correctly described as a hybrid (or, as I prefer, a big fat kludge).
Now what about 64bit Windows NT based operating systems, such as Vista and Server 2008 x64? Are they a similar kludge as Win9x, given they run both 32bit and 64bit code? The short answer is no. The long answer requires that we delve into how X86-64, or as it’s more commonly called, x64, works.
The x64 CPU supports 3 modes of operation; Real Mode – the legacy x86 segmented memory model used by DOS, Protected Mode – the 32bit linear address space mode with hardware memory management introduced with the 80386 (also includes 16bit 286 protected mode and v86 mode), and Long Mode – the 64bit linear address space mode, also with hardware memory management, introduced by AMD with the Hammer architecture based CPU’s.
Long Mode is interesting because, when active, it actually encapsulates 32bit Protected Mode and 64bit Native Long Mode. When a 64bit OS switches the CPU to Long Mode, the first stop is an intermediate “compatibility sub-mode”. This sub-mode is essentially identical to “legacy” 32bit Protected Mode, but without the virtual-8086 (v86) sub-mode support used to run DOS/BIOS 16bit Real Mode code in a Protected Mode, 32bit OS. It is a further step to actually switch the CPU to full 64bit Long Mode, but this step is actually a critical part of AMD’s well thought out compatibility strategy. By allowing a nearly unmodified 32bit code-base to run in a default 32-bit “sub-mode”, AMD solved many problems for OS developers and, as we shall see in a few paragraphs, actually made it possible for Apple, and others, to get to a 64bit world via an elegant shortcut.
But first, how does Windows support x64? 64bit (x64) variants of XP, Server 2003, Server 2008 and Vista run in full 64bit Long Mode, meaning that the system boots all the way to full Long Mode, supports and uses pointers 64bits in length and subsequently supports 64bit virtual addressing along with 64bit datatypes (for example, in Windows data model, LLP64, longlong is natively a 64bit data type). Furthermore all data registers used are 64bits in length and 8 additional general purpose and XMM registers are available for use via the full Long Mode instruction set architecture. This is a point that needs repeating, with 64bit Windows, the OS, kernel, drivers, shell and all major libraries (Win32, COM, GDI, Direct X, .Net, etc.) are all true, native 64bit code all running in 64bit full Long Mode with access to the 8 extra GP and XMM registers and gobs of address space. Top to bottom, 64bit Windows (NT) is a 64bit OS.
So if, the OS is 64bit, how does it run 32bit applications? Well, first of all, x64 versions of Windows do not use an “emulation” environment, such as the NTVDM used in 32bit Windows NT based operating systems for running 16bit code. Instead, 64bit NT uses a translation layer called WOW64 which leverages a very cool feature of the x64 architecture that AMD had the foresight to add when developing x64, namely the ability, once the CPU is in Long Mode, to dynamically switch the CPU’s sub mode from either 32bit compatibility (e.g. protected) mode or full Long Mode based on the code segment (CS) value loaded in the CS register.
In 64bit Windows, this works as follows: when loading a 32bit process, 64bit DLL’s named NTDLL.DLL, WOW64.DLL and WOWWIN64.DLL are loaded into the address space and then, WOW64.DLL proceeds to load a 32bit version of ntdll.dll and calls its initialization routine which loads all the required DLL’s for the application, including 32bit system DLL’s that, if they make system calls, are modified to call into WOW64.DLL (or WOWWIN64.DLL), rather than the standard call path. For all 32bit code loaded in the address space, the memory manager sets the L&D bits of the Code Segment to its corresponding 32bit mode indicator (L0, D1). When (and anytime) the program begins executing, e.g. there is a context switch to a thread executing 32bit code in the programs process, that code segment value is loaded into the CS register, per the transfer of control, and the CPU switches on the fly to 32bit Compatibility Mode.
When a 32bit program needs to make a system call or, as is more often the case, a function in a system DLL needs to make a core OS function call, the modified DLL calls the WOW64 stub libraries, which, based on loading the CS value for WOW64, causes the CS register values used for CPU mode selection to be set to L&D values L1, D0. From this point on, until returning execution back to 32bit code, the processor is in 64bit long mode. Before passing (thunking) the system call, WOW64 also performs stack translation for 32bit values/arguments.
So, from the above description, you can now understand how a 64bit Long Mode OS, on x64, executes 32bit protected mode code and it should be fairly clear how x64 CPU’s make the transition between 32 and 64bit modes. I will add, that the same WOW64 subsystem is used on the other major 64bit platform supported by Windows, IA64 (Itanium) with an additional DLL that provides x86 to IA64 instruction translation, making the Itanium version of WOW64 an actual emulation environment, versus a translation (thunking) layer as it is in x64 version of Windows.
Okay, so how do other OS’s, such as OS X Leopard, run 64bit code on a 32bit kernel? The answer is the exact inverse of what is described above. Using OS X as an example, the “XNU” kernel is not 64bit code; it is a 32bit PAE enabled kernel and, as discussed a few paragraphs back, a 32bit system can bootstrap into Long Mode’s 32bit compatibility mode. Furthermore none of XNU/Darwin kernel mode drivers or components are 64bit as one can’t safely mix x86 and x64 code in a common address space due to differences in pointer lengths (32bit vs. 64bit), argument/variable management (stack vs. register) and register counts (8 vs. 16) except when it’s very tightly controlled, such as the WOW64 DLL’s mapped into a 32bit processes user-mode address space. Doing so in kernel mode would be a very dangerous and make kernel mode development extremely difficult to debug.
However, the OS does support 64bit processes and has certain libraries that are coded as 64bit native for supporting 64bit programs (processes). Just as Windows switches the CPU back to 64bit long mode via WOW64 when an application makes a system call, a system call from a 64bit long mode process in OS X will cause whatever library invokes the system call to follow a call path that results in some code specifying the CS for a 32bit code segment descriptor, thus setting the L&D bits of the Code Segment register to its corresponding 32bit mode indicator (L0,D1). Subsequently, a function, prior to passing arguments to the native kernel mode system call, will truncate/reformat any 64bit values that are being passed to 32bit x86 compatible values. Returning from the system call back to the 64bit process, causes CS register values to be set to L1,D0 and the processor is magically back in full 64bit Long Mode.
Now, if that sounded disparaging of OS X, then you have bought into the “more bits” is better “measuring” contest. Reality is that the benefits of a fully 64bit OS are dependent on a lot of factors. Apple using the x64 architecture in the manner they did is a valid way to support 64bit applications. It is interesting to note, that once upon a time, this was the direction NT was headed in with NT 5.0 (Windows 2000) on the Alpha architecture (not to mention this is how several other OS’s on other architectures made the move to 64bit support). The likely reason that Windows (NT) ended up becoming a fully 64bit platform all at once is because, with the end of Alpha, development of a 64bit system was focused entirely on Itanium which had no real 32bit mode. As a result, the x64 version of Windows is actually a port of the Itanium version and even carries over the 44bit address map from that platform. Had 64bit efforts started later, with more focus initially on x86, it is very likely that Windows would have travelled the more relaxed route to a 64bit world.
That said, eventually those OS vendors using a 64bit process on a 32bit kernel model afforded to them via x64’s clever compatibility, will have to port their core codebase to native 64bit since, in the near future, contraints in the amount of addressing allowed in 32bit compatibility mode will limit the total physically addressable system memory (even for 64bit processes) and the memory accessible to the kernel itself.
http://makfu.wordpress.com/2008/03/07/prepare-to-be-rocked/
Fullquote:
One of the recurring bits of misinformation that I see floating about message forums is how OS X has supposedly better “64bit support” with comments stating that OS X Leopard is “more 64bit” than Vista. I find this assertion amusing because it is, contrary to Apple marketing, completely wrong.
First, let me state that, their Windows software notwithstanding, Apple makes terrific products and OS X is an excellent operating system. However, the question at hand is whether OS X Leopard is a 64bit OS, and the answer to that is an unequivocal no.
First, to make myself clear, I define the OS as the core kernel, drivers, system services, shell and primary UI libraries. An OS can support application code of different word lengths, via subsystems, without actually being natively said word length. For example, DOS, via the 4GW runtime environment, could run 32bit protected mode code, however this did not make DOS itself a 32bit OS (though some would claim that DOS4GW provided so many functions that it was itself an OS).
Some operating systems, like Windows 9x, are legitimately hybrid systems, as its core kernel was derived from a v86 VMM (virtual machine monitor that, yes, you could almost call a hypervisor). The Windows V86 VMM was a true 32bit preemptive multitasking, virtual memory, Ring 0 kernel that managed 1 “system VM”, where your 16bit windows apps and 16bit Windows OS code lived, and other DOS v86 vm’s. In fact, even as far back as Windows 2.11 /386, the system VM (where your 16bit windows apps ran using cooperative multitasking) was preemptively multitasked alongside all DOS applications running in their separate dos “box” v86 virtual machines.
The VMM was extended in Windows95 (not by much) to support preemptive thread scheduling and memory management for 32bit protected-mode Windows processes. This meant that Windows 9x was not a pure 32bit system, since much functionality was derived from the 16bit components in the system VM and, even in a few occurrences, DOS int21h and BIOS int13h 16bit real mode functions were invoked. Because of its 32 bit kernel, however, it was also not a 16bit OS in the strict sense and is correctly described as a hybrid (or, as I prefer, a big fat kludge).
Now what about 64bit Windows NT based operating systems, such as Vista and Server 2008 x64? Are they a similar kludge as Win9x, given they run both 32bit and 64bit code? The short answer is no. The long answer requires that we delve into how X86-64, or as it’s more commonly called, x64, works.
The x64 CPU supports 3 modes of operation; Real Mode – the legacy x86 segmented memory model used by DOS, Protected Mode – the 32bit linear address space mode with hardware memory management introduced with the 80386 (also includes 16bit 286 protected mode and v86 mode), and Long Mode – the 64bit linear address space mode, also with hardware memory management, introduced by AMD with the Hammer architecture based CPU’s.
Long Mode is interesting because, when active, it actually encapsulates 32bit Protected Mode and 64bit Native Long Mode. When a 64bit OS switches the CPU to Long Mode, the first stop is an intermediate “compatibility sub-mode”. This sub-mode is essentially identical to “legacy” 32bit Protected Mode, but without the virtual-8086 (v86) sub-mode support used to run DOS/BIOS 16bit Real Mode code in a Protected Mode, 32bit OS. It is a further step to actually switch the CPU to full 64bit Long Mode, but this step is actually a critical part of AMD’s well thought out compatibility strategy. By allowing a nearly unmodified 32bit code-base to run in a default 32-bit “sub-mode”, AMD solved many problems for OS developers and, as we shall see in a few paragraphs, actually made it possible for Apple, and others, to get to a 64bit world via an elegant shortcut.
But first, how does Windows support x64? 64bit (x64) variants of XP, Server 2003, Server 2008 and Vista run in full 64bit Long Mode, meaning that the system boots all the way to full Long Mode, supports and uses pointers 64bits in length and subsequently supports 64bit virtual addressing along with 64bit datatypes (for example, in Windows data model, LLP64, longlong is natively a 64bit data type). Furthermore all data registers used are 64bits in length and 8 additional general purpose and XMM registers are available for use via the full Long Mode instruction set architecture. This is a point that needs repeating, with 64bit Windows, the OS, kernel, drivers, shell and all major libraries (Win32, COM, GDI, Direct X, .Net, etc.) are all true, native 64bit code all running in 64bit full Long Mode with access to the 8 extra GP and XMM registers and gobs of address space. Top to bottom, 64bit Windows (NT) is a 64bit OS.
So if, the OS is 64bit, how does it run 32bit applications? Well, first of all, x64 versions of Windows do not use an “emulation” environment, such as the NTVDM used in 32bit Windows NT based operating systems for running 16bit code. Instead, 64bit NT uses a translation layer called WOW64 which leverages a very cool feature of the x64 architecture that AMD had the foresight to add when developing x64, namely the ability, once the CPU is in Long Mode, to dynamically switch the CPU’s sub mode from either 32bit compatibility (e.g. protected) mode or full Long Mode based on the code segment (CS) value loaded in the CS register.
In 64bit Windows, this works as follows: when loading a 32bit process, 64bit DLL’s named NTDLL.DLL, WOW64.DLL and WOWWIN64.DLL are loaded into the address space and then, WOW64.DLL proceeds to load a 32bit version of ntdll.dll and calls its initialization routine which loads all the required DLL’s for the application, including 32bit system DLL’s that, if they make system calls, are modified to call into WOW64.DLL (or WOWWIN64.DLL), rather than the standard call path. For all 32bit code loaded in the address space, the memory manager sets the L&D bits of the Code Segment to its corresponding 32bit mode indicator (L0, D1). When (and anytime) the program begins executing, e.g. there is a context switch to a thread executing 32bit code in the programs process, that code segment value is loaded into the CS register, per the transfer of control, and the CPU switches on the fly to 32bit Compatibility Mode.
When a 32bit program needs to make a system call or, as is more often the case, a function in a system DLL needs to make a core OS function call, the modified DLL calls the WOW64 stub libraries, which, based on loading the CS value for WOW64, causes the CS register values used for CPU mode selection to be set to L&D values L1, D0. From this point on, until returning execution back to 32bit code, the processor is in 64bit long mode. Before passing (thunking) the system call, WOW64 also performs stack translation for 32bit values/arguments.
So, from the above description, you can now understand how a 64bit Long Mode OS, on x64, executes 32bit protected mode code and it should be fairly clear how x64 CPU’s make the transition between 32 and 64bit modes. I will add, that the same WOW64 subsystem is used on the other major 64bit platform supported by Windows, IA64 (Itanium) with an additional DLL that provides x86 to IA64 instruction translation, making the Itanium version of WOW64 an actual emulation environment, versus a translation (thunking) layer as it is in x64 version of Windows.
Okay, so how do other OS’s, such as OS X Leopard, run 64bit code on a 32bit kernel? The answer is the exact inverse of what is described above. Using OS X as an example, the “XNU” kernel is not 64bit code; it is a 32bit PAE enabled kernel and, as discussed a few paragraphs back, a 32bit system can bootstrap into Long Mode’s 32bit compatibility mode. Furthermore none of XNU/Darwin kernel mode drivers or components are 64bit as one can’t safely mix x86 and x64 code in a common address space due to differences in pointer lengths (32bit vs. 64bit), argument/variable management (stack vs. register) and register counts (8 vs. 16) except when it’s very tightly controlled, such as the WOW64 DLL’s mapped into a 32bit processes user-mode address space. Doing so in kernel mode would be a very dangerous and make kernel mode development extremely difficult to debug.
However, the OS does support 64bit processes and has certain libraries that are coded as 64bit native for supporting 64bit programs (processes). Just as Windows switches the CPU back to 64bit long mode via WOW64 when an application makes a system call, a system call from a 64bit long mode process in OS X will cause whatever library invokes the system call to follow a call path that results in some code specifying the CS for a 32bit code segment descriptor, thus setting the L&D bits of the Code Segment register to its corresponding 32bit mode indicator (L0,D1). Subsequently, a function, prior to passing arguments to the native kernel mode system call, will truncate/reformat any 64bit values that are being passed to 32bit x86 compatible values. Returning from the system call back to the 64bit process, causes CS register values to be set to L1,D0 and the processor is magically back in full 64bit Long Mode.
Now, if that sounded disparaging of OS X, then you have bought into the “more bits” is better “measuring” contest. Reality is that the benefits of a fully 64bit OS are dependent on a lot of factors. Apple using the x64 architecture in the manner they did is a valid way to support 64bit applications. It is interesting to note, that once upon a time, this was the direction NT was headed in with NT 5.0 (Windows 2000) on the Alpha architecture (not to mention this is how several other OS’s on other architectures made the move to 64bit support). The likely reason that Windows (NT) ended up becoming a fully 64bit platform all at once is because, with the end of Alpha, development of a 64bit system was focused entirely on Itanium which had no real 32bit mode. As a result, the x64 version of Windows is actually a port of the Itanium version and even carries over the 44bit address map from that platform. Had 64bit efforts started later, with more focus initially on x86, it is very likely that Windows would have travelled the more relaxed route to a 64bit world.
That said, eventually those OS vendors using a 64bit process on a 32bit kernel model afforded to them via x64’s clever compatibility, will have to port their core codebase to native 64bit since, in the near future, contraints in the amount of addressing allowed in 32bit compatibility mode will limit the total physically addressable system memory (even for 64bit processes) and the memory accessible to the kernel itself.
http://makfu.wordpress.com/2008/03/07/prepare-to-be-rocked/