higher half kernel and identity-mapping

If you are new to OS Development, plan on spending some time here first before going into the other forums.

Moderator:Moderators

Post Reply
vjain20
Posts:12
Joined:Fri Apr 06, 2012 7:24 am
higher half kernel and identity-mapping

Post by vjain20 » Tue Apr 17, 2012 3:05 am

Hi,

I have queries about memory management and higher half kernels which have been troubling me for quite some time.
  1. The tutorial mentions -
    A Higher Half Kernel is a kernel that has a virtual base address of 2GB or above.
    My understanding is that if the kernel has virtual addresses V then all the addresses generated (referenced) by the kernel will be (V + offset).
    Am I right?
  2. Virtual base address of the kernel can be mentioned in assembly file using ORG directive or in a linker script using location counter .(a dot).
    Is this correct?
  3. If the kernel is linked to a virtual base address of 3GB and the boot-loader places it in the physical address 1MB, how will the kernel even start to run
    when paging is disabled ? Won't all the addresses (3GB + offset ) be treated as physical addresses and result in fault.
  4. If the kernel has virtual base address of 3GB and it is loaded at 1MB , what is the use of identity-mapping. As far as I understand
    identity-mapping will map 3GB virtual to 3GB physical. However there is nothing there at 3GB physical while the kernel will
    only reference addresses above 3GB. I would appreciate if somebody could explain the use of identity-mapping in general.

halofreak1990
Posts:92
Joined:Thu May 27, 2010 8:54 pm
Location:Netherlands

Re: higher half kernel and identity-mapping

Post by halofreak1990 » Sat Apr 21, 2012 11:48 pm

vjain20 wrote:Hi,
I have queries about memory management and higher half kernels which have been troubling me for quite some time.
vjain20 wrote: 1. The tutorial mentions -
A Higher Half Kernel is a kernel that has a virtual base address of 2GB or above.
My understanding is that if the kernel has virtual addresses V then all the addresses generated (referenced) by the kernel will be (V + offset).
Am I right?
Yes

vjain20 wrote: 2. Virtual base address of the kernel can be mentioned in assembly file using ORG directive or in a linker script using location counter .(a dot).
Is this correct?
Yes. However, the address is not necessarily virtual. It's just that- an address.

vjain20 wrote: 3. If the kernel is linked to a virtual base address of 3GB and the boot-loader places it in the physical address 1MB, how will the kernel even start to run
when paging is disabled ? Won't all the addresses (3GB + offset ) be treated as physical addresses and result in fault.
The tutorial is slightly misleading in this part. The bootloader used by the tutorials cannot load anything above 1 MB. Because of this, the kernel is loaded at address 0x3000, instead of 1MB.
However, the page mappings are done in such a way that the kernel still runs at 3GB virtual when paging is enabled.
Paging is enabled just before the bootloader jumps into the kernel, therefore jumping to 3GB, before the kernel disables paging again so it can set up the physical memory manager.

vjain20 wrote: 4. If the kernel has virtual base address of 3GB and it is loaded at 1MB , what is the use of identity-mapping. As far as I understand
identity-mapping will map 3GB virtual to 3GB physical. However there is nothing there at 3GB physical while the kernel will
only reference addresses above 3GB. I would appreciate if somebody could explain the use of identity-mapping in general.
The identity mapping mentioned in the tutorial is about identity mapping the first 4 MB, so the kernel does not need page translations to find, for example, the video address 0xB8000 in virtual memory.

vjain20
Posts:12
Joined:Fri Apr 06, 2012 7:24 am

Re: higher half kernel and identity-mapping

Post by vjain20 » Mon Apr 23, 2012 11:09 am

Thanks a lot!!

vjain20
Posts:12
Joined:Fri Apr 06, 2012 7:24 am

Re: higher half kernel and identity-mapping

Post by vjain20 » Wed Apr 25, 2012 10:48 pm

The tutorial is slightly misleading in this part. The bootloader used by the tutorials cannot load anything above 1 MB. Because of this, the kernel is loaded at address 0x3000, instead of 1MB. However, the page mappings are done in such a way that the kernel still runs at 3GB virtual when paging is enabled.Paging is enabled just before the bootloader jumps into the kernel, therefore jumping to 3GB, before the kernel disables paging again so it can set up the physical memory manager.
Do you mean that it is not possible for the bootloader to load the kernel at 1MB? If it is so could you please explain why?
I looked at the code from the part of the tutorial in which kernel is being loaded and no paging is setup and I can see the code
for copying kernel from 0x3000 to 1MB.

Andyhhp
Moderator
Posts:387
Joined:Tue Oct 23, 2007 10:05 am
Location:127.0.0.1
Contact:

Re: higher half kernel and identity-mapping

Post by Andyhhp » Fri Apr 27, 2012 8:49 pm

It is certainly possible to use memory above the 1Mb limit in Real mode.

This is known as Unreal Mode, and works because of a quirk with how the segment registers work in 32bit mode.

In 32bit mode, a lot of information is cached in non-accessable bits in the CPU. This saves the CPU needing to repeatedly read the GDT entry for the base, limit and permissions.

However, these non-accessable bits are only ever updated when you write to the segment registers, even if you switch between 16 and 32bit mode in the meantime.

Therefore, to get BIOS service interrupts to use memory above the 1MB mark:

1) Construct a GDT with one segment set up with a base address of 1MB, limit of 4GB (and other relevent permissions)
2) Jump to 32bit mode
3) Immediatly load ds or es (as appropriate) with the segment you have set up
4) Immediatly jump back into 16bit mode

At this point, you can use ds (or es as a segment override) as the destination target for instructions, and voila - you are suddently writing to memory above 1MB

Caveats:
1) Any time you load a segment register in 16 bit mode, you will loose its base address
2) This method can only address 64K at a time (si/di registers are 16 bits, referencing bytes)

To load more than 64K,
1) Set base to 1MB
2) Use BIOS routines to copy up to 64K of data
3) Update segment base to point to 1MB+64K
4) Reload ds (or es) with new segment
5) Repeat from 2)

So yes - it is possible to get the BIOS interrupt routines to use memory above 1MB, but it is a tad hacky. Nevertheless, this is certainly 1 technique grub-legacy uses to load binaries into memory.

~Andrew
Image

vjain20
Posts:12
Joined:Fri Apr 06, 2012 7:24 am

Re: higher half kernel and identity-mapping

Post by vjain20 » Sat Apr 28, 2012 10:20 am

It is certainly possible to use memory above the 1Mb limit in Real mode.
Thanks a lot for explaining how unreal mode works! I am happy after reading it :) However here the question was whether the bootloader
in the tutorial can access memory over 1 MB or not and my understanding is that the bootloader enters protected mode
and enables A20 line. So it should not have any problem loading the kernel at a higher address.

Andyhhp
Moderator
Posts:387
Joined:Tue Oct 23, 2007 10:05 am
Location:127.0.0.1
Contact:

Re: higher half kernel and identity-mapping

Post by Andyhhp » Sat Apr 28, 2012 12:23 pm

The problem is that BIOS service routines can not be used in 32bit mode, which means that when the bootloader jumps into 32bit mode, it can no longer load anything from disk without a proper scsi driver.
Image

Post Reply