virtual memory

If you are new to development, plan on spending some time here before visiting the other forums.

Moderator:Moderators

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm
virtual memory

Post by xixpsychoxix » Fri Feb 19, 2010 10:55 pm

ok, so i got my memory allocator working but i now want to start my heap at 0xD0000000 (virtual memory) and have it end at 0xDFFFFFFF. However, I cannot adapt the virtual memory manager initializing code to correctly map these addresses to physical addresses. How do I use the virtual memory manager to map these addresses to physical addresses?

Fanael
Posts:8
Joined:Sat Feb 20, 2010 12:55 am
Location:Poland

Re: virtual memory

Post by Fanael » Sat Feb 20, 2010 5:56 pm

xixpsychoxix wrote:How do I use the virtual memory manager to map these addresses to physical addresses?
You're using it incorrectly.

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Sat Feb 20, 2010 9:57 pm

oh. well i kinda knew that. im not really sure how to use it then i suppose... i'll re-read the chapter and see what happens. can anyone give me advice on how to accomplish what i am trying to do?

User avatar
Mike
Site Admin
Posts:465
Joined:Sat Oct 20, 2007 7:58 pm
Contact:

Re: virtual memory

Post by Mike » Sun Feb 21, 2010 12:09 am

Hello,

1) The demo has a serious bug: It initializes the VMM before the PMM. Very silly error on my behalf but can cause some headaches. Make sure the call to vmmngr_initialize() is done after initializing the PMM memory regions.

2) The VMM demo maps the kernel to 3GB, however the kernels VMM does not take this into account, it only maps the first 4MB's. This would have caused a triple fault. Went on unnoticed do to the error #1.

To fix, all that needs to be done is to map two page tables: One for the kernel (at 0xc0000000) and one for the first 4 MB. I can post an example of the fixed routine if you like (Ill update the demo with it as well.)

...With that in mind, the series memory manager is a little overcomplicated. A generic mapping routine just needs to get the page table index and page directory index from the virtual address that it is mapping to. This is a little more difficult do to the complexity nature of the series VMM (it can be made easier to work with. I plan a rewrite on that portion of it.)

Here is an example tested implementation that does what you want using the series VMM. Please keep in mind that it will NOT work unless the above 2 errors are fixed. (I can post fixed source code if needed.)

This routine maps any physical address to a virtual address. Be careful of mapping pages twice

Code: Select all

#define PAGE_DIRECTORY_INDEX(x) (((x) >> 22) & 0x3ff)
#define PAGE_TABLE_INDEX(x) (((x) >> 12) & 0x3ff)
#define PAGE_GET_PHYSICAL_ADDRESS(x) (*x & ~0xfff)

void MmMapPage (void* phys, void* virt) {

	//! get page directory
	pdirectory* pageDirectory = vmmngr_get_directory ();

	//! get page table
	pd_entry* e = &pageDirectory->m_entries [PAGE_DIRECTORY_INDEX ((uint32_t) virt) ];
	if ( (*e & I86_PTE_PRESENT) != I86_PTE_PRESENT) {

		//! page table not present, allocate it
		ptable* table = (ptable*) pmmngr_alloc_block ();
		if (!table)
			return;

		//! clear page table
		vmmngr_ptable_clear (table);

		//! create a new entry
		pd_entry* entry =
			&pageDirectory->m_entries [PAGE_DIRECTORY_INDEX ( (uint32_t) virt) ];

		//! map in the table (Can also just do *entry |= 3) to enable these bits
		pd_entry_add_attrib (entry, I86_PDE_PRESENT);
		pd_entry_add_attrib (entry, I86_PDE_WRITABLE);
		pd_entry_set_frame (entry, (physical_addr)table);
	}

	//! get table
	ptable* table = (ptable*) PAGE_GET_PHYSICAL_ADDRESS ( e );

	//! get page
	pt_entry* page = &table->m_entries [ PAGE_TABLE_INDEX ( (uint32_t) virt) ];

	//! map it in (Can also do (*page |= 3 to enable..)
	pt_entry_set_frame ( page, (physical_addr) phys);
	pt_entry_add_attrib ( page, I86_PTE_PRESENT);
}
Lead Programmer for BrokenThorn Entertainment, Co.
Website: http://www.brokenthorn.com
Email: webmaster@brokenthorn.com

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Sun Feb 21, 2010 12:13 am

Hey thanks alot. if you can post the fixed routine i would be very appreciative! or is that the one you already posted?

User avatar
Mike
Site Admin
Posts:465
Joined:Sat Oct 20, 2007 7:58 pm
Contact:

Re: virtual memory

Post by Mike » Sun Feb 21, 2010 12:17 am

Hello,

The above routine I wrote as a solution for your original problem.

Here is the updated vmmngr_initialize() routine. Insure that it is called after initializing the PMM or it will not work. It also uses the #defines I posted in the previous post. This goes in mmngr_virtual.cpp:

Code: Select all

void vmmngr_initialize () {

	//! allocate default page table
	ptable* table = (ptable*) pmmngr_alloc_block ();
	if (!table)
		return;

	ptable* table2 = (ptable*) pmmngr_alloc_block ();
	if (!table2)
		return;

	//! clear page table
	vmmngr_ptable_clear (table);

	//! 1st 4mb are idenitity mapped
	for (int i=0, frame=0x0, virt=0x00000000; i<1024; i++, frame+=4096, virt+=4096) {

		//! create a new page
		pt_entry page=0;
		pt_entry_add_attrib (&page, I86_PTE_PRESENT);
		pt_entry_set_frame (&page, frame);

		//! ...and add it to the page table
		table2->m_entries [PAGE_TABLE_INDEX (virt) ] = page;
	}

	//! map 1mb to 3gb (where we are at)
	for (int i=0, frame=0x100000, virt=0xc0000000; i<1024; i++, frame+=4096, virt+=4096) {

		//! create a new page
		pt_entry page=0;
		pt_entry_add_attrib (&page, I86_PTE_PRESENT);
		pt_entry_set_frame (&page, frame);

		//! ...and add it to the page table
		table->m_entries [PAGE_TABLE_INDEX (virt) ] = page;
	}

	//! create default directory table
	pdirectory*	dir = (pdirectory*) pmmngr_alloc_blocks (3);
	if (!dir)
		return;

	//! clear directory table and set it as current
	vmmngr_pdirectory_clear (dir);

	//! get first entry in dir table and set it up to point to our table
	pd_entry* entry = &dir->m_entries [PAGE_DIRECTORY_INDEX (0xc0000000) ];
	pd_entry_add_attrib (entry, I86_PDE_PRESENT);
	pd_entry_add_attrib (entry, I86_PDE_WRITABLE);
	pd_entry_set_frame (entry, (physical_addr)table);

	pd_entry* entry2 = &dir->m_entries [PAGE_DIRECTORY_INDEX (0x00000000) ];
	pd_entry_add_attrib (entry2, I86_PDE_PRESENT);
	pd_entry_add_attrib (entry2, I86_PDE_WRITABLE);
	pd_entry_set_frame (entry2, (physical_addr)table2);

	//! store current PDBR
	_cur_pdbr = (physical_addr) &dir->m_entries;

	//! switch to our page directory
	vmmngr_switch_pdirectory (dir);

	//! enable paging
	pmmngr_paging_enable (true);
}
Lead Programmer for BrokenThorn Entertainment, Co.
Website: http://www.brokenthorn.com
Email: webmaster@brokenthorn.com

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Sun Feb 21, 2010 12:21 am

thank you very much. i have been looking forward to finishing my code for FAT12 but couldn't get this stuff to work for me. One more question. Does the routine for mapping virtual addresses to physical addresses only map 4kb pages?

User avatar
Mike
Site Admin
Posts:465
Joined:Sat Oct 20, 2007 7:58 pm
Contact:

Re: virtual memory

Post by Mike » Sun Feb 21, 2010 12:22 am

No problem :) Yes, the memory manager does not support large (4MB) pages. 4KB pages only.
Lead Programmer for BrokenThorn Entertainment, Co.
Website: http://www.brokenthorn.com
Email: webmaster@brokenthorn.com

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Sun Feb 21, 2010 12:29 am

so then in order to map a 4mb section i would have to map all of the entries in a page table to consecutive addresses correct? also, do you have to call that routine after the virtual memory manager is initialized?
Last edited by xixpsychoxix on Sun Feb 21, 2010 12:42 am, edited 1 time in total.

User avatar
Mike
Site Admin
Posts:465
Joined:Sat Oct 20, 2007 7:58 pm
Contact:

Re: virtual memory

Post by Mike » Sun Feb 21, 2010 12:35 am

That is correct. :)
Lead Programmer for BrokenThorn Entertainment, Co.
Website: http://www.brokenthorn.com
Email: webmaster@brokenthorn.com

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Mon Feb 22, 2010 11:00 pm

So i tried to use the new mapping routine to map pages to 0xD0000000 like so:

Code: Select all


MmMapPage (pmmgr_alloc_block (), (void *) 0xD0000000);

but this would not work correctly. I narrowed down the problem to be that pmmgr_alloc_block is returning NULL even though the system has blocks available. Have there been other issues with this function not working after virtual memory has been initialized?

User avatar
Mike
Site Admin
Posts:465
Joined:Sat Oct 20, 2007 7:58 pm
Contact:

Re: virtual memory

Post by Mike » Tue Feb 23, 2010 1:38 am

Hello,

pmmgr_alloc_block() should work fine. However, check to make sure that your PMM bitmap is not getting overwritten. If you have verified that its fine, please post the routine that initializes the PMM and VMM.
Lead Programmer for BrokenThorn Entertainment, Co.
Website: http://www.brokenthorn.com
Email: webmaster@brokenthorn.com

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Tue Feb 23, 2010 3:15 am

ok, i found one thing in the pmm code. I had:

Code: Select all


#define PMMGR_BLOCK_ALIGN pmmgr_BLOCK_SIZE

but im not sure if that's it or not. im gonna keep checking.

xixpsychoxix
Posts:59
Joined:Tue Oct 13, 2009 8:49 pm

Re: virtual memory

Post by xixpsychoxix » Tue Feb 23, 2010 3:23 am

ok, i dont think that the bitmap is being overwritten because the only code that could be doing it is after my trying to set the frames. If it would help i could post all of my kernel code too. here is the pmmgr initialize routine:

Code: Select all


void pmmgr_init (size_t memsize, physical_addr bitmap)
{
	mmgr_memory_size = memsize;
	mmgr_memory_map  = (uint32_t *) bitmap;
	mmgr_max_blocks  = (pmmgr_get_memory_size ()*1024) / PMMGR_BLOCK_SIZE;
	mmgr_used_blocks = pmmgr_get_block_count ();

	memset (mmgr_memory_map, 0xf, pmmgr_get_block_count() / PMMGR_BLOCKS_PER_BYTE);
}

and the vmmgr initialization is the same as you posted earlier in this bulletin.

User avatar
Mike
Site Admin
Posts:465
Joined:Sat Oct 20, 2007 7:58 pm
Contact:

Re: virtual memory

Post by Mike » Tue Feb 23, 2010 3:28 am

Hello,

I was actually looking for the code that calls pmmgr_init and vmmngr_initialize.
Lead Programmer for BrokenThorn Entertainment, Co.
Website: http://www.brokenthorn.com
Email: webmaster@brokenthorn.com

Post Reply