/* $Id$ */
/** @file
* PGM - Page Manager and Monitor. (Mixing stuff here, not good?)
*/
/*
* Copyright (C) 2006-2013 Oracle Corporation
*
* This file is part of VirtualBox Open Source Edition (OSE), as
* available from http://www.virtualbox.org. This file is free software;
* General Public License (GPL) as published by the Free Software
* Foundation, in version 2 as it comes in the "COPYING" file of the
* VirtualBox OSE distribution. VirtualBox OSE is distributed in the
* hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
*/
/** @page pg_pgm PGM - The Page Manager and Monitor
*
* @see grp_pgm,
* @ref pg_pgm_pool,
* @ref pg_pgm_phys.
*
*
* @section sec_pgm_modes Paging Modes
*
* There are three memory contexts: Host Context (HC), Guest Context (GC)
* and intermediate context. When talking about paging HC can also be referred
* to as "host paging", and GC referred to as "shadow paging".
*
* We define three basic paging modes: 32-bit, PAE and AMD64. The host paging mode
* is defined by the host operating system. The mode used in the shadow paging mode
* depends on the host paging mode and what the mode the guest is currently in. The
* following relation between the two is defined:
*
* @verbatim
Host > 32-bit | PAE | AMD64 |
Guest | | | |
==v================================
32-bit 32-bit PAE PAE
-------|--------|--------|--------|
PAE PAE PAE PAE
-------|--------|--------|--------|
AMD64 AMD64 AMD64 AMD64
-------|--------|--------|--------| @endverbatim
*
* All configuration except those in the diagonal (upper left) are expected to
* require special effort from the switcher (i.e. a bit slower).
*
*
*
*
* @section sec_pgm_shw The Shadow Memory Context
*
*
* [..]
*
* Because of guest context mappings requires PDPT and PML4 entries to allow
* writing on AMD64, the two upper levels will have fixed flags whatever the
* guest is thinking of using there. So, when shadowing the PD level we will
* calculate the effective flags of PD and all the higher levels. In legacy
* PAE mode this only applies to the PWT and PCD bits (the rest are
*
*
*
* @section sec_pgm_int The Intermediate Memory Context
*
* The world switch goes thru an intermediate memory context which purpose it is
* to provide different mappings of the switcher code. All guest mappings are also
* present in this context.
*
* The switcher code is mapped at the same location as on the host, at an
* identity mapped location (physical equals virtual address), and at the
* hypervisor location. The identity mapped location is for when the world
* switches that involves disabling paging.
*
* PGM maintain page tables for 32-bit, PAE and AMD64 paging modes. This
* simplifies switching guest CPU mode and consistency at the cost of more
* code to do the work. All memory use for those page tables is located below
* 4GB (this includes page tables for guest context mappings).
*
* Note! The intermediate memory context is also used for 64-bit guest
* execution on 32-bit hosts. Because we need to load 64-bit registers
* prior to switching to guest context, we need to be in 64-bit mode
* first. So, HM has some 64-bit worker routines in VMMRC.rc that get
* invoked via the special world switcher code in LegacyToAMD64.asm.
*
*
* @subsection subsec_pgm_int_gc Guest Context Mappings
*
* During assignment and relocation of a guest context mapping the intermediate
* memory context is used to verify the new location.
*
* Guest context mappings are currently restricted to below 4GB, for reasons
* of simplicity. This may change when we implement AMD64 support.
*
*
*
*
* @section sec_pgm_misc Misc
*
*
* @subsection sec_pgm_misc_A20 The A20 Gate
*
* PGM implements the A20 gate masking when translating a virtual guest address
* into a physical address for CPU access, i.e. PGMGstGetPage (and friends) and
* the code reading the guest page table entries during shadowing. The masking
* is done consistenly for all CPU modes, paged ones included. Large pages are
* also masked correctly. (On current CPUs, experiments indicates that AMD does
* not apply A20M in paged modes and intel only does it for the 2nd MB of
* memory.)
*
* The A20 gate implementation is per CPU core. It can be configured on a per
* core basis via the keyboard device and PC architecture device. This is
* probably not exactly how real CPUs do it, but SMP and A20 isn't a place where
* guest OSes try pushing things anyway, so who cares. (On current real systems
* the A20M signal is probably only sent to the boot CPU and it affects all
* thread and probably all cores in that package.)
*
* The keyboard device and the PC architecture device doesn't OR their A20
* config bits together, rather they are currently implemented such that they
* mirror the CPU state. So, flipping the bit in either of them will change the
* A20 state. (On real hardware the bits of the two devices should probably be
* ORed together to indicate enabled, i.e. both needs to be cleared to disable
* A20 masking.)
*
* The A20 state will change immediately, transmeta fashion. There is no delays
* due to buses, wiring or other physical stuff. (On real hardware there are
* normally delays, the delays differs between the two devices and probably also
* between chipsets and CPU generations. Note that it's said that transmeta CPUs
* port accesses in microcode. Neat.)
*
* @sa http://en.wikipedia.org/wiki/A20_line#The_80286_and_the_high_memory_area
*
*
* @subsection subsec_pgm_misc_diff Differences Between Legacy PAE and Long Mode PAE
*
* The differences between legacy PAE and long mode PAE are:
* -# PDPE bits 1, 2, 5 and 6 are defined differently. In leagcy mode they are
* all marked down as must-be-zero, while in long mode 1, 2 and 5 have the
* usual meanings while 6 is ignored (AMD). This means that upon switching to
* legacy PAE mode we'll have to clear these bits and when going to long mode
* they must be set. This applies to both intermediate and shadow contexts,
* however we don't need to do it for the intermediate one since we're
* executing with CR0.WP at that time.
* -# CR3 allows a 32-byte aligned address in legacy mode, while in long mode
* a page aligned one is required.
*
*
* @section sec_pgm_handlers Access Handlers
*
* Placeholder.
*
*
* @subsection sec_pgm_handlers_virt Virtual Access Handlers
*
* Placeholder.
*
*
* @subsection sec_pgm_handlers_virt Virtual Access Handlers
*
* We currently implement three types of virtual access handlers: ALL, WRITE
* and HYPERVISOR (WRITE). See PGMVIRTHANDLERTYPE for some more details.
*
* The HYPERVISOR access handlers is kept in a separate tree since it doesn't apply
* to physical pages (PGMTREES::HyperVirtHandlers) and only needs to be consulted in
* a special \#PF case. The ALL and WRITE are in the PGMTREES::VirtHandlers tree, the
* rest of this section is going to be about these handlers.
*
* We'll go thru the life cycle of a handler and try make sense of it all, don't know
* how successful this is gonna be...
*
* 1. A handler is registered thru the PGMR3HandlerVirtualRegister and
* PGMHandlerVirtualRegisterEx APIs. We check for conflicting virtual handlers
* and create a new node that is inserted into the AVL tree (range key). Then
* a full PGM resync is flagged (clear pool, sync cr3, update virtual bit of PGMPAGE).
*
* 2. The following PGMSyncCR3/SyncCR3 operation will first make invoke HandlerVirtualUpdate.
*
* 2a. HandlerVirtualUpdate will will lookup all the pages covered by virtual handlers
* via the current guest CR3 and update the physical page -> virtual handler
* translation. Needless to say, this doesn't exactly scale very well. If any changes
* are detected, it will flag a virtual bit update just like we did on registration.
* PGMPHYS pages with changes will have their virtual handler state reset to NONE.
*
* 2b. The virtual bit update process will iterate all the pages covered by all the
* virtual handlers and update the PGMPAGE virtual handler state to the max of all
* virtual handlers on that page.
*
* 2c. Back in SyncCR3 we will now flush the entire shadow page cache to make sure
* we don't miss any alias mappings of the monitored pages.
*
* 2d. SyncCR3 will then proceed with syncing the CR3 table.
*
* 3. \#PF(np,read) on a page in the range. This will cause it to be synced
* read-only and resumed if it's a WRITE handler. If it's an ALL handler we
* will call the handlers like in the next step. If the physical mapping has
* changed we will - some time in the future - perform a handler callback
* (optional) and update the physical -> virtual handler cache.
*
* 4. \#PF(,write) on a page in the range. This will cause the handler to
* be invoked.
*
* 5. The guest invalidates the page and changes the physical backing or
* unmaps it. This should cause the invalidation callback to be invoked
* (it might not yet be 100% perfect). Exactly what happens next... is
* this where we mess up and end up out of sync for a while?
*
* 6. The handler is deregistered by the client via PGMHandlerVirtualDeregister.
* We will then set all PGMPAGEs in the physical -> virtual handler cache for
* this handler to NONE and trigger a full PGM resync (basically the same
* as int step 1). Which means 2 is executed again.
*
*
* @subsubsection sub_sec_pgm_handler_virt_todo TODOs
*
* There is a bunch of things that needs to be done to make the virtual handlers
* work 100% correctly and work more efficiently.
*
* The first bit hasn't been implemented yet because it's going to slow the
* whole mess down even more, and besides it seems to be working reliably for
* our current uses. OTOH, some of the optimizations might end up more or less
* implementing the missing bits, so we'll see.
*
* On the optimization side, the first thing to do is to try avoid unnecessary
* cache flushing. Then try team up with the shadowing code to track changes
* in mappings by means of access to them (shadow in), updates to shadows pages,
* invlpg, and shadow PT discarding (perhaps).
*
* Some idea that have popped up for optimization for current and new features:
* - bitmap indicating where there are virtual handlers installed.
* (4KB => 2**20 pages, page 2**12 => covers 32-bit address space 1:1!)
* - Shadow page table entry bit (if any left)?
*
*/
/** @page pg_pgm_phys PGM Physical Guest Memory Management
*
*
* Objectives:
* - Guest RAM over-commitment using memory ballooning,
* zero pages and general page sharing.
* - Moving or mirroring a VM onto a different physical machine.
*
*
* @subsection subsec_pgmPhys_Definitions Definitions
*
* Allocation chunk - A RTR0MemObjAllocPhysNC object and the tracking
* machinery associated with it.
*
*
*
*
* @subsection subsec_pgmPhys_AllocPage Allocating a page.
*
* Initially we map *all* guest memory to the (per VM) zero page, which
* means that none of the read functions will cause pages to be allocated.
*
* Exception, access bit in page tables that have been shared. This must
* be handled, but we must also make sure PGMGst*Modify doesn't make
* unnecessary modifications.
*
* Allocation points:
* - PGMPhysSimpleWriteGCPhys and PGMPhysWrite.
* - Replacing a zero page mapping at \#PF.
* - Replacing a shared page mapping at \#PF.
* - ROM registration (currently MMR3RomRegister).
* - VM restore (pgmR3Load).
*
* For the first three it would make sense to keep a few pages handy
* until we've reached the max memory commitment for the VM.
*
* For the ROM registration, we know exactly how many pages we need
* and will request these from ring-0. For restore, we will save
* the number of non-zero pages in the saved state and allocate
* them up front. This would allow the ring-0 component to refuse
* the request if the isn't sufficient memory available for VM use.
*
* Btw. for both ROM and restore allocations we won't be requiring
* zeroed pages as they are going to be filled instantly.
*
*
* @subsection subsec_pgmPhys_FreePage Freeing a page
*
* There are a few points where a page can be freed:
* - After being replaced by the zero page.
* - After being replaced by a shared page.
* - After being ballooned by the guest additions.
* - At reset.
* - At restore.
*
* When freeing one or more pages they will be returned to the ring-0
* component and replaced by the zero page.
*
* The reasoning for clearing out all the pages on reset is that it will
* return us to the exact same state as on power on, and may thereby help
* us reduce the memory load on the system. Further it might have a
* (temporary) positive influence on memory fragmentation (@see subsec_pgmPhys_Fragmentation).
*
* On restore, as mention under the allocation topic, pages should be
* freed / allocated depending on how many is actually required by the
* new VM state. The simplest approach is to do like on reset, and free
* all non-ROM pages and then allocate what we need.
*
* A measure to prevent some fragmentation, would be to let each allocation
* chunk have some affinity towards the VM having allocated the most pages
* from it. Also, try make sure to allocate from allocation chunks that
* are almost full. Admittedly, both these measures might work counter to
* our intentions and its probably not worth putting a lot of effort,
* cpu time or memory into this.
*
*
* @subsection subsec_pgmPhys_SharePage Sharing a page
*
* The basic idea is that there there will be a idle priority kernel
* thread walking the non-shared VM pages hashing them and looking for
* pages with the same checksum. If such pages are found, it will compare
* them byte-by-byte to see if they actually are identical. If found to be
* identical it will allocate a shared page, copy the content, check that
* the page didn't change while doing this, and finally request both the
* VMs to use the shared page instead. If the page is all zeros (special
* checksum and byte-by-byte check) it will request the VM that owns it
* to replace it with the zero page.
*
* To make this efficient, we will have to make sure not to try share a page
* that will change its contents soon. This part requires the most work.
* A simple idea would be to request the VM to write monitor the page for
* a while to make sure it isn't modified any time soon. Also, it may
* make sense to skip pages that are being write monitored since this
* information is readily available to the thread if it works on the
* per-VM guest memory structures (presently called PGMRAMRANGE).
*
*
* @subsection subsec_pgmPhys_Fragmentation Fragmentation Concerns and Counter Measures
*
* The pages are organized in allocation chunks in ring-0, this is a necessity
* if we wish to have an OS agnostic approach to this whole thing. (On Linux we
* could easily work on a page-by-page basis if we liked. Whether this is possible
* or efficient on NT I don't quite know.) Fragmentation within these chunks may
* become a problem as part of the idea here is that we wish to return memory to
* the host system.
*
* For instance, starting two VMs at the same time, they will both allocate the
* guest memory on-demand and if permitted their page allocations will be
* intermixed. Shut down one of the two VMs and it will be difficult to return
* any memory to the host system because the page allocation for the two VMs are
* mixed up in the same allocation chunks.
*
* To further complicate matters, when pages are freed because they have been
* to be reused by another VM or returned to the host system. This will cause
* allocation chunks to contain pages belonging to different VMs and prevent
* returning memory to the host when one of those VM shuts down.
*
* The only way to really deal with this problem is to move pages. This can
* either be done at VM shutdown and or by the idle priority worker thread
* involved for coercing a VM to move a page (or to do it for it) will be
*
*
* @subsection subsec_pgmPhys_Tracking Tracking Structures And Their Cost
*
* There's a difficult balance between keeping the per-page tracking structures
* (global and guest page) easy to use and keeping them from eating too much
* memory. We have limited virtual memory resources available when operating in
* 32-bit kernel space (on 64-bit there'll it's quite a different story). The
* tracking structures will be attempted designed such that we can deal with up
* to 32GB of memory on a 32-bit system and essentially unlimited on 64-bit ones.
*
*
* @subsubsection subsubsec_pgmPhys_Tracking_Kernel Kernel Space
*
* @see pg_GMM
*
* @subsubsection subsubsec_pgmPhys_Tracking_PerVM Per-VM
*
* Fixed info is the physical address of the page (HCPhys) and the page id
* (described above). Theoretically we'll need 48(-12) bits for the HCPhys part.
* Today we've restricting ourselves to 40(-12) bits because this is the current
* restrictions of all AMD64 implementations (I think Barcelona will up this
* to 48(-12) bits, not that it really matters) and I needed the bits for
* tracking mappings of a page. 48-12 = 36. That leaves 28 bits, which means a
* decent range for the page id: 2^(28+12) = 1024TB.
*
* In additions to these, we'll have to keep maintaining the page flags as we
* currently do. Although it wouldn't harm to optimize these quite a bit, like
* for instance the ROM shouldn't depend on having a write handler installed
* that the page syncing code doesn't have to mess about checking multiple
* flag combinations (ROM || RW handler || write monitored) in order to
* figure out how to setup a shadow PTE. But this of course, is second
* priority at present. Current this requires 12 bits, but could probably
* be optimized to ~8.
*
* Then there's the 24 bits used to track which shadow page tables are
* currently mapping a page for the purpose of speeding up physical
* access handlers, and thereby the page pool cache. More bit for this
* purpose wouldn't hurt IIRC.
*
* Then there is a new bit in which we need to record what kind of page
* this is, shared, zero, normal or write-monitored-normal. This'll
* require 2 bits. One bit might be needed for indicating whether a
* write monitored page has been written to. And yet another one or
* two for tracking migration status. 3-4 bits total then.
*
* Whatever is left will can be used to record the sharabilitiy of a
* page. The page checksum will not be stored in the per-VM table as
* the idle thread will not be permitted to do modifications to it.
* It will instead have to keep its own working set of potentially
* shareable pages and their check sums and stuff.
*
* For the present we'll keep the current packing of the
* PGMRAMRANGE::aHCPhys to keep the changes simple, only of course,
* we'll have to change it to a struct with a total of 128-bits at
* our disposal.
*
* The initial layout will be like this:
* @verbatim
RTHCPHYS HCPhys; The current stuff.
63:40 Current shadow PT tracking stuff.
39:12 The physical page frame number.
11:0 The current flags.
uint32_t u28PageId : 28; The page id.
uint32_t u2State : 2; The page state { zero, shared, normal, write monitored }.
uint32_t fWrittenTo : 1; Whether a write monitored page was written to.
uint32_t u1Reserved : 1; Reserved for later.
uint32_t u32Reserved; Reserved for later, mostly sharing stats.
@endverbatim
*
* The final layout will be something like this:
* @verbatim
RTHCPHYS HCPhys; The current stuff.
63:48 High page id (12+).
47:12 The physical page frame number.
11:0 Low page id.
uint32_t fReadOnly : 1; Whether it's readonly page (rom or monitored in some way).
uint32_t u3Type : 3; The page type {RESERVED, MMIO, MMIO2, ROM, shadowed ROM, RAM}.
uint32_t u2PhysMon : 2; Physical access handler type {none, read, write, all}.
uint32_t u2VirtMon : 2; Virtual access handler type {none, read, write, all}..
uint32_t u2State : 2; The page state { zero, shared, normal, write monitored }.
uint32_t fWrittenTo : 1; Whether a write monitored page was written to.
uint32_t u20Reserved : 20; Reserved for later, mostly sharing stats.
uint32_t u32Tracking; The shadow PT tracking stuff, roughly.
@endverbatim
*
* Cost wise, this means we'll double the cost for guest memory. There isn't anyway
* around that I'm afraid. It means that the cost of dealing out 32GB of memory
* to one or more VMs is: (32GB >> PAGE_SHIFT) * 16 bytes, or 128MBs. Or another
* example, the VM heap cost when assigning 1GB to a VM will be: 4MB.
*
* A couple of cost examples for the total cost per-VM + kernel.
* 32-bit Windows and 32-bit linux:
* 1GB guest ram, 256K pages: 4MB + 2MB(+) = 6MB
* 4GB guest ram, 1M pages: 16MB + 8MB(+) = 24MB
* 32GB guest ram, 8M pages: 128MB + 64MB(+) = 192MB
* 64-bit Windows and 64-bit linux:
* 1GB guest ram, 256K pages: 4MB + 3MB(+) = 7MB
* 4GB guest ram, 1M pages: 16MB + 12MB(+) = 28MB
* 32GB guest ram, 8M pages: 128MB + 96MB(+) = 224MB
*
* UPDATE - 2007-09-27:
* trust the guest 100% and reporting the same page as ballooned more
* than once will put the GMM off balance.
*
*
* @subsection subsec_pgmPhys_Serializing Serializing Access
*
* Initially, we'll try a simple scheme:
*
* - The per-VM RAM tracking structures (PGMRAMRANGE) is only modified
* by the EMT thread of that VM while in the pgm critsect.
* - Other threads in the VM process that needs to make reliable use of
* the per-VM RAM tracking structures will enter the critsect.
* - No process external thread or kernel thread will ever try enter
* the pgm critical section, as that just won't work.
* - The idle thread (and similar threads) doesn't not need 100% reliable
* data when performing it tasks as the EMT thread will be the one to
* do the actual changes later anyway. So, as long as it only accesses
* the main ram range, it can do so by somehow preventing the VM from
* being destroyed while it works on it...
*
* - The over-commitment management, including the allocating/freeing
* chunks, is serialized by a ring-0 mutex lock (a fast one since the
* more mundane mutex implementation is broken on Linux).
* - A separate mutex is protecting the set of allocation chunks so
* allocating more chunks. This mutex can be take from under the other
* one, but not the other way around.
*
*
* @subsection subsec_pgmPhys_Request VM Request interface
*
* When in ring-0 it will become necessary to send requests to a VM so it can
* for instance move a page while defragmenting during VM destroy. The idle
* thread will make use of this interface to request VMs to setup shared
* pages and to perform write monitoring of pages.
*
* I would propose an interface similar to the current VMReq interface, similar
* in that it doesn't require locking and that the one sending the request may
* wait for completion if it wishes to. This shouldn't be very difficult to
* realize.
*
* The requests themselves are also pretty simple. They are basically:
* -# Check that some precondition is still true.
* -# Do the update.
* -# Update all shadow page tables involved with the page.
*
* The 3rd step is identical to what we're already doing when updating a
* physical handler, see pgmHandlerPhysicalSetRamFlagsAndFlushShadowPTs.
*
*
*
* @section sec_pgmPhys_MappingCaches Mapping Caches
*
* In order to be able to map in and out memory and to be able to support
* guest with more RAM than we've got virtual address space, we'll employing
* a mapping cache. Normally ring-0 and ring-3 can share the same cache,
* however on 32-bit darwin the ring-0 code is running in a different memory
* context and therefore needs a separate cache. In raw-mode context we also
* need a separate cache. The 32-bit darwin mapping cache and the one for
* raw-mode context share a lot of code, see PGMRZDYNMAP.
*
*
* @subsection subsec_pgmPhys_MappingCaches_R3 Ring-3
*
* We've considered implementing the ring-3 mapping cache page based but found
* that this was bother some when one had to take into account TLBs+SMP and
* portability (missing the necessary APIs on several platforms). There were
* also some performance concerns with this approach which hadn't quite been
* worked out.
*
* Instead, we'll be mapping allocation chunks into the VM process. This simplifies
* matters greatly quite a bit since we don't need to invent any new ring-0 stuff,
* only some minor RTR0MEMOBJ mapping stuff. The main concern here is that mapping
* compared to the previous idea is that mapping or unmapping a 1MB chunk is more
* costly than a single page, although how much more costly is uncertain. We'll
* try address this by using a very big cache, preferably bigger than the actual
* VM RAM size if possible. The current VM RAM sizes should give some idea for
* 32-bit boxes, while on 64-bit we can probably get away with employing an
* unlimited cache.
*
* The cache have to parts, as already indicated, the ring-3 side and the
* ring-0 side.
*
* The ring-0 will be tied to the page allocator since it will operate on the
* memory objects it contains. It will therefore require the first ring-0 mutex
* discussed in @ref subsec_pgmPhys_Serializing. We
* some double house keeping wrt to who has mapped what I think, since both
* VMMR0.r0 and RTR0MemObj will keep track of mapping relations
*
* The ring-3 part will be protected by the pgm critsect. For simplicity, we'll
* require anyone that desires to do changes to the mapping cache to do that
* from within this critsect. Alternatively, we could employ a separate critsect
* for serializing changes to the mapping cache as this would reduce potential
* contention with other threads accessing mappings unrelated to the changes
* that are in process. We can see about this later, contention will show
* up in the statistics anyway, so it'll be simple to tell.
*
* The organization of the ring-3 part will be very much like how the allocation
* chunks are organized in ring-0, that is in an AVL tree by chunk id. To avoid
* having to walk the tree all the time, we'll have a couple of lookaside entries
* like in we do for I/O ports and MMIO in IOM.
*
* The simplified flow of a PGMPhysRead/Write function:
* -# Enter the PGM critsect.
* -# Lookup GCPhys in the ram ranges and get the Page ID.
* -# Calc the Allocation Chunk ID from the Page ID.
* -# Check the lookaside entries and then the AVL tree for the Chunk ID.
* If not found in cache:
* -# Call ring-0 and request it to be mapped and supply
* a chunk to be unmapped if the cache is maxed out already.
* -# Insert the new mapping into the AVL tree (id + R3 address).
* -# Update the relevant lookaside entry and return the mapping address.
* -# Leave the critsect.
*
*
* @section sec_pgmPhys_Fallback Fallback
*
* Current all the "second tier" hosts will not support the RTR0MemObjAllocPhysNC
* API and thus require a fallback.
*
* So, when RTR0MemObjAllocPhysNC returns VERR_NOT_SUPPORTED the page allocator
* will return to the ring-3 caller (and later ring-0) and asking it to seed
* the page allocator with some fresh pages (VERR_GMM_SEED_ME). Ring-3 will
* then perform an SUPR3PageAlloc(cbChunk >> PAGE_SHIFT) call and make a
* "SeededAllocPages" call to ring-0.
*
* The first time ring-0 sees the VERR_NOT_SUPPORTED failure it will disable
* all page sharing (zero page detection will continue). It will also force
* all allocations to come from the VM which seeded the page. Both these
* measures are taken to make sure that there will never be any need for
* mapping anything into ring-3 - everything will be mapped already.
*
* Whether we'll continue to use the current MM locked memory management
* for this I don't quite know (I'd prefer not to and just ditch that all
* together), we'll see what's simplest to do.
*
*
*
* @section sec_pgmPhys_Changes Changes
*
* Breakdown of the changes involved?
*/
/*******************************************************************************
* Header Files *
*******************************************************************************/
#ifdef VBOX_WITH_REM
#endif
#include "PGMInternal.h"
#include "PGMInline.h"
#include <iprt/asm-amd64-x86.h>
/*******************************************************************************
* Internal Functions *
*******************************************************************************/
#ifdef VBOX_STRICT
#endif
static PGMMODE pgmR3CalcShadowMode(PVM pVM, PGMMODE enmGuestMode, SUPPAGINGMODE enmHostMode, PGMMODE enmShadowMode, VMMSWITCHER *penmSwitcher);
#ifdef VBOX_WITH_DEBUGGER
# ifdef VBOX_STRICT
# endif
#endif
/*******************************************************************************
* Global Variables *
*******************************************************************************/
#ifdef VBOX_WITH_DEBUGGER
/** Argument descriptors for '.pgmerror' and '.pgmerroroff'. */
{
/* cTimesMin, cTimesMax, enmCategory, fFlags, pszName, pszDescription */
};
{
/* cTimesMin, cTimesMax, enmCategory, fFlags, pszName, pszDescription */
};
# ifdef DEBUG_sandervl
{
/* cTimesMin, cTimesMax, enmCategory, fFlags, pszName, pszDescription */
};
# endif
/** Command descriptors. */
{
/* pszCmd, cArgsMin, cArgsMax, paArgDesc, cArgDescs, fFlags, pfnHandler pszSyntax, ....pszDescription */
{ "pgmerror", 0, 1, &g_aPgmErrorArgs[0], 1, 0, pgmR3CmdError, "", "Enables inject runtime of errors into parts of PGM." },
{ "pgmerroroff", 0, 1, &g_aPgmErrorArgs[0], 1, 0, pgmR3CmdError, "", "Disables inject runtime errors into parts of PGM." },
# ifdef VBOX_STRICT
# ifdef VBOX_WITH_PAGE_SHARING
{ "pgmcheckduppages", 0, 0, NULL, 0, 0, pgmR3CmdCheckDuplicatePages, "", "Check for duplicate pages in all running VMs." },
{ "pgmsharedmodules", 0, 0, NULL, 0, 0, pgmR3CmdShowSharedModules, "", "Print shared modules info." },
# endif
# endif
{ "pgmphystofile", 1, 2, &g_aPgmPhysToFileArgs[0], 2, 0, pgmR3CmdPhysToFile, "", "Save the physical memory to file." },
};
#endif
/*
* Shadow - 32-bit mode
*/
#include "PGMShw.h"
/* Guest - real mode */
#include "PGMBth.h"
#include "PGMGstDefs.h"
#include "PGMGst.h"
/* Guest - protected mode */
#include "PGMBth.h"
#include "PGMGstDefs.h"
#include "PGMGst.h"
/* Guest - 32-bit mode */
#include "PGMBth.h"
#include "PGMGstDefs.h"
#include "PGMGst.h"
/*
* Shadow - PAE mode
*/
#include "PGMShw.h"
/* Guest - real mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - protected mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - 32-bit mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - PAE mode */
#include "PGMBth.h"
#include "PGMGstDefs.h"
#include "PGMGst.h"
/*
* Shadow - AMD64 mode
*/
#include "PGMShw.h"
#ifdef VBOX_WITH_64_BITS_GUESTS
/* Guest - AMD64 mode */
# include "PGMBth.h"
# include "PGMGstDefs.h"
# include "PGMGst.h"
#endif /* VBOX_WITH_64_BITS_GUESTS */
/*
* Shadow - Nested paging mode
*/
#include "PGMShw.h"
/* Guest - real mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - protected mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - 32-bit mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - PAE mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
#ifdef VBOX_WITH_64_BITS_GUESTS
/* Guest - AMD64 mode */
# include "PGMGstDefs.h"
# include "PGMBth.h"
#endif /* VBOX_WITH_64_BITS_GUESTS */
/*
* Shadow - EPT
*/
#include "PGMShw.h"
/* Guest - real mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - protected mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - 32-bit mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
/* Guest - PAE mode */
#include "PGMGstDefs.h"
#include "PGMBth.h"
#ifdef VBOX_WITH_64_BITS_GUESTS
/* Guest - AMD64 mode */
# include "PGMGstDefs.h"
# include "PGMBth.h"
#endif /* VBOX_WITH_64_BITS_GUESTS */
/**
* Initiates the paging of VM.
*
* @returns VBox status code.
* @param pVM Pointer to VM structure.
*/
{
LogFlow(("PGMR3Init:\n"));
int rc;
/*
* Assert alignment and sizes.
*/
/*
* Init the structure.
*/
{
}
{
}
/* Init the per-CPU part. */
{
#ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
#endif
{
#ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
#endif
}
pPGM->fA20Enabled = true;
}
true
#else
false
#endif
);
#if HC_ARCH_BITS == 32
# ifdef RT_OS_DARWIN
rc = CFGMR3QueryU32Def(pCfgPGM, "MaxRing3Chunks", &pVM->pgm.s.ChunkR3Map.cMax, _1G / GMM_CHUNK_SIZE * 3);
# else
rc = CFGMR3QueryU32Def(pCfgPGM, "MaxRing3Chunks", &pVM->pgm.s.ChunkR3Map.cMax, _1G / GMM_CHUNK_SIZE);
# endif
#else
#endif
/*
* Get the configured RAM size - to estimate saved state size.
*/
if (rc == VERR_CFGM_VALUE_NOT_FOUND)
cbRam = 0;
else if (RT_SUCCESS(rc))
{
cbRam = 0;
}
else
{
return rc;
}
/*
* Check for PCI pass-through.
*/
AssertMsgRCReturn(rc, ("Configuration error: Failed to query integer \"PciPassThrough\", rc=%Rrc.\n", rc), rc);
#ifdef VBOX_WITH_STATISTICS
/*
* Allocate memory for the statistics before someone tries to use them.
*/
size_t cbTotalStats = RT_ALIGN_Z(sizeof(PGMSTATS), 64) + RT_ALIGN_Z(sizeof(PGMCPUSTATS), 64) * pVM->cCpus;
void *pv;
{
}
#endif /* VBOX_WITH_STATISTICS */
/*
* Register callbacks, string formatters and the saved state data unit.
*/
#ifdef VBOX_STRICT
#endif
if (RT_FAILURE(rc))
return rc;
/*
* Initialize the PGM critical section and flush the phys TLBs
*/
/*
* For the time being we sport a full set of handy pages in addition to the base
* memory to simplify things.
*/
rc = MMR3ReserveHandyPages(pVM, RT_ELEMENTS(pVM->pgm.s.aHandyPages)); /** @todo this should be changed to PGM_HANDY_PAGES_MIN but this needs proper testing... */
/*
* Trees
*/
if (RT_SUCCESS(rc))
{
/*
* Allocate the zero page.
*/
}
if (RT_SUCCESS(rc))
{
/*
* Allocate the invalid MMIO page.
* (The invalid bits in HCPhysInvMmioPg are set later on init complete.)
*/
}
if (RT_SUCCESS(rc))
{
/*
* Init the paging.
*/
}
if (RT_SUCCESS(rc))
{
/*
* Init the page pool.
*/
}
if (RT_SUCCESS(rc))
{
{
if (RT_FAILURE(rc))
break;
}
}
if (RT_SUCCESS(rc))
{
/*
* Info & statistics
*/
"Shows the current paging mode. "
"Recognizes 'all', 'guest', 'shadow' and 'host' as arguments, defaulting to 'all' if nothing is given.",
"Dumps all the entries in the top level paging table. No arguments.",
"Dumps all the physical address ranges. No arguments.",
"Dumps physical, virtual and hyper virtual handlers. "
"Pass 'phys', 'virt', 'hyper' as argument if only one kind is wanted."
"Add 'nost' if the statistics are unwanted, use together with 'all' or explicit selection.",
"Dumps guest mappings.",
#ifdef VBOX_WITH_DEBUGGER
/*
* Debugger commands.
*/
static bool s_fRegisteredCmds = false;
if (!s_fRegisteredCmds)
{
if (RT_SUCCESS(rc2))
s_fRegisteredCmds = true;
}
#endif
return VINF_SUCCESS;
}
/* Almost no cleanup necessary, MM frees all memory. */
return rc;
}
/**
* Init paging.
*
* Since we need to check what mode the host is operating in before we can choose
* the right paging functions for the host we have to delay this until R0 has
* been initialized.
*
* @returns VBox status code.
* @param pVM Pointer to the VM.
*/
{
/*
* Force a recalculation of modes and switcher so everyone gets notified.
*/
{
}
/*
* Allocate static mapping space for whatever the cr3 register
* points to and in the case of PAE mode to the 4 PDs.
*/
if (RT_FAILURE(rc))
{
return rc;
}
/*
* Allocate pages for the three possible intermediate contexts
* (AMD64, PAE and plain 32-Bit). We maintain all three contexts
* for the sake of simplicity. The AMD64 uses the PAE for the
* lower levels, making the total number of pages 11 (3 + 7 + 1).
*
* We assume that two page tables will be enought for the core code
* mappings (HC virtual and identity).
*/
pVM->pgm.s.pInterPD = (PX86PD)MMR3PageAllocLow(pVM); AssertReturn(pVM->pgm.s.pInterPD, VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPTs[0] = (PX86PT)MMR3PageAllocLow(pVM); AssertReturn(pVM->pgm.s.apInterPTs[0], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPTs[1] = (PX86PT)MMR3PageAllocLow(pVM); AssertReturn(pVM->pgm.s.apInterPTs[1], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPaePTs[0] = (PX86PTPAE)MMR3PageAlloc(pVM); AssertReturn(pVM->pgm.s.apInterPaePTs[0], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPaePTs[1] = (PX86PTPAE)MMR3PageAlloc(pVM); AssertReturn(pVM->pgm.s.apInterPaePTs[1], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPaePDs[0] = (PX86PDPAE)MMR3PageAlloc(pVM); AssertReturn(pVM->pgm.s.apInterPaePDs[0], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPaePDs[1] = (PX86PDPAE)MMR3PageAlloc(pVM); AssertReturn(pVM->pgm.s.apInterPaePDs[1], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPaePDs[2] = (PX86PDPAE)MMR3PageAlloc(pVM); AssertReturn(pVM->pgm.s.apInterPaePDs[2], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.apInterPaePDs[3] = (PX86PDPAE)MMR3PageAlloc(pVM); AssertReturn(pVM->pgm.s.apInterPaePDs[3], VERR_NO_PAGE_MEMORY);
pVM->pgm.s.pInterPaePDPT = (PX86PDPT)MMR3PageAllocLow(pVM); AssertReturn(pVM->pgm.s.pInterPaePDPT, VERR_NO_PAGE_MEMORY);
pVM->pgm.s.pInterPaePDPT64 = (PX86PDPT)MMR3PageAllocLow(pVM); AssertReturn(pVM->pgm.s.pInterPaePDPT64, VERR_NO_PAGE_MEMORY);
pVM->pgm.s.pInterPaePML4 = (PX86PML4)MMR3PageAllocLow(pVM); AssertReturn(pVM->pgm.s.pInterPaePML4, VERR_NO_PAGE_MEMORY);
AssertRelease(pVM->pgm.s.HCPhysInterPD != NIL_RTHCPHYS && !(pVM->pgm.s.HCPhysInterPD & PAGE_OFFSET_MASK));
AssertRelease(pVM->pgm.s.HCPhysInterPaePDPT != NIL_RTHCPHYS && !(pVM->pgm.s.HCPhysInterPaePDPT & PAGE_OFFSET_MASK));
AssertRelease(pVM->pgm.s.HCPhysInterPaePML4 != NIL_RTHCPHYS && !(pVM->pgm.s.HCPhysInterPaePML4 & PAGE_OFFSET_MASK) && pVM->pgm.s.HCPhysInterPaePML4 < 0xffffffff);
/*
* Initialize the pages, setting up the PML4 and PDPT for repetitive 4GB action.
*/
{
}
{
pVM->pgm.s.pInterPaePDPT64->a[i].u = X86_PDPE_P | X86_PDPE_RW | X86_PDPE_US | X86_PDPE_A | PGM_PLXFLAGS_PERMANENT
}
pVM->pgm.s.pInterPaePML4->a[i].u = X86_PML4E_P | X86_PML4E_RW | X86_PML4E_US | X86_PML4E_A | PGM_PLXFLAGS_PERMANENT
/*
* Initialize paging workers and mode from current host mode
* and the guest running in real mode.
*/
{
case SUPPAGINGMODE_32_BIT:
case SUPPAGINGMODE_PAE:
case SUPPAGINGMODE_PAE_GLOBAL:
case SUPPAGINGMODE_PAE_NX:
break;
case SUPPAGINGMODE_AMD64:
case SUPPAGINGMODE_AMD64_NX:
#ifndef VBOX_WITH_HYBRID_32BIT_KERNEL
if (ARCH_BITS != 64)
{
AssertMsgFailed(("Host mode %d (64-bit) is not supported by non-64bit builds\n", pVM->pgm.s.enmHostMode));
LogRel(("PGM: Host mode %d (64-bit) is not supported by non-64bit builds\n", pVM->pgm.s.enmHostMode));
}
#endif
break;
default:
}
if (RT_SUCCESS(rc))
{
LogFlow(("pgmR3InitPaging: returns successfully\n"));
#if HC_ARCH_BITS == 64
LogRel(("PGM: HCPhysInterPD=%RHp HCPhysInterPaePDPT=%RHp HCPhysInterPaePML4=%RHp\n",
LogRel(("PGM: apInterPTs={%RHp,%RHp} apInterPaePTs={%RHp,%RHp} apInterPaePDs={%RHp,%RHp,%RHp,%RHp} pInterPaePDPT64=%RHp\n",
MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[0]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[1]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[2]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[3]),
#endif
/*
* Log the host paging mode. It may come in handy.
*/
const char *pszHostMode;
{
default: pszHostMode = "???"; break;
}
return VINF_SUCCESS;
}
return rc;
}
/**
* Init statistics
* @returns VBox status code.
*/
{
int rc;
/*
* Release statistics.
*/
/* Common - misc variables */
STAM_REL_REG(pVM, &pPGM->cAllPages, STAMTYPE_U32, "/PGM/Page/cAllPages", STAMUNIT_COUNT, "The total number of pages.");
STAM_REL_REG(pVM, &pPGM->cPrivatePages, STAMTYPE_U32, "/PGM/Page/cPrivatePages", STAMUNIT_COUNT, "The number of private pages.");
STAM_REL_REG(pVM, &pPGM->cSharedPages, STAMTYPE_U32, "/PGM/Page/cSharedPages", STAMUNIT_COUNT, "The number of shared pages.");
STAM_REL_REG(pVM, &pPGM->cReusedSharedPages, STAMTYPE_U32, "/PGM/Page/cReusedSharedPages", STAMUNIT_COUNT, "The number of reused shared pages.");
STAM_REL_REG(pVM, &pPGM->cZeroPages, STAMTYPE_U32, "/PGM/Page/cZeroPages", STAMUNIT_COUNT, "The number of zero backed pages.");
STAM_REL_REG(pVM, &pPGM->cPureMmioPages, STAMTYPE_U32, "/PGM/Page/cPureMmioPages", STAMUNIT_COUNT, "The number of pure MMIO pages.");
STAM_REL_REG(pVM, &pPGM->cMonitoredPages, STAMTYPE_U32, "/PGM/Page/cMonitoredPages", STAMUNIT_COUNT, "The number of write monitored pages.");
STAM_REL_REG(pVM, &pPGM->cWrittenToPages, STAMTYPE_U32, "/PGM/Page/cWrittenToPages", STAMUNIT_COUNT, "The number of previously write monitored pages that have been written to.");
STAM_REL_REG(pVM, &pPGM->cWriteLockedPages, STAMTYPE_U32, "/PGM/Page/cWriteLockedPages", STAMUNIT_COUNT, "The number of write(/read) locked pages.");
STAM_REL_REG(pVM, &pPGM->cReadLockedPages, STAMTYPE_U32, "/PGM/Page/cReadLockedPages", STAMUNIT_COUNT, "The number of read (only) locked pages.");
STAM_REL_REG(pVM, &pPGM->cBalloonedPages, STAMTYPE_U32, "/PGM/Page/cBalloonedPages", STAMUNIT_COUNT, "The number of ballooned pages.");
STAM_REL_REG(pVM, &pPGM->cHandyPages, STAMTYPE_U32, "/PGM/Page/cHandyPages", STAMUNIT_COUNT, "The number of handy pages (not included in cAllPages).");
STAM_REL_REG(pVM, &pPGM->cLargePages, STAMTYPE_U32, "/PGM/Page/cLargePages", STAMUNIT_COUNT, "The number of large pages allocated (includes disabled).");
STAM_REL_REG(pVM, &pPGM->cLargePagesDisabled, STAMTYPE_U32, "/PGM/Page/cLargePagesDisabled", STAMUNIT_COUNT, "The number of disabled large pages.");
STAM_REL_REG(pVM, &pPGM->cRelocations, STAMTYPE_COUNTER, "/PGM/cRelocations", STAMUNIT_OCCURENCES,"Number of hypervisor relocations.");
STAM_REL_REG(pVM, &pPGM->ChunkR3Map.c, STAMTYPE_U32, "/PGM/ChunkR3Map/c", STAMUNIT_COUNT, "Number of mapped chunks.");
STAM_REL_REG(pVM, &pPGM->ChunkR3Map.cMax, STAMTYPE_U32, "/PGM/ChunkR3Map/cMax", STAMUNIT_COUNT, "Maximum number of mapped chunks.");
STAM_REL_REG(pVM, &pPGM->cMappedChunks, STAMTYPE_U32, "/PGM/ChunkR3Map/Mapped", STAMUNIT_COUNT, "Number of times we mapped a chunk.");
STAM_REL_REG(pVM, &pPGM->cUnmappedChunks, STAMTYPE_U32, "/PGM/ChunkR3Map/Unmapped", STAMUNIT_COUNT, "Number of times we unmapped a chunk.");
STAM_REL_REG(pVM, &pPGM->StatLargePageReused, STAMTYPE_COUNTER, "/PGM/LargePage/Reused", STAMUNIT_OCCURENCES, "The number of times we've reused a large page.");
STAM_REL_REG(pVM, &pPGM->StatLargePageRefused, STAMTYPE_COUNTER, "/PGM/LargePage/Refused", STAMUNIT_OCCURENCES, "The number of times we couldn't use a large page.");
STAM_REL_REG(pVM, &pPGM->StatLargePageRecheck, STAMTYPE_COUNTER, "/PGM/LargePage/Recheck", STAMUNIT_OCCURENCES, "The number of times we've rechecked a disabled large page.");
STAM_REL_REG(pVM, &pPGM->StatShModCheck, STAMTYPE_PROFILE, "/PGM/ShMod/Check", STAMUNIT_TICKS_PER_CALL, "Profiles the shared module checking.");
/* Live save */
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.fActive, STAMTYPE_U8, "/PGM/LiveSave/fActive", STAMUNIT_COUNT, "Active or not.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.cIgnoredPages, STAMTYPE_U32, "/PGM/LiveSave/cIgnoredPages", STAMUNIT_COUNT, "The number of ignored pages in the RAM ranges (i.e. MMIO, MMIO2 and ROM).");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.cDirtyPagesLong, STAMTYPE_U32, "/PGM/LiveSave/cDirtyPagesLong", STAMUNIT_COUNT, "Longer term dirty page average.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.cDirtyPagesShort, STAMTYPE_U32, "/PGM/LiveSave/cDirtyPagesShort", STAMUNIT_COUNT, "Short term dirty page average.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.cPagesPerSecond, STAMTYPE_U32, "/PGM/LiveSave/cPagesPerSecond", STAMUNIT_COUNT, "Pages per second.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.cSavedPages, STAMTYPE_U64, "/PGM/LiveSave/cSavedPages", STAMUNIT_COUNT, "The total number of saved pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Ram.cReadyPages, STAMTYPE_U32, "/PGM/LiveSave/Ram/cReadPages", STAMUNIT_COUNT, "RAM: Ready pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Ram.cDirtyPages, STAMTYPE_U32, "/PGM/LiveSave/Ram/cDirtyPages", STAMUNIT_COUNT, "RAM: Dirty pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Ram.cZeroPages, STAMTYPE_U32, "/PGM/LiveSave/Ram/cZeroPages", STAMUNIT_COUNT, "RAM: Ready zero pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Ram.cMonitoredPages, STAMTYPE_U32, "/PGM/LiveSave/Ram/cMonitoredPages", STAMUNIT_COUNT, "RAM: Write monitored pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Rom.cReadyPages, STAMTYPE_U32, "/PGM/LiveSave/Rom/cReadPages", STAMUNIT_COUNT, "ROM: Ready pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Rom.cDirtyPages, STAMTYPE_U32, "/PGM/LiveSave/Rom/cDirtyPages", STAMUNIT_COUNT, "ROM: Dirty pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Rom.cZeroPages, STAMTYPE_U32, "/PGM/LiveSave/Rom/cZeroPages", STAMUNIT_COUNT, "ROM: Ready zero pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Rom.cMonitoredPages, STAMTYPE_U32, "/PGM/LiveSave/Rom/cMonitoredPages", STAMUNIT_COUNT, "ROM: Write monitored pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Mmio2.cReadyPages, STAMTYPE_U32, "/PGM/LiveSave/Mmio2/cReadPages", STAMUNIT_COUNT, "MMIO2: Ready pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Mmio2.cDirtyPages, STAMTYPE_U32, "/PGM/LiveSave/Mmio2/cDirtyPages", STAMUNIT_COUNT, "MMIO2: Dirty pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Mmio2.cZeroPages, STAMTYPE_U32, "/PGM/LiveSave/Mmio2/cZeroPages", STAMUNIT_COUNT, "MMIO2: Ready zero pages.");
STAM_REL_REG_USED(pVM, &pPGM->LiveSave.Mmio2.cMonitoredPages,STAMTYPE_U32, "/PGM/LiveSave/Mmio2/cMonitoredPages",STAMUNIT_COUNT, "MMIO2: Write monitored pages.");
#ifdef VBOX_WITH_STATISTICS
# define PGM_REG_COUNTER(a, b, c) \
# define PGM_REG_COUNTER_BYTES(a, b, c) \
# define PGM_REG_PROFILE(a, b, c) \
rc = STAMR3RegisterF(pVM, a, STAMTYPE_PROFILE, STAMVISIBILITY_ALWAYS, STAMUNIT_TICKS_PER_CALL, c, b); \
PGM_REG_PROFILE(&pStats->StatAllocLargePage, "/PGM/LargePage/Prof/Alloc", "Time spent by the host OS for large page allocation.");
PGM_REG_PROFILE(&pStats->StatClearLargePage, "/PGM/LargePage/Prof/Clear", "Time spent clearing the newly allocated large pages.");
PGM_REG_COUNTER(&pStats->StatLargePageOverflow, "/PGM/LargePage/Overflow", "The number of times allocating a large page took too long.");
PGM_REG_PROFILE(&pStats->StatR3IsValidLargePage, "/PGM/LargePage/Prof/R3/IsValid", "pgmPhysIsValidLargePage profiling - R3.");
PGM_REG_PROFILE(&pStats->StatRZIsValidLargePage, "/PGM/LargePage/Prof/RZ/IsValid", "pgmPhysIsValidLargePage profiling - RZ.");
PGM_REG_COUNTER(&pStats->StatR3DetectedConflicts, "/PGM/R3/DetectedConflicts", "The number of times PGMR3CheckMappingConflicts() detected a conflict.");
PGM_REG_PROFILE(&pStats->StatR3ResolveConflict, "/PGM/R3/ResolveConflict", "pgmR3SyncPTResolveConflict() profiling (includes the entire relocation).");
PGM_REG_COUNTER(&pStats->StatR3PhysRead, "/PGM/R3/Phys/Read", "The number of times PGMPhysRead was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatR3PhysReadBytes, "/PGM/R3/Phys/Read/Bytes", "The number of bytes read by PGMPhysRead.");
PGM_REG_COUNTER(&pStats->StatR3PhysWrite, "/PGM/R3/Phys/Write", "The number of times PGMPhysWrite was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatR3PhysWriteBytes, "/PGM/R3/Phys/Write/Bytes", "The number of bytes written by PGMPhysWrite.");
PGM_REG_COUNTER(&pStats->StatR3PhysSimpleRead, "/PGM/R3/Phys/Simple/Read", "The number of times PGMPhysSimpleReadGCPtr was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatR3PhysSimpleReadBytes, "/PGM/R3/Phys/Simple/Read/Bytes", "The number of bytes read by PGMPhysSimpleReadGCPtr.");
PGM_REG_COUNTER(&pStats->StatR3PhysSimpleWrite, "/PGM/R3/Phys/Simple/Write", "The number of times PGMPhysSimpleWriteGCPtr was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatR3PhysSimpleWriteBytes, "/PGM/R3/Phys/Simple/Write/Bytes", "The number of bytes written by PGMPhysSimpleWriteGCPtr.");
PGM_REG_PROFILE(&pStats->StatChunkFindCandidate, "/PGM/ChunkR3Map/Map/Find", "Chunk unmap find profiling.");
PGM_REG_PROFILE(&pStats->StatChunkUnmap, "/PGM/ChunkR3Map/Map/Unmap", "Chunk unmap of address space profiling.");
PGM_REG_PROFILE(&pStats->StatChunkMap, "/PGM/ChunkR3Map/Map/Map", "Chunk map of address space profiling.");
PGM_REG_COUNTER(&pStats->StatPageMapTlbFlushes, "/PGM/R3/Page/MapTlbFlushes", "TLB flushes (all contexts).");
PGM_REG_COUNTER(&pStats->StatPageMapTlbFlushEntry, "/PGM/R3/Page/MapTlbFlushEntry", "TLB entry flushes (all contexts).");
PGM_REG_PROFILE(&pStats->StatRZSyncCR3HandlerVirtualUpdate, "/PGM/RZ/SyncCR3/Handlers/VirtualUpdate", "Profiling of the virtual handler updates.");
PGM_REG_PROFILE(&pStats->StatRZSyncCR3HandlerVirtualReset, "/PGM/RZ/SyncCR3/Handlers/VirtualReset", "Profiling of the virtual handler resets.");
PGM_REG_PROFILE(&pStats->StatR3SyncCR3HandlerVirtualUpdate, "/PGM/R3/SyncCR3/Handlers/VirtualUpdate", "Profiling of the virtual handler updates.");
PGM_REG_PROFILE(&pStats->StatR3SyncCR3HandlerVirtualReset, "/PGM/R3/SyncCR3/Handlers/VirtualReset", "Profiling of the virtual handler resets.");
PGM_REG_COUNTER(&pStats->StatRZPhysHandlerReset, "/PGM/RZ/PhysHandlerReset", "The number of times PGMHandlerPhysicalReset is called.");
PGM_REG_COUNTER(&pStats->StatR3PhysHandlerReset, "/PGM/R3/PhysHandlerReset", "The number of times PGMHandlerPhysicalReset is called.");
PGM_REG_COUNTER(&pStats->StatRZPhysHandlerLookupHits, "/PGM/RZ/PhysHandlerLookupHits", "The number of cache hits when looking up physical handlers.");
PGM_REG_COUNTER(&pStats->StatR3PhysHandlerLookupHits, "/PGM/R3/PhysHandlerLookupHits", "The number of cache hits when looking up physical handlers.");
PGM_REG_COUNTER(&pStats->StatRZPhysHandlerLookupMisses, "/PGM/RZ/PhysHandlerLookupMisses", "The number of cache misses when looking up physical handlers.");
PGM_REG_COUNTER(&pStats->StatR3PhysHandlerLookupMisses, "/PGM/R3/PhysHandlerLookupMisses", "The number of cache misses when looking up physical handlers.");
PGM_REG_PROFILE(&pStats->StatRZVirtHandlerSearchByPhys, "/PGM/RZ/VirtHandlerSearchByPhys", "Profiling of pgmHandlerVirtualFindByPhysAddr.");
PGM_REG_PROFILE(&pStats->StatR3VirtHandlerSearchByPhys, "/PGM/R3/VirtHandlerSearchByPhys", "Profiling of pgmHandlerVirtualFindByPhysAddr.");
PGM_REG_COUNTER(&pStats->StatRZPageReplaceShared, "/PGM/RZ/Page/ReplacedShared", "Times a shared page was replaced.");
PGM_REG_COUNTER(&pStats->StatRZPageReplaceZero, "/PGM/RZ/Page/ReplacedZero", "Times the zero page was replaced.");
/// @todo PGM_REG_COUNTER(&pStats->StatRZPageHandyAllocs, "/PGM/RZ/Page/HandyAllocs", "Number of times we've allocated more handy pages.");
PGM_REG_COUNTER(&pStats->StatR3PageReplaceShared, "/PGM/R3/Page/ReplacedShared", "Times a shared page was replaced.");
PGM_REG_COUNTER(&pStats->StatR3PageReplaceZero, "/PGM/R3/Page/ReplacedZero", "Times the zero page was replaced.");
/// @todo PGM_REG_COUNTER(&pStats->StatR3PageHandyAllocs, "/PGM/R3/Page/HandyAllocs", "Number of times we've allocated more handy pages.");
PGM_REG_COUNTER(&pStats->StatRZPhysRead, "/PGM/RZ/Phys/Read", "The number of times PGMPhysRead was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRZPhysReadBytes, "/PGM/RZ/Phys/Read/Bytes", "The number of bytes read by PGMPhysRead.");
PGM_REG_COUNTER(&pStats->StatRZPhysWrite, "/PGM/RZ/Phys/Write", "The number of times PGMPhysWrite was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRZPhysWriteBytes, "/PGM/RZ/Phys/Write/Bytes", "The number of bytes written by PGMPhysWrite.");
PGM_REG_COUNTER(&pStats->StatRZPhysSimpleRead, "/PGM/RZ/Phys/Simple/Read", "The number of times PGMPhysSimpleReadGCPtr was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRZPhysSimpleReadBytes, "/PGM/RZ/Phys/Simple/Read/Bytes", "The number of bytes read by PGMPhysSimpleReadGCPtr.");
PGM_REG_COUNTER(&pStats->StatRZPhysSimpleWrite, "/PGM/RZ/Phys/Simple/Write", "The number of times PGMPhysSimpleWriteGCPtr was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRZPhysSimpleWriteBytes, "/PGM/RZ/Phys/Simple/Write/Bytes", "The number of bytes written by PGMPhysSimpleWriteGCPtr.");
/* GC only: */
PGM_REG_COUNTER(&pStats->StatRCInvlPgConflict, "/PGM/RC/InvlPgConflict", "Number of times PGMInvalidatePage() detected a mapping conflict.");
PGM_REG_COUNTER(&pStats->StatRCInvlPgSyncMonCR3, "/PGM/RC/InvlPgSyncMonitorCR3", "Number of times PGMInvalidatePage() ran into PGM_SYNC_MONITOR_CR3.");
PGM_REG_COUNTER(&pStats->StatRCPhysRead, "/PGM/RC/Phys/Read", "The number of times PGMPhysRead was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRCPhysReadBytes, "/PGM/RC/Phys/Read/Bytes", "The number of bytes read by PGMPhysRead.");
PGM_REG_COUNTER(&pStats->StatRCPhysWrite, "/PGM/RC/Phys/Write", "The number of times PGMPhysWrite was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRCPhysWriteBytes, "/PGM/RC/Phys/Write/Bytes", "The number of bytes written by PGMPhysWrite.");
PGM_REG_COUNTER(&pStats->StatRCPhysSimpleRead, "/PGM/RC/Phys/Simple/Read", "The number of times PGMPhysSimpleReadGCPtr was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRCPhysSimpleReadBytes, "/PGM/RC/Phys/Simple/Read/Bytes", "The number of bytes read by PGMPhysSimpleReadGCPtr.");
PGM_REG_COUNTER(&pStats->StatRCPhysSimpleWrite, "/PGM/RC/Phys/Simple/Write", "The number of times PGMPhysSimpleWriteGCPtr was called.");
PGM_REG_COUNTER_BYTES(&pStats->StatRCPhysSimpleWriteBytes, "/PGM/RC/Phys/Simple/Write/Bytes", "The number of bytes written by PGMPhysSimpleWriteGCPtr.");
PGM_REG_COUNTER(&pStats->StatTrackVirgin, "/PGM/Track/Virgin", "The number of first time shadowings");
PGM_REG_COUNTER(&pStats->StatTrackAliased, "/PGM/Track/Aliased", "The number of times switching to cRef2, i.e. the page is being shadowed by two PTs.");
PGM_REG_COUNTER(&pStats->StatTrackAliasedMany, "/PGM/Track/AliasedMany", "The number of times we're tracking using cRef2.");
PGM_REG_COUNTER(&pStats->StatTrackAliasedLots, "/PGM/Track/AliasedLots", "The number of times we're hitting pages which has overflowed cRef2");
PGM_REG_COUNTER(&pStats->StatTrackOverflows, "/PGM/Track/Overflows", "The number of times the extent list grows too long.");
PGM_REG_COUNTER(&pStats->StatTrackNoExtentsLeft, "/PGM/Track/NoExtentLeft", "The number of times the extent list was exhausted.");
PGM_REG_PROFILE(&pStats->StatTrackDeref, "/PGM/Track/Deref", "Profiling of SyncPageWorkerTrackDeref (expensive).");
#endif
/*
* Note! The layout below matches the member layout exactly!
*/
/*
* Common - stats
*/
{
#define PGM_REG_COUNTER(a, b, c) \
rc = STAMR3RegisterF(pVM, a, STAMTYPE_COUNTER, STAMVISIBILITY_ALWAYS, STAMUNIT_OCCURENCES, c, b, idCpu); \
#define PGM_REG_PROFILE(a, b, c) \
rc = STAMR3RegisterF(pVM, a, STAMTYPE_PROFILE, STAMVISIBILITY_ALWAYS, STAMUNIT_TICKS_PER_CALL, c, b, idCpu); \
PGM_REG_COUNTER(&pPgmCpu->cGuestModeChanges, "/PGM/CPU%u/cGuestModeChanges", "Number of guest mode changes.");
#ifdef VBOX_WITH_STATISTICS
# if 0 /* rarely useful; leave for debugging. */
STAMR3RegisterF(pVM, &pCpuStats->StatSyncPtPD[i], STAMTYPE_COUNTER, STAMVISIBILITY_USED, STAMUNIT_OCCURENCES,
STAMR3RegisterF(pVM, &pCpuStats->StatSyncPagePD[i], STAMTYPE_COUNTER, STAMVISIBILITY_USED, STAMUNIT_OCCURENCES,
"The number of SyncPage per PD n.", "/PGM/CPU%u/PDSyncPage/%04X", i, j);
# endif
/* R0 only: */
PGM_REG_PROFILE(&pCpuStats->StatR0NpMiscfg, "/PGM/CPU%u/R0/NpMiscfg", "PGMR0Trap0eHandlerNPMisconfig() profiling.");
PGM_REG_COUNTER(&pCpuStats->StatR0NpMiscfgSyncPage, "/PGM/CPU%u/R0/NpMiscfgSyncPage", "SyncPage calls from PGMR0Trap0eHandlerNPMisconfig().");
/* RZ only: */
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0e, "/PGM/CPU%u/RZ/Trap0e", "Profiling of the PGMTrap0eHandler() body.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2Ballooned, "/PGM/CPU%u/RZ/Trap0e/Time2/Ballooned", "Profiling of the Trap0eHandler body when the cause is read access to a ballooned page.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2CSAM, "/PGM/CPU%u/RZ/Trap0e/Time2/CSAM", "Profiling of the Trap0eHandler body when the cause is CSAM.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2DirtyAndAccessed, "/PGM/CPU%u/RZ/Trap0e/Time2/DirtyAndAccessedBits", "Profiling of the Trap0eHandler body when the cause is dirty and/or accessed bit emulation.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2GuestTrap, "/PGM/CPU%u/RZ/Trap0e/Time2/GuestTrap", "Profiling of the Trap0eHandler body when the cause is a guest trap.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2HndPhys, "/PGM/CPU%u/RZ/Trap0e/Time2/HandlerPhysical", "Profiling of the Trap0eHandler body when the cause is a physical handler.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2HndVirt, "/PGM/CPU%u/RZ/Trap0e/Time2/HandlerVirtual", "Profiling of the Trap0eHandler body when the cause is a virtual handler.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2HndUnhandled, "/PGM/CPU%u/RZ/Trap0e/Time2/HandlerUnhandled", "Profiling of the Trap0eHandler body when the cause is access outside the monitored areas of a monitored page.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2InvalidPhys, "/PGM/CPU%u/RZ/Trap0e/Time2/InvalidPhys", "Profiling of the Trap0eHandler body when the cause is access to an invalid physical guest address.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2MakeWritable, "/PGM/CPU%u/RZ/Trap0e/Time2/MakeWritable", "Profiling of the Trap0eHandler body when the cause is that a page needed to be made writeable.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2Mapping, "/PGM/CPU%u/RZ/Trap0e/Time2/Mapping", "Profiling of the Trap0eHandler body when the cause is related to the guest mappings.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2Misc, "/PGM/CPU%u/RZ/Trap0e/Time2/Misc", "Profiling of the Trap0eHandler body when the cause is not known.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2OutOfSync, "/PGM/CPU%u/RZ/Trap0e/Time2/OutOfSync", "Profiling of the Trap0eHandler body when the cause is an out-of-sync page.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2OutOfSyncHndPhys, "/PGM/CPU%u/RZ/Trap0e/Time2/OutOfSyncHndPhys", "Profiling of the Trap0eHandler body when the cause is an out-of-sync physical handler page.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2OutOfSyncHndVirt, "/PGM/CPU%u/RZ/Trap0e/Time2/OutOfSyncHndVirt", "Profiling of the Trap0eHandler body when the cause is an out-of-sync virtual handler page.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2OutOfSyncHndObs, "/PGM/CPU%u/RZ/Trap0e/Time2/OutOfSyncObsHnd", "Profiling of the Trap0eHandler body when the cause is an obsolete handler page.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2SyncPT, "/PGM/CPU%u/RZ/Trap0e/Time2/SyncPT", "Profiling of the Trap0eHandler body when the cause is lazy syncing of a PT.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2WPEmulation, "/PGM/CPU%u/RZ/Trap0e/Time2/WPEmulation", "Profiling of the Trap0eHandler body when the cause is CR0.WP emulation.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2Wp0RoUsHack, "/PGM/CPU%u/RZ/Trap0e/Time2/WP0R0USHack", "Profiling of the Trap0eHandler body when the cause is CR0.WP and netware hack to be enabled.");
PGM_REG_PROFILE(&pCpuStats->StatRZTrap0eTime2Wp0RoUsUnhack, "/PGM/CPU%u/RZ/Trap0e/Time2/WP0R0USUnhack", "Profiling of the Trap0eHandler body when the cause is CR0.WP and netware hack to be disabled.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eConflicts, "/PGM/CPU%u/RZ/Trap0e/Conflicts", "The number of times #PF was caused by an undetected conflict.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersMapping, "/PGM/CPU%u/RZ/Trap0e/Handlers/Mapping", "Number of traps due to access handlers in mappings.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersOutOfSync, "/PGM/CPU%u/RZ/Trap0e/Handlers/OutOfSync", "Number of traps due to out-of-sync handled pages.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersPhysAll, "/PGM/CPU%u/RZ/Trap0e/Handlers/PhysAll", "Number of traps due to physical all-access handlers.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersPhysAllOpt, "/PGM/CPU%u/RZ/Trap0e/Handlers/PhysAllOpt", "Number of the physical all-access handler traps using the optimization.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersPhysWrite, "/PGM/CPU%u/RZ/Trap0e/Handlers/PhysWrite", "Number of traps due to physical write-access handlers.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersVirtual, "/PGM/CPU%u/RZ/Trap0e/Handlers/Virtual", "Number of traps due to virtual access handlers.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersVirtualByPhys, "/PGM/CPU%u/RZ/Trap0e/Handlers/VirtualByPhys", "Number of traps due to virtual access handlers by physical address.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersVirtualUnmarked,"/PGM/CPU%u/RZ/Trap0e/Handlers/VirtualUnmarked","Number of traps due to virtual access handlers by virtual address (without proper physical flags).");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersUnhandled, "/PGM/CPU%u/RZ/Trap0e/Handlers/Unhandled", "Number of traps due to access outside range of monitored page(s).");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eHandlersInvalid, "/PGM/CPU%u/RZ/Trap0e/Handlers/Invalid", "Number of traps due to access to invalid physical memory.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eUSNotPresentRead, "/PGM/CPU%u/RZ/Trap0e/Err/User/NPRead", "Number of user mode not present read page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eUSNotPresentWrite, "/PGM/CPU%u/RZ/Trap0e/Err/User/NPWrite", "Number of user mode not present write page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eUSWrite, "/PGM/CPU%u/RZ/Trap0e/Err/User/Write", "Number of user mode write page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eUSReserved, "/PGM/CPU%u/RZ/Trap0e/Err/User/Reserved", "Number of user mode reserved bit page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eUSNXE, "/PGM/CPU%u/RZ/Trap0e/Err/User/NXE", "Number of user mode NXE page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eUSRead, "/PGM/CPU%u/RZ/Trap0e/Err/User/Read", "Number of user mode read page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eSVNotPresentRead, "/PGM/CPU%u/RZ/Trap0e/Err/Supervisor/NPRead", "Number of supervisor mode not present read page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eSVNotPresentWrite, "/PGM/CPU%u/RZ/Trap0e/Err/Supervisor/NPWrite", "Number of supervisor mode not present write page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eSVWrite, "/PGM/CPU%u/RZ/Trap0e/Err/Supervisor/Write", "Number of supervisor mode write page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eSVReserved, "/PGM/CPU%u/RZ/Trap0e/Err/Supervisor/Reserved", "Number of supervisor mode reserved bit page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eSNXE, "/PGM/CPU%u/RZ/Trap0e/Err/Supervisor/NXE", "Number of supervisor mode NXE page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eGuestPF, "/PGM/CPU%u/RZ/Trap0e/GuestPF", "Number of real guest page faults.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eGuestPFMapping, "/PGM/CPU%u/RZ/Trap0e/GuestPF/InMapping", "Number of real guest page faults in a mapping.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eWPEmulInRZ, "/PGM/CPU%u/RZ/Trap0e/WP/InRZ", "Number of guest page faults due to X86_CR0_WP emulation.");
PGM_REG_COUNTER(&pCpuStats->StatRZTrap0eWPEmulToR3, "/PGM/CPU%u/RZ/Trap0e/WP/ToR3", "Number of guest page faults due to X86_CR0_WP emulation (forward to R3 for emulation).");
#if 0 /* rarely useful; leave for debugging. */
STAMR3RegisterF(pVM, &pCpuStats->StatRZTrap0ePD[i], STAMTYPE_COUNTER, STAMVISIBILITY_USED, STAMUNIT_OCCURENCES,
#endif
PGM_REG_COUNTER(&pCpuStats->StatRZGuestCR3WriteHandled, "/PGM/CPU%u/RZ/CR3WriteHandled", "The number of times the Guest CR3 change was successfully handled.");
PGM_REG_COUNTER(&pCpuStats->StatRZGuestCR3WriteUnhandled, "/PGM/CPU%u/RZ/CR3WriteUnhandled", "The number of times the Guest CR3 change was passed back to the recompiler.");
PGM_REG_COUNTER(&pCpuStats->StatRZGuestCR3WriteConflict, "/PGM/CPU%u/RZ/CR3WriteConflict", "The number of times the Guest CR3 monitoring detected a conflict.");
PGM_REG_COUNTER(&pCpuStats->StatRZGuestROMWriteHandled, "/PGM/CPU%u/RZ/ROMWriteHandled", "The number of times the Guest ROM change was successfully handled.");
PGM_REG_COUNTER(&pCpuStats->StatRZGuestROMWriteUnhandled, "/PGM/CPU%u/RZ/ROMWriteUnhandled", "The number of times the Guest ROM change was passed back to the recompiler.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapMigrateInvlPg, "/PGM/CPU%u/RZ/DynMap/MigrateInvlPg", "invlpg count in PGMR0DynMapMigrateAutoSet.");
PGM_REG_PROFILE(&pCpuStats->StatRZDynMapGCPageInl, "/PGM/CPU%u/RZ/DynMap/PageGCPageInl", "Calls to pgmR0DynMapGCPageInlined.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapGCPageInlHits, "/PGM/CPU%u/RZ/DynMap/PageGCPageInl/Hits", "Hash table lookup hits.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapGCPageInlMisses, "/PGM/CPU%u/RZ/DynMap/PageGCPageInl/Misses", "Misses that falls back to the code common.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapGCPageInlRamHits, "/PGM/CPU%u/RZ/DynMap/PageGCPageInl/RamHits", "1st ram range hits.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapGCPageInlRamMisses, "/PGM/CPU%u/RZ/DynMap/PageGCPageInl/RamMisses", "1st ram range misses, takes slow path.");
PGM_REG_PROFILE(&pCpuStats->StatRZDynMapHCPageInl, "/PGM/CPU%u/RZ/DynMap/PageHCPageInl", "Calls to pgmRZDynMapHCPageInlined.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapHCPageInlHits, "/PGM/CPU%u/RZ/DynMap/PageHCPageInl/Hits", "Hash table lookup hits.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapHCPageInlMisses, "/PGM/CPU%u/RZ/DynMap/PageHCPageInl/Misses", "Misses that falls back to the code common.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPage, "/PGM/CPU%u/RZ/DynMap/Page", "Calls to pgmR0DynMapPage");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapSetOptimize, "/PGM/CPU%u/RZ/DynMap/Page/SetOptimize", "Calls to pgmRZDynMapOptimizeAutoSet.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapSetSearchFlushes, "/PGM/CPU%u/RZ/DynMap/Page/SetSearchFlushes", "Set search restoring to subset flushes.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapSetSearchHits, "/PGM/CPU%u/RZ/DynMap/Page/SetSearchHits", "Set search hits.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapSetSearchMisses, "/PGM/CPU%u/RZ/DynMap/Page/SetSearchMisses", "Set search misses.");
PGM_REG_PROFILE(&pCpuStats->StatRZDynMapHCPage, "/PGM/CPU%u/RZ/DynMap/Page/HCPage", "Calls to pgmRZDynMapHCPageCommon (ring-0).");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageHits0, "/PGM/CPU%u/RZ/DynMap/Page/Hits0", "Hits at iPage+0");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageHits1, "/PGM/CPU%u/RZ/DynMap/Page/Hits1", "Hits at iPage+1");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageHits2, "/PGM/CPU%u/RZ/DynMap/Page/Hits2", "Hits at iPage+2");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageInvlPg, "/PGM/CPU%u/RZ/DynMap/Page/InvlPg", "invlpg count in pgmR0DynMapPageSlow.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageSlow, "/PGM/CPU%u/RZ/DynMap/Page/Slow", "Calls to pgmR0DynMapPageSlow - subtract this from pgmR0DynMapPage to get 1st level hits.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageSlowLoopHits, "/PGM/CPU%u/RZ/DynMap/Page/SlowLoopHits" , "Hits in the loop path.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageSlowLoopMisses, "/PGM/CPU%u/RZ/DynMap/Page/SlowLoopMisses", "Misses in the loop path. NonLoopMisses = Slow - SlowLoopHit - SlowLoopMisses");
//PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPageSlowLostHits, "/PGM/CPU%u/R0/DynMap/Page/SlowLostHits", "Lost hits.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapSubsets, "/PGM/CPU%u/RZ/DynMap/Subsets", "Times PGMRZDynMapPushAutoSubset was called.");
PGM_REG_COUNTER(&pCpuStats->StatRZDynMapPopFlushes, "/PGM/CPU%u/RZ/DynMap/SubsetPopFlushes", "Times PGMRZDynMapPopAutoSubset flushes the subset.");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[0], "/PGM/CPU%u/RZ/DynMap/SetFilledPct000..09", "00-09% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[1], "/PGM/CPU%u/RZ/DynMap/SetFilledPct010..19", "10-19% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[2], "/PGM/CPU%u/RZ/DynMap/SetFilledPct020..29", "20-29% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[3], "/PGM/CPU%u/RZ/DynMap/SetFilledPct030..39", "30-39% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[4], "/PGM/CPU%u/RZ/DynMap/SetFilledPct040..49", "40-49% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[5], "/PGM/CPU%u/RZ/DynMap/SetFilledPct050..59", "50-59% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[6], "/PGM/CPU%u/RZ/DynMap/SetFilledPct060..69", "60-69% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[7], "/PGM/CPU%u/RZ/DynMap/SetFilledPct070..79", "70-79% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[8], "/PGM/CPU%u/RZ/DynMap/SetFilledPct080..89", "80-89% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[9], "/PGM/CPU%u/RZ/DynMap/SetFilledPct090..99", "90-99% filled (RC: min(set-size, dynmap-size))");
PGM_REG_COUNTER(&pCpuStats->aStatRZDynMapSetFilledPct[10], "/PGM/CPU%u/RZ/DynMap/SetFilledPct100", "100% filled (RC: min(set-size, dynmap-size))");
/* HC only: */
/* RZ & R3: */
PGM_REG_PROFILE(&pCpuStats->StatRZSyncCR3, "/PGM/CPU%u/RZ/SyncCR3", "Profiling of the PGMSyncCR3() body.");
PGM_REG_PROFILE(&pCpuStats->StatRZSyncCR3Handlers, "/PGM/CPU%u/RZ/SyncCR3/Handlers", "Profiling of the PGMSyncCR3() update handler section.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3Global, "/PGM/CPU%u/RZ/SyncCR3/Global", "The number of global CR3 syncs.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3NotGlobal, "/PGM/CPU%u/RZ/SyncCR3/NotGlobal", "The number of non-global CR3 syncs.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3DstCacheHit, "/PGM/CPU%u/RZ/SyncCR3/DstChacheHit", "The number of times we got some kind of a cache hit.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3DstFreed, "/PGM/CPU%u/RZ/SyncCR3/DstFreed", "The number of times we've had to free a shadow entry.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3DstFreedSrcNP, "/PGM/CPU%u/RZ/SyncCR3/DstFreedSrcNP", "The number of times we've had to free a shadow entry for which the source entry was not present.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3DstNotPresent, "/PGM/CPU%u/RZ/SyncCR3/DstNotPresent", "The number of times we've encountered a not present shadow entry for a present guest entry.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3DstSkippedGlobalPD, "/PGM/CPU%u/RZ/SyncCR3/DstSkippedGlobalPD", "The number of times a global page directory wasn't flushed.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncCR3DstSkippedGlobalPT, "/PGM/CPU%u/RZ/SyncCR3/DstSkippedGlobalPT", "The number of times a page table with only global entries wasn't flushed.");
PGM_REG_PROFILE(&pCpuStats->StatRZSyncPT, "/PGM/CPU%u/RZ/SyncPT", "Profiling of the pfnSyncPT() body.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncPTFailed, "/PGM/CPU%u/RZ/SyncPT/Failed", "The number of times pfnSyncPT() failed.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncPagePDNAs, "/PGM/CPU%u/RZ/SyncPagePDNAs", "The number of time we've marked a PD not present from SyncPage to virtualize the accessed bit.");
PGM_REG_COUNTER(&pCpuStats->StatRZSyncPagePDOutOfSync, "/PGM/CPU%u/RZ/SyncPagePDOutOfSync", "The number of time we've encountered an out-of-sync PD in SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatRZAccessedPage, "/PGM/CPU%u/RZ/AccessedPage", "The number of pages marked not present for accessed bit emulation.");
PGM_REG_PROFILE(&pCpuStats->StatRZDirtyBitTracking, "/PGM/CPU%u/RZ/DirtyPage", "Profiling the dirty bit tracking in CheckPageFault().");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtyPage, "/PGM/CPU%u/RZ/DirtyPage/Mark", "The number of pages marked read-only for dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtyPageBig, "/PGM/CPU%u/RZ/DirtyPage/MarkBig", "The number of 4MB pages marked read-only for dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtyPageSkipped, "/PGM/CPU%u/RZ/DirtyPage/Skipped", "The number of pages already dirty or readonly.");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtyPageTrap, "/PGM/CPU%u/RZ/DirtyPage/Trap", "The number of traps generated for dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtyPageStale, "/PGM/CPU%u/RZ/DirtyPage/Stale", "The number of traps generated for dirty bit tracking (stale tlb entries).");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtiedPage, "/PGM/CPU%u/RZ/DirtyPage/SetDirty", "The number of pages marked dirty because of write accesses.");
PGM_REG_COUNTER(&pCpuStats->StatRZDirtyTrackRealPF, "/PGM/CPU%u/RZ/DirtyPage/RealPF", "The number of real pages faults during dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatRZPageAlreadyDirty, "/PGM/CPU%u/RZ/DirtyPage/AlreadySet", "The number of pages already marked dirty because of write accesses.");
PGM_REG_PROFILE(&pCpuStats->StatRZInvalidatePage, "/PGM/CPU%u/RZ/InvalidatePage", "PGMInvalidatePage() profiling.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePage4KBPages, "/PGM/CPU%u/RZ/InvalidatePage/4KBPages", "The number of times PGMInvalidatePage() was called for a 4KB page.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePage4MBPages, "/PGM/CPU%u/RZ/InvalidatePage/4MBPages", "The number of times PGMInvalidatePage() was called for a 4MB page.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePage4MBPagesSkip, "/PGM/CPU%u/RZ/InvalidatePage/4MBPagesSkip","The number of times PGMInvalidatePage() skipped a 4MB page.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePagePDMappings, "/PGM/CPU%u/RZ/InvalidatePage/PDMappings", "The number of times PGMInvalidatePage() was called for a page directory containing mappings (no conflict).");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePagePDNAs, "/PGM/CPU%u/RZ/InvalidatePage/PDNAs", "The number of times PGMInvalidatePage() was called for a not accessed page directory.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePagePDNPs, "/PGM/CPU%u/RZ/InvalidatePage/PDNPs", "The number of times PGMInvalidatePage() was called for a not present page directory.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePagePDOutOfSync, "/PGM/CPU%u/RZ/InvalidatePage/PDOutOfSync", "The number of times PGMInvalidatePage() was called for an out of sync page directory.");
PGM_REG_COUNTER(&pCpuStats->StatRZInvalidatePageSkipped, "/PGM/CPU%u/RZ/InvalidatePage/Skipped", "The number of times PGMInvalidatePage() was skipped due to not present shw or pending pending SyncCR3.");
PGM_REG_COUNTER(&pCpuStats->StatRZPageOutOfSyncSupervisor, "/PGM/CPU%u/RZ/OutOfSync/SuperVisor", "Number of traps due to pages out of sync (P) and times VerifyAccessSyncPage calls SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatRZPageOutOfSyncUser, "/PGM/CPU%u/RZ/OutOfSync/User", "Number of traps due to pages out of sync (P) and times VerifyAccessSyncPage calls SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatRZPageOutOfSyncSupervisorWrite,"/PGM/CPU%u/RZ/OutOfSync/SuperVisorWrite", "Number of traps due to pages out of sync (RW) and times VerifyAccessSyncPage calls SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatRZPageOutOfSyncUserWrite, "/PGM/CPU%u/RZ/OutOfSync/UserWrite", "Number of traps due to pages out of sync (RW) and times VerifyAccessSyncPage calls SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatRZPageOutOfSyncBallloon, "/PGM/CPU%u/RZ/OutOfSync/Balloon", "The number of times a ballooned page was accessed (read).");
PGM_REG_PROFILE(&pCpuStats->StatRZPrefetch, "/PGM/CPU%u/RZ/Prefetch", "PGMPrefetchPage profiling.");
PGM_REG_PROFILE(&pCpuStats->StatRZFlushTLB, "/PGM/CPU%u/RZ/FlushTLB", "Profiling of the PGMFlushTLB() body.");
PGM_REG_COUNTER(&pCpuStats->StatRZFlushTLBNewCR3, "/PGM/CPU%u/RZ/FlushTLB/NewCR3", "The number of times PGMFlushTLB was called with a new CR3, non-global. (switch)");
PGM_REG_COUNTER(&pCpuStats->StatRZFlushTLBNewCR3Global, "/PGM/CPU%u/RZ/FlushTLB/NewCR3Global", "The number of times PGMFlushTLB was called with a new CR3, global. (switch)");
PGM_REG_COUNTER(&pCpuStats->StatRZFlushTLBSameCR3, "/PGM/CPU%u/RZ/FlushTLB/SameCR3", "The number of times PGMFlushTLB was called with the same CR3, non-global. (flush)");
PGM_REG_COUNTER(&pCpuStats->StatRZFlushTLBSameCR3Global, "/PGM/CPU%u/RZ/FlushTLB/SameCR3Global", "The number of times PGMFlushTLB was called with the same CR3, global. (flush)");
PGM_REG_PROFILE(&pCpuStats->StatRZGstModifyPage, "/PGM/CPU%u/RZ/GstModifyPage", "Profiling of the PGMGstModifyPage() body.");
PGM_REG_PROFILE(&pCpuStats->StatR3SyncCR3, "/PGM/CPU%u/R3/SyncCR3", "Profiling of the PGMSyncCR3() body.");
PGM_REG_PROFILE(&pCpuStats->StatR3SyncCR3Handlers, "/PGM/CPU%u/R3/SyncCR3/Handlers", "Profiling of the PGMSyncCR3() update handler section.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3Global, "/PGM/CPU%u/R3/SyncCR3/Global", "The number of global CR3 syncs.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3NotGlobal, "/PGM/CPU%u/R3/SyncCR3/NotGlobal", "The number of non-global CR3 syncs.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3DstCacheHit, "/PGM/CPU%u/R3/SyncCR3/DstChacheHit", "The number of times we got some kind of a cache hit.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3DstFreed, "/PGM/CPU%u/R3/SyncCR3/DstFreed", "The number of times we've had to free a shadow entry.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3DstFreedSrcNP, "/PGM/CPU%u/R3/SyncCR3/DstFreedSrcNP", "The number of times we've had to free a shadow entry for which the source entry was not present.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3DstNotPresent, "/PGM/CPU%u/R3/SyncCR3/DstNotPresent", "The number of times we've encountered a not present shadow entry for a present guest entry.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3DstSkippedGlobalPD, "/PGM/CPU%u/R3/SyncCR3/DstSkippedGlobalPD", "The number of times a global page directory wasn't flushed.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncCR3DstSkippedGlobalPT, "/PGM/CPU%u/R3/SyncCR3/DstSkippedGlobalPT", "The number of times a page table with only global entries wasn't flushed.");
PGM_REG_PROFILE(&pCpuStats->StatR3SyncPT, "/PGM/CPU%u/R3/SyncPT", "Profiling of the pfnSyncPT() body.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncPTFailed, "/PGM/CPU%u/R3/SyncPT/Failed", "The number of times pfnSyncPT() failed.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncPagePDNAs, "/PGM/CPU%u/R3/SyncPagePDNAs", "The number of time we've marked a PD not present from SyncPage to virtualize the accessed bit.");
PGM_REG_COUNTER(&pCpuStats->StatR3SyncPagePDOutOfSync, "/PGM/CPU%u/R3/SyncPagePDOutOfSync", "The number of time we've encountered an out-of-sync PD in SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatR3AccessedPage, "/PGM/CPU%u/R3/AccessedPage", "The number of pages marked not present for accessed bit emulation.");
PGM_REG_PROFILE(&pCpuStats->StatR3DirtyBitTracking, "/PGM/CPU%u/R3/DirtyPage", "Profiling the dirty bit tracking in CheckPageFault().");
PGM_REG_COUNTER(&pCpuStats->StatR3DirtyPage, "/PGM/CPU%u/R3/DirtyPage/Mark", "The number of pages marked read-only for dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatR3DirtyPageBig, "/PGM/CPU%u/R3/DirtyPage/MarkBig", "The number of 4MB pages marked read-only for dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatR3DirtyPageSkipped, "/PGM/CPU%u/R3/DirtyPage/Skipped", "The number of pages already dirty or readonly.");
PGM_REG_COUNTER(&pCpuStats->StatR3DirtyPageTrap, "/PGM/CPU%u/R3/DirtyPage/Trap", "The number of traps generated for dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatR3DirtiedPage, "/PGM/CPU%u/R3/DirtyPage/SetDirty", "The number of pages marked dirty because of write accesses.");
PGM_REG_COUNTER(&pCpuStats->StatR3DirtyTrackRealPF, "/PGM/CPU%u/R3/DirtyPage/RealPF", "The number of real pages faults during dirty bit tracking.");
PGM_REG_COUNTER(&pCpuStats->StatR3PageAlreadyDirty, "/PGM/CPU%u/R3/DirtyPage/AlreadySet", "The number of pages already marked dirty because of write accesses.");
PGM_REG_PROFILE(&pCpuStats->StatR3InvalidatePage, "/PGM/CPU%u/R3/InvalidatePage", "PGMInvalidatePage() profiling.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePage4KBPages, "/PGM/CPU%u/R3/InvalidatePage/4KBPages", "The number of times PGMInvalidatePage() was called for a 4KB page.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePage4MBPages, "/PGM/CPU%u/R3/InvalidatePage/4MBPages", "The number of times PGMInvalidatePage() was called for a 4MB page.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePage4MBPagesSkip, "/PGM/CPU%u/R3/InvalidatePage/4MBPagesSkip","The number of times PGMInvalidatePage() skipped a 4MB page.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePagePDMappings, "/PGM/CPU%u/R3/InvalidatePage/PDMappings", "The number of times PGMInvalidatePage() was called for a page directory containing mappings (no conflict).");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePagePDNAs, "/PGM/CPU%u/R3/InvalidatePage/PDNAs", "The number of times PGMInvalidatePage() was called for a not accessed page directory.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePagePDNPs, "/PGM/CPU%u/R3/InvalidatePage/PDNPs", "The number of times PGMInvalidatePage() was called for a not present page directory.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePagePDOutOfSync, "/PGM/CPU%u/R3/InvalidatePage/PDOutOfSync", "The number of times PGMInvalidatePage() was called for an out of sync page directory.");
PGM_REG_COUNTER(&pCpuStats->StatR3InvalidatePageSkipped, "/PGM/CPU%u/R3/InvalidatePage/Skipped", "The number of times PGMInvalidatePage() was skipped due to not present shw or pending pending SyncCR3.");
PGM_REG_COUNTER(&pCpuStats->StatR3PageOutOfSyncSupervisor, "/PGM/CPU%u/R3/OutOfSync/SuperVisor", "Number of traps due to pages out of sync and times VerifyAccessSyncPage calls SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatR3PageOutOfSyncUser, "/PGM/CPU%u/R3/OutOfSync/User", "Number of traps due to pages out of sync and times VerifyAccessSyncPage calls SyncPage.");
PGM_REG_COUNTER(&pCpuStats->StatR3PageOutOfSyncBallloon, "/PGM/CPU%u/R3/OutOfSync/Balloon", "The number of times a ballooned page was accessed (read).");
PGM_REG_PROFILE(&pCpuStats->StatR3Prefetch, "/PGM/CPU%u/R3/Prefetch", "PGMPrefetchPage profiling.");
PGM_REG_PROFILE(&pCpuStats->StatR3FlushTLB, "/PGM/CPU%u/R3/FlushTLB", "Profiling of the PGMFlushTLB() body.");
PGM_REG_COUNTER(&pCpuStats->StatR3FlushTLBNewCR3, "/PGM/CPU%u/R3/FlushTLB/NewCR3", "The number of times PGMFlushTLB was called with a new CR3, non-global. (switch)");
PGM_REG_COUNTER(&pCpuStats->StatR3FlushTLBNewCR3Global, "/PGM/CPU%u/R3/FlushTLB/NewCR3Global", "The number of times PGMFlushTLB was called with a new CR3, global. (switch)");
PGM_REG_COUNTER(&pCpuStats->StatR3FlushTLBSameCR3, "/PGM/CPU%u/R3/FlushTLB/SameCR3", "The number of times PGMFlushTLB was called with the same CR3, non-global. (flush)");
PGM_REG_COUNTER(&pCpuStats->StatR3FlushTLBSameCR3Global, "/PGM/CPU%u/R3/FlushTLB/SameCR3Global", "The number of times PGMFlushTLB was called with the same CR3, global. (flush)");
PGM_REG_PROFILE(&pCpuStats->StatR3GstModifyPage, "/PGM/CPU%u/R3/GstModifyPage", "Profiling of the PGMGstModifyPage() body.");
#endif /* VBOX_WITH_STATISTICS */
}
return VINF_SUCCESS;
}
/**
* Init the PGM bits that rely on VMMR0 and MM to be fully initialized.
*
* The dynamic mapping area will also be allocated and initialized at this
* time. We could allocate it during PGMR3Init of course, but the mapping
* wouldn't be allocated at that time preventing us from setting up the
* page table entries with the dummy page.
*
* @returns VBox status code.
* @param pVM Pointer to the VM.
*/
{
int rc;
/*
* Reserve space for the dynamic mappings.
*/
if (RT_SUCCESS(rc))
if ( RT_SUCCESS(rc)
&& (pVM->pgm.s.pbDynPageMapBaseGC >> X86_PD_PAE_SHIFT) != ((pVM->pgm.s.pbDynPageMapBaseGC + MM_HYPER_DYNAMIC_SIZE - 1) >> X86_PD_PAE_SHIFT))
{
if (RT_SUCCESS(rc))
}
if (RT_SUCCESS(rc))
{
AssertRelease((pVM->pgm.s.pbDynPageMapBaseGC >> X86_PD_PAE_SHIFT) == ((pVM->pgm.s.pbDynPageMapBaseGC + MM_HYPER_DYNAMIC_SIZE - 1) >> X86_PD_PAE_SHIFT));
}
return rc;
}
/**
* Ring-3 init finalizing.
*
* @returns VBox status code.
* @param pVM Pointer to the VM.
*/
{
int rc;
/*
* Reserve space for the dynamic mappings.
* Initialize the dynamic mapping pages with dummy pages to simply the cache.
*/
/* get the pointer to the page table entries. */
pVM->pgm.s.paDynPageMap32BitPTEsGC = pMapping->aPTs[iPT].pPTRC + iPG * sizeof(pMapping->aPTs[0].pPTR3->a[0]);
pVM->pgm.s.paDynPageMapPaePTEsGC = pMapping->aPTs[iPT].paPaePTsRC + iPG * sizeof(pMapping->aPTs[0].paPaePTsR3->a[0]);
/* init cache area */
{
}
/*
* Determine the max physical address width (MAXPHYADDR) and apply it to
* all the mask members and stuff.
*/
if ( uMaxExtLeaf >= 0x80000008
&& uMaxExtLeaf <= 0x80000fff)
{
}
else
{
cMaxPhysAddrWidth = 48;
}
/** @todo query from CPUM. */
/*
* Initialize the invalid paging entry masks, assuming NX is disabled.
*/
{
/** @todo The manuals are not entirely clear whether the physical
* address width is relevant. See table 5-9 in the intel
* manual vs the PDE4M descriptions. Write testcase (NP). */
pVCpu->pgm.s.fGst32BitMbzBigPdeMask = ((uint32_t)(fMbzPageFrameMask >> (32 - 13)) & X86_PDE4M_PG_HIGH_MASK)
pVCpu->pgm.s.fGst64ShadowedPteMask = X86_PTE_P | X86_PTE_RW | X86_PTE_US | X86_PTE_G | X86_PTE_A | X86_PTE_D;
}
/*
* Note that AMD uses all the 8 reserved bits for the address (so 40 bits in total);
* Intel only goes up to 36 bits, so we stick to 36 as well.
* Update: More recent intel manuals specifies 40 bits just like AMD.
*/
else
/*
* Allocate memory if we're supposed to do that.
*/
return rc;
}
/**
* Init phase completed callback.
*
* @returns VBox status code.
* @param pVM Pointer to the VM.
* @param enmWhat What has been completed.
* @thread EMT(0)
*/
{
switch (enmWhat)
{
case VMINITCOMPLETED_HM:
#ifdef VBOX_WITH_PCI_PASSTHROUGH
{
/*
* Report assignments to the IOMMU (hope that's good enough for now).
*/
{
}
}
#else
#endif
break;
default:
/* shut up gcc */
break;
}
return VINF_SUCCESS;
}
/**
* Applies relocations to data and code managed by this component.
*
* This function will be called at init and whenever the VMM need to relocate it
* self inside the GC.
*
* @param pVM The VM.
* @param offDelta Relocation delta relative to old location.
*/
{
LogFlow(("PGMR3Relocate %RGv to %RGv\n", pVM->pgm.s.GCPtrCR3Mapping, pVM->pgm.s.GCPtrCR3Mapping + offDelta));
/*
* Paging stuff.
*/
/* Shadow, guest and both mode switch & relocation for each VCPU. */
{
}
/*
* Trees.
*/
/*
* Ram ranges.
*/
{
/* Update the pSelfRC pointers and relink them. */
/* Flush the RC TLB. */
for (unsigned i = 0; i < PGM_RAMRANGE_TLB_ENTRIES; i++)
}
/*
* Update the pSelfRC pointer of the MMIO2 ram ranges since they might not
* be mapped and thus not included in the above exercise.
*/
/*
* Update the two page directories with all page table mappings.
* (One or more of them have changed, that's why we're here.)
*/
/* Relocate GC addresses of Page Tables. */
{
{
}
}
/*
* Dynamic page mapping area.
*/
{
{
}
}
/*
* The Zero page.
*/
#ifdef VBOX_WITH_2X_4GB_ADDR_SPACE
#else
#endif
/*
* Physical and virtual handlers.
*/
RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysHandlers, true, pgmR3RelocatePhysHandler, &offDelta);
RTAvlroGCPtrDoWithAll(&pVM->pgm.s.pTreesR3->VirtHandlers, true, pgmR3RelocateVirtHandler, &offDelta);
RTAvlroGCPtrDoWithAll(&pVM->pgm.s.pTreesR3->HyperVirtHandlers, true, pgmR3RelocateHyperVirtHandler, &offDelta);
/*
* The page pool.
*/
#ifdef VBOX_WITH_STATISTICS
/*
* Statistics.
*/
#endif
}
/**
* Callback function for relocating a physical access handler.
*
* @returns 0 (continue enum)
* @param pNode Pointer to a PGMPHYSHANDLER node.
* @param pvUser Pointer to the offDelta. This is a pointer to the delta since we're
* not certain the delta will fit in a void pointer for all possible configs.
*/
{
if (pHandler->pfnHandlerRC)
return 0;
}
/**
* Callback function for relocating a virtual access handler.
*
* @returns 0 (continue enum)
* @param pNode Pointer to a PGMVIRTHANDLER node.
* @param pvUser Pointer to the offDelta. This is a pointer to the delta since we're
* not certain the delta will fit in a void pointer for all possible configs.
*/
{
return 0;
}
/**
* Callback function for relocating a virtual access handler for the hypervisor mapping.
*
* @returns 0 (continue enum)
* @param pNode Pointer to a PGMVIRTHANDLER node.
* @param pvUser Pointer to the offDelta. This is a pointer to the delta since we're
* not certain the delta will fit in a void pointer for all possible configs.
*/
{
return 0;
}
/**
* Resets a virtual CPU when unplugged.
*
* @param pVM Pointer to the VM.
* @param pVCpu Pointer to the VMCPU.
*/
{
/*
* Re-init other members.
*/
/*
* Clear the FFs PGM owns.
*/
}
/**
* The VM is being reset.
*
* For the PGM component this means that any PD write monitors
* needs to be removed.
*
* @param pVM Pointer to the VM.
*/
{
LogFlow(("PGMR3Reset:\n"));
/*
* Unfix any fixed mappings and disable CR3 monitoring.
*/
/*
* Exit the guest paging mode before the pgm pool gets reset.
* Important to clean up the amd64 case.
*/
{
}
#ifdef DEBUG
#endif
/*
* Switch mode back to real mode. (before resetting the pgm pool!)
*/
{
}
/*
* Reset the shadow page pool.
*/
/*
* Re-init various other members and clear the FFs that PGM owns.
*/
{
PGMNotifyNxeChanged(pVCpu, false);
{
#ifdef PGM_WITH_A20
#endif
}
}
}
/**
* Memory setup after VM construction or reset.
*
* @param pVM Pointer to the VM.
* @param fAtReset Indicates the context, after reset if @c true or after
* construction if @c false.
*/
{
if (fAtReset)
{
}
}
#ifdef VBOX_STRICT
/**
* VM state change callback for clearing fNoMorePhysWrites after
* a snapshot has been created.
*/
static DECLCALLBACK(void) pgmR3ResetNoMorePhysWritesFlag(PUVM pUVM, VMSTATE enmState, VMSTATE enmOldState, void *pvUser)
{
if ( enmState == VMSTATE_RUNNING
|| enmState == VMSTATE_RESUMING)
}
#endif
/**
* Private API to reset fNoMorePhysWrites.
*/
{
}
/**
* Terminates the PGM.
*
* @returns VBox status code.
* @param pVM Pointer to VM structure.
*/
{
/* Must free shared pages here. */
}
/**
* Show paging mode.
*
* @param pVM Pointer to the VM.
* @param pHlp The info helpers.
* @param pszArgs "all" (default), "guest", "shadow" or "host".
*/
{
/* digest argument. */
if (pszArgs)
else
{
fGuest = true;
fShadow = true;
fHost = true;
}
/** @todo SMP support! */
/* print info. */
if (fGuest)
pHlp->pfnPrintf(pHlp, "Guest paging mode: %s (changed %RU64 times), A20 %s (changed %RU64 times)\n",
if (fShadow)
pHlp->pfnPrintf(pHlp, "Shadow paging mode: %s\n", PGMGetModeName(pVM->aCpus[0].pgm.s.enmShadowMode));
if (fHost)
{
const char *psz;
{
default: psz = "unknown"; break;
}
}
}
/**
* Dump registered MMIO ranges to the log.
*
* @param pVM Pointer to the VM.
* @param pHlp The info helpers.
* @param pszArgs Arguments, ignored.
*/
{
"RAM ranges (pVM=%p)\n"
"%.*s %.*s\n",
pVM,
"%RGp-%RGp %RHv %s\n",
}
/**
* Dump the page directory to the log.
*
* @param pVM Pointer to the VM.
* @param pHlp The info helpers.
* @param pszArgs Arguments, ignored.
*/
{
/** @todo SMP support!! */
/** @todo fix this! Convert the PGMR3DumpHierarchyHC functions to do guest stuff. */
/* Big pages supported? */
/* Global pages supported? */
/*
* Get page directory addresses.
*/
/*
* Iterate the page directory.
*/
{
{
"%04X - %RGp P=%d U=%d RW=%d G=%d - BIG\n",
iPD,
else
"%04X - %RGp P=%d U=%d RW=%d [G=%d]\n",
iPD,
}
}
}
/**
* Service a VMMCALLRING3_PGM_LOCK call.
*
* @returns VBox status code.
* @param pVM Pointer to the VM.
*/
{
return rc;
}
/**
* Converts a PGMMODE value to a PGM_TYPE_* \#define.
*
* @returns PGM_TYPE_*.
* @param pgmMode The mode value to convert.
*/
{
switch (pgmMode)
{
case PGMMODE_REAL: return PGM_TYPE_REAL;
case PGMMODE_PROTECTED: return PGM_TYPE_PROT;
case PGMMODE_32_BIT: return PGM_TYPE_32BIT;
case PGMMODE_PAE:
case PGMMODE_PAE_NX: return PGM_TYPE_PAE;
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX: return PGM_TYPE_AMD64;
case PGMMODE_NESTED: return PGM_TYPE_NESTED;
case PGMMODE_EPT: return PGM_TYPE_EPT;
default:
}
}
/**
* Gets the index into the paging mode data array of a SHW+GST mode.
*
* @returns PGM::paPagingData index.
* @param uShwType The shadow paging mode type.
* @param uGstType The guest paging mode type.
*/
{
+ (uGstType - PGM_TYPE_REAL);
}
/**
* Gets the index into the paging mode data array of a SHW+GST mode.
*
* @returns PGM::paPagingData index.
* @param enmShw The shadow paging mode.
* @param enmGst The guest paging mode.
*/
{
}
/**
* Calculates the max data index.
* @returns The number of entries in the paging data array.
*/
{
}
/**
* Initializes the paging mode data kept in PGM::paModeData.
*
* @param pVM Pointer to the VM.
* @param fResolveGCAndR0 Indicate whether or not GC and Ring-0 symbols can be resolved now.
* This is used early in the init process to avoid trouble with PDM
* not being initialized yet.
*/
{
int rc;
/*
* Allocate the array on the first call.
*/
{
pVM->pgm.s.paModeData = (PPGMMODEDATA)MMR3HeapAllocZ(pVM, MM_TAG_PGM, sizeof(PGMMODEDATA) * pgmModeDataMaxIndex());
}
/*
* Initialize the array entries.
*/
#ifdef VBOX_WITH_64_BITS_GUESTS
#endif
/* The nested paging mode. */
#ifdef VBOX_WITH_64_BITS_GUESTS
#endif
/* The shadow part of the nested callback mode depends on the host paging mode (AMD-V only). */
{
#if HC_ARCH_BITS == 32
case SUPPAGINGMODE_32_BIT:
for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_PAE; i++)
{
}
# ifdef VBOX_WITH_64_BITS_GUESTS
# endif
break;
case SUPPAGINGMODE_PAE:
case SUPPAGINGMODE_PAE_NX:
case SUPPAGINGMODE_PAE_GLOBAL:
for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_PAE; i++)
{
}
# ifdef VBOX_WITH_64_BITS_GUESTS
# endif
break;
#endif /* HC_ARCH_BITS == 32 */
case SUPPAGINGMODE_AMD64:
case SUPPAGINGMODE_AMD64_NX:
# ifdef VBOX_WITH_64_BITS_GUESTS
for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_AMD64; i++)
# else
for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_PAE; i++)
# endif
{
}
break;
#endif /* HC_ARCH_BITS == 64 || RT_OS_DARWIN */
default:
AssertFailed();
break;
}
/* Extended paging (EPT) / Intel VT-x */
#ifdef VBOX_WITH_64_BITS_GUESTS
#endif
return VINF_SUCCESS;
}
/**
* Switch to different (or relocated in the relocate case) mode data.
*
* @param pVM Pointer to the VM.
* @param pVCpu Pointer to the VMCPU.
* @param enmShw The shadow paging mode.
* @param enmGst The guest paging mode.
*/
{
/* shadow */
/* guest */
/* both */
#ifdef VBOX_STRICT
#endif
#ifdef VBOX_STRICT
#endif
#ifdef VBOX_STRICT
#endif
}
/**
* Calculates the shadow paging mode.
*
* @returns The shadow paging mode.
* @param pVM Pointer to the VM.
* @param enmGuestMode The guest mode.
* @param enmHostMode The host mode.
* @param enmShadowMode The current shadow mode.
* @param penmSwitcher Where to store the switcher to use.
* VMMSWITCHER_INVALID means no change.
*/
static PGMMODE pgmR3CalcShadowMode(PVM pVM, PGMMODE enmGuestMode, SUPPAGINGMODE enmHostMode, PGMMODE enmShadowMode, VMMSWITCHER *penmSwitcher)
{
switch (enmGuestMode)
{
/*
* When switching to real or protected mode we don't change
* anything since it's likely that we'll switch back pretty soon.
*
* During pgmR3InitPaging we'll end up here with PGMMODE_INVALID
* and is supposed to determine which shadow paging and switcher to
* use during init.
*/
case PGMMODE_REAL:
case PGMMODE_PROTECTED:
if ( enmShadowMode != PGMMODE_INVALID
break; /* (no change) */
switch (enmHostMode)
{
case SUPPAGINGMODE_32_BIT:
break;
case SUPPAGINGMODE_PAE:
case SUPPAGINGMODE_PAE_NX:
case SUPPAGINGMODE_PAE_GLOBAL:
#ifdef DEBUG_bird
if (RTEnvExist("VBOX_32BIT"))
{
}
#endif
break;
case SUPPAGINGMODE_AMD64:
case SUPPAGINGMODE_AMD64_NX:
#ifdef DEBUG_bird
if (RTEnvExist("VBOX_32BIT"))
{
}
#endif
break;
}
break;
case PGMMODE_32_BIT:
switch (enmHostMode)
{
case SUPPAGINGMODE_32_BIT:
break;
case SUPPAGINGMODE_PAE:
case SUPPAGINGMODE_PAE_NX:
case SUPPAGINGMODE_PAE_GLOBAL:
#ifdef DEBUG_bird
if (RTEnvExist("VBOX_32BIT"))
{
}
#endif
break;
case SUPPAGINGMODE_AMD64:
case SUPPAGINGMODE_AMD64_NX:
#ifdef DEBUG_bird
if (RTEnvExist("VBOX_32BIT"))
{
}
#endif
break;
}
break;
case PGMMODE_PAE:
case PGMMODE_PAE_NX: /** @todo This might require more switchers and guest+both modes. */
switch (enmHostMode)
{
case SUPPAGINGMODE_32_BIT:
break;
case SUPPAGINGMODE_PAE:
case SUPPAGINGMODE_PAE_NX:
case SUPPAGINGMODE_PAE_GLOBAL:
break;
case SUPPAGINGMODE_AMD64:
case SUPPAGINGMODE_AMD64_NX:
break;
}
break;
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
switch (enmHostMode)
{
case SUPPAGINGMODE_32_BIT:
break;
case SUPPAGINGMODE_PAE:
case SUPPAGINGMODE_PAE_NX:
case SUPPAGINGMODE_PAE_GLOBAL:
break;
case SUPPAGINGMODE_AMD64:
case SUPPAGINGMODE_AMD64_NX:
break;
}
break;
default:
return PGMMODE_INVALID;
}
/* Override the shadow mode is nested paging is active. */
return enmShadowMode;
}
/**
* Performs the actual mode change.
* This is called by PGMChangeMode and pgmR3InitPaging().
*
* @returns VBox status code. May suspend or power off the VM on error, but this
* will trigger using FFs and not status codes.
*
* @param pVM Pointer to the VM.
* @param pVCpu Pointer to the VMCPU.
* @param enmGuestMode The new guest mode. This is assumed to be different from
* the current mode.
*/
{
#if HC_ARCH_BITS == 32
#endif
Log(("PGMR3ChangeMode: Guest mode: %s -> %s\n", PGMGetModeName(pVCpu->pgm.s.enmGuestMode), PGMGetModeName(enmGuestMode)));
/*
* Calc the shadow mode and switcher.
*/
enmShadowMode = pgmR3CalcShadowMode(pVM, enmGuestMode, pVM->pgm.s.enmHostMode, pVCpu->pgm.s.enmShadowMode, &enmSwitcher);
#ifdef VBOX_WITH_RAW_MODE
if ( enmSwitcher != VMMSWITCHER_INVALID
&& !HMIsEnabled(pVM))
{
/*
* Select new switcher.
*/
if (RT_FAILURE(rc))
{
return rc;
}
}
#endif
/*
* Exit old mode(s).
*/
#if HC_ARCH_BITS == 32
/* The nested shadow paging mode for AMD-V does change when running 64 bits guests on 32 bits hosts; typically PAE <-> AMD64 */
&& enmShadowMode == PGMMODE_NESTED);
#else
const bool fForceShwEnterExit = false;
#endif
/* shadow */
{
LogFlow(("PGMR3ChangeMode: Shadow mode: %s -> %s\n", PGMGetModeName(pVCpu->pgm.s.enmShadowMode), PGMGetModeName(enmShadowMode)));
{
if (RT_FAILURE(rc))
{
return rc;
}
}
}
else
LogFlow(("PGMR3ChangeMode: Shadow mode remains: %s\n", PGMGetModeName(pVCpu->pgm.s.enmShadowMode)));
/* guest */
{
if (RT_FAILURE(rc))
{
return rc;
}
}
/*
* Load new paging mode data.
*/
/*
* Enter new shadow mode (if changed).
*/
{
int rc;
switch (enmShadowMode)
{
case PGMMODE_32_BIT:
break;
case PGMMODE_PAE:
case PGMMODE_PAE_NX:
break;
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
break;
case PGMMODE_NESTED:
break;
case PGMMODE_EPT:
break;
case PGMMODE_REAL:
case PGMMODE_PROTECTED:
default:
return VERR_INTERNAL_ERROR;
}
if (RT_FAILURE(rc))
{
return rc;
}
}
/*
* Always flag the necessary updates
*/
/*
* Enter the new guest and shadow+guest modes.
*/
switch (enmGuestMode)
{
case PGMMODE_REAL:
{
case PGMMODE_32_BIT:
break;
case PGMMODE_PAE:
case PGMMODE_PAE_NX:
break;
case PGMMODE_NESTED:
break;
case PGMMODE_EPT:
break;
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
AssertMsgFailed(("Should use PAE shadow mode!\n"));
default: AssertFailed(); break;
}
break;
case PGMMODE_PROTECTED:
{
case PGMMODE_32_BIT:
break;
case PGMMODE_PAE:
case PGMMODE_PAE_NX:
break;
case PGMMODE_NESTED:
break;
case PGMMODE_EPT:
break;
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
AssertMsgFailed(("Should use PAE shadow mode!\n"));
default: AssertFailed(); break;
}
break;
case PGMMODE_32_BIT:
{
case PGMMODE_32_BIT:
break;
case PGMMODE_PAE:
case PGMMODE_PAE_NX:
break;
case PGMMODE_NESTED:
break;
case PGMMODE_EPT:
break;
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
AssertMsgFailed(("Should use PAE shadow mode!\n"));
default: AssertFailed(); break;
}
break;
case PGMMODE_PAE_NX:
case PGMMODE_PAE:
{
if (!(u32Features & X86_CPUID_FEATURE_EDX_PAE))
N_("The guest is trying to switch to the PAE mode which is currently disabled by default in VirtualBox. PAE support can be enabled using the VM settings (System/Processor)"));
{
case PGMMODE_PAE:
case PGMMODE_PAE_NX:
break;
case PGMMODE_NESTED:
break;
case PGMMODE_EPT:
break;
case PGMMODE_32_BIT:
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
AssertMsgFailed(("Should use PAE shadow mode!\n"));
default: AssertFailed(); break;
}
break;
}
#ifdef VBOX_WITH_64_BITS_GUESTS
case PGMMODE_AMD64_NX:
case PGMMODE_AMD64:
{
case PGMMODE_AMD64:
case PGMMODE_AMD64_NX:
break;
case PGMMODE_NESTED:
break;
case PGMMODE_EPT:
break;
case PGMMODE_32_BIT:
case PGMMODE_PAE:
case PGMMODE_PAE_NX:
AssertMsgFailed(("Should use AMD64 shadow mode!\n"));
default: AssertFailed(); break;
}
break;
#endif
default:
break;
}
/* status codes. */
if (RT_SUCCESS(rc))
{
rc = VINF_SUCCESS;
}
/* Notify HM as well. */
return rc;
}
/**
* Called by pgmPoolFlushAllInt prior to flushing the pool.
*
* @returns VBox status code, fully asserted.
* @param pVCpu Pointer to the VMCPU.
*/
{
/* Unmap the old CR3 value before flushing everything. */
/* Exit the current shadow paging mode as well; nested paging and EPT use a root CR3 which will get flushed here. */
return rc;
}
/**
* Called by pgmPoolFlushAllInt after flushing the pool.
*
* @returns VBox status code, fully asserted.
* @param pVM Pointer to the VM.
* @param pVCpu Pointer to the VMCPU.
*/
{
("%RHp != %RHp %s\n", (RTHCPHYS)CPUMGetHyperCR3(pVCpu), PGMGetHyperCR3(pVCpu), PGMGetModeName(pVCpu->pgm.s.enmShadowMode)));
return rc;
}
/**
* Called by PGMR3PhysSetA20 after changing the A20 state.
*
* @param pVCpu Pointer to the VMCPU.
*/
{
/** @todo Probably doing a bit too much here. */
}
#ifdef VBOX_WITH_DEBUGGER
/**
* @callback_method_impl{FNDBGCCMD, The '.pgmerror' and '.pgmerroroff' commands.}
*/
static DECLCALLBACK(int) pgmR3CmdError(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PUVM pUVM, PCDBGCVAR paArgs, unsigned cArgs)
{
/*
* Validate input.
*/
DBGC_CMDHLP_ASSERT_PARSER_RET(pCmdHlp, pCmd, 0, cArgs == 0 || (cArgs == 1 && paArgs[0].enmType == DBGCVAR_TYPE_STRING));
if (!cArgs)
{
/*
* Print the list of error injection locations with status.
*/
}
else
{
/*
* String switch on where to inject the error.
*/
else
}
return VINF_SUCCESS;
}
/**
* @callback_method_impl{FNDBGCCMD, The '.pgmsync' command.}
*/
static DECLCALLBACK(int) pgmR3CmdSync(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PUVM pUVM, PCDBGCVAR paArgs, unsigned cArgs)
{
/*
* Validate input.
*/
if (!pVCpu)
/*
* Force page directory sync.
*/
if (RT_FAILURE(rc))
return rc;
return VINF_SUCCESS;
}
#ifdef VBOX_STRICT
/**
* EMT callback for pgmR3CmdAssertCR3.
*
* @returns VBox status code.
* @param pUVM The user mode VM handle.
* @param pcErrors Where to return the error count.
*/
{
return VINF_SUCCESS;
}
/**
* @callback_method_impl{FNDBGCCMD, The '.pgmassertcr3' command.}
*/
static DECLCALLBACK(int) pgmR3CmdAssertCR3(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PUVM pUVM, PCDBGCVAR paArgs, unsigned cArgs)
{
/*
* Validate input.
*/
if (RT_FAILURE(rc))
return rc;
unsigned cErrors = 0;
rc = VMR3ReqCallWaitU(pUVM, DBGCCmdHlpGetCurrentCpu(pCmdHlp), (PFNRT)pgmR3CmdAssertCR3EmtWorker, 2, pUVM, &cErrors);
if (RT_FAILURE(rc))
if (cErrors > 0)
}
#endif /* VBOX_STRICT */
/**
* @callback_method_impl{FNDBGCCMD, The '.pgmsyncalways' command.}
*/
static DECLCALLBACK(int) pgmR3CmdSyncAlways(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PUVM pUVM, PCDBGCVAR paArgs, unsigned cArgs)
{
/*
* Validate input.
*/
if (!pVCpu)
/*
* Force page directory sync.
*/
int rc;
{
}
else
{
}
return rc;
}
/**
* @callback_method_impl{FNDBGCCMD, The '.pgmphystofile' command.}
*/
static DECLCALLBACK(int) pgmR3CmdPhysToFile(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PUVM pUVM, PCDBGCVAR paArgs, unsigned cArgs)
{
/*
* Validate input.
*/
if (cArgs == 2)
{
return DBGCCmdHlpFail(pCmdHlp, pCmd, "Invalid 2nd argument '%s', must be 'nozero'.\n", paArgs[1].u.pszString);
}
/*
* Open the output file and get the ram parameters.
*/
int rc = RTFileOpen(&hFile, paArgs[0].u.pszString, RTFILE_O_WRITE | RTFILE_O_CREATE_REPLACE | RTFILE_O_DENY_WRITE);
if (RT_FAILURE(rc))
return DBGCCmdHlpPrintf(pCmdHlp, "error: RTFileOpen(,'%s',) -> %Rrc.\n", paArgs[0].u.pszString, rc);
/*
* Dump the physical memory, page by page.
*/
{
/* fill the gap */
{
{
}
}
{
if ( PGM_PAGE_IS_ZERO(pPage)
{
if (fIncZeroPgs)
{
if (RT_FAILURE(rc))
}
}
else
{
switch (PGM_PAGE_GET_TYPE(pPage))
{
case PGMPAGETYPE_RAM:
case PGMPAGETYPE_ROM_SHADOW: /* trouble?? */
case PGMPAGETYPE_ROM:
case PGMPAGETYPE_MMIO2:
{
void const *pvPage;
if (RT_SUCCESS(rc))
{
if (RT_FAILURE(rc))
}
else
DBGCCmdHlpPrintf(pCmdHlp, "error: PGMPhysGCPhys2CCPtrReadOnly -> %Rrc at GCPhys=%RGp.\n", rc, GCPhys);
break;
}
default:
AssertFailed();
case PGMPAGETYPE_MMIO:
if (fIncZeroPgs)
{
if (RT_FAILURE(rc))
}
break;
}
}
/* advance */
pPage++;
}
}
if (RT_SUCCESS(rc))
return DBGCCmdHlpPrintf(pCmdHlp, "Successfully saved physical memory to '%s'.\n", paArgs[0].u.pszString);
return VINF_SUCCESS;
}
#endif /* VBOX_WITH_DEBUGGER */
/**
* pvUser argument of the pgmR3CheckIntegrity*Node callbacks.
*/
typedef struct PGMCHECKINTARGS
{
/**
* Validate a node in the physical handler tree.
*
* @returns 0 on if ok, other wise 1.
* @param pNode The handler node.
* @param pvUser pVM.
*/
static DECLCALLBACK(int) pgmR3CheckIntegrityPhysHandlerNode(PAVLROGCPHYSNODECORE pNode, void *pvUser)
{
AssertReleaseMsg(pCur->Core.Key <= pCur->Core.KeyLast,("pCur=%p %RGp-%RGp %s\n", pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc));
|| (pArgs->fLeftToRight ? pArgs->pPrevPhys->Core.KeyLast < pCur->Core.Key : pArgs->pPrevPhys->Core.KeyLast > pCur->Core.Key),
("pPrevPhys=%p %RGp-%RGp %s\n"
" pCur=%p %RGp-%RGp %s\n",
pArgs->pPrevPhys, pArgs->pPrevPhys->Core.Key, pArgs->pPrevPhys->Core.KeyLast, pArgs->pPrevPhys->pszDesc,
return 0;
}
/**
* Validate a node in the virtual handler tree.
*
* @returns 0 on if ok, other wise 1.
* @param pNode The handler node.
* @param pvUser pVM.
*/
static DECLCALLBACK(int) pgmR3CheckIntegrityVirtHandlerNode(PAVLROGCPTRNODECORE pNode, void *pvUser)
{
AssertReleaseMsg(pCur->Core.Key <= pCur->Core.KeyLast,("pCur=%p %RGv-%RGv %s\n", pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc));
|| (pArgs->fLeftToRight ? pArgs->pPrevVirt->Core.KeyLast < pCur->Core.Key : pArgs->pPrevVirt->Core.KeyLast > pCur->Core.Key),
("pPrevVirt=%p %RGv-%RGv %s\n"
" pCur=%p %RGv-%RGv %s\n",
pArgs->pPrevVirt, pArgs->pPrevVirt->Core.Key, pArgs->pPrevVirt->Core.KeyLast, pArgs->pPrevVirt->pszDesc,
{
AssertReleaseMsg(pCur->aPhysToVirt[iPage].offVirtHandler == -RT_OFFSETOF(PGMVIRTHANDLER, aPhysToVirt[iPage]),
("pCur=%p %RGv-%RGv %s\n"
"iPage=%d offVirtHandle=%#x expected %#x\n",
}
return 0;
}
/**
* Validate a node in the virtual handler tree.
*
* @returns 0 on if ok, other wise 1.
* @param pNode The handler node.
* @param pvUser pVM.
*/
static DECLCALLBACK(int) pgmR3CheckIntegrityPhysToVirtHandlerNode(PAVLROGCPHYSNODECORE pNode, void *pvUser)
{
AssertReleaseMsg(pCur->Core.Key <= pCur->Core.KeyLast,("pCur=%p %RGp-%RGp\n", pCur, pCur->Core.Key, pCur->Core.KeyLast));
|| (pArgs->fLeftToRight ? pArgs->pPrevPhys2Virt->Core.KeyLast < pCur->Core.Key : pArgs->pPrevPhys2Virt->Core.KeyLast > pCur->Core.Key),
("pPrevPhys2Virt=%p %RGp-%RGp\n"
" pCur=%p %RGp-%RGp\n",
|| (pArgs->fLeftToRight ? pArgs->pPrevPhys2Virt->Core.KeyLast < pCur->Core.Key : pArgs->pPrevPhys2Virt->Core.KeyLast > pCur->Core.Key),
("pPrevPhys2Virt=%p %RGp-%RGp\n"
" pCur=%p %RGp-%RGp\n",
AssertReleaseMsg((pCur->offNextAlias & (PGMPHYS2VIRTHANDLER_IN_TREE | PGMPHYS2VIRTHANDLER_IS_HEAD)) == (PGMPHYS2VIRTHANDLER_IN_TREE | PGMPHYS2VIRTHANDLER_IS_HEAD),
("pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
{
for (;;)
{
pCur2 = (PPGMPHYS2VIRTHANDLER)((intptr_t)pCur + (pCur->offNextAlias & PGMPHYS2VIRTHANDLER_OFF_MASK));
(" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
AssertReleaseMsg((pCur2->offNextAlias & (PGMPHYS2VIRTHANDLER_IN_TREE | PGMPHYS2VIRTHANDLER_IS_HEAD)) == PGMPHYS2VIRTHANDLER_IN_TREE,
(" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n"
"pCur2=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
(" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n"
"pCur2=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
(" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n"
"pCur2=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
break;
}
}
return 0;
}
/**
* Perform an integrity check on the PGM component.
*
* @returns VINF_SUCCESS if everything is fine.
* @returns VBox error status after asserting on integrity breach.
* @param pVM Pointer to the VM.
*/
{
/*
* Check the trees.
*/
int cErrors = 0;
cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysHandlers, true, pgmR3CheckIntegrityPhysHandlerNode, &Args);
cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysHandlers, false, pgmR3CheckIntegrityPhysHandlerNode, &Args);
cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->VirtHandlers, true, pgmR3CheckIntegrityVirtHandlerNode, &Args);
cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->VirtHandlers, false, pgmR3CheckIntegrityVirtHandlerNode, &Args);
cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->HyperVirtHandlers, true, pgmR3CheckIntegrityVirtHandlerNode, &Args);
cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->HyperVirtHandlers, false, pgmR3CheckIntegrityVirtHandlerNode, &Args);
cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysToVirtHandlers, true, pgmR3CheckIntegrityPhysToVirtHandlerNode, &Args);
cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysToVirtHandlers, false, pgmR3CheckIntegrityPhysToVirtHandlerNode, &Args);
}