GMMR0.cpp revision d2de6dfa4cbd1c25dcab48cce02d0af5cf12b2ae
248c89033c87fed7229aa29bbbc4f4698fb13687vboxsync * GMM - Global Memory Manager.
248c89033c87fed7229aa29bbbc4f4698fb13687vboxsync * Copyright (C) 2007 Oracle Corporation
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This file is part of VirtualBox Open Source Edition (OSE), as
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * available from http://www.virtualbox.org. This file is free software;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * you can redistribute it and/or modify it under the terms of the GNU
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * General Public License (GPL) as published by the Free Software
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Foundation, in version 2 as it comes in the "COPYING" file of the
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * VirtualBox OSE distribution. VirtualBox OSE is distributed in the
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @page pg_gmm GMM - The Global Memory Manager
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * As the name indicates, this component is responsible for global memory
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * management. Currently only guest RAM is allocated from the GMM, but this
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * may change to include shadow page tables and other bits later.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Guest RAM is managed as individual pages, but allocated from the host OS
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * in chunks for reasons of portability / efficiency. To minimize the memory
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * footprint all tracking structure must be as small as possible without
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * unnecessary performance penalties.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The allocation chunks has fixed sized, the size defined at compile time
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * by the #GMM_CHUNK_SIZE \#define.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Each chunk is given an unquie ID. Each page also has a unique ID. The
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * relation ship between the two IDs is:
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * GMM_CHUNK_SHIFT = log2(GMM_CHUNK_SIZE / PAGE_SIZE);
c10a6f0c7041e4d1ee50ad38425aab9d43c55522vboxsync * idPage = (idChunk << GMM_CHUNK_SHIFT) | iPage;
c10a6f0c7041e4d1ee50ad38425aab9d43c55522vboxsync * Where iPage is the index of the page within the chunk. This ID scheme
c10a6f0c7041e4d1ee50ad38425aab9d43c55522vboxsync * permits for efficient chunk and page lookup, but it relies on the chunk size
c10a6f0c7041e4d1ee50ad38425aab9d43c55522vboxsync * to be set at compile time. The chunks are organized in an AVL tree with their
c10a6f0c7041e4d1ee50ad38425aab9d43c55522vboxsync * IDs being the keys.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * The physical address of each page in an allocation chunk is maintained by
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * the #RTR0MEMOBJ and obtained using #RTR0MemObjGetPagePhysAddr. There is no
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * need to duplicate this information (it'll cost 8-bytes per page if we did).
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * So what do we need to track per page? Most importantly we need to know
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * which state the page is in:
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * - Private - Allocated for (eventually) backing one particular VM page.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * - Shared - Readonly page that is used by one or more VMs and treated
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * as COW by PGM.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * - Free - Not used by anyone.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * For the page replacement operations (sharing, defragmenting and freeing)
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * to be somewhat efficient, private pages needs to be associated with a
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * particular page in a particular VM.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Tracking the usage of shared pages is impractical and expensive, so we'll
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * settle for a reference counting system instead.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Free pages will be chained on LIFOs
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * On 64-bit systems we will use a 64-bit bitfield per page, while on 32-bit
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * systems a 32-bit bitfield will have to suffice because of address space
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * limitations. The #GMMPAGE structure shows the details.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @section sec_gmm_alloc_strat Page Allocation Strategy
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The strategy for allocating pages has to take fragmentation and shared
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * pages into account, or we may end up with with 2000 chunks with only
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * a few pages in each. Shared pages cannot easily be reallocated because
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * of the inaccurate usage accounting (see above). Private pages can be
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * reallocated by a defragmentation thread in the same manner that sharing
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * The first approach is to manage the free pages in two sets depending on
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * whether they are mainly for the allocation of shared or private pages.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * In the initial implementation there will be almost no possibility for
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * mixing shared and private pages in the same chunk (only if we're really
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * stressed on memory), but when we implement forking of VMs and have to
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * deal with lots of COW pages it'll start getting kind of interesting.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * The sets are lists of chunks with approximately the same number of
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * free pages. Say the chunk size is 1MB, meaning 256 pages, and a set
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * consists of 16 lists. So, the first list will contain the chunks with
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * 1-7 free pages, the second covers 8-15, and so on. The chunks will be
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * moved between the lists as pages are freed up or allocated.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * @section sec_gmm_costs Costs
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * The per page cost in kernel space is 32-bit plus whatever RTR0MEMOBJ
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * entails. In addition there is the chunk cost of approximately
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * (sizeof(RT0MEMOBJ) + sizof(CHUNK)) / 2^CHUNK_SHIFT bytes per page.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * On Windows the per page #RTR0MEMOBJ cost is 32-bit on 32-bit windows
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * and 64-bit on 64-bit windows (a PFN_NUMBER in the MDL). So, 64-bit per page.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The cost on Linux is identical, but here it's because of sizeof(struct page *).
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @section sec_gmm_legacy Legacy Mode for Non-Tier-1 Platforms
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * In legacy mode the page source is locked user pages and not
ffb50166c9adb4ae583b914d405197035cf890advboxsync * #RTR0MemObjAllocPhysNC, this means that a page can only be allocated
ffb50166c9adb4ae583b914d405197035cf890advboxsync * by the VM that locked it. We will make no attempt at implementing
ffb50166c9adb4ae583b914d405197035cf890advboxsync * page sharing on these systems, just do enough to make it all work.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @subsection sub_gmm_locking Serializing
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * One simple fast mutex will be employed in the initial implementation, not
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * two as metioned in @ref subsec_pgmPhys_Serializing.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @see @ref subsec_pgmPhys_Serializing
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @section sec_gmm_overcommit Memory Over-Commitment Management
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The GVM will have to do the system wide memory over-commitment
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * management. My current ideas are:
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * - Per VM oc policy that indicates how much to initially commit
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * to it and what to do in a out-of-memory situation.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * - Prevent overtaxing the host.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * There are some challenges here, the main ones are configurability and
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * security. Should we for instance permit anyone to request 100% memory
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * commitment? Who should be allowed to do runtime adjustments of the
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * config. And how to prevent these settings from being lost when the last
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * VM process exits? The solution is probably to have an optional root
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * daemon the will keep VMMR0.r0 in memory and enable the security measures.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @section sec_gmm_numa NUMA
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * NUMA considerations will be designed and implemented a bit later.
248c89033c87fed7229aa29bbbc4f4698fb13687vboxsync * The preliminary guesses is that we will have to try allocate memory as
248c89033c87fed7229aa29bbbc4f4698fb13687vboxsync * close as possible to the CPUs the VM is executed on (EMT and additional CPU
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * threads). Which means it's mostly about allocation and sharing policies.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Both the scheduler and allocator interface will to supply some NUMA info
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * and we'll need to have a way to calc access costs.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/*******************************************************************************
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync* Header Files *
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync*******************************************************************************/
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync/*******************************************************************************
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync* Structures and Typedefs *
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync*******************************************************************************/
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync/** Pointer to set of free chunks. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync/** Pointer to a GMM allocation chunk. */
3fb3de312d1ff675e0f7cc62a7d46cbb1d5d9353vboxsync * The per-page tracking structure employed by the GMM.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * On 32-bit hosts we'll some trickery is necessary to compress all
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * the information into 32-bits. When the fSharedFree member is set,
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * the 30th bit decides whether it's a free page or not.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * Because of the different layout on 32-bit and 64-bit hosts, macros
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * are used to get and set some of the data.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsynctypedef union GMMPAGE
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** Unsigned integer view. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The common view. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The page state. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The view of a private page. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The guest page frame number. (Max addressable: 2 ^ 44 - 16) */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The GVM handle. (64K VMs) */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** Reserved. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The page state. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The view of a shared page. */
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /** The host page frame number. (Max addressable: 2 ^ 44 - 16) */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The reference count (64K VMs). */
11b175175a0ed424b8e8354acda681ad0adde0f8vboxsync /** Reserved. Checksum or something? Two hGVMs for forking? */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The page state. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The view of a free page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The index of the next page in the free list. UINT16_MAX is NIL. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Reserved. Checksum or something? */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Reserved. Checksum or something? */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The page state. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#else /* 32-bit */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Unsigned integer view. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The common view. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The page state. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The view of a private page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The guest page frame number. (Max addressable: 2 ^ 36) */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The GVM handle. (127 VMs) */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The top page state bit, MBZ. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The view of a shared page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The reference count. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The page state. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The view of a free page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The index of the next page in the free list. UINT16_MAX is NIL. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Reserved. Checksum or something? */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The page state. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Pointer to a GMMPAGE. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @name The Page States.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** A private page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** A private page - alternative value used on the 32-bit implemenation.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This will never be used on 64-bit hosts. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** A shared page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** A free page. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @def GMM_PAGE_IS_PRIVATE
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns true if private, false if not.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pPage The GMM page.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_PAGE_IS_PRIVATE(pPage) ( (pPage)->Common.u2State == GMM_PAGE_STATE_PRIVATE )
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_PAGE_IS_PRIVATE(pPage) ( (pPage)->Private.fZero == 0 )
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @def GMM_PAGE_IS_SHARED
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns true if shared, false if not.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * @param pPage The GMM page.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#define GMM_PAGE_IS_SHARED(pPage) ( (pPage)->Common.u2State == GMM_PAGE_STATE_SHARED )
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @def GMM_PAGE_IS_FREE
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * @returns true if free, false if not.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * @param pPage The GMM page.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync#define GMM_PAGE_IS_FREE(pPage) ( (pPage)->Common.u2State == GMM_PAGE_STATE_FREE )
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync/** @def GMM_PAGE_PFN_LAST
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * The last valid guest pfn range.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * @remark Some of the values outside the range has special meaning,
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * see GMM_PAGE_PFN_UNSHAREABLE.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsyncAssertCompile(GMM_PAGE_PFN_LAST == (GMM_GCPHYS_LAST >> PAGE_SHIFT));
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync/** @def GMM_PAGE_PFN_UNSHAREABLE
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Indicates that this page isn't used for normal guest memory and thus isn't shareable.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_PAGE_PFN_UNSHAREABLE UINT32_C(0xfffffff1)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_PAGE_PFN_UNSHAREABLE UINT32_C(0x00fffff1)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncAssertCompile(GMM_PAGE_PFN_UNSHAREABLE == (GMM_GCPHYS_UNSHAREABLE >> PAGE_SHIFT));
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * A GMM allocation chunk ring-3 mapping record.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This should really be associated with a session and not a VM, but
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * it's simpler to associated with a VM and cleanup with the VM object
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * is destroyed.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsynctypedef struct GMMCHUNKMAP
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The mapping object. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The VM owning the mapping. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Pointer to a GMM allocation chunk mapping. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync GMMCHUNKTYPE_CONTINUOUS = 2, /* one 2 MB continuous physical range. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * A GMM allocation chunk.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsynctypedef struct GMMCHUNK
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The AVL node core.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The Key is the chunk ID. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The memory object.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Either from RTR0MemObjAllocPhysNC or RTR0MemObjLockUser depending on
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * what the host can dish up with. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** Pointer to the next chunk in the free list. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Pointer to the previous chunk in the free list. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Pointer to the free set this chunk belongs to. NULL for
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * chunks with no free pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Pointer to an array of mappings. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of mappings. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The head of the list of free pages. UINT16_MAX is the NIL value. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The number of free pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The GVM handle of the VM that first allocated pages from this chunk, this
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * is used as a preference when there are several chunks to choose from.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * When in bound memory mode this isn't a preference any longer. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of private pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of shared pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Chunk type */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The pages. */
b8bb9c9f6b8ebfd0a7d6df0c0289f9fe80241750vboxsync * An allocation chunk TLB entry.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsynctypedef struct GMMCHUNKTLBE
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The chunk id. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** Pointer to the chunk. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync/** Pointer to an allocation chunk TLB entry. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** The number of entries tin the allocation chunk TLB. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Gets the TLB entry index for the given Chunk ID. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#define GMM_CHUNKTLB_IDX(idChunk) ( (idChunk) & (GMM_CHUNKTLB_ENTRIES - 1) )
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * An allocation chunk TLB.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsynctypedef struct GMMCHUNKTLB
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The TLB entries. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Pointer to an allocation chunk TLB. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** The GMMCHUNK::cFree shift count. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** The GMMCHUNK::cFree mask for use when considering relinking a chunk. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** The number of lists in set. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#define GMM_CHUNK_FREE_SET_LISTS (GMM_CHUNK_NUM_PAGES >> GMM_CHUNK_FREE_SET_SHIFT)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * A set of free chunks.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of free pages in the set. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** Chunks ordered by increasing number of free pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The GMM instance data.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsynctypedef struct GMM
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Magic / eye catcher. GMM_MAGIC */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The fast mutex protecting the GMM.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * More fine grained locking can be implemented later if necessary. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The chunk tree. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The chunk TLB. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The private free set. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The shared free set. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** Shared module tree (global). */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** todo seperate trees for distinctly different guest OSes. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The maximum number of pages we're allowed to allocate.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * @gcfgm 64-bit GMM/MaxPages Direct.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @gcfgm 32-bit GMM/PctPages Relative to the number of host pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of pages that has been reserved.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The deal is that cReservedPages - cOverCommittedPages <= cMaxPages. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The number of pages that we have over-committed in reservations. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of actually allocated (committed if you like) pages. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The number of pages that are shared. A subset of cAllocatedPages. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The number of pages that are shared that has been left behind by
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * VMs not doing proper cleanups. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of allocation chunks.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * (The number of pages we've allocated from the host can be derived from this.) */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The number of current ballooned pages. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The legacy allocation mode indicator.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This is determined at initialization time. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The bound memory mode indicator.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * When set, the memory will be bound to a specific VM and never
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * shared. This is always set if fLegacyAllocationMode is set.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * (Also determined at initialization time.) */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /** The number of registered VMs. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** The previous allocated Chunk ID.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Used as a hint to avoid scanning the whole bitmap. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** Chunk ID allocation bitmap.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Bits of allocated IDs are set, free ones are clear.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The NIL id (0) is marked allocated. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync uint32_t bmChunkId[(GMM_CHUNKID_LAST + 1 + 31) / 32];
c10a6f0c7041e4d1ee50ad38425aab9d43c55522vboxsync/** Pointer to the GMM instance. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** The value of GMM::u32Magic (Katsuhiro Otomo). */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/*******************************************************************************
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync* Global Variables *
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync*******************************************************************************/
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Pointer to the GMM instance data. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Macro for obtaining and validating the g_pGMM pointer.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * On failure it will return from the invoking function with the specified return value.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGMM The name of the pGMM variable.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param rc The return value on failure. Use VERR_INTERNAL_ERROR for
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * VBox status codes.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync AssertMsgReturn((pGMM)->u32Magic == GMM_MAGIC, ("%p - %#x\n", (pGMM), (pGMM)->u32Magic), (rc)); \
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync } while (0)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** Macro for obtaining and validating the g_pGMM pointer, void function variant.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * On failure it will return from the invoking function.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGMM The name of the pGMM variable.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync AssertMsgReturnVoid((pGMM)->u32Magic == GMM_MAGIC, ("%p - %#x\n", (pGMM), (pGMM)->u32Magic)); \
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync } while (0)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @def GMM_CHECK_SANITY_UPON_ENTERING
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Checks the sanity of the GMM instance data before making changes.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This is macro is a stub by default and must be enabled manually in the code.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns true if sane, false if not.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGMM The name of the pGMM variable.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#if defined(VBOX_STRICT) && 0
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_CHECK_SANITY_UPON_ENTERING(pGMM) (gmmR0SanityCheck((pGMM), __PRETTY_FUNCTION__, __LINE__) == 0)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_CHECK_SANITY_UPON_ENTERING(pGMM) (true)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @def GMM_CHECK_SANITY_UPON_LEAVING
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Checks the sanity of the GMM instance data after making changes.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This is macro is a stub by default and must be enabled manually in the code.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns true if sane, false if not.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGMM The name of the pGMM variable.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#if defined(VBOX_STRICT) && 0
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync# define GMM_CHECK_SANITY_UPON_LEAVING(pGMM) (gmmR0SanityCheck((pGMM), __PRETTY_FUNCTION__, __LINE__) == 0)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/** @def GMM_CHECK_SANITY_IN_LOOPS
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Checks the sanity of the GMM instance in the allocation loops.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This is macro is a stub by default and must be enabled manually in the code.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns true if sane, false if not.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGMM The name of the pGMM variable.
c99b597540585068d22dde4c9f74730305f24097vboxsync#if defined(VBOX_STRICT) && 0
c99b597540585068d22dde4c9f74730305f24097vboxsync# define GMM_CHECK_SANITY_IN_LOOPS(pGMM) (gmmR0SanityCheck((pGMM), __PRETTY_FUNCTION__, __LINE__) == 0)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/*******************************************************************************
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync* Internal Functions *
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync*******************************************************************************/
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic DECLCALLBACK(int) gmmR0TermDestroyChunk(PAVLU32NODECORE pNode, void *pvGMM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic DECLCALLBACK(int) gmmR0CleanupVMScanChunk(PAVLU32NODECORE pNode, void *pvGMM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic DECLCALLBACK(int) gmmR0CleanupSharedModule(PAVLGCPTRNODECORE pNode, void *pvGVM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/*static*/ DECLCALLBACK(int) gmmR0CleanupVMDestroyChunk(PAVLU32NODECORE pNode, void *pvGVM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncDECLINLINE(void) gmmR0LinkChunk(PGMMCHUNK pChunk, PGMMCHUNKFREESET pSet);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncDECLINLINE(void) gmmR0UnlinkChunk(PGMMCHUNK pChunk);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic uint32_t gmmR0SanityCheck(PGMM pGMM, const char *pszFunction, unsigned uLineNo);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic void gmmR0FreeChunk(PGMM pGMM, PGVM pGVM, PGMMCHUNK pChunk);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic void gmmR0FreeSharedPage(PGMM pGMM, uint32_t idPage, PGMMPAGE pPage);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic int gmmR0UnmapChunk(PGMM pGMM, PGVM pGVM, PGMMCHUNK pChunk);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Initializes the GMM component.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This is called when the VMMR0.r0 module is loaded and protected by the
c99b597540585068d22dde4c9f74730305f24097vboxsync * loader semaphore.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns VBox status code.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * Allocate the instance data and the lock(s).
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync for (unsigned i = 0; i < RT_ELEMENTS(pGMM->ChunkTLB.aEntries); i++)
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync pGMM->ChunkTLB.aEntries[i].idChunk = NIL_GMM_CHUNKID;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Check and see if RTR0MemObjAllocPhysNC works.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#if 0 /* later, see #3170. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync rc = RTR0MemObjAllocPhysNC(&MemObj, _64K, NIL_RTHCPHYS);
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync pGMM->fLegacyAllocationMode = pGMM->fBoundMemoryMode = true;
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync SUPR0Printf("GMMR0Init: RTR0MemObjAllocPhysNC(,64K,Any) -> %d!\n", rc);
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync# if defined(RT_OS_WINDOWS) || defined(RT_OS_SOLARIS) || defined(RT_OS_LINUX) || defined(RT_OS_FREEBSD)
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync /* Don't reuse possibly partial chunks because of the virtual address space limitation. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * Query system page count and guess a reasonable cMaxPages value.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pGMM->cMaxPages = UINT32_MAX; /** @todo IPRT function for query ram size and such. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync LogFlow(("GMMInit: pGMM=%p fLegacyAllocationMode=%RTbool fBoundMemoryMode=%RTbool\n", pGMM, pGMM->fLegacyAllocationMode, pGMM->fBoundMemoryMode));
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * Terminates the GMM component.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * Take care / be paranoid...
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("GMMR0Term: u32Magic=%#x\n", pGMM->u32Magic);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Undo what init did and free all the resources we've acquired.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* Destroy the fundamentals. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* free any chunks still hanging around. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync RTAvlU32Destroy(&pGMM->pChunks, gmmR0TermDestroyChunk, pGMM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* finally the instance data itself. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * RTAvlU32Destroy callback.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns 0
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pNode The node to destroy.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pvGMM The GMM handle.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsyncstatic DECLCALLBACK(int) gmmR0TermDestroyChunk(PAVLU32NODECORE pNode, void *pvGMM)
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync if (pChunk->cFree != (GMM_CHUNK_SIZE >> PAGE_SHIFT))
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("GMMR0Term: %p/%#x: cFree=%d cPrivate=%d cShared=%d cMappings=%d\n", pChunk,
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pChunk->Core.Key, pChunk->cFree, pChunk->cPrivate, pChunk->cShared, pChunk->cMappings);
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync int rc = RTR0MemObjFree(pChunk->MemObj, true /* fFreeMappings */);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("GMMR0Term: %p/%#x: RTRMemObjFree(%p,true) -> %d (cMappings=%d)\n", pChunk,
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pChunk->Core.Key, pChunk->MemObj, rc, pChunk->cMappings);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Initializes the per-VM data for the GMM.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This is called from within the GVMM lock (from GVMMR0CreateVM)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * and should only initialize the data members so GMMR0CleanupVM
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * can deal with them. We reserve no memory or anything here,
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * that's done later in GMMR0InitVM.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGVM Pointer to the Global VM structure.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync AssertCompile(RT_SIZEOFMEMB(GVM,gmm.s) <= RT_SIZEOFMEMB(GVM,gmm.padding));
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Cleans up when a VM is terminating.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGVM Pointer to the Global VM structure.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync LogFlow(("GMMR0CleanupVM: pGVM=%p:{.pVM=%p, .hSelf=%#x}\n", pGVM, pGVM->pVM, pGVM->hSelf));
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* Clean up all registered shared modules. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync RTAvlGCPtrDestroy(&pGVM->gmm.s.pSharedModuleTree, gmmR0CleanupSharedModule, pGVM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The policy is 'INVALID' until the initial reservation
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * request has been serviced.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * If it's the last VM around, we can skip walking all the chunk looking
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * for the pages owned by this VM and instead flush the whole shebang.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * This takes care of the eventuality that a VM has left shared page
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * references behind (shouldn't happen of course, but you never know).
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync#if 0 /* disabled so it won't hide bugs. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync RTAvlU32Destroy(&pGMM->pChunks, gmmR0CleanupVMDestroyChunk, pGMM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync for (unsigned i = 0; i < RT_ELEMENTS(pGMM->ChunkTLB.aEntries); i++)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pGMM->ChunkTLB.aEntries[i].idChunk = NIL_GMM_CHUNKID;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync memset(&pGMM->bmChunkId[0], 0, sizeof(pGMM->bmChunkId));
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Walk the entire pool looking for pages that belongs to this VM
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * and left over mappings. (This'll only catch private pages, shared
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * pages will be 'left behind'.)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync uint64_t cPrivatePages = pGVM->gmm.s.cPrivatePages; /* save */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync RTAvlU32DoWithAll(&pGMM->pChunks, true /* fFromLeft */, gmmR0CleanupVMScanChunk, pGVM);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("GMMR0CleanupVM: hGVM=%#x has %#x private pages that cannot be found!\n", pGVM->hSelf, pGVM->gmm.s.cPrivatePages);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* free empty chunks. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync PGMMCHUNK pCur = pGMM->Private.apLists[RT_ELEMENTS(pGMM->Private.apLists) - 1];
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* account for shared pages that weren't freed. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync Assert(pGMM->cSharedPages >= pGVM->gmm.s.cSharedPages);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("GMMR0CleanupVM: hGVM=%#x left %#x shared pages behind!\n", pGVM->hSelf, pGVM->gmm.s.cSharedPages);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pGMM->cLeftBehindSharedPages += pGVM->gmm.s.cSharedPages;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Update the over-commitment management statistics.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pGMM->cReservedPages -= pGVM->gmm.s.Reserved.cBasePages
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /** @todo Update GMM->cOverCommittedPages */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync /* zap the GVM data. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * RTAvlU32DoWithAll callback.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns 0
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pNode The node to search.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param pvGVM Pointer to the shared VM structure.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic DECLCALLBACK(int) gmmR0CleanupVMScanChunk(PAVLU32NODECORE pNode, void *pvGVM)
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * Look for pages belonging to the VM.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * (Perform some internal checks while we're scanning.)
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync if (pChunk->cFree != (GMM_CHUNK_SIZE >> PAGE_SHIFT))
b8bb9c9f6b8ebfd0a7d6df0c0289f9fe80241750vboxsync unsigned cPrivate = 0;
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync unsigned cShared = 0;
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync unsigned cFree = 0;
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync gmmR0UnlinkChunk(pChunk); /* avoiding cFreePages updates. */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync while (iPage-- > 0)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Free the page.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * The reason for not using gmmR0FreePrivatePage here is that we
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * must *not* cause the chunk to be freed from under us - we're in
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * an AVL tree walk here.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pChunk->aPages[iPage].Free.iNext = pChunk->iFreeHead;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync pChunk->aPages[iPage].Free.u2State = GMM_PAGE_STATE_FREE;
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync gmmR0LinkChunk(pChunk, pChunk->cShared ? &g_pGMM->Shared : &g_pGMM->Private);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Did it add up?
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("gmmR0CleanupVMScanChunk: Chunk %p/%#x has bogus stats - free=%d/%d private=%d/%d shared=%d/%d\n",
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync pChunk->cFree, cFree, pChunk->cPrivate, cPrivate, pChunk->cShared, cShared);
d3b1b01528fe21777281edf167f8deca06f86e39vboxsync * Look for the mapping belonging to the terminating VM.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync pChunk->paMappings[i] = pChunk->paMappings[pChunk->cMappings];
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync pChunk->paMappings[pChunk->cMappings].MapObj = NIL_RTR0MEMOBJ;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync int rc = RTR0MemObjFree(MemObj, false /* fFreeMappings (NA) */);
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("gmmR0CleanupVMScanChunk: %p/%#x: mapping #%x: RTRMemObjFree(%p,false) -> %d \n",
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * If not in bound memory mode, we should reset the hGVM field
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * if it has our handle in it.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync SUPR0Printf("gmmR0CleanupVMScanChunk: %p/%#x: cFree=%#x - it should be 0 in bound mode!\n",
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync AssertMsgFailed(("%p/%#x: cFree=%#x - it should be 0 in bound mode!\n", pChunk, pChunk->Core.Key, pChunk->cFree));
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync gmmR0LinkChunk(pChunk, pChunk->cShared ? &g_pGMM->Shared : &g_pGMM->Private);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * RTAvlU32Destroy callback for GMMR0CleanupVM.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @returns 0
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pNode The node (allocation chunk) to destroy.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pvGVM Pointer to the shared VM structure.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync/*static*/ DECLCALLBACK(int) gmmR0CleanupVMDestroyChunk(PAVLU32NODECORE pNode, void *pvGVM)
0dd3967035b8a02985920baa57f948dc542b9388vboxsync SUPR0Printf("gmmR0CleanupVMDestroyChunk: %p/%#x: mapping #%x: pGVM=%p exepcted %p\n", pChunk,
0dd3967035b8a02985920baa57f948dc542b9388vboxsync pChunk->Core.Key, i, pChunk->paMappings[i].pGVM, pGVM);
0dd3967035b8a02985920baa57f948dc542b9388vboxsync int rc = RTR0MemObjFree(pChunk->paMappings[i].MapObj, false /* fFreeMappings (NA) */);
0dd3967035b8a02985920baa57f948dc542b9388vboxsync SUPR0Printf("gmmR0CleanupVMDestroyChunk: %p/%#x: mapping #%x: RTRMemObjFree(%p,false) -> %d \n", pChunk,
0dd3967035b8a02985920baa57f948dc542b9388vboxsync pChunk->Core.Key, i, pChunk->paMappings[i].MapObj, rc);
6475559a7e0e52892efbab4fbdedc879f6866109vboxsync int rc = RTR0MemObjFree(pChunk->MemObj, true /* fFreeMappings */);
0dd3967035b8a02985920baa57f948dc542b9388vboxsync SUPR0Printf("gmmR0CleanupVMDestroyChunk: %p/%#x: RTRMemObjFree(%p,true) -> %d (cMappings=%d)\n", pChunk,
0dd3967035b8a02985920baa57f948dc542b9388vboxsync pChunk->Core.Key, pChunk->MemObj, rc, pChunk->cMappings);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * The initial resource reservations.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * This will make memory reservations according to policy and priority. If there aren't
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * sufficient resources available to sustain the VM this function will fail and all
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * future allocations requests will fail as well.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * These are just the initial reservations made very very early during the VM creation
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsync * process and will be adjusted later in the GMMR0UpdateReservation call after the
02651f98b4320e70a300ba1ebe95270096ebfd4dvboxsync * ring-3 init has completed.
0dd3967035b8a02985920baa57f948dc542b9388vboxsync * @returns VBox status code.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @retval VERR_GMM_MEMORY_RESERVATION_DECLINED
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @retval VERR_GMM_
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pVM Pointer to the shared VM structure.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param idCpu VCPU id
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param cBasePages The number of pages that may be allocated for the base RAM and ROMs.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * This does not include MMIO2 and similar.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param cShadowPages The number of pages that may be allocated for shadow pageing structures.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param cFixedPages The number of pages that may be allocated for fixed objects like the
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * hyper heap, MMIO2 and similar.
3c6306a66deef467e3c13483dd6529e1e1c6b822vboxsync * @param enmPolicy The OC policy to use on this VM.
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsync * @param enmPriority The priority in an out-of-memory situation.
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsync * @thread The creator thread / EMT.
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsyncGMMR0DECL(int) GMMR0InitialReservation(PVM pVM, VMCPUID idCpu, uint64_t cBasePages, uint32_t cShadowPages, uint32_t cFixedPages,
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsync LogFlow(("GMMR0InitialReservation: pVM=%p cBasePages=%#llx cShadowPages=%#x cFixedPages=%#x enmPolicy=%d enmPriority=%d\n",
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsync pVM, cBasePages, cShadowPages, cFixedPages, enmPolicy, enmPriority));
f75c6db919d277952ca03b7acf643e5e3ac96cafvboxsync * Validate, get basics and take the semaphore.
805a319b88bdf29b369da48402c58897a5e8b65dvboxsync AssertReturn(cShadowPages, VERR_INVALID_PARAMETER);
805a319b88bdf29b369da48402c58897a5e8b65dvboxsync AssertReturn(enmPolicy > GMMOCPOLICY_INVALID && enmPolicy < GMMOCPOLICY_END, VERR_INVALID_PARAMETER);
805a319b88bdf29b369da48402c58897a5e8b65dvboxsync AssertReturn(enmPriority > GMMPRIORITY_INVALID && enmPriority < GMMPRIORITY_END, VERR_INVALID_PARAMETER);
805a319b88bdf29b369da48402c58897a5e8b65dvboxsync * Check if we can accomodate this.
805a319b88bdf29b369da48402c58897a5e8b65dvboxsync /* ... later ... */
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Update the records.
805a319b88bdf29b369da48402c58897a5e8b65dvboxsync pGMM->cReservedPages += cBasePages + cFixedPages + cShadowPages;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync LogFlow(("GMMR0InitialReservation: returns %Rrc\n", rc));
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * VMMR0 request wrapper for GMMR0InitialReservation.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @returns see GMMR0InitialReservation.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pVM Pointer to the shared VM structure.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param idCpu VCPU id
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pReq The request packet.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncGMMR0DECL(int) GMMR0InitialReservationReq(PVM pVM, VMCPUID idCpu, PGMMINITIALRESERVATIONREQ pReq)
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Validate input and pass it on.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync AssertMsgReturn(pReq->Hdr.cbReq == sizeof(*pReq), ("%#x != %#x\n", pReq->Hdr.cbReq, sizeof(*pReq)), VERR_INVALID_PARAMETER);
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync return GMMR0InitialReservation(pVM, idCpu, pReq->cBasePages, pReq->cShadowPages, pReq->cFixedPages, pReq->enmPolicy, pReq->enmPriority);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * This updates the memory reservation with the additional MMIO2 and ROM pages.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @returns VBox status code.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @retval VERR_GMM_MEMORY_RESERVATION_DECLINED
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pVM Pointer to the shared VM structure.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param idCpu VCPU id
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param cBasePages The number of pages that may be allocated for the base RAM and ROMs.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * This does not include MMIO2 and similar.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param cShadowPages The number of pages that may be allocated for shadow pageing structures.
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * @param cFixedPages The number of pages that may be allocated for fixed objects like the
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * hyper heap, MMIO2 and similar.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @thread EMT.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsyncGMMR0DECL(int) GMMR0UpdateReservation(PVM pVM, VMCPUID idCpu, uint64_t cBasePages, uint32_t cShadowPages, uint32_t cFixedPages)
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync LogFlow(("GMMR0UpdateReservation: pVM=%p cBasePages=%#llx cShadowPages=%#x cFixedPages=%#x\n",
3c6306a66deef467e3c13483dd6529e1e1c6b822vboxsync * Validate, get basics and take the semaphore.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync AssertReturn(cShadowPages, VERR_INVALID_PARAMETER);
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * Check if we can accomodate this.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync /* ... later ... */
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync * Update the records.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync pGMM->cReservedPages -= pGVM->gmm.s.Reserved.cBasePages
5050fc8de0b121eab1b738d7c1007cde4903284dvboxsync pGMM->cReservedPages += cBasePages + cFixedPages + cShadowPages;
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync LogFlow(("GMMR0UpdateReservation: returns %Rrc\n", rc));
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * VMMR0 request wrapper for GMMR0UpdateReservation.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @returns see GMMR0UpdateReservation.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pVM Pointer to the shared VM structure.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param idCpu VCPU id
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pReq The request packet.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsyncGMMR0DECL(int) GMMR0UpdateReservationReq(PVM pVM, VMCPUID idCpu, PGMMUPDATERESERVATIONREQ pReq)
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Validate input and pass it on.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync AssertMsgReturn(pReq->Hdr.cbReq == sizeof(*pReq), ("%#x != %#x\n", pReq->Hdr.cbReq, sizeof(*pReq)), VERR_INVALID_PARAMETER);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync return GMMR0UpdateReservation(pVM, idCpu, pReq->cBasePages, pReq->cShadowPages, pReq->cFixedPages);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Performs sanity checks on a free set.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @returns Error count.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pGMM Pointer to the GMM instance.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pSet Pointer to the set.
0dd3967035b8a02985920baa57f948dc542b9388vboxsync * @param pszSetName The set name.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pszFunction The function from which it was called.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param uLine The line number.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsyncstatic uint32_t gmmR0SanityCheckSet(PGMM pGMM, PGMMCHUNKFREESET pSet, const char *pszSetName,
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Count the free pages in all the chunks and match it against pSet->cFreePages.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync for (unsigned i = 0; i < RT_ELEMENTS(pSet->apLists); i++)
ffb50166c9adb4ae583b914d405197035cf890advboxsync for (PGMMCHUNK pCur = pSet->apLists[i]; pCur; pCur = pCur->pFreeNext)
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync /** @todo check that the chunk is hash into the right set. */
b4d7b4dbcc45b8bde7502aa129440d92d7ffd038vboxsync SUPR0Printf("GMM insanity: found %#x pages in the %s set, expected %#x. (%s, line %u)\n",
ffb50166c9adb4ae583b914d405197035cf890advboxsync cPages, pszSetName, pSet->cFreePages, pszFunction, uLineNo);
ffb50166c9adb4ae583b914d405197035cf890advboxsync * Performs some sanity checks on the GMM while owning lock.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @returns Error count.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param pGMM Pointer to the GMM instance.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pszFunction The function from which it is called.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param uLineNo The line number.
ffb50166c9adb4ae583b914d405197035cf890advboxsyncstatic uint32_t gmmR0SanityCheck(PGMM pGMM, const char *pszFunction, unsigned uLineNo)
ffb50166c9adb4ae583b914d405197035cf890advboxsync cErrors += gmmR0SanityCheckSet(pGMM, &pGMM->Private, "private", pszFunction, uLineNo);
ffb50166c9adb4ae583b914d405197035cf890advboxsync cErrors += gmmR0SanityCheckSet(pGMM, &pGMM->Shared, "shared", pszFunction, uLineNo);
ffb50166c9adb4ae583b914d405197035cf890advboxsync /** @todo add more sanity checks. */
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Looks up a chunk in the tree and fill in the TLB entry for it.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * This is not expected to fail and will bitch if it does.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @returns Pointer to the allocation chunk, NULL if not found.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param pGMM Pointer to the GMM instance.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param idChunk The ID of the chunk to find.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param pTlbe Pointer to the TLB entry.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsyncstatic PGMMCHUNK gmmR0GetChunkSlow(PGMM pGMM, uint32_t idChunk, PGMMCHUNKTLBE pTlbe)
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync PGMMCHUNK pChunk = (PGMMCHUNK)RTAvlU32Get(&pGMM->pChunks, idChunk);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync AssertMsgReturn(pChunk, ("Chunk %#x not found!\n", idChunk), NULL);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * Finds a allocation chunk.
b4d7b4dbcc45b8bde7502aa129440d92d7ffd038vboxsync * This is not expected to fail and will bitch if it does.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @returns Pointer to the allocation chunk, NULL if not found.
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync * @param pGMM Pointer to the GMM instance.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync * @param idChunk The ID of the chunk to find.
ffb50166c9adb4ae583b914d405197035cf890advboxsyncDECLINLINE(PGMMCHUNK) gmmR0GetChunk(PGMM pGMM, uint32_t idChunk)
ffb50166c9adb4ae583b914d405197035cf890advboxsync * Do a TLB lookup, branch if not in the TLB.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsync PGMMCHUNKTLBE pTlbe = &pGMM->ChunkTLB.aEntries[GMM_CHUNKTLB_IDX(idChunk)];
ffb50166c9adb4ae583b914d405197035cf890advboxsync * Finds a page.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * This is not expected to fail and will bitch if it does.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @returns Pointer to the page, NULL if not found.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param pGMM Pointer to the GMM instance.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param idPage The ID of the page to find.
750df3fe104e01cadbc3d5bd20243055d283d4e5vboxsyncDECLINLINE(PGMMPAGE) gmmR0GetPage(PGMM pGMM, uint32_t idPage)
ffb50166c9adb4ae583b914d405197035cf890advboxsync PGMMCHUNK pChunk = gmmR0GetChunk(pGMM, idPage >> GMM_CHUNKID_SHIFT);
c89333d3e41e439ed9e74768000edc399d3e72e6vboxsync return &pChunk->aPages[idPage & GMM_PAGEID_IDX_MASK];
ffb50166c9adb4ae583b914d405197035cf890advboxsync * Unlinks the chunk from the free list it's currently on (if any).
ffb50166c9adb4ae583b914d405197035cf890advboxsync * @param pChunk The allocation chunk.
6475559a7e0e52892efbab4fbdedc879f6866109vboxsync pSet->apLists[(pChunk->cFree - 1) >> GMM_CHUNK_FREE_SET_SHIFT] = pNext;
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * Links the chunk onto the appropriate free list in the specified free set.
ffb50166c9adb4ae583b914d405197035cf890advboxsync * If no free entries, it's not linked into any list.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * @param pChunk The allocation chunk.
462e60a19d02a99b2b1a5c08dff74bb0808d707cvboxsync * @param pSet The free set.
6475559a7e0e52892efbab4fbdedc879f6866109vboxsyncDECLINLINE(void) gmmR0LinkChunk(PGMMCHUNK pChunk, PGMMCHUNKFREESET pSet)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync unsigned iList = (pChunk->cFree - 1) >> GMM_CHUNK_FREE_SET_SHIFT;
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * Frees a Chunk ID.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param pGMM Pointer to the GMM instance.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync * @param idChunk The Chunk ID to free.
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsyncstatic void gmmR0FreeChunkId(PGMM pGMM, uint32_t idChunk)
553a2f0d8ef91a6dad8de4eef206ff093af53a5dvboxsync AssertMsg(ASMBitTest(&pGMM->bmChunkId[0], idChunk), ("%#x\n", idChunk));
return idChunk;
AssertMsgReturn(!ASMAtomicBitTestAndSet(&pGMM->bmChunkId[0], idChunk), ("%#x\n", idChunk), NIL_GMM_CHUNKID);
AssertMsgReturn(!ASMAtomicBitTestAndSet(&pGMM->bmChunkId[0], idChunk), ("%#x\n", idChunk), NIL_GMM_CHUNKID);
static int gmmR0RegisterChunk(PGMM pGMM, PGMMCHUNKFREESET pSet, RTR0MEMOBJ MemObj, uint16_t hGVM, GMMCHUNKTYPE enmChunkType, PGMMCHUNK *ppChunk = NULL)
int rc;
if (pChunk)
LogFlow(("gmmR0RegisterChunk: pChunk=%p id=%#x cChunks=%d\n", pChunk, pChunk->Core.Key, pGMM->cChunks));
if (ppChunk)
return VINF_SUCCESS;
return rc;
static int gmmR0AllocateOneChunk(PGMM pGMM, PGMMCHUNKFREESET pSet, uint16_t hGVM, GMMCHUNKTYPE enmChunkType, PGMMCHUNK *ppChunk = NULL)
int rc;
AssertReturn(enmChunkType == GMMCHUNKTYPE_NON_CONTINUOUS || enmChunkType == GMMCHUNKTYPE_CONTINUOUS, VERR_INVALID_PARAMETER);
return rc;
return VERR_INTERNAL_ERROR_4;
if (!pChunk)
return rc;
return VERR_INTERNAL_ERROR_5;
return rc;
return VERR_INTERNAL_ERROR_5;
return VINF_SUCCESS;
pPage->u = 0;
static int gmmR0AllocatePages(PGMM pGMM, PGVM pGVM, uint32_t cPages, PGMMPAGEDESC paPages, GMMACCOUNT enmAccount)
return VERR_GMM_HIT_GLOBAL_LIMIT;
switch (enmAccount)
case GMMACCOUNT_BASE:
if (RT_UNLIKELY(pGVM->gmm.s.Allocated.cBasePages + pGVM->gmm.s.cBalloonedPages + cPages > pGVM->gmm.s.Reserved.cBasePages))
pGVM->gmm.s.Reserved.cBasePages, pGVM->gmm.s.Allocated.cBasePages, pGVM->gmm.s.cBalloonedPages, cPages));
return VERR_GMM_HIT_VM_ACCOUNT_LIMIT;
case GMMACCOUNT_SHADOW:
return VERR_GMM_HIT_VM_ACCOUNT_LIMIT;
case GMMACCOUNT_FIXED:
return VERR_GMM_HIT_VM_ACCOUNT_LIMIT;
return VERR_GMM_SEED_ME;
return VERR_GMM_SEED_ME;
switch (enmAccount)
return VINF_SUCCESS;
GMMR0DECL(int) GMMR0AllocateHandyPages(PVM pVM, VMCPUID idCpu, uint32_t cPagesToUpdate, uint32_t cPagesToAlloc, PGMMPAGEDESC paPages)
return rc;
unsigned iPage = 0;
AssertMsgReturn(paPages[iPage].HCPhysGCPhys == NIL_RTHCPHYS, ("#%#x: %RHp\n", iPage, paPages[iPage].HCPhysGCPhys), VERR_INVALID_PARAMETER);
AssertMsgReturn(paPages[iPage].idPage == NIL_GMM_PAGEID, ("#%#x: %#x\n", iPage, paPages[iPage].idPage), VERR_INVALID_PARAMETER);
AssertMsgReturn(paPages[iPage].idSharedPage == NIL_GMM_PAGEID, ("#%#x: %#x\n", iPage, paPages[iPage].idSharedPage), VERR_INVALID_PARAMETER);
Log(("GMMR0AllocateHandyPages: #%#x/%#x: Not private! %.*Rhxs\n", iPage, paPages[iPage].idPage, sizeof(*pPage), pPage));
Log(("GMMR0AllocateHandyPages: #%#x/%#x: Not found! (shared)\n", iPage, paPages[iPage].idSharedPage));
return rc;
GMMR0DECL(int) GMMR0AllocatePages(PVM pVM, VMCPUID idCpu, uint32_t cPages, PGMMPAGEDESC paPages, GMMACCOUNT enmAccount)
LogFlow(("GMMR0AllocatePages: pVM=%p cPages=%#x paPages=%p enmAccount=%d\n", pVM, cPages, paPages, enmAccount));
return rc;
AssertMsgReturn(enmAccount > GMMACCOUNT_INVALID && enmAccount < GMMACCOUNT_END, ("%d\n", enmAccount), VERR_INVALID_PARAMETER);
AssertMsgReturn(cPages > 0 && cPages < RT_BIT(32 - PAGE_SHIFT), ("%#x\n", cPages), VERR_INVALID_PARAMETER);
AssertMsgReturn(paPages[iPage].idPage == NIL_GMM_PAGEID, ("#%#x: %#x\n", iPage, paPages[iPage].idPage), VERR_INVALID_PARAMETER);
AssertMsgReturn(paPages[iPage].idSharedPage == NIL_GMM_PAGEID, ("#%#x: %#x\n", iPage, paPages[iPage].idSharedPage), VERR_INVALID_PARAMETER);
return rc;
GMMR0DECL(int) GMMR0AllocateLargePage(PVM pVM, VMCPUID idCpu, uint32_t cbPage, uint32_t *pIdPage, RTHCPHYS *pHCPhys)
return rc;
return VERR_NOT_SUPPORTED;
if (RT_UNLIKELY(pGVM->gmm.s.Allocated.cBasePages + pGVM->gmm.s.cBalloonedPages + cPages > pGVM->gmm.s.Reserved.cBasePages))
return VERR_GMM_HIT_VM_ACCOUNT_LIMIT;
return rc;
return rc;
return rc;
return VERR_NOT_SUPPORTED;
Log(("GMMR0FreeLargePage: allocated=%#llx cPages=%#x!\n", pGVM->gmm.s.Allocated.cBasePages, cPages));
return rc;
&& pGVM)
pPage, pPage - &pChunk->aPages[0], idPage, pPage->Common.u2State, pChunk->iFreeHead)); NOREF(idPage);
pPage->u = 0;
#ifdef VBOX_WITH_PAGE_SHARING
DECLINLINE(void) gmmR0ConvertToSharedPage(PGMM pGMM, PGVM pGVM, RTHCPHYS HCPhys, uint32_t idPage, PGMMPAGE pPage)
static int gmmR0FreePages(PGMM pGMM, PGVM pGVM, uint32_t cPages, PGMMFREEPAGEDESC paPages, GMMACCOUNT enmAccount)
switch (enmAccount)
case GMMACCOUNT_BASE:
case GMMACCOUNT_SHADOW:
case GMMACCOUNT_FIXED:
switch (enmAccount)
return rc;
GMMR0DECL(int) GMMR0FreePages(PVM pVM, VMCPUID idCpu, uint32_t cPages, PGMMFREEPAGEDESC paPages, GMMACCOUNT enmAccount)
LogFlow(("GMMR0FreePages: pVM=%p cPages=%#x paPages=%p enmAccount=%d\n", pVM, cPages, paPages, enmAccount));
return rc;
AssertMsgReturn(enmAccount > GMMACCOUNT_INVALID && enmAccount < GMMACCOUNT_END, ("%d\n", enmAccount), VERR_INVALID_PARAMETER);
AssertMsgReturn(cPages > 0 && cPages < RT_BIT(32 - PAGE_SHIFT), ("%#x\n", cPages), VERR_INVALID_PARAMETER);
return rc;
GMMR0DECL(int) GMMR0BalloonedPages(PVM pVM, VMCPUID idCpu, GMMBALLOONACTION enmAction, uint32_t cBalloonedPages)
AssertMsgReturn(cBalloonedPages < RT_BIT(32 - PAGE_SHIFT), ("%#x\n", cBalloonedPages), VERR_INVALID_PARAMETER);
return rc;
switch (enmAction)
case GMMBALLOONACTION_INFLATE:
/* Codepath never taken. Might be interesting in the future to request ballooned memory from guests in low memory conditions.. */
AssertFailed();
Log(("GMMR0BalloonedPages: +%#x - Global=%#llx / VM: Total=%#llx Req=%#llx Actual=%#llx (pending)\n", cBalloonedPages,
pGMM->cBalloonedPages, pGVM->gmm.s.cBalloonedPages, pGVM->gmm.s.cReqBalloonedPages, pGVM->gmm.s.cReqActuallyBalloonedPages));
case GMMBALLOONACTION_DEFLATE:
cBalloonedPages, pGMM->cBalloonedPages, pGVM->gmm.s.cBalloonedPages, pGVM->gmm.s.cReqDeflatePages));
case GMMBALLOONACTION_RESET:
return rc;
return VINF_SUCCESS;
return rc;
return rc;
return rc;
return VINF_SUCCESS;
Log(("gmmR0MapChunk: Chunk %#x is not mapped into pGVM=%p/%#x\n", pChunk->Core.Key, pGVM, pGVM->hSelf));
return VERR_GMM_CHUNK_NOT_MAPPED;
return VERR_GMM_CHUNK_NOT_FOUND;
return VINF_SUCCESS;
return VERR_GMM_CHUNK_ALREADY_MAPPED;
int rc = RTR0MemObjMapUser(&MapObj, pChunk->MemObj, (RTR3PTR)-1, 0, RTMEM_PROT_READ | RTMEM_PROT_WRITE, NIL_RTR0PROCESS);
void *pvMappings = RTMemRealloc(pChunk->paMappings, (pChunk->cMappings + 2 /*8*/) * sizeof(pChunk->paMappings[0]));
return VERR_NO_MEMORY;
return rc;
GMMR0DECL(int) GMMR0MapUnmapChunk(PVM pVM, VMCPUID idCpu, uint32_t idChunkMap, uint32_t idChunkUnmap, PRTR3PTR ppvR3)
return rc;
return VERR_INVALID_PARAMETER;
return rc;
AssertMsgReturn(pReq->Hdr.cbReq == sizeof(*pReq), ("%#x != %#x\n", pReq->Hdr.cbReq, sizeof(*pReq)), VERR_INVALID_PARAMETER);
return rc;
return VERR_NOT_SUPPORTED;
rc = RTR0MemObjLockUser(&MemObj, pvR3, GMM_CHUNK_SIZE, RTMEM_PROT_READ | RTMEM_PROT_WRITE, NIL_RTR0PROCESS);
return rc;
GMMR0DECL(int) GMMR0RegisterSharedModule(PVM pVM, VMCPUID idCpu, VBOXOSFAMILY enmGuestOS, char *pszModuleName, char *pszVersion, RTGCPTR GCBaseAddr, uint32_t cbModule,
#ifdef VBOX_WITH_PAGE_SHARING
return rc;
bool fNewModule = false;
PGMMSHAREDMODULEPERVM pRecVM = (PGMMSHAREDMODULEPERVM)RTAvlGCPtrGet(&pGVM->gmm.s.pSharedModuleTree, GCBaseAddr);
if (!pRecVM)
if (!pRecVM)
AssertFailed();
goto end;
fNewModule = true;
PGMMSHAREDMODULE pGlobalModule = (PGMMSHAREDMODULE)RTAvlGCPtrGet(&pGMM->pGlobalSharedModuleTree, GCBaseAddr);
if (!pGlobalModule)
if (!pGlobalModule)
AssertFailed();
goto end;
for (unsigned i = 0; i < cRegions; i++)
if ( fNewModule
|| pRecVM->fCollision == true) /* colliding module unregistered and new one registerd since the last check */
Log(("GMMR0RegisterSharedModule: using existing module %s cUser=%d!\n", pszModuleName, pGlobalModule->cUsers));
goto end;
end:
return rc;
return VERR_NOT_IMPLEMENTED;
GMMR0DECL(int) GMMR0RegisterSharedModuleReq(PVM pVM, VMCPUID idCpu, PGMMREGISTERSHAREDMODULEREQ pReq)
AssertMsgReturn(pReq->Hdr.cbReq >= sizeof(*pReq) && pReq->Hdr.cbReq == RT_UOFFSETOF(GMMREGISTERSHAREDMODULEREQ, aRegions[pReq->cRegions]), ("%#x != %#x\n", pReq->Hdr.cbReq, sizeof(*pReq)), VERR_INVALID_PARAMETER);
return GMMR0RegisterSharedModule(pVM, idCpu, pReq->enmGuestOS, pReq->szName, pReq->szVersion, pReq->GCBaseAddr, pReq->cbModule, pReq->cRegions, pReq->aRegions);
GMMR0DECL(int) GMMR0UnregisterSharedModule(PVM pVM, VMCPUID idCpu, char *pszModuleName, char *pszVersion, RTGCPTR GCBaseAddr, uint32_t cbModule)
#ifdef VBOX_WITH_PAGE_SHARING
return rc;
Log(("GMMR0UnregisterSharedModule %s %s base=%RGv size %x\n", pszModuleName, pszVersion, GCBaseAddr, cbModule));
PGMMSHAREDMODULEPERVM pRecVM = (PGMMSHAREDMODULEPERVM)RTAvlGCPtrGet(&pGVM->gmm.s.pSharedModuleTree, GCBaseAddr);
if (!pRecVM)
goto end;
/* Free the ranges, but leave the pages intact as there might still be references; they will be cleared by the COW mechanism. */
end:
return rc;
return VERR_NOT_IMPLEMENTED;
GMMR0DECL(int) GMMR0UnregisterSharedModuleReq(PVM pVM, VMCPUID idCpu, PGMMUNREGISTERSHAREDMODULEREQ pReq)
AssertMsgReturn(pReq->Hdr.cbReq == sizeof(*pReq), ("%#x != %#x\n", pReq->Hdr.cbReq, sizeof(*pReq)), VERR_INVALID_PARAMETER);
return GMMR0UnregisterSharedModule(pVM, idCpu, pReq->szName, pReq->szVersion, pReq->GCBaseAddr, pReq->cbModule);
#ifdef VBOX_WITH_PAGE_SHARING
* - if a shared page is new, then it changes the GMM page type to shared and returns it in the paPageDesc array
* - if a shared page already exists, then it checks if the VM page is identical and if so frees the VM page and returns the shared page in the paPageDesc array
GMMR0DECL(int) GMMR0SharedModuleCheckRange(PGVM pGVM, PGMMSHAREDMODULE pModule, unsigned idxRegion, unsigned cPages, PGMMSHAREDPAGEDESC paPageDesc)
AssertReturn(cPages == (pModule->aRegions[idxRegion].cbRegion >> PAGE_SHIFT), VERR_INVALID_PARAMETER);
Log(("GMMR0SharedModuleCheckRange %s base %RGv region %d cPages %d\n", pModule->szName, pModule->Core.Key, idxRegion, cPages));
pGlobalRegion->paHCPhysPageID = (uint32_t *)RTMemAlloc(cPages * sizeof(*pGlobalRegion->paHCPhysPageID));
AssertFailed();
goto end;
for (unsigned i = 0; i < cPages; i++)
for (unsigned i = 0; i < cPages; i++)
if (!pPage)
AssertFailed();
goto end;
AssertMsg(paPageDesc[i].GCPhys == (pPage->Private.pfn << 12), ("desc %RGp gmm %RGp\n", paPageDesc[i].HCPhys, (pPage->Private.pfn << 12)));
if (!pPage)
AssertFailed();
goto end;
Log(("Replace existing page guest %RGp host %RHp -> %RHp\n", paPageDesc[i].GCPhys, paPageDesc[i].HCPhys, pPage->Shared.pfn << PAGE_SHIFT));
if (pChunk)
AssertFailed();
goto end;
AssertFailed();
goto end;
/* Get the virtual address of the physical page; map the chunk into the VM process if not already done. */
goto end;
end:
return rc;
#ifdef VBOX_WITH_PAGE_SHARING
return rc;
return rc;
return VERR_NOT_IMPLEMENTED;
#ifdef VBOX_WITH_PAGE_SHARING
Log(("gmmR0CheckSharedModule: check %s %s base=%RGv size=%x collision=%d\n", pGlobalModule->szName, pGlobalModule->szVersion, pGlobalModule->Core.Key, pGlobalModule->cbModule, pLocalModule->fCollision));
#ifdef VBOX_WITH_PAGE_SHARING
return rc;
RTAvlGCPtrDoWithAll(&pGVM->gmm.s.pSharedModuleTree, true /* fFromLeft */, gmmR0CheckSharedModule, &Info);
return rc;
return VERR_NOT_IMPLEMENTED;