hment.c revision aac11643c466386309a97e46ac9f9a4cad538e5c
2N/A * The contents of this file are subject to the terms of the 2N/A * Common Development and Distribution License (the "License"). 2N/A * You may not use this file except in compliance with the License. 2N/A * See the License for the specific language governing permissions 2N/A * and limitations under the License. 2N/A * When distributing Covered Code, include this CDDL HEADER in each 2N/A * If applicable, add the following below this CDDL HEADER, with the 2N/A * fields enclosed by brackets "[]" replaced with your own identifying 2N/A * information: Portions Copyright [yyyy] [name of copyright owner] 2N/A * Copyright 2007 Sun Microsystems, Inc. All rights reserved. 2N/A * Use is subject to license terms. 2N/A#
pragma ident "%Z%%M% %I% %E% SMI" 2N/A * When pages are shared by more than one mapping, a list of these 2N/A * structs hangs off of the page_t connected by the hm_next and hm_prev 2N/A * fields. Every hment is also indexed by a system-wide hash table, using 2N/A * hm_hashnext to connect it to the chain of hments in a single hash 2N/A * Value returned by hment_walk() when dealing with a single mapping 2N/A * embedded in the page_t. 2N/A * The hment reserve is similar to the htable reserve, with the following 2N/A * exception. Hment's are never needed for HAT kmem allocs. 2N/A * The hment_reserve_amount variable is used, so that you can change it's 2N/A * value to zero via a kernel debugger to force stealing to get tested. 2N/A * Possible performance RFE: we might need to make this dynamic, perhaps 2N/A * based on the number of pages in the system. 2N/A * Lots of highly shared pages will have the same value for "entry" (consider 2N/A * the starting address of "xterm" or "sh"). So we'll distinguish them by 2N/A * adding the pfn of the page table into both the high bits. 2N/A * The shift by 9 corresponds to the range of values for entry (0..511). * "mlist_lock" is a hashed mutex lock for protecting per-page mapping * lists and "hash_lock" is a similar lock protecting the hment hash * table. The hashed approach is taken to avoid the spatial overhead of * maintaining a separate lock for each page, while still achieving better * scalability than a single lock would allow. * the shift by 9 is so that all large pages don't use the same hash bucket * put one hment onto the reserves list * Take one hment from the reserve. * We rely on a "donation system" to refill the hment reserve * list, which only takes place when we are allocating hments for * user mappings. It is theoretically possible that an incredibly * long string of kernel hment_alloc()s with no intervening user * hment_alloc()s could exhaust that pool. * If we aren't using the reserves, try using kmem to get an hment. * Donate any successful allocations to reserves if low. * If we're in panic, resort to using the reserves. * If allocation failed, we need to tap the reserves or steal * If we still haven't gotten an hment, attempt to steal one by * victimizing a mapping in a user htable. * we're in dire straights, try the reserve * still no hment is a serious problem. panic(
"hment_alloc(): no reserve, couldn't steal");
* Free an hment, possibly to the reserves list when called from the * thread using the reserves. For example, when freeing an hment during an * htable_steal(), we can't recurse into the kmem allocator, so we just * push the hment onto the reserve list. * zero out all fields to try and force any race conditions to segfault * Internal routine to add a full hment to a page_t mapping list * Add the hment to the page's mapping list. * Add the hment to the system-wide hash table. * Prepare a mapping list entry to the given page. * There are 4 different situations to deal with: * - Adding the first mapping to a page_t as an embedded hment * - Refaulting on an existing embedded mapping * - Upgrading an embedded mapping when adding a 2nd mapping * - Adding another mapping to a page_t that already has multiple mappings * note we don't optimized for the refaulting case here. * same page and the need to drop all locks while allocating hments, any or * all of the 3 situations can occur (and in almost any order) in any given * The most common case is establishing the first mapping to a * page, so check that first. This doesn't need any allocated * we had an hment already, so free it and retry * If there is an embedded mapping, we may need to * convert it to an hment. /* should point to htable */ * If we are faulting on a pre-existing mapping * This happens a lot due to segmap. * If we have an hment allocated, use it to promote the * existing embedded mapping. * We either didn't have an hment allocated or we just * used it for the embedded mapping. In either case, * allocate another hment and restart. * Last possibility is that we're adding an hment to a list * we allocated an hment already, free it and retry * Record a mapping list entry for the htable/entry to the given page. * hment_prepare() should have properly set up the situation. * The most common case is establishing the first mapping to a * page, so check that first. This doesn't need any allocated * We should never get here with a pre-existing embedded maping * add the new hment to the mapping list * Walk through the mappings for a page. * must already have done an x86_hm_enter() * Remove a mapping to a page from its mapping list. Must have * the corresponding mapping list locked. * Finds the mapping list entry with the given pte_t and * unlinks it from the mapping list. * Check if we have only one mapping embedded in the page_t. * Otherwise it must be in the list of hments. * Find the hment in the system-wide hash table and remove it. panic(
"hment_remove() missing in hash table pp=%lx, ht=%lx," * Remove the hment from the page's mapping list * Put initial hment's in the reserve pool. * Readjust the hment reserves after they may have been used. * Free up any excess reserves * initialize hment data structures * Initialize kmem caches. On 32 bit kernel's we shut off * debug information to save on precious kernel VA usage. * return the number of mappings to a page * Note there is no ASSERT() that the MUTEX is held for this. * Hence the return value might be inaccurate if this is called without * doing an x86_hm_enter(). * walk through all larger mapping sizes counting mappings if (
larger ==
pp)
/* don't double count large mappings */ * We need to steal an hment. Walk through all the page_t's until we * find one that has multiple mappings. Unload one of the mappings * and reclaim that hment. Note that we'll save/restart the starting * page to try and spread the pain. * The loop and function exit here if nothing found to steal. * Only lock the page_t if it has hments. * Search the mapping list for a usable mapping. * Steal the mapping we found. Note that hati_page_unmap() will