cmd_cpu.h revision 9ee0dd10a0351d0f2cb5138380ec1017c92bdd4f
* The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * See the License for the specific language governing permissions * and limitations under the License. * When distributing Covered Code, include this CDDL HEADER in each * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * Copyright 2006 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. #
pragma ident "%Z%%M% %I% %E% SMI" * Each CPU of interest has a cmd_cpu_t structure. CPUs become of interest when * they are the focus of ereports, or when they detect UEs. CPUs may be the * target of several different kinds of ereport, each of which is tracked * differently. cpu_cases lists the types of cases that can be open against a * given CPU. The life of a CPU is complicated by the fact that xxCs and xxUs * received by the DE may in fact be side-effects of earlier UEs, xxCs, or xxUs. * Causes of side-effects, and actions taken to resolve them, can be found below * ________ CMD_PTR_CPU_ICACHE * / \ ,--------. CMD_PTR_CPU_DCACHE * |CPU | <---- |case_ptr| (one or more of CMD_PTR_CPU_PCACHE ) * | | `--------' CMD_PTR_CPU_ITLB * |,-------| ,-------. CMD_PTR_CPU_DTLB * ||asru | ----> |fmri_t | CMD_PTR_CPU_L2DATA * |:-------| :-------: CMD_PTR_CPU_L2DATA_UERETRY * ||fru | ----> |fmri_t | CMD_PTR_CPU_L2TAG * |`-------| `-------' CMD_PTR_CPU_L3DATA * | | ,---------. CMD_PTR_CPU_L3DATA_UERETRY * | uec | ----> |UE cache | CMD_PTR_CPU_L3TAG * \________/ `---------' CMD_PTR_CPU_FPU * | xr | <---- |case_ptr| (CMD_PTR_XR_WAITER) * ||rsrc | ----> |fmri_t | * | cpu | ----> detecting CPU * Data structure P? Case- Notes * ---------------- --- ----- -------------------------------------- * cmd_cpu_t Yes No Name is derived from CPU ID ("cpu_%d") * cmd_case_ptr_t Yes Yes Name is case's UUID * cpu_asru (fmri_t) Yes No Name is derived from CPU ID ("cpu_asru_%d") * cpu_fru (fmri_t) Yes No Name is derived from CPU ID ("cpu_fru_%d") * cpu_uec Yes No Name is derived from CPU ID ("cpu_uec_%d") * cmd_xr_t Yes Yes Name is `redelivery' * xr_rsrc (fmri_t) Yes No Name is derived from case's UUID ("%s_rsrc") * The UE cache. We actually have two UE caches - the current one and the old * one. When it's time to flush the UE cache, we move the current UE cache to * the old position and flush the E$. Then, we schedule the removal of the old * UE cache. This allows a) xxUs triggered by the flush to match against the * old cache, while b) still allowing new UEs to be added to the current UE * cache. UE matches will always search in both caches (if present), but * additions will only end up in the current cache. We go to all of this * effort because the cost of a missed ereport (discarding due to a false match * in the cache) is much less than that of a missed match. In the latter case, * the CPU will be erroneously offlined. * A special case is triggered if we see a UE with a not valid AFAR. Without * the AFAR, we aren't able to properly match subsequent xxU's. As a result, * we need to throw the cache into all-match mode, wherein all subsequent match * attempts will succeed until the UE cache is flushed. * Certain types of xxC and xxU can trigger other types as side-effects. These * secondary ereports need to be discarded, as treating them as legitimate * ereports in their own right will cause erroneous diagnosis. As an example * (see cmd_xxcu_trains for more), an L2$ UCC will usually trigger an L2$ WDC * resulting from the trap handler's flushing of the L2$. If we treat both as * legitimate, we'll end up adding two ereports to the SERD engine, * significantly cutting the threshold for retiring the CPU. * Our saving grace is the fact that the side-effect ereports will have the same * ENA as the primary. As such, we can keep track of groups of ereports by ENA. * These groups, which we'll call trains, can then be matched against a list of * known trains. The list (an array of cmd_xxcu_train_t structures) has both a * description of the composition of the train and an indication as to which of * the received ereports is the primary. * The cmd_xxcu_trw_t is used to gather the members of the train. When the * first member comes in, we allocate a trw, recording the ENA of the ereport, * as well as noting its class in trw_mask. We then reschedule the delivery of * the ereport for some configurable time in the future, trusting that all * members of the train will have arrived by that time. Subsequent ereports in * the same train match the recorded ENA, and add themselves to the mask. * When the first ereport is redelivered, trw_mask is used to determine whether * or not a train has been seen. An exact match is required. If a match is * made, the ereport indicated as the primary cause is used for diagnosis. * We don't have access to ereport nvlists when they are redelivered via timer. * As such, we have to retrieve everything we might need for diagnosis when we * first receive the ereport. The retrieved information is stored in the * cmd_xr_t, which is persisted. * xr_hdlr can't be persisted, so we use these in xr_hdlrid to indicate the * handler to be used. xr_hdlr is then updated so it can be used directly. * For sun4v, the size of xr_synd is expanded to 32 bits in order to * accomodate the Niagara L2 syndrome (4x7 bits). id_t xr_id;
/* ID of timer used for redelivery */ * The master structure containing or referencing all of the state for a given * We periodically flush the E$, thus allowing us to flush the UE cache (see * above for a description of the UE cache). In particular, we flush it * whenever we see a UE with a non-valid AFAR. To keep from overflushing the * CPU, we cap the number of flushes that we'll do in response to UEs with * non-valid AFARs. The cap is the number of permitted flushes per GC/restart * cycle, and was determined arbitrarily. * The CPU structure started life without a version number. Making things more * complicated, the version number in the new struct occupies the space used for * cpu_cpuid in the non-versioned struct. We therefore have to use somewhat * unorthodox version numbers to distinguish between the two types of struct * (pre- and post-versioning) -- version numbers that can't be mistaken for * CPUIDs. Our version numbers, therefore, will be negative. * For future expansion, the version member must always stay where it is. At * some point in the future, when more structs get versions, the version member * should move into the cmd_header_t. /* Portion of the bank structure which must be persisted */ /* Persistent and dynamic CPU data */ * L2$ and L3$ Data errors * ------ ----------- ------------------------------- * xxC l2cachedata fault.cpu.<cputype>.l2cachedata * xxU - fault.cpu.<cputype>.l2cachedata * L3_xxC l3cachedata fault.cpu.<cputype>.l3cachedata * L3_xxU - fault.cpu.<cputype>.l3cachedata * NOTE: For the purposes of the discussion below, xxC and xxU refer to both * L2$ and L3$ data errors. * These ereports will be dropped if (among other things) they are side-effects * of UEs (xxUs only) or other xxCs or xxUs. Whenever UEs are detected, they * are added to a per-CPU cache. xxUs are then compared to this cache. If a * xxU's AFAR refers to an address which recently saw a UE, the xxU is dropped, * as it was most likely caused by the UE. When multiple xxCs and xxUs are seen * with the same ENA, all save one are generally side-effects. We track these * groups (referred to as trains), matching them against a premade list. If one * of the trains matches, we drop all but the primary, which is indicated in the * The expected resolution of l2cachedata and l3cachedata faults is the * disabling of the indicated CPU. * ------- ----------- ------------------------------- * TxCE l2cachetag fault.cpu.<cputype>.l2cachetag * L3_THCE l3cachetag fault.cpu.<cputype>.l3cachetag * LTC l2cachetag fault.cpu.<cputype>.l2cachetag * We'll never see the uncorrectable Tag errors - they'll cause the machine to * reset, and we'll be ne'er the wiser. * The expected resolution of l2cachetag and l3cachetag faults is the disabling * ------- --------- ------------------------------- * IPE icache fault.cpu.<cputype>.icache * IxSPE icache fault.cpu.<cputype>.icache * DPE dcache fault.cpu.<cputype>.dcache * DxSPE dcache fault.cpu.<cputype>.dcache * PDSPE pcache fault.cpu.<cputype>.pcache * The I$, D$, and P$ are clean, and thus have no uncorrectable errors. * The expected resolution of icache, dcache, and pcache faults is the disabling * ------ --------- ------------------------------- * ITLBPE itlb fault.cpu.<cputype>.itlb * DTLBPE dtlb fault.cpu.<cputype>.dtlb * The expected resolution of itlb and dtlb faults is the disabling of the * ------ --------- ------------------------------- * The expected resolution of FPU faults is the disabling of the indicated CPU. * ------ --------- ------------------------------- * IRC ireg fault.cpu.<cputype>.ireg * The expected resolution of ireg faults is the disabling of the indicated CPU. * ------ --------- ------------------------------- * FRC freg fault.cpu.ultraSPARC-T1.frc * The expected resolution of freg faults is the repair of the indicated CPU. * ------ --------- ------------------------------- * The expected resolution of mau faults is the repair of the indicated CPU. * ------ --------- ------------------------------- * L2CTL - fault.cpu.<cputype>.l2ctl * The expected resolution of l2ctl faults is the repair of the indicated CPU. * CPUs are described by FMRIs. This routine will retrieve the CPU state * structure (creating a new one if necessary) described by the detector * FMRI in the passed ereport.