manifest.py revision 2843
39N/A# The contents of this file are subject to the terms of the 39N/A# Common Development and Distribution License (the "License"). 39N/A# You may not use this file except in compliance with the License. 39N/A# See the License for the specific language governing permissions 39N/A# and limitations under the License. 39N/A# When distributing Covered Code, include this CDDL HEADER in each 39N/A# If applicable, add the following below this CDDL HEADER, with the 39N/A# fields enclosed by brackets "[]" replaced with your own identifying 39N/A# information: Portions Copyright [yyyy] [name of copyright owner] 2854N/A# Copyright (c) 2007, 2012, Oracle and/or its affiliates. All rights reserved. 2690N/A """Returns the serialized state of this object in a format 2690N/A that that can be easily stored using JSON, pickle, etc.""" 2690N/A """Allocate a new object using previously serialized state 2690N/A obtained via getstate().""" 2690N/A # decode serialized state into python objects 1713N/A """A Manifest is the representation of the actions composing a specific 39N/A package version on both the client and the repository. Both purposes 39N/A utilize the same storage format. 39N/A The serialized structure of a manifest is an unordered list of actions. 205N/A The special action, "set", represents a package attribute. 205N/A The reserved attribute, "fmri", represents the package and version 39N/A described by this manifest. It is available as a string via the 39N/A attributes dictionary, and as an FMRI object from the fmri member. 39N/A The list of manifest-wide reserved attributes is 39N/A base_directory Default base directory, for non-user images. 39N/A isa Package is intended for a list of ISAs. 39N/A platform Package is intended for a list of platforms. 39N/A relocatable Suitable for User Image. 39N/A All non-prefixed attributes are reserved to the framework. Third 39N/A parties may prefix their attributes with a reversed domain name, domain 39N/A name, or stock symbol. An example might be 39N/A com.example,supported 39N/A as an indicator that a specific package version is supported by the 39N/A vendor, example.com. 39N/A manifest.null is provided as the null manifest. Differences against the 39N/A null manifest result in the complete set of attributes and actions of 48N/A the non-null manifest, meaning that all operations can be viewed as 48N/A tranitions between the manifest being installed and the manifest already 48N/A present in the image (which may be the null manifest). 1352N/A """A generator function that returns the unsorted manifest 1352N/A contents as lines of text.""" 926N/A """Return three lists of action pairs representing origin and 203N/A destination actions. The first list contains the pairs 203N/A representing additions, the second list contains the pairs 203N/A representing updates, and the third list contains the pairs 203N/A representing removals. All three lists are in the order in 1045N/A which they should be executed.""" 1045N/A # XXX Do we need to find some way to assert that the keys are 2054N/A # No origin was provided, so nothing has been changed or 1045N/A # removed; only added. In addition, this doesn't need 1045N/A # to be sorted since the caller likely already does 2084N/A """handle key values that may be lists""" 2842N/A # Transform list of actions into a dictionary keyed by 2842N/A # action key attribute, key attribute and mediator, or 2842N/A # id if there is no key attribute. 59N/A # XXX Do changed actions need to be sorted at all? This is 72N/A # likely to be the largest list, so we might save significant 72N/A # time by not sorting. Should we sort above? Insert into a 59N/A # singlesort = lambda x: x[0] or x[1] 2240N/A """Like the unix utility comm, except that this function 838N/A takes an arbitrary number of manifests and compares them, 838N/A returning a tuple consisting of each manifest's actions 838N/A that are not the same for all manifests, followed by a 926N/A list of actions that are the same in each manifest.""" 838N/A # Must specify at least one manifest. 2240N/A # construct list of dictionaries of actions in each 838N/A # manifest, indexed by unique key and variant combination 2240N/A # The unique key for each action is based on its 2240N/A # type, key attribute, and unique variants set 2240N/A # If there is no key attribute for the 2240N/A # action, then fallback to the object 2240N/A # id for the action as its identifier. 2240N/A # catch duplicate actions here... 2284N/A # construct list of key sets in each dict 926N/A # determine which common_keys have common actions 926N/A """Where difference() returns three lists, combined_difference() 203N/A returns a single list of the concatenation of the three.""" 838N/A """Output expects that self is newer than other. Use of sets 48N/A requires that we convert the action objects into some marshalled 48N/A form, otherwise set member identities are derived from the 48N/A object pointers, rather than the contents.""" 2453N/A """Generate contents of dircache file containing all dirctories 2453N/A referenced explicitly or implicitly from self.actions. Include 2453N/A variants as values; collapse variants where possible.""" 2339N/A yield "dir path=%s %s\n" % \
2453N/A """Generate contents of mediatorcache file containing all 2453N/A mediators referenced explicitly or implicitly from self.actions. 2453N/A Include variants as values; collapse variants where possible.""" 2453N/A a =
"set name=pkg.mediator " \
2910N/A """create dictionary of all actions referenced explicitly or 2910N/A implicitly from self.actions... include variants as values; 2910N/A collapse variants where possible""" 2910N/A # build a dictionary containing all directories tagged w/ 2910N/A # remove any tags if any entries are always delivered (NULL) 2910N/A # could collapse refs where all variants are present 2910N/A # (the current logic only collapses them if at least 2910N/A # one reference is delivered without a facet or 2910N/A """ return a list of directories implicitly or 2910N/A explicitly referenced by this object""" 2910N/A # generate actions that contain directories 2910N/A """A generator function that yields tuples of the form (mediator, 2910N/A mediations) expressing the set of possible mediations for this 2910N/A package, where 'mediations' is a set() of possible mediations for 2910N/A the mediator. Each mediation is a tuple of the form (priority, 2910N/A # generate actions that contain mediators 2910N/A # Consider this mediation unavailable 2910N/A # if it can't be parsed for whatever 2910N/A """Generate actions in manifest through ordered callable list""" 2910N/A """Generate actions in the manifest of type "type" 2910N/A through ordered callable list""" 2910N/A """Generate actions in the manifest of types "atypes" 2910N/A through ordered callable list.""" 2910N/A """Generate the value of the key attribute for each action 2910N/A of type "type" in the manifest.""" 2910N/A """Find actions in the manifest which are duplicates (i.e., 2910N/A represent the same object) but which are not identical (i.e., 2910N/A have all the same attributes).""" 2910N/A """Return a key on which actions can be sorted.""" 2910N/A # Get an iterable for the string. 2910N/A if not l
or l[
0] ==
"#":
# ignore blank lines & comments 2910N/A # Accumulate errors and continue so that as 2910N/A # much of the action data as possible can be 2910N/A """Populate the manifest with actions. 2910N/A 'content' is an optional value containing either the text 2910N/A representation of the manifest or an iterable of 2910N/A 'excludes' is optional. If provided it must be a length two 2910N/A list with the variants to be excluded as the first element and 2910N/A the facets to be excluded as the second element. 2910N/A 'pathname' is an optional filename containing the location of 2910N/A 'signatures' is an optional boolean value that indicates whether 2910N/A a manifest signature should be generated. This is only possible 2910N/A when 'content' is a string or 'pathname' is provided. 2910N/A # sdict and odict in difference() above, and have that be our 2910N/A # main datastore, rather than the simple list we have now. If 2910N/A # we do that here, we can even assert that the "same" action 2910N/A # can't be in a manifest twice. (The problem of having the same 2910N/A # action more than once in packages that can be installed 2910N/A # together has to be solved somewhere else, though.) 2910N/A # Generate manifest signature based upon 2910N/A # input content, but only if signatures 2453N/A # Make sure that either no excludes were provided or that both 2453N/A # variants and facet excludes were. 2339N/A """Remove any actions from the manifest which should be 2453N/A """Performs any needed transformations on the action then adds 2339N/A The "action" parameter is the action object that should be 2453N/A The "excludes" parameter is the variants to exclude from the 2453N/A # XXX handle legacy transition issues; not needed once support 2453N/A # for upgrading images from older releases (< build 151) has 2608N/A # Translate old action to new. 2339N/A # then both are set, so only need to check for one). 2910N/A # add any set actions to attributes 2910N/A """Fill attribute array w/ set action contents.""" 2910N/A """Produces the search dictionary for a specific manifest. 2910N/A A dictionary is constructed which maps a tuple of token, 2910N/A action type, key, and the value that matched the token to 2910N/A the byte offset into the manifest file. file_path is the 2910N/A path to the manifest file. excludes is the variants which 2910N/A should be allowed in this image. return_line is a debugging 2910N/A flag which makes the function map the information to the 2910N/A string of the line, rather than the byte offset to allow 2910N/A """Translates what actions.generate_indices produces 2910N/A into a dictionary mapping token, action_name, key, and 2910N/A the value that should be displayed for matching that 2910N/A token to byte offsets into the manifest file. 2910N/A The "lst" parameter is the data to be converted. 2910N/A The "cp" parameter is the byte offset into the file 2910N/A for the action which produced lst.""" 2608N/A """This method takes a string representing the on-disk 2608N/A manifest content, and returns a hash value.""" 2453N/A # Byte stream expected, so pass encoded. 2453N/A """Verifies whether the signatures for the contents of 2453N/A the manifest match the specified signature data. Raises 2639N/A the 'BadManifestSignatures' exception on failure.""" 2453N/A """Store the manifest contents to disk.""" 838N/A # We specifically avoid sorting manifests before writing 2608N/A # them to disk-- there's really no point in doing so, since 2608N/A # we'll sort actions globally during packaging operations. 926N/A """Return a dictionary mapping variant tags to their values.""" 838N/A """Returns the boolean of the value of the attribute 'key'.""" 1500N/A """Returns an integer representing the total size, in bytes, of 1432N/A the Manifest's data payload. 1970N/A 'excludes' is a list of variants which should be allowed when 1432N/A """Private helper function to populate list of facets and 50N/A variants on-demand.""" 1636N/A # append any variants and facets to manifest dict 2073N/A # Lists can't be set elements. 1713N/A _(
"%(forv)s '%(v)s' specified multiple times") %
2073N/A """Return the value for the package attribute 'key'.""" 2073N/A """Set the value for the package attribute 'key' to 'value'.""" 2073N/A """This class serves as a wrapper for the Manifest class for callers 2073N/A that need efficient access to package data on a per-action type basis. 1890N/A It achieves this by partitioning the manifest into multiple files (one 1500N/A per action type) and then storing an on-disk cache of the directories 2073N/A explictly and implicitly referenced by the manifest each tagged with 1500N/A """Raises KeyError exception if factored manifest is not present 1500N/A and contents are None; delays reading of manifest until required 926N/A if cache file is present. 2026N/A 'fmri' is a PkgFmri object representing the identity of the 2608N/A 'cache_root' is the pathname of the directory where the manifest 2608N/A and cache files should be stored or loaded from. 2026N/A 'contents' is an optional string to use as the contents of the 2026N/A manifest if a cached copy does not already exist. 2026N/A 'excludes' is optional. If provided it must be a length two 2073N/A list with the variants to be excluded as the first element and 2026N/A the facets to be exclduded as the second element. 1500N/A 'pathname' is an optional string containing the pathname of a 1500N/A manifest. If not provided, it is assumed that the manifest is 1500N/A stored in a file named 'manifest' in the directory indicated by 1500N/A 'cache_root'. If provided, and contents is also provided, then 1500N/A 'contents' will be stored in 'pathname' if it does not already 1500N/A # Make sure that either no excludes were provided or that both 2639N/A # variants and facet excludes were. 2639N/A # Do we have a cached copy? 2639N/A # we have no cached copy; save one 1500N/A # don't specify excludes so on-disk copy has 2639N/A # we have a cached copy of the manifest 2639N/A # have we computed the dircache? 2910N/A """Load all manifest contents from on-disk copy of manifest""" 2910N/A """Unload manifest; used to reduce peak memory comsumption 2910N/A when downloading new manifests""" 2910N/A """Finish loading.... this part of initialization is common 2910N/A """ store the current action set; also create per-type 2910N/A caches. Return True if data was saved, False if not""" 2910N/A # this allows us to try to cache new manifests 2910N/A # when non-root w/o failures. 2910N/A """ create manifest.<typename> files to accelerate partial 2910N/A parsing of manifests. Separate from __storeback code to 2910N/A allow upgrade to reuse existing on disk manifests""" 2910N/A # Ensure target cache directory and intermediates exist. 2910N/A # create per-action type cache; use rename to avoid 2910N/A # corrupt files if ^C'd in the middle 941N/A """Remove all manifest cache files found in the given directory 941N/A (excluding the manifest itself). 941N/A # Only raise error if failure wasn't due to 941N/A # cache directory not existing. 941N/A """Private helper function for loading arbitrary cached manifest 429N/A # need to load from disk 941N/A # we have cached copy on disk; use it 1007N/A """ return a list of directories implicitly or explicitly 1007N/A """ generate actions of the specified type; 1007N/A use already in-memory stuff if already loaded, 1007N/A otherwise use per-action types files""" 1007N/A # invoke subclass method to generate action by action 1007N/A # This checks if we've already written out the factored 1007N/A # manifest files. If so, we'll use it, and if not, then 1007N/A # we'll load the full manifest. 144N/A # get manifest from disk 1472N/A # invoke subclass method to generate action by action 1890N/A # we have a cached copy - use it 1352N/A return # no such action in this manifest 429N/A """A generator function that yields set actions expressing the 144N/A set of possible mediations for this package. 1636N/A """Load attributes dictionary from cached set actions; 1636N/A this speeds up pkg info a lot""" 315N/A """No assignments to factored manifests allowed.""" 315N/A assert "FactoredManifests are not dicts" 1044N/A """The absolute pathname of the file containing the manifest.""" 1044N/A """Special class for pkgplan's need for a empty manifest; 2910N/A the regular null manifest doesn't support get_directories 2910N/A and making the factored manifest code handle this case is 2910N/A """Return three lists of action pairs representing origin and 2910N/A destination actions. The first list contains the pairs 2910N/A representing additions, the second list contains the pairs 2910N/A representing updates, and the third list contains the pairs 2910N/A representing removals. All three lists are in the order in 2910N/A which they should be executed.""" 2910N/A # The difference for this case is simply everything in the 2286N/A # origin has been removed. This is an optimization for 2910N/A # This method is overridden so that self.excludes is never set 2910N/A # on the singleton NullFactoredManifest. 2910N/A """Simple Exception class to handle manifest specific errors"""