manifest.py revision 2238
39N/A# The contents of this file are subject to the terms of the 39N/A# Common Development and Distribution License (the "License"). 39N/A# You may not use this file except in compliance with the License. 39N/A# See the License for the specific language governing permissions 39N/A# and limitations under the License. 39N/A# When distributing Covered Code, include this CDDL HEADER in each 39N/A# If applicable, add the following below this CDDL HEADER, with the 39N/A# fields enclosed by brackets "[]" replaced with your own identifying 39N/A# information: Portions Copyright [yyyy] [name of copyright owner] 2238N/A# Copyright (c) 2007, 2011, Oracle and/or its affiliates. All rights reserved. 39N/A """A Manifest is the representation of the actions composing a specific 39N/A package version on both the client and the repository. Both purposes 39N/A utilize the same storage format. 205N/A The serialized structure of a manifest is an unordered list of actions. 205N/A The special action, "set", represents a package attribute. 39N/A The reserved attribute, "fmri", represents the package and version 39N/A described by this manifest. It is available as a string via the 39N/A attributes dictionary, and as an FMRI object from the fmri member. 39N/A The list of manifest-wide reserved attributes is 39N/A base_directory Default base directory, for non-user images. 39N/A isa Package is intended for a list of ISAs. 39N/A platform Package is intended for a list of platforms. 39N/A relocatable Suitable for User Image. 39N/A All non-prefixed attributes are reserved to the framework. Third 39N/A parties may prefix their attributes with a reversed domain name, domain 39N/A name, or stock symbol. An example might be 39N/A com.example,supported 39N/A as an indicator that a specific package version is supported by the 39N/A vendor, example.com. 48N/A manifest.null is provided as the null manifest. Differences against the 48N/A null manifest result in the complete set of attributes and actions of 48N/A the non-null manifest, meaning that all operations can be viewed as 48N/A tranitions between the manifest being installed and the manifest already 48N/A present in the image (which may be the null manifest). 1352N/A """A generator function that returns the unsorted manifest 1352N/A contents as lines of text.""" 203N/A """Return three lists of action pairs representing origin and 203N/A destination actions. The first list contains the pairs 203N/A representing additions, the second list contains the pairs 203N/A representing updates, and the third list contains the pairs 1045N/A representing removals. All three lists are in the order in 1045N/A which they should be executed.""" 72N/A # XXX Do we need to find some way to assert that the keys are 1045N/A # No origin was provided, so nothing has been changed or 1045N/A # removed; only added. In addition, this doesn't need 1045N/A # to be sorted since the caller likely already does 2084N/A """handle key values that may be lists""" 72N/A # XXX Do changed actions need to be sorted at all? This is 72N/A # likely to be the largest list, so we might save significant 72N/A # time by not sorting. Should we sort above? Insert into a 72N/A # singlesort = lambda x: x[0] or x[1] 838N/A """Like the unix utility comm, except that this function 838N/A takes an arbitrary number of manifests and compares them, 838N/A returning a tuple consisting of each manifest's actions 926N/A that are not the same for all manifests, followed by a 838N/A list of actions that are the same in each manifest.""" 838N/A # construct list of dictionaries of actions in each 838N/A # manifest, indexed by unique keys 838N/A # construct list of key sets in each dict 838N/A # determine which common_keys have common actions 203N/A """Where difference() returns three lists, combined_difference() 315N/A returns a single list of the concatenation of the three.""" 48N/A """Output expects that self is newer than other. Use of sets 48N/A requires that we convert the action objects into some marshalled 48N/A form, otherwise set member identities are derived from the 48N/A object pointers, rather than the contents.""" 838N/A """Generate actions in manifest through ordered callable list""" 838N/A """Generate actions in the manifest of type "type" 838N/A through ordered callable list""" 615N/A """Generate the value of the key atrribute for each action 615N/A of type "type" in the manifest.""" 111N/A """Find actions in the manifest which are duplicates (i.e., 111N/A represent the same object) but which are not identical (i.e., 111N/A have all the same attributes).""" 111N/A """Return a key on which actions can be sorted.""" 2073N/A # Get an iterable for the string. 1636N/A if not l
or l[
0] ==
"#":
# ignore blank lines & comments 1970N/A # Accumulate errors and continue so that as 1970N/A # much of the action data as possible can be 1713N/A """Populate the manifest with actions. 2073N/A 'content' is an optional value containing either the text 2073N/A representation of the manifest or an iterable of 2073N/A 'excludes' is an optional list of variants to exclude from the 2073N/A 'pathname' is an optional filename containing the location of 2073N/A 'signatures' is an optional boolean value that indicates whether 2073N/A a manifest signature should be generated. This is only possible 2073N/A when 'content' is a string or 'pathname' is provided. 1500N/A # sdict and odict in difference() above, and have that be our 1500N/A # main datastore, rather than the simple list we have now. If 1500N/A # we do that here, we can even assert that the "same" action 1500N/A # can't be in a manifest twice. (The problem of having the same 1500N/A # action more than once in packages that can be installed 1500N/A # together has to be solved somewhere else, though.) 2073N/A # Generate manifest signature based upon 2073N/A # input content, but only if signatures 2026N/A """Remove any actions from the manifest which should be 1500N/A """Performs any needed transformations on the action then adds 1500N/A The "action" parameter is the action object that should be 1500N/A The "excludes" parameter is the variants to exclude from the 1500N/A # XXX handle legacy transition issues; not needed after 1500N/A # 2009.06 release & republication are complete. 1500N/A # Translate old action to new. 1500N/A # add any set actions to attributes 1500N/A # append any variants and facets to manifest dict 2238N/A # Lists can't be set elements. 2238N/A _(
"%(forv)s '%(v)s' specified multiple times") %
956N/A """Fill attribute array w/ set action contents.""" 1007N/A """Produces the search dictionary for a specific manifest. 1007N/A A dictionary is constructed which maps a tuple of token, 1007N/A action type, key, and the value that matched the token to 1007N/A the byte offset into the manifest file. file_path is the 1007N/A path to the manifest file. excludes is the variants which 1007N/A should be allowed in this image. return_line is a debugging 1007N/A flag which makes the function map the information to the 1007N/A string of the line, rather than the byte offset to allow 1100N/A """Translates what actions.generate_indices produces 1100N/A into a dictionary mapping token, action_name, key, and 1100N/A the value that should be displayed for matching that 1100N/A token to byte offsets into the manifest file. 1100N/A The "lst" parameter is the data to be converted. 1100N/A The "cp" parameter is the byte offset into the file 1100N/A for the action which produced lst.""" 1472N/A """This method takes a string representing the on-disk 1472N/A manifest content, and returns a hash value.""" 1890N/A # Byte stream expected, so pass encoded. 1352N/A """Verifies whether the signatures for the contents of 1352N/A the manifest match the specified signature data. Raises 1352N/A the 'BadManifestSignatures' exception on failure.""" 429N/A """Store the manifest contents to disk.""" 315N/A # We specifically avoid sorting manifests before writing 315N/A # them to disk-- there's really no point in doing so, since 315N/A # we'll sort actions globally during packaging operations. 1231N/A """Return a dictionary mapping variant tags to their values.""" 1461N/A """Returns the boolean of the value of the attribute 'key'.""" 1044N/A """Returns an integer representing the total size, in bytes, of 1044N/A the Manifest's data payload. 1044N/A 'excludes' is a list of variants which should be allowed when 838N/A """Return the value for the package attribute 'key'.""" 296N/A """Set the value for the package attribute 'key' to 'value'.""" 2054N/A """This class serves as a wrapper for the Manifest class for callers 2054N/A that need efficient access to package data on a per-action type basis. 2054N/A It achieves this by partitioning the manifest into multiple files (one 2054N/A per action type) and then storing an on-disk cache of the directories 2054N/A explictly and implicitly referenced by the manifest each tagged with 2073N/A """Raises KeyError exception if factored manifest is not present 2073N/A and contents are None; delays reading of manifest until required 2073N/A 'fmri' is a PkgFmri object representing the identity of the 2073N/A 'cache_root' is the pathname of the directory where the manifest 2073N/A and cache files should be stored or loaded from. 2073N/A 'contents' is an optional string to use as the contents of the 2073N/A manifest if a cached copy does not already exist. 2073N/A 'excludes' is an optional list of excludes to apply to the 2073N/A 'pathname' is an optional string containing the pathname of a 2073N/A manifest. If not provided, it is assumed that the manifest is 2073N/A stored in a file named 'manifest' in the directory indicated by 2144N/A 'cache_root'. If provided, and contents is also provided, then 2144N/A 'contents' will be stored in 'pathname' if it does not already 956N/A # Do we have a cached copy? 956N/A # we have no cached copy; save one 956N/A # don't specify excludes so on-disk copy has 956N/A # we have a cached copy of the manifest 956N/A # have we computed the dircache? 956N/A """Load all manifest contents from on-disk copy of manifest""" 967N/A """Unload manifest; used to reduce peak memory comsumption 967N/A when downloading new manifests""" 1507N/A """Finish loading.... this part of initialization is common 956N/A to multiple code paths""" 956N/A """ store the current action set; also create per-type 967N/A caches. Return True if data was saved, False if not""" 956N/A # this allows us to try to cache new manifests 1672N/A # when non-root w/o failures. 956N/A """ create manifest.<typename> files to accelerate partial 1507N/A parsing of manifests. Separate from __storeback code to 956N/A allow upgrade to reuse existing on disk manifests""" 2144N/A # Ensure target cache directory and intermediates exist. 956N/A # create per-action type cache; use rename to avoid 1507N/A # corrupt files if ^C'd in the middle 1507N/A """ from a dictionary of paths, generate contents of dircache 956N/A yield "dir path=%s %s\n" % \
1507N/A """ create dictionary of all directories referenced 1507N/A by actions explicitly or implicitly from self.actions... 956N/A include variants as values; collapse variants where possible""" 1507N/A # build a dictionary containing all directories tagged w/ 956N/A # remove any tags if any entries are always installed (NULL) 956N/A # could collapse dirs where all variants are present 2073N/A """Remove all manifest cache files found in the given directory 2073N/A (excluding the manifest itself). 956N/A """ return a list of directories implicitly or 956N/A explicitly referenced by this object""" 956N/A # need to load from disk 956N/A # generate actions that contain directories 956N/A # we have cached copy on disk; use it 956N/A """ generate actions of the specified type; 956N/A use already in-memory stuff if already loaded, 956N/A otherwise use per-action types files""" 956N/A # invoke subclass method to generate action by action 2205N/A # This checks if we've already written out the factorerd 2205N/A # manifest files. If so, we'll use it, and if not, then 2205N/A # we'll load the full manifest. 956N/A # get manifest from disk 956N/A # invoke subclass method to generate action by action 956N/A # we have a cached copy - use it 956N/A return # no such action in this manifest 956N/A """Load attributes dictionary from cached set actions; 956N/A this speeds up pkg info a lot""" 2054N/A """No assignments to factored manifests allowed.""" 2054N/A assert "FactoredManifests are not dicts" 2073N/A """The absolute pathname of the file containing the manifest.""" 964N/A """Special class for pkgplan's need for a empty manifest; 964N/A the regular null manifest doesn't support get_directories 2054N/A and making the factored manifest code handle this case is 1045N/A """Return three lists of action pairs representing origin and 1045N/A destination actions. The first list contains the pairs 1045N/A representing additions, the second list contains the pairs 1045N/A representing updates, and the third list contains the pairs 1045N/A representing removals. All three lists are in the order in 1045N/A which they should be executed.""" 1045N/A # The difference for this case is simply everything in the 1045N/A # origin has been removed. This is an optimization for