packaging.txt revision 7034
46N/A
46N/A Userland Consolidation Packaging Guidelines.
46N/A
46N/A Each component that integrates into the Userland consolidation must have at
46N/Aleast one package manifest that describes the content to be delivered. In some
46N/Acases components *may* deliver through multiple packages. Canonical component
46N/Apackage manifests must be placed in the component's build directory. They also
46N/Amust be named *.p5m.
46N/A
46N/A In order to understand what must go in the content of a package manifest,
46N/Ait's useful to have an understanding of how a canonical manifest is transformed
46N/Ainto a final manifest used for package publication. Manifest transformation
46N/Atakes the following basic path:
46N/A
46N/A canonical manifest
46N/A (.../{component}/{component}.p5m)
46N/A |
46N/A v
46N/A mogrified manifest
166N/A (.../{component}/{build-dir}/manifest-$(MACH)-{component}.mogrified)
46N/A |
46N/A v
181N/A mangled manifest file contents
181N/A (.../{component}/{build-dir}/manifest-$(ARCH)-{component}.mangled)
181N/A |
181N/A v
46N/A dependencies generated
166N/A (.../{component}/{build-dir}/manifest-$(MACH)-{component}.depend)
46N/A |
46N/A v
46N/A dependencies resolved
185N/A (.../{component}/{build-dir}/manifest-$(MACH)-{component}.depend.res)
46N/A |
46N/A v
46N/A manifest validation
185N/A (.../{component}/{build-dir}/.linted-$(MACH))
46N/A |
46N/A v
46N/A publication manifest
166N/A (.../{component}/{build-dir}/manifest-$(MACH)-{component}.published)
46N/A |
46N/A v
46N/A publication
46N/A
46N/A
46N/ACanonical Manifest
46N/A The canonical manifest contains actions that can't otherwise be generated
46N/A automatically from the data encapsulated in the component Makefile, gate
46N/A transformations, build tree, and packaging tools. This includes actions
46N/A for license information, some path related attributes, legacy actions,
185N/A non-discoverable dependencies, users, groups, drivers, and others.
46N/A
46N/A Actions that are associated with objects that are specific to a single
46N/A architecture should be tagged with a 'variant.arch' attribute specific to
46N/A the architecture that applied to the action. Ex:
46N/A file path=/usr/lib/$(MACH64)/libx86onlybits.so variant.arch=i386
46N/A
46N/A Actions for editable files must include an appropriate 'preserve' attribute:
151N/A file path=etc/gnu/a2ps.cfg preserve=true mode=0644
46N/A
46N/A license actions should be placed in the canonical manifest.
46N/A
3468N/AManually generated actions
3468N/A * com.oracle.info.description is a terse description of what utilities,
3468N/A libraries and/or services the package provides. This should be short,
3468N/A specific, concise text, identifying the technology covered by the
3468N/A associated license(s). It should fit naturally in the sentence "This
3468N/A package may contain XXX." For example, "XXX" might be "the tar command"
3468N/A or "bzip2 compression software." When appropriate, this may begin with
3468N/A "portions of" or another, more specific qualifying clause.
3468N/A * com.oracle.info.tpno is the Oracle 3rd party license number.
3468N/A * info.classification is "org.opensolaris.category.2008:FOO" where FOO
3468N/A varies according to the sorts of utilities, libraries and/or services
3468N/A that the package provides. Existing packages contain most useful
3468N/A values; check them out to find the closest match. For a complete
3468N/A list of allowed values, refer to the Solaris system file
3468N/A /usr/share/lib/pkg/opensolaris.org.sections .
3468N/A * org.opensolaris.arc-caseid is typically "PSARC/YYYY/###" and multiple
3468N/A different values are allowed.
3468N/A * pkg.summary is a short synopsis of what the package provides.
3468N/A * org.opensolaris.consolidation is the name of the consolidation delivering
3468N/A the package. In Userland, this is $(CONSOLIDATION) (which expands to
3468N/A "Userland" during the build). Manifests in the Userland gate can also
3468N/A decorate this package attribute with an 'incorporate={incorporation-name}'
3468N/A decoration to specify where the package should be incorporated at the end
3468N/A of the userland build. A special value of 'none' will cause the package
3468N/A to be unincorporated and float freely from the rest of the rest of the
3468N/A packages. Note that unincorporated packages don't automatically get
3468N/A updated with the rest of the system when 'pkg update' is run unless the
3468N/A unincorporated package(s) are specified on the command line.
3468N/A
46N/AMogrified Manifest
166N/A The canonical manifest is combined with a set of the transforms
166N/A in $(WS_TOP)/transforms, and a set of macros to more complete
46N/A package manifest using pkgmogrify(1). The transforms apply default
46N/A attributes to the various actions in the canonical manifest(s). More
46N/A detail about the attributes can be found in the transform file themselves.
46N/A The macros applied at the time of mogrification are as follows:
46N/A $(MACH)
46N/A $(MACH32)
46N/A $(MACH64)
46N/A $(PUBLISHER)
46N/A $(CONSOLIDATION)
46N/A $(BUILD_VERSION)
46N/A $(SOLARIS_VERSION)
46N/A $(OS_VERSION)
166N/A $(IPS_COMPONENT_VERSION)
166N/A $(COMPONENT_VERSION)
46N/A $(COMPONENT_PROJECT_URL)
46N/A $(COMPONENT_ARCHIVE_URL)
46N/A
46N/ADependencies Generated
46N/A The mogrified manifest and the prototype install tree are passed through
46N/A pkgdepend(1) to generate a set of dependencies for the package content.
46N/A These dependencies are only those that "pkgdepend generate" can determine
185N/A on its own. Additional dependencies that cannot be automatically
46N/A determined by pkgdepend(1) should be placed in the canonical manifest.
166N/A Statically defined dependencies should be described in a canonical manifest
46N/A in an unresolved form (ie. the form generated by "pkgdepend generate").
46N/A Ex:
46N/A depend fmri=__TBD pkg.debug.depend.file=etc/passwd \
46N/A pkg.debug.reason=usr/bin/vipw type=require
46N/A
46N/A depend fmri=__TBD pkg.debug.depend.file=sh \
46N/A pkg.debug.depend.path=usr/bin \
46N/A pkg.debug.depend.reason=usr/bin/psmandup \
46N/A pkg.debug.depend.type=script type=require
46N/A
46N/A This will allow the next step to resolve all dependencies to their proper
46N/A package(s).
46N/A
46N/ADependencies Resolved
185N/A The manifest with unresolved dependencies is passed through pkgdepend(1)
1640N/A again to resolve dependencies against the package repositories. The result
1640N/A is a manifest that is suitable for publication. All these manifests are
1640N/A processed together in a single step, which is more efficient than resolving
1640N/A dependencies in each manifest separately. While each manifest ends up with
1640N/A a .depend.res copy in the build directory, the umbrella dependency
1640N/A resolution target is {build-dir}/.resolved-$(MACH).
1640N/A
1640N/A The resolution step is also set up to use the -e flag to pkgdepend resolve,
1640N/A which limits the set of packages it looks at to resolve the dependencies it
1640N/A generated in the previous step. This makes the resolution step a great deal
1640N/A faster, but requires that you keep a static list of these packages checked
1640N/A into the workspace, and update it when packages are added to it. Having
1640N/A extra packages in there is safe.
1640N/A
1640N/A In order to create this list, build and publish your component (or at least
1640N/A through the resolution stage) without a file "resolve.deps" in the component
1640N/A directory, and run "gmake sample-resolve.deps". If the file is empty (that
1640N/A is, no computed dependencies were found), a warning will be emitted and the
1640N/A file will be removed, as pkgdepend currently errors out in that case.
1640N/A
1640N/A To test, run "gmake clean" and re-publish.
1640N/A
1640N/A Don't forget to "hg add resolve.deps"!
1640N/A
1640N/A Note that there is a possibility the list of dependencies will be different
1640N/A on different architectures, so you should run this on both sparc and x86,
1640N/A and combine the two lists. Please keep the files sorted.
46N/A
46N/AManifest Validation
46N/A The resolved manifest(s) and prototype install tree are passed through
46N/A a set of validations. This includes running pkglint(1), comparing the
46N/A manifest content to the prototype install tree, and validation of the file
185N/A content of the prototype install tree. Any anomalies are reported.
46N/A Content validation is performed by extension to pkglint(1) in
46N/A $(WS_TOP)/tools/python/userland-lint
46N/A
3778N/A Note that when integrating new packages, and one or more of them depends
3778N/A on or more of the others, then this may result:
3778N/A WARNING pkglint.action005.1 obsolete dependency check skipped:
3778N/A unable to find dependency (target pkg) for (source pkg)
3778N/A This means that the target package was not found in the reference repo,
3778N/A which was the source of the pkglint cache that was created when 'gmake
3778N/A setup' was run after the workspace was created; thus the warning is
3778N/A harmless and can be ignored in this circumstance.
3778N/A
46N/APublication.
46N/A Once manifest validation has occurred, the package(s) is/are finally
46N/A published to the workspace package repository.
46N/A
3778N/ARenames
3778N/A
3778N/A Renames in IPS are tricky. We will use a case study to illustrate how
3778N/A this needs to work.
3778N/A
3778N/A All of the library/python-2/FOO modules that we had in Userland at the
3778N/A time were renamed to library/python/FOO in s12_41. This involved:
3778N/A * changing the FMRI to drop the "-2"
3778N/A * adding an optional dependency on the old name at the version and build
3778N/A in which the rename occurred; this forces the rename
3778N/A * depend type=optional fmri=library/python-2/FOO-$(PYV)@VERSION,BUILD
3778N/A where FOO is the component name (e.g., "alembic", "amqp", etc.),
3778N/A "$(PYV)" is that literal string, VERSION was the expanded value of
3778N/A COMPONENT_VERSION from each component's Makefile (e.g., "0.6.0"
3778N/A for alembic, "1.0.12" for amqp, etc.), and BUILD was set to
3778N/A "5.12-5.12.0.0.0.41.0". Note that BUILD needs to be set to
3778N/A the value of the build you are integrating into.
5630N/A * creation of the file "history" (or adding to it if it already exists)
5630N/A in each affected component directory; one or more new lines should be
5630N/A added (typically one for the versionless package, plus one for each
5630N/A versioned instance of the package) with syntax:
5630N/A OLD-NAME@VERSION,BUILD NEW-NAME
5630N/A for renames and:
5630N/A OLD-NAME@VERSION,BUILD
5630N/A for obsoletions, where FOO, VERSION and BUILD are all as above. Most
5630N/A Python modules, including the two mentioned above, have such files
5630N/A which can serve as examples.
3778N/A
7034N/A By default, packages created from the history files will be part of
7034N/A the userland-incorporation. If a package needs to be included in a
7034N/A different incorporation, then incorporate=INCORPORATION may be
7034N/A placed on the end of either the rename or obsoletion line.
7034N/A Examples with incorporate=consolidation/desktop/gnome-incorporation
7034N/A may be found in components/meta-packages/history/history.
7034N/A
7034N/A It's possible to specify whether the record applies to just i386 or
7034N/A sparc by providing arch=i386 or arch=sparc.
7034N/A
3778N/A So far, not too bad. This gets slightly more complicated when back-
3778N/A porting in that BUILD needs to be set to the back-port build (e.g.,
3778N/A "5.11-0.175.3.0.0.8.0", as was the case for tkinter-27) rather than
3778N/A the S12 build.
3778N/A
3778N/A If the component is at the same version as when the rename was done, then
3778N/A that's it. But if the component has since been upgraded, then care must
3778N/A be taken to set VERSION in both places above to the value that it was in
3778N/A s12_41 when the rename was originally done. In particular, the latest
3778N/A version of a package name in an earlier release branch must not be greater
3778N/A than the latest version of that same package name in the newer release
3778N/A branch. Otherwise the audits from Release Engineering will complain and
3778N/A our gatekeeper will make you do a follow-up push to fix this.
46N/A
5631N/A=============================
5631N/AComponent EOF (End Of Feature)
5631N/A=============================
5631N/AThis is a brief description on how to remove a component from Userland for
5631N/Aan EOF.
5631N/A
5631N/ADeleting the Component
5631N/A----------------------
5631N/A
5631N/AAt some point, you will need to do a "hg remove" of the component's directory.
5631N/AThis can be done before or after you make the necessary entry in the
5631N/Acomponents/meta-packages/history/history file. In this file you will see plenty
5631N/Aof examples like this:
5631N/A
5631N/Abackup/areca@7.1-5.12.0.0.0.85.0
5631N/A
5631N/AYou will need to make sure you have the correct package name, version, and
5631N/Abuild you are targeting for the EOF for your new entry. Also check for an old
5631N/ASUNW entry at the top of the file. If there is one you will need to change
5631N/Athis to match the targeted build. So something like this:
5631N/A
5631N/ASUNWareca@7.1,5.11-0.133 backup/areca@7.1-0.133
5631N/A
5631N/AShould be changed to:
5631N/A
5631N/ASUNWareca@7.1-5.12.0.0.0.85.0
5631N/A
5631N/ARemember that the package may have had numerous names in its history.
5631N/AGenerally the way to find this is by using:
5631N/A
5631N/A pkg search -o search.match_type,pkg.name depend::<package name>
5631N/A
5631N/AFirst with the package to be EOF'ed, and then examining each of the dependent
5631N/Apackages. If there are require dependencies on any of the dependents, then the
5631N/Alatter should be examined to see if it need to be either EOF'ed at the same
5631N/Atime, or at least updated to remove the dependency you're trying to EOF.
5631N/A
5631N/ACheck to see if the package you are removing is referenced by a group package
5631N/A(pkg list -n 'group/*'). See the 'EOFs and Removals' and 'Group Packages'
5631N/Asections in /usr/src/pkg/README.pkg from the on-gate for further details.
5631N/A
5631N/ATesting
5631N/A-------
5631N/A
5631N/ATo test your changes the first thing you will need to do is to invoke
5631N/A'gmake publish' from the components/meta-packages/history directory or at the
5631N/Atop level (a full Userland build) if you are removing multiple components. Once
5631N/Athis is complete you will need to setup a client with the EOF package(s)
5631N/Ainstalled.
5631N/A
5631N/AThen perform the following steps:
5631N/A
5631N/A1. # pkg set-publisher \
5631N/A -P -p file:///net/strax/builds/your_name/your_dir/i386/repo
5631N/A
5631N/A
5631N/AIf you are removing just a single component:
5631N/A
5631N/A2. pkg change-facet 'facet.version-lock.<eof_package_name_goes_here>=false'
5631N/A3. pkg update pkg://nightly/<eof_package_name_goes_here>
5631N/A
5631N/AAfter the update the component package should be removed from the test system.
5631N/A
5631N/A
5631N/AIf you are removing multiple components:
5631N/A
5631N/A2. pkg change-facet \
5631N/A version-lock.consolidation/userland/userland-incorporation=false
5631N/A3. pkg update pkg://nightly/consolidation/userland/userland-incorporation
5631N/A
5631N/AAfter the update the EOF component packages should be removed from the test
5631N/Asystem.
5631N/A
5631N/AFor languages and anything else that has modules or is otherwise part of the
5631N/AUserland build environment you will need to do a full build, of both ISAs, on
5631N/Athe test machines you just updated.
46N/A# vi:set fdm=marker expandtab ts=4: