1923N/A.. The contents of this file are subject to the terms of the
1923N/A Common Development and Distribution License (the "License").
1923N/A You may not use this file except in compliance with the License.
1923N/A See the License for the specific language governing permissions
1923N/A and limitations under the License.
1923N/A.. When distributing Covered Code, include this CDDL HEADER in each
1923N/A If applicable, add the following below this CDDL HEADER, with the
1923N/A fields enclosed by brackets "[]" replaced with your own identifying
1923N/A information: Portions Copyright [yyyy] [name of copyright owner]
1923N/A.. Copyright (c) 2010, Oracle
and/or its affiliates. All rights reserved.
1923N/A=================================
1923N/AImage Packaging Developer's Guide
1923N/A=================================
1923N/A^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1923N/A*PSARC/2008/190:* pkg(5): image packaging system
1923N/A^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1923N/A:Authors: David Comay, Danek Duvall, Tim Foster, Stephen Hahn, Krister Johansen,
1923N/A Dan Price, Brock Pytlik, Bart Smaalders, Shawn Walker
1923N/A:Contact: pkg-discuss@opensolaris.org
1923N/A.. contents:: Table of Contents
1923N/A For the purposes of review, this document--a draft of the
1923N/A developer guide--contains additional information that may be omitted
1923N/A from the final guide. Such information is generally set off from the
1923N/AIn this text, *The Image Packaging Developer's Guide*, we cover a range
1923N/Aof topics covering technical aspects of software delivery using |pkg5|,
1923N/Athe image packaging system. As you complete Parts I, II, and III of the
1923N/Adocument, you should feel comfortable
1923N/A* with basic |pkg5| principles of operation and use,
1923N/A* with authoring packages, decorated with correct metadata and named
1923N/A* renaming previously published packages, and
1923N/A* managing simple and complex |depotd1m| deployments.
1923N/AParts IV and V focus on implementing components capable of replacing
1923N/Athe default image packaging retrieval clients and depot servers,
1923N/AThe Image Packaging System, or |pkg5|, is a software delivery mechanism
1923N/Athat enables the publication and retrieval of versioned, interdependent
1923N/Asoftware components. The goal of this mechanism is to always produce a
1923N/Afunctioning image, in which each component in that image has its
1923N/Adependencies on other components met in a self-consistent fashion.
1923N/AUsually, when we refer to an image, we are referring to the *system
1923N/Aimage* associated with an instance of the |OS_Name| operating system,
1923N/Abut an image can also be used to deliver an independent set of software
1923N/Acomponents, or a set linked in some fashion to yet another image.
1923N/AInter-image relationships allow |pkg5| to be used in a wide variety of
1923N/Asoftware deployment contexts.
1923N/AThe idea of a versioned component comes from recognizing that, in
1923N/Ageneral, software components change over time, as bugs are fixed, new
1923N/Afeatures introduced, component boundaries changed, or the component
1923N/Aitself becomes irrelevant. We can look at the history of a component as
1923N/Aa series of versions of that component, persisting indefinitely. In the
1923N/Afollowing chapters, the relationship over time of a component to other
1923N/Acomponents, and to itself, will be revisited. The intent is to
1923N/Aemphasize that each software component has a lifecycle, and that there
1923N/Aare representative states and operations in |pkg5| as a component moves
1923N/AThere are a number of features in |pkg5| that make it particularly
1923N/Aappealing in a network context. In general, all |pkg5| metadata is
1923N/Asearchable remotely, easing the discovery of available software. |pkg5|
1923N/Acontent delivery is bandwidth-efficient, sending only needed data to the
1923N/Aclient, and always in a compressed form. Each publisher's depot
1923N/Asoftware has public information, which allows the identification of
1923N/Aother locations for software retrieval, such as mirrors for that depot
1923N/Aor publishers of related software. These features are discussed as they
1923N/Aapply to each of the components.
1923N/A|pkg5|, although it has a command line client, was developed to support
1923N/Aboth interactive use with other clients and programmatic use by higher
1923N/Alevel management software. We generally illustrate operations by
1923N/Areviewing the command line client. For developers and package
1923N/Apublishers, we will identify where package metadata, such as additional
1923N/Ainformation about the package or a specific object delivered by the
1923N/Apackage, can influence these different client uses. We, in addition,
1923N/Adescribe the more complex publication scenarios, such as signed
1923N/Amanifests and obsolescence. For operators and administrators, we
1923N/Adiscuss security, in the form of access control, and depot management.
1923N/AOur final sections focus on |pkg5| extensions, such as entitlement
1923N/AIn this section, we attempt to review the key concepts and indicate
1923N/Awhere further discussion and examples can be found in this Guide.
1923N/APackages may be installed into images; image contents are the
1923N/Aconsequence of package installations.
1923N/APackages, package versions, actions, and actuators
1923N/A--------------------------------------------------
1923N/A A package represents a set of related resources that can be
1923N/A installed into a file system location on a computing system. In the
1923N/A image packaging system, packages are distinguished by their package
1923N/A names and by their package versions. Only a single version of a
1923N/A package may be installed to a file system location.
1923N/A Since the name of a package often is the easiest way to identify its
1923N/A contents, the selection of a package name is an important step in
1923N/A package publication. The set of package names, or *package
1923N/A namespace* follows conventions to allow publishers to have uniquely
1923N/A named packages. As an example, the package name for the image
1923N/A names can be matched on unique substrings, which is convenient for
1923N/A interactive use, so we will often abbreviate package names to their
1923N/A unique portion. Thus, for command line invocations, we will often
1923N/A A package version is a multiple component object. Two versions of a
1923N/A particular package can always be ordered via a comparison: they
1923N/A will either be equal, or one will be less than the other. In the
1923N/A image packaging system, a version consists of three sub-versions and
1923N/A ``0.5.11,5.11-0.138:20100430T013957Z``.
1923N/A Each package version has a specific state in each image. For
1923N/A instance a package version may be *installed* or merely *known* by
1923N/A having an entry in a catalog.
1923N/APackage tags and file attributes
1923N/A As briefly mentioned, packages and their content may have additional
1923N/A information associated with them. There are both mandatory and
1923N/A optional metadata items that package publisher can provide and
1923N/A with which client software is expected to comply. Additionally,
1923N/A specific publishers may wish to provide additional metadata for
1923N/A their own use, or for use with the search facility.
1923N/A The resources that a package delivers are called actions, as
1923N/A they cause changes to the system on which the package is installed.
1923N/A Actions that differ between the current and proposed package
1923N/A versions are delivered; actions that do not differ are not
1923N/A There are a number of action types for delivering content to the
1923N/A file system, such as ``file``, ``directory``, and ``link``, as well
1923N/A as action types that deliver new system metadata, like the ``user``
1923N/A action, which delivers a new or modified user account. The file
1923N/A within the package that contains its set of actions is called a
1923N/A *manifest*. An arbitrary amount of metadata may be added to any
1923N/A action; such metadata items are called *tags*.
1923N/A A special kind of action tag is the actuator, which identifies a
1923N/A side effect this action will have when installed on the in-use image
1923N/A on a running system (a "live image"). Actuators have no effect on
1923N/A non-live images. One typical actuator use is to request a restart
1923N/A of a particular |smf5| service instance. This request is made by
1923N/A specifying the FMRI as the value for the ``restart_fmri`` actuator.
1923N/A We refer to a file system location configured to receive package
1923N/A installations as an image. Each collection of installed package
1923N/A instances is some kind of image. We envision three kinds:
1923N/A An entire image contains (and can contain) all appropriate
1923N/A packages from a repository. It has a machine type and release.
1923N/A A partial image contains (and can contain) all appropriate
1923N/A packages from a repository. An partial image is tied to an
1923N/A entire image and has an identified list of non-writeable
1923N/A directory paths. It inherits its machine type and release.
1923N/A A user image consists entirely of relocatable packages. It is
1923N/A tied to an entire image for dependency satisfaction of
1923N/A non-relocatable packages. It inherits its machine type and
1923N/APublishers, catalogs, and repositories
1923N/A--------------------------------------
1923N/A Each package has a publisher, which represents the person or
1923N/A organization that has assembled the package and makes it available
1923N/A Since a publisher can make one or more packages available, each
1923N/A publisher generally provides a list of all packages and versions
1923N/A currently available for retrieval. The file that contains this list
1923N/A Although a package [will be able to] be distributed independently,
1923N/A publishers with many packages may find it easier to publish that
1923N/A collection to a repository, which is a structured set of directories
1923N/A and files that contains the packages, their metadata, and their
1923N/A contents. A *depot* is a network server that can respond to
1923N/A requests for specific operations on [one or more] repositories.
1923N/ADependencies and constraints
1923N/A----------------------------
1923N/APackages can have relationships with other packages, such as a
1923N/A``require`` relationship, where the presence of a package is mandatory
1923N/Afor a second package to be installed. Each declaration of a
1923N/Arelationship in a package's metadata is done using a ``depend`` action,
1923N/Awhich expresses the relationship as a *dependency*. Dependency types
1923N/A A require dependency states that the package mentioned by the
1923N/A ``fmri`` attribute of the ``depend`` action must also be installed
1923N/A into the image, with version at or above the specified version. If
1923N/A no version is given, any version is allowed.
1923N/A An optional dependency states that the package mentioned by the
1923N/A ``fmri`` attribute, if a version is installed in the image, that
1923N/A version must be at or above the specified version.
1923N/A An exclude dependency states that, if the package version or a
1923N/A successor mentioned by the ``fmri`` attribute is installed in the
1923N/A image, the current package (containing the ``depend`` action) cannot
1923N/A An incorporation dependency expresses a special type of
1923N/A relationship, where the presence of the dependent package is
1923N/A constrained to the version range implied by the ``fmri`` portion of
1923N/A the ``depend`` action. Incorporation dependencies are used to
1923N/A restrict the set of package versions to a known and, presumably,
1923N/A tested subset of all available versions.
2084N/A A require-any dependency acts as a require dependency, except that
2084N/A multiple fmris may be specified, and any one being present will satisfy
2084N/A A conditional dependency states that the package mentioned by the
2084N/A ``fmri`` attribute of the ``depend`` action must also be installed
2084N/A into the image, with version at or above the specified version if the
2084N/A package named by the ``predicate`` attribute is present on the system
2084N/A at the specified level or higher.
2084N/A A origin dependency states that the the package mentioned by
2084N/A the ``fmri`` attribute must be at the specified level or higher (if present)
2084N/A prior to installation of the containing package. If the attribute ``root-image``
2084N/A is present and has the value ``true``, the image that is checked is the
1923N/A-----------------------------
1923N/A |pkg5| supplants the historical Solaris Operating System packaging
1923N/A tools, although these are still supported for compatibility. Moving
1923N/A a software component to the Image Packaging System requires some
1923N/A|smf5| configuration transition
1923N/A Certain resource types, such as manual pages and desktop icons, are
1923N/A delivered by multiple applications. New |smf5| service instances
1923N/A have been provided to simplify the integration of standard resources.
1923N/A For some applications, specific operations must be taken after
1923N/A package installation or removal. In addition to general techniques
1923N/A for handling such operations, we review specific utilities and
1923N/A services introduced for handling typical cases.
1923N/A - **Retrieval clients.** |pkg1|, PackageManager. UpdateManager.
1923N/A - relationship with |beadm1m|, |libbe3lib|, ZFS
1923N/A - **Publication and manipulation clients.** |pkgsend1| and |pkgrecv1|
1923N/A - |pkgdepend1|, |pkgdiff1|, |pkgfmt1|, |pkgmogrify1|
1923N/A - **Depot servers.** |depotd1m|
1923N/A---------------------------------
1923N/AAuthoring and publishing packages
1923N/A---------------------------------
1923N/APick a publisher name, based on a DNS domain you control.
1923N/AIf we examine the process of publishing a particular version of a
1923N/Apackage, we can see there are three sets of decisions:
1923N/A* package contents, or what files or other resources are delivered by
1923N/A this version of the package;
1923N/A* package name, or what the package is called; and
1923N/A* package metadata, or what additional information is provided to
1923N/A describe the package's purpose within a larger system.
1923N/ABefore we discuss these topics, we review how publication works.
1923N/A Additional options for access control to publication operations are
1923N/ATo publish a package, we use the |pkgsend1| command to open a transaction
1923N/Awith an active package depot, to send actions, which are the resources
1923N/Aand metadata delivered by the package, and finally to close the
1923N/Atransaction. Upon the request for the transaction to be closed, the
1923N/Adepot daemon will, for a valid submission, update its catalog and search
1923N/Aindices to reflect the newly published package version. Clients, such
1923N/Aas |pkg1|, can then refresh their local catalogs and retrieve the new
1923N/ALet's work through the above description in the form of an example.
1923N/AIn the following example, we wish to publish a simple package that
1923N/Asystem, contains some descriptive information about this particular
1923N/Arelease of the operating system.
1923N/A|pkg5| supports an extensible set of "actions", which are defined as
1923N/Areversible operations that a package can request to enable its later
1923N/Afunction on the target image.
1923N/A Packages need a limited set of operations on individual files to
1923N/A manipulate the configuration. The current class actions are given in
1923N/A Appendix A. It appears that if "manifest" and "rbac" were supported,
1923N/A along with some management of editable files (preserve, renamenew,
1923N/A initd, renameold), then the remaining operations could be deferred to
1923N/AWe can analyze the typical delivered components of a working operating
1923N/Asystem to identify the set of common actions. The decision to provide
1923N/Aan action for a specific class of resource is strongly influenced by the
1923N/Aneed for elements of that class to be fully configured for system boot to
1923N/Acomplete. Resources that can be configured after initial boot are
1923N/Agenerally not provided with actions, and are expected to use ``file``
1923N/Aactions to deliver content and an |smf5| service to derive and assemble
1923N/Aany necessary local configuration.
1923N/A Declare dependency on other packages.
1923N/A Package contains device driver Module loading will be disabled
1923N/A during operations on live images.
1923N/A All other files. Preservation and rename handling are managed as
1923N/A All hard and symbolic links.
1923N/A Package requires user, group, or other package-reference managed
1923N/A Record package attributes into legacy packaging metadata.
1923N/A License files, which deliver into the image metadata rather than
1923N/A Deliver a cryptographic signature for the containing manifest.
1923N/A <action name="dependency" payload="false" commitment="Committed" />
1923N/A <action name="directory" payload="false" commitment="Committed" />
1923N/A <action name="hardlink" payload="false" commitment="Committed" />
1923N/A <action name="legacy" payload="false" commitment="Committed" />
1923N/A <action name="license" payload="true" commitment="Committed" />
1923N/A <action name="link" payload="false" commitment="Committed" />
1923N/A <action name="driver" payload="false" commitment="Committed" />
1923N/A <action name="file" payload="true" commitment="Committed" />
1923N/A <action name="group" payload="false" commitment="Committed" />
1923N/A <action name="user" payload="false" commitment="Committed" />
1923N/A <action name="signature" payload="false" commitment="Committed" />
1923N/A It is discouraged, but certainly possible to deliver custom actions
1923N/A including those actions in a separate package that the new package
1923N/A requires, and invoking the |pkg1| client twice--once to deliver the
1923N/A custom actions and once to use them to install the new package.
1923N/A XXX The deployer may wish to deny such actions from operating. For this
1923N/A case, the set of known actions is fixed elsewhere in the pkg modules
1923N/A and updated with subsequent versions. A global and per-image policy,
1923N/A known-actions-only, allows the deployer to disallow operations on
1923N/A packages utilizing actions of unknown provenance.
1923N/A <policy name="known-actions-only" scope="global,image"
1923N/A type="boolean" commitment="Committed">
1923N/A Deployer control over execution of unknown actions.
1923N/AThose system configuration steps which can be deferred.
1923N/AMany of the good packaging criteria present trade-offs among themselves. It
1923N/Awill often be difficult to satisfy all requirements equally. These criteria are
1923N/Apresented in order of importance; however, this sequence is meant to serve as a
1923N/Aflexible guide depending on the circumstances. Although each of these criteria
1923N/Ais important, it is up to you to optimize these requirements to produce a good
1923N/AOptimize for Client-Server Configurations
1923N/A-----------------------------------------
1923N/AYou should consider the various patterns of software use
1923N/A(client and server) when laying out packages. Good packaging
1923N/Adesign divides the affected files to optimize installation of each
1923N/Aconfiguration type. For example, for a network protocol implementation,
1923N/Ait should be possible to install the client without necessarily
1923N/APackage by Functional Boundaries
1923N/A--------------------------------
1923N/APackages should be self-contained and distinctly identified with a set of
1923N/Afunctionality. For example, a package containing UFS should contain all UFS
1923N/Autilities and be limited to only UFS binaries.
1923N/APackages should be organized from a customer's point of view into functional
1923N/APackage Along License or Royalty Boundaries
1923N/A-------------------------------------------
1923N/APut code that requires royalty payments due to contractual agreements or
1923N/Athat has distinct software license terms in a dedicated package or group
1923N/Aof packages. Do not to disperse the code into more packages than
1923N/AWhen constructing the packages, ensure that duplicate files are eliminated when
1923N/Apossible. Unnecessary duplication of files results in support and version
1923N/Adifficulties. If your product has multiple packages, constantly compare the
1923N/Acontents of these packages for redundancies.
1923N/ASize is package-specific and depends on other criteria. For example, the
1923N/Amaximum size of /opt should be considered. When possible, a good package should
1923N/Anot contain only one or two files or contain extremely large numbers of files.
1923N/AThere are cases where a smaller or larger package might be appropriate to
1923N/ALicensing Considerations for Packages
1923N/A-------------------------------------
1923N/AIf you are distributing software that uses licensing, there are several things
1923N/A*Business Operations.* Before you begin distributing licensed software, set up
1923N/Ayour business operations to distribute, price, and track licenses. There are a
1923N/Avariety of ways to distribute licenses, such as fax, electronic mail, or an 800
1923N/Atelephone number. You need to choose a method of distribution and set up all
1923N/Athe necessary processes. You also need to consider whether licenses need to be
1923N/Aupgraded with the software and how this will be done.
1923N/APricing policy and types of licenses must also be considered. You must consider
1923N/Ahow the product is used and what kinds of licenses your users will need to use
1923N/Athe product effectively. Single user licenses may not be appropriate for many
1923N/A*Communication with Users.* Before you implement licensing, you need to inform
1923N/Ayour users, particularly if the product has not been licensed in the past.
1923N/AWhen you do implement licensing, you may want to consider implementing it
1923N/Agradually. The first step would be monitoring the use of licenses, followed by
1923N/Awarning that the software is being used without a license, and finally, denying
1923N/A*Technology.* If you are going to use a commercial product for licensing, there
1923N/Aare many things to consider when making your choice. You need to decide what
1923N/Ayour priorities are. For example, is ease of administration and use most
1923N/Aimportant? Or is enforcing the licensing policy more important?
1923N/AYou also need to consider whether the software will be used in a heterogeneous
1923N/Aor homogeneous environment and whether standards are important. You may also
1923N/Awant to look at the security provided by the product. Is it easy to get around
1923N/Athe enforcement of licenses?
1923N/AThe issues involved in choosing a commercial product will vary depending on
1923N/Athe kind of application and your reasons for implementing licensing.
1923N/AIn the following sections, we give examples of delivering specific
1923N/Aresources via the image packaging system.
1923N/AXXX pick a specific Gnome application
1923N/ACoordinating groups of packages using incorporations
1923N/A----------------------------------------------------
1923N/AMaking a package version obsolete
1923N/A---------------------------------
1923N/AMoving a file between packages
1923N/A------------------------------
1923N/AMigrating an existing package
1923N/A=============================
1923N/AMigrating a System V Package
1923N/A----------------------------
1923N/APublishing from an installation directory
1923N/A-----------------------------------------
1923N/Apkgmogrify, pkgdepend, pkgdiff, and pkgfmt.
1923N/ADistributing packages with a depot
1923N/A==================================
1923N/AUsing Apache HTTPD as a reverse proxy cache
1923N/A-------------------------------------------
1923N/ASplitting and spreading package retrieval load
1923N/A~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1923N/APublishing packages with a depot
1923N/A================================
1923N/AXXX could we automate snapshots?
293N/A- protocol / network format
1923N/A - publication side REST API
1923N/ARetrieval protocol operations
1923N/A=============================
1923N/APublication protocol operations
1923N/A===============================
1923N/A A GET operation that retrieves the current set of operations
1923N/A offered by the contacted depot.
1923N/A List of operations and versions, one operation per line, space
1923N/A separated list of versions for each operation.
1923N/A A GET operation that presents a query to the search index
1923N/A capability of the contacted depot.
1923N/A--------------------------------
1923N/APackage depots and other servers
1923N/A--------------------------------
1923N/A--------------------------------
1923N/AAppendix: Reference manual pages
1923N/A--------------------------------
1923N/A------------------------------------------
1923N/AAppendix: Architectural process materials
1923N/A------------------------------------------