#

# CDDL HEADER START

#

# The contents of this file are subject to the terms of the

# Common Development and Distribution License (the "License").

# You may not use this file except in compliance with the License.

#

# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE

# or http://www.opensolaris.org/os/licensing.

# See the License for the specific language governing permissions

# and limitations under the License.

#

# When distributing Covered Code, include this CDDL HEADER in each

# file and include the License file at usr/src/OPENSOLARIS.LICENSE.

# If applicable, add the following below this CDDL HEADER, with the

# fields enclosed by brackets "[]" replaced with your own identifying

# information: Portions Copyright [yyyy] [name of copyright owner]

#

# CDDL HEADER END

#

#

# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

#

Introduction

------------

This README describes some basics about creating, modifying, and

building packages in ON. All package creation in ON is done using

tools available to our customers. If terminology in this document

is confusing, you may wish to review the pkg(5) manpage.

Packaging Overview

------------------

usr/src/pkg/ contains the definitions and rules needed to build an IPS

repository which contain the deliverables from an ON build.

The manifests directory contains all manifests, and has one file

per package. For most packaging changes you only need to add or

change the packaging manifests themselves.

The build rules create a package repository. Separate DEBUG and

non-DEBUG versions are built, and are available at:

$CODEMGR_WS/packages/$MACH/nightly[-nd]/repo.redist

Building Packages

-----------------

The -p option to nightly will build the IPS repository.

Alternatively, in usr/src/pkg/Makefile there are make targets for:

all

Process manifests into their final form with unresolved

dependencies before publication.

install

Publish packages.

gendeps

Run `pkgdepend resolve`. See Dependencies section.

protocmp

Compare the proto area against package definitions for

missing or incorrect files.

pmodes

Check file and directory modes for best practices.

check

Run protocmp and pmodes.

The build behavior may modified by the following variables:

SUPPRESSPKGDEP

If SUPPRESSPKGDEP is set to "true" in your environment,

package dependencies will not be generated. This variable

should not be set in normal builds as it will mask product

bugs.

PKGDEBUG

If PKGDEBUG is set in your environment, $MAKE will print

detailed information about the commands it executes to

process and publish packages.

ONNV_BUILDNUM

If ONNV_BUILDNUM is set to an older ON build number,

your packages will be published at that version instead

of the one defined in usr/src/Makefile.buildnum, which

is managed by the gatekeepers.

A set of intermediate build products are placed in

usr/src/pkg/packages.$MACH/. These can be useful during development.

.mog

The resulting package manifest after running pkgmogrify(1)

on the supplied manifest. See below for details on

pkgmogrify(1) use in ON.

.dep

The resulting manifest after running `pkgdepend generate`

on the .mog manifest.

.res

The resulting manifest after running `pkgdepend resolve`

on the .dep manifest.

Incremental Builds

------------------

If you want to process a subset of manifests, simply set PKGS on the

make command line and specify the "all" target. If you want to process

them all, simply specify the "all" target.

% dmake -e PKGS="BRCMbnx BRCMbnxe" all

% dmake -e all

If you want to publish a subset of packages, simply set PKGS on the

make command line and specify the "install" target. Overriding PKGS

will cause dependency resolution to be limited to PKGS from the

current build, with a fallback to packages installed on the build

system. If you want to publish them all, simply specify the

"install" target.

% dmake -e PKGS="BRCMbnx BRCMbnxe" install

% dmake -e install

You can also use package names, or package names with ".pub"

extensions, as build targets. This will cause processing or

publication of the specified package(s).

The on-disk repository will be initialized when it does not exist,

or when you run nightly -p. If you build incrementally,

packages will simply be added to the existing repository.

To do cross-platform packaging, prefix your target with (for

example) "sparc/", as in "dmake sparc/install". Note that we

currently pull some license files directly from a built source

tree, so if you do this in a workspace that had proto area copied

in via nightly -U, then you'll need to set $SRC to point to the

workspace that was actually built.

Testing Packages

----------------

To test the generated repository, you should use the "onu" tool

available from /opt/onbld/bin or usr/src/tools/scripts/ to setup and

upgrade your system. A manpage is available in /opt/onbld/man

or usr/src/tools/scripts/onu.1.

Alternatively, you can manually start a pkg.depot(1M) server to

serve the generated repository to multiple test machines.

Start up a depot on your build machine.

cd $CODEMGR_WS/packages/$MACH/nightly[-nd]

/usr/lib/pkg.depotd -d repo.redist -p <port> &

Make SURE you choose an unused port and the depot

starts successfully.

The depot can be started across NFS as well if you

have a fast pipe.

Configure your test system.

pkg set-publisher -P -g http://<your server host>:<port> on-nightly

pkg set-publisher --non-sticky opensolaris.org

pkg uninstall entire

pkg image-update your test system.

pkg image-update will upgrade all packages on your test system

Always make sure your bits are installed with image-update.

Check your versions.

pkg info osnet-incorporation

Multiple packages should be updated.

If you did a full build, all ON packages will be updated.

When image-update tells you that only one or two packages has

been updated, you likely did not get the updates you expected.

There are various tactics for troubleshooting a failed upgrade.

Make sure entire is uninstalled.

pkg install -nv osnet-incorporation@<your version>

Ask IPS to explicitly check if ON can be installed, and

if it can't, tell you why not.

Obsolete and renamed packages can cause trouble.

pkg search -l ::pkg.renamed:true

pkg search -l ::pkg.obsolete:true

Making Packaging Changes

------------------------

Package definitions are in usr/src/pkg/manifests/, and have one

file per package, including for multi-architecture packages. For

most packaging changes you only need to add or change the packaging

manifests themselves. No packaging metadata may be kept outside of the

manifests. If you find yourself needing to modify usr/src/pkg/Makefile,

you're almost certainly doing something wrong.

Remember that the "check" target is available to check many of

your packaging changes.

We run pkgmogrify(1) over the manifests before publication. This

allows us to apply a set of macros and package transformations in

order to make the manifests themselves easier to maintain.

We supply a set of commonly-used macros for use in package manifests.

These are the PKGMOG_DEFINES in usr/src/pkg/Makefile.

$(i386_ONLY)

$(sparc_ONLY)

$(ARCH)

$(ARCH32)

$(ARCH64)

$(PKGVERS), which is set to

$(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-0.$(PKGVERS_BRANCH)

pkgmogrify(1) also allows us to include a set of default transformations.

The definitions for these transforms are in usr/src/pkg/transforms/. An

overview of their use is supplied here, but for a full accounting, please

read pkmogrify(1) and the files themselves.

defaults

This transform is applied to all manifests. It specifies

a set of sensible default permissions, a set of

directory locations for which the reboot-needed actuator

is always applied to files, and some other basic defaults.

publish

This transform is applied to all manifests. It ensures

that manifest lines which don't apply to the current

architecture are stripped.

restart_fmri

This transform is applied to all manifests. It modifies

all package manifest lines for SMF manifests in standard

locations to include an actuator which runs manifest-import

on installation/update/removal, as well as some others. If

you're adding a new class of file that would benefit from

a restart or refresh of a specific SMF service, please add

it here.

extract_metadata

This transform is applied to all manifests. It deals with

manipulations required for packaging metadata like

pkg.renamed, and pkg.obsolete.

hollow_zone_pkg

This transform is available for explicit inclusion in

some manifests. It ensures that all contents of the

package are not installed within a non-global zone, but the

package and its metadata are available in order to satisfy

packaging dependencies.

pkgmogrify(1) also allows us to use comments and continuation lines

in our manifests.

Zones and Packages

------------------

pkg(5) uses variants to implement zones. If a package is marked

with both global and non-global zone variants, the package contents will

be installed in both global and non-global by default.

set name=variant.opensolaris.zone value=global value=nonglobal

Specific actions within a package can be tagged as applying to only

the global zone or only the non-global zones.

The hollow_zone_pkg transform described above is also available to

simplify a common packaging scenario.

Dependencies

------------

Package dependencies are automatically calculated during build time

using pkgdepend(1). After you've done a build, you can verify your

dependencies in the <package>.res file described above. If the

file is missing a dependency that you believe could be auto-detected,

please file a bug against pkgdepend(1).

Dependencies can be added manually using the "depend" action. Please

add a comment describing why the dependency is required.

Moving Content Between Packages and Removing Content

----------------------------------------------------

pkg(5) tracks when content is removed from packages, and automatically

removes it.

If you need to move content between packages, there are two primary

things to do.

"preserve" files must be marked with original_name.

The first time a "preserve" file moves between packages,

you must set original_name=<original package>:<file>

in that file's action. Subsequent moves do not require

modification.

Consider adding a dependency on the new package.

The only way a system will end up with a new package

after upgrade is if an existing package depends on it.

Renaming a Package

------------------

To rename a package, leave the old package manifest in place, but

empty it of all delivered content. The old package should include:

set name=pkg.fmri with the version set explicitly to the

build you're integrating into. For example, if you wanted

to rename SUNWrmodu in build 133 you'd change the pkg.fmri

line to read

set name=pkg.fmri value=pkg:/SUNWrmodu@0.5.11,5.11-0.133

set name=pkg.renamed value=true

The architectures and variants you're renaming. These

should just be copied from your old package, as you

must rename a package on all architectures and

variants simultaneously.

A dependency on the new package.

If there were "preserve" files in the package you're renaming, make

sure to create original_name settings in the new package.

If there was a org.opensolaris.noincorp property in the package being

renamed, make sure you keep it in both the original and the renamed

packages.

EOFs and Removals

-----------------

To remove files, directories, drivers, or anything else within a package,

simply stop delivering them in the package. IPS will manage the removal

of no longer delivered content.

Package removals have impact on the rest of the system. Before marking

a package as obsolete, search in the OpenSolaris development

repository (http://pkg.opensolaris.org/dev or http://ipkg.sfbay/dev)

to find out if any other packages depend on the software you intend

to EOF. If any packages do, you need to coordinate with those

consolidations.

The "slim_install" package may depend on ON packages. If it does,

you must send a FLAG DAY message for ON users and PIT who test

install. You must also file an installation bug to get that package

updated in the same build or earlier than you intend to integrate.

You should also test install yourself. You can do this by replacing

the "slim_install" in your Distro Constructor manifest with the

explicit list of packages to install.

To remove a package, you must mark it as obsolete. You must *also* mark

as obsolete any packages which are renamed ancestors of this package, and

remove their rename dependencies. Here is what you must do.

To find rename ancestors, select all of the manifests which are renames,

then look for the one which was renamed to the package you care about.

For example, to find rename ancestors of 'system/zones', do the following:

$ cd usr/src/pkg/manifests

$ mypkgname=system/zones

$ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.mf` /dev/null

SUNWzone.mf

Make sure to check that the package has not undergone multiple renames!

$ mypkgname=SUNWzone

$ grep -l "fmri=pkg:/$mypkgname@" `grep -l pkg.renamed *.mf` /dev/null

$

Once you have the renamed ancestor list, for *each* of the packages (the

newly obsolete package, and its renamed ancestors), edit the package as

follows:

Update 'set name=pkg.fmri' with the version set explicitly to the

build you're integrating into. For example, if you wanted

to remove SUNWwbsd in build 133 you'd change the pkg.fmri

line to read:

'set name=pkg.fmri value=pkg:/SUNWwbsd@0.5.11,5.11-0.133'

Add 'set name=pkg.obsolete value=true'.

Maintain the architecture and variant declarations in the

package you are obsoleting. Note that you must obsolete a

package on all architectures and variants simultaneously.

Delete everything else.

If the package is a renamed ancestor, leave a comment behind as

follows:

# Was renamed to <other-pkg-name>, both now obsolete.

Here is a complete example. SUNWfoobar was a package which was renamed

to system/foobar in build 140, and then later obsoleted in build 150.

Note that in all cases the package FMRI is updated to the obsoletion

build, 150.

SUNWfoobar.mf:

# Was renamed to system/foobar, both now obsolete.

set name=pkg.fmri value=pkg:/SUNWfoobar@0.5.11,5.11-0.150

set name=pkg.obsolete value=true

set name=variant.arch value=$(ARCH)

system-foobar.mf:

set name=pkg.fmri value=pkg:/system/foobar@0.5.11,5.11-0.150

set name=pkg.obsolete value=true

set name=variant.arch value=$(ARCH)

Creating a Package

------------------

The easiest thing is to copy a package similar to the one you're

trying to create. Note that packages are no longer split on the

/usr and / boundary.

The following actions are required for all packages in ON.

set name=pkg.fmri

Every package must have an FMRI. That is the package's

name.

set name=pkg.summary

Every package must have a short summary of its purpose.

set name=pkg.description

Every package must have a one-sentence description of

its purpose.

set name=variant.arch

Every package must specify which architectures it delivers.

set name=info.classification

Every package must specify a category for the packaging GUI.

You must use an existing category, and may not invent new ones.

Existing categories and their subcategories are listed

in /usr/share/package-manager/data/opensolaris.org.sections.

license

All packages must specify a set of license actions. If

you're adding items here that are not already included in

usr/src/pkg/license_files, then you will also need to modify

usr/src/tools/opensolaris/license-list.

You don't need to set the following. They're taken care of for all OS/Net

packages in the transforms/common_actions file.

set name=variant.opensolaris.zone

Every package must specify whether it can be installed in

global zones, non-global zones, or both. All ON packages are

delivered in both global and non-global zones.

set name=org.opensolaris.consolidation value=osnet

All packages from OS/Net come from OS/Net...

Drivers and Packages

--------------------

Scripting is no longer required to deal with addition or removal of

drivers in ON. A "driver" action should be specified for each driver,

and IPS will handle updates to all the driver files.