Makefile revision ae39239e04452933e89d03a1fb502e3e2836a332
2N/A# The contents of this file are subject to the terms of the 2N/A# Common Development and Distribution License (the "License"). 2N/A# You may not use this file except in compliance with the License. 2N/A# See the License for the specific language governing permissions 2N/A# and limitations under the License. 2N/A# When distributing Covered Code, include this CDDL HEADER in each 2N/A# If applicable, add the following below this CDDL HEADER, with the 2N/A# fields enclosed by brackets "[]" replaced with your own identifying 2N/A# information: Portions Copyright [yyyy] [name of copyright owner] 2N/A# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 2N/A# Make sure we're getting a consistent execution environment for the 2N/A# To suppress package dependency generation on any system, regardless 2N/A# of how it was installed, set SUPPRESSPKGDEP=true in the build 2N/A# Comment this line out or set "PKGDEBUG=" in your build environment 2N/A# to get more verbose output from the make processes in usr/src/pkg 2N/A# Cross platform packaging notes 2N/A# By default, we package the proto area from the same architecture as 2N/A# the packaging build. In other words, if you're running nightly or 2N/A# bldenv on an x86 platform, it will take objects from the x86 proto 2N/A# area and use them to create x86 repositories. 2N/A# If you want to create repositories for an architecture that's 2N/A# different from $(uname -p), you do so by setting PKGMACH in your 2N/A# For this to work correctly, the following must all happen: 2N/A# 1. You need the desired proto area, which you can get either by 2N/A# doing a gatekeeper-style build with the -U option to 2N/A# nightly(1), or by using rsync. If you don't do this, you will 2N/A# get packaging failures building all packages, because pkgsend 2N/A# is unable to find the required binaries. 2N/A# 2. You need the desired tools proto area, which you can get in the 2N/A# same ways as the normal proto area. If you don't do this, you 2N/A# will get packaging failures building SUNWonbld, because pkgsend is 2N/A# unable to find the tools binaries. 2N/A# 3. You need to have built the appropriate third party license 2N/A# files in $SRC, which generally means you should override SRC in 2N/A# your build environment. If you don't do this, you will get 2N/A# packaging failures because pkgsend is unable to find various 2N/A# THIRDPARTYLICENSE files. 2N/A# 4. The remainder of this Makefile should never refer directly to 2N/A# $(MACH). Instead, $(PKGMACH) should be used whenever an 2N/A# architecture-specific path or token is needed. If this is done 2N/A# incorrectly, then packaging will fail, and you will see the 2N/A# value of $(uname -p) instead of the value of $(PKGMACH) in the 2N/A# commands that fail. 2N/A# 5. Each time a rule in this Makefile invokes $(MAKE), it should 2N/A# pass PKGMACH=$(PKGMACH) explicitly on the command line. If 2N/A# this is done incorrectly, then packaging will fail, and you 2N/A# will see the value of $(uname -p) instead of the value of 2N/A# $(PKGMACH) in the commands that fail. 2N/A# Refer also to the convenience targets defined later in this 2N/A# ROOT, TOOLS_PROTO, and PKGARCHIVE should be set by nightly or 2N/A# bldenv. These macros translate them into terms of $PKGMACH, instead 2N/A# The publish transforms, EXCEPTIONS list, and some manifests need to 2N/A# know when we're building open-only and when we're using internal 2N/A# We only use internal crypto when we're doing a closed build, the 2N/A# CODESIGN_USER env variable is not set, and ON_CRYPTO_BINS is not set. 2N/A# This matches the conditions under which the internal key and cert 2N/A# are needed for the packaged objects. 2N/A# We use X_FLAG, as exported by nightly and bldenv, to decide when we 2N/A# need IHV-related exceptions for protocmp. 2N/A# Always build the redistributable repository, but only build the 2N/A# nonredistributable bits if we have access to closed source. 2N/A# Some objects that result from the closed build are still 2N/A# redistributable, and should be packaged as part of an open-only 2N/A# build. Access to those objects is provided via the closed-bins 2N/A# The packages directory will contain the processed manifests as 2N/A# direct build targets and subdirectories for package metadata extracted 2N/A# incidentally during manifest processing. 2N/A# Nothing underneath $(PDIR) should ever be managed by SCM. 2N/A# The tools proto must be specified for dependency generation. 2N/A# Publication from the tools proto area is managed in the 2N/A# To get these defaults, manifests should simply refer to $(PKGVERS). 2N/A# The ARCH32 and ARCH64 macros are used in the manifests to express 2N/A# architecture-specific subdirectories in the installation paths 2N/A# for isaexec'd commands. 2N/A# We can't simply use $(MACH32) and $(MACH64) here, because they're 2N/A# only defined for the build architecture. To do cross-platform 2N/A# packaging, we need both values. 2N/A# macros and transforms needed by pkgmogrify 2N/A# If you append to this list using target-specific assignments (:=), 2N/A# be very careful that the targets are of the form $(PDIR)/pkgname. If 2N/A# you use a higher level target, or a package list, you'll trigger a 2N/A# complete reprocessing of all manifests because they'll fail command 2N/A# dependency checking. 2N/A# The package lists are generated with $(PKGDEP_TYPE) as their 2N/A# dependency types, so that they can be included by either an 2N/A# incorporation or a group package. 2N/A# All packaging build products should go into $(PDIR), so they don't 2N/A# need to be included separately in CLOBBERFILES. 2N/A# By default, PKGS will list all manifests. To build and/or publish a 2N/A# subset of packages, override this on the command line or in the 2N/A# build environment and then reference (implicitly or explicitly) the all 2N/A# or install targets. 2N/A# Track the synthetic manifests separately so we can properly express 2N/A# build rules and dependencies. The synthetic and real packages use 2N/A# different sets of transforms and macros for pkgmogrify. 2N/A# For each package, we determine the target repository based on 2N/A# manifest-embedded metadata. Because we make that determination on 2N/A# the fly, the publication target cannot be expressed as a 2N/A# subdirectory inside the unknown-by-the-makefile target repository. 2N/A# In order to limit the target set to real files in known locations, 2N/A# we use a ".pub" file in $(PDIR) for each processed manifest, regardless 2N/A# of content or target repository. 2N/A# Any given repository- and status-specific package list may be empty, 2N/A# but we can only determine that dynamically, so we always generate all 2N/A# lists for each repository we're building. 2N/A# The meanings of each package status are as follows: 2N/A# ---------- ---------------------------------------------------- 2N/A# noincorp Do not include in incorporation or group package 2N/A# obsolete Include in incorporation, but not group package 2N/A# renamed Include in incorporation, but not group package 2N/A# current Include in incorporation and group package 2N/A# Since the semantics of the "noincorp" package status dictate that 2N/A# such packages are not included in the incorporation or group packages, 2N/A# there is no need to build noincorp package lists. 2N/A# Since packages depend on their incorporation, noincorp necessarily 2N/A# implies nodepend, or it would inadvertently pull in other packages. 2N/A# For a single manifest, the dependency chain looks like this: 2N/A# | use pkgmogrify to process raw manifest 2N/A# * | use pkgdepend generate to generate dependencies 2N/A# % | use pkgdepend resolve to resolve dependencies 2N/A# | use pkgsend to publish the package 2N/A# * This may be suppressed via SUPPRESSPKGDEP. The resulting 2N/A# packages will install correctly, but care must be taken to 2N/A# install all dependencies, because pkg will not have the input 2N/A# it needs to determine this automatically. 2N/A# % This is included in this diagram to make the picture complete, but 2N/A# this is a point of synchronization in the build process. 2N/A# Dependency resolution is actually done once on the entire set of 2N/A# manifests, not on a per-package basis. 2N/A# The full dependency chain for generating everything that needs to be 2N/A# published, without actually publishing it, looks like this: 2N/A# processed synthetic packages 2N/A# package lists synthetic package manifests 2N/A# processed real packages 2N/A# package dir real package manifests 2N/A# Here, each item is a set of real or synthetic packages. For this 2N/A# portion of the build, no reference is made to the proto area. It is 2N/A# therefore suitable for the "all" target, as opposed to "install." 2N/A# Since each of these steps is expressed explicitly, "all" need only 2N/A# depend on the head of the chain. 2N/A# From the end of manifest processing, the publication dependency 2N/A# chain looks like this: 2N/A# repository metadata (catalogs and search indices) 2N/A# | | pkgsend publish 2N/A# repositories resolved dependencies 2N/A# pkgsend | | pkgdepend resolve 2N/A# create-repository | 2N/A# | generated dependencies 2N/A# processed manifests 2N/A# Due to limitations in pkgdepend, we cannot simply treat synthetic 2N/A# and real manifests identically. But we don't really want to 2N/A# maintain a separate chain for synthetic manifests, so for the left 2N/A# side of this diagram, we actually do faux dependency generation and 2N/A# resolution, so we end up with the expected set of files in $(PDIR), 2N/A# per the individual file chain described above: mf, mog, dep, res, 2N/A# and pub files for each manifest. 2N/A# This will build the directory to contain the processed manifests 2N/A# and the metadata symlinks. 2N/A @
print "Creating $(@)" 2N/A# This rule resolves dependencies across all published manifests. 2N/A# We should be able to do this with 2N/A# pkgdepend resolve -m $(PUB_PKGS:%.pub=%.dep) 2N/A# but until 14113 is fixed, the incorporations confuse pkgdepend, so we 2N/A# just create the .res file for DEP_SYNTH_PKGS directly. 2N/A# We also shouldn't have to ignore the error from pkgdepend, but 2N/A# until at least 14110 is resolved, pkgdepend will always exit with 2N/A print "Suppressing dependency resolution"; \
2N/A print "Resolving dependencies"; \
2N/A @
print "Creating repository metadata" 2N/A# Since we create zero-length processed manifests for a graceful abort 2N/A# from pkgmogrify, we need to detect that here and make no effort to 2N/A# publish the package. 2N/A# For all other packages, we publish them regardless of status. We 2N/A# derive the target repository as a component of the metadata-derived 2N/A# symlink for each package. 2N/A# Initialize the empty on-disk repositories 2N/A @
print "Initializing $(@F)" 2N/A# rule to process real manifests 2N/A# To allow redistributability and package status to change, we must 2N/A# remove not only the actual build target (the processed manifest), but 2N/A# also the incidental ones (the metadata-derived symlinks). 2N/A# If pkgmogrify exits cleanly but fails to create the specified output 2N/A# file, it means that it encountered an abort directive. That means 2N/A# that this package should not be published for this particular build 2N/A# environment. Since we can't prune such packages from $(PKGS) 2N/A# retroactively, we need to create an empty target file to keep make 2N/A# from trying to rebuild it every time. For these empty targets, we 2N/A# do not create metadata symlinks. 2N/A# Automatic dependency resolution to files is also done at this phase of 2N/A# processing. The skipped packages are skipped due to existing bugs 2N/A# The incorporation dependency is tricky: it needs to go into all 2N/A# current and renamed manifests (ie all incorporated packages), but we 2N/A# don't know which those are until after we run pkgmogrify. So 2N/A# instead of expressing it as a transform, we tack it on ex post facto. 2N/A# Implementation notes: 2N/A# - The first $(RM) must not match other manifests, or we'll run into 2N/A# race conditions with parallel manifest processing. 2N/A# - The make macros [ie $(MACRO)] are evaluated when the makefile is 2N/A# read in, and will result in a fixed, macro-expanded rule for each 2N/A# target enumerated in $(PROC_PKGS). 2N/A# - The shell variables (ie $$VAR) are assigned on the fly, as the rule 2N/A# is executed. The results may only be referenced in the shell in 2N/A# which they are assigned, so from the perspective of make, all code 2N/A# that needs these variables needs to be part of the same line of 2N/A# code. Hence the use of command separators and line continuation 2N/A# - The extract_metadata transforms are designed to spit out shell 2N/A# variable assignments to stdout. So the eval statement should 2N/A# begin with the default values, and any output from pkgmogrify 2N/A# should be in the form of a variable assignment to override those 2N/A# - When this rule completes execution, it must leave an updated 2N/A# target file ($@) in place, or make will reprocess the package 2N/A# every time it encounters it as a dependency. Hence the "touch" 2N/A# statement to ensure that the target is created, even when 2N/A# pkgmogrify encounters an abort in the publish transforms. This 2N/A# will not cause publication failures when switching build 2N/A# environments, because $(CLOSED_BUILD) and $(OPEN_ONLY) are 2N/A# referenced in $(PKGMOG_DEFINES), and changes will therefore 2N/A# trigger a rebuild for command dependency failure. (Command 2N/A# dependency checking is turned on by .KEEP_STATE: above.) 2N/A @
print "Processing manifest $(<F)" 2N/A if [ -f
$(@) ];
then \
2N/A @
print "Generating dependencies for $(<F)" 2N/A# The full chain implies that there should be a .dep.res suffix rule, 2N/A# but dependency generation is done on a set of manifests, rather than 2N/A# on a per-manifest basis. Instead, see the gendeps rule above. 2N/A r=$${m
#$(@F:%.pub=%.metadata.)+(?).}; \ 2N/A if [ -s
$(<) ];
then \
2N/A print "Publishing $(@F:%.pub=%) to $$r repository"; \
2N/A# rule to build the synthetic manifests 2N/A# This rule necessarily has PKGDEP_TYPE that changes according to 2N/A# the specific synthetic manifest. Rather than escape command 2N/A# dependency checking for the real manifest processing, or failing to 2N/A# express the (indirect) dependency of synthetic manifests on real 2N/A# manifests, we simply split this rule out from the one above. 2N/A# The implementation notes from the previous rule are applicable 2N/A @
print "Processing synthetic manifest $(@F:%.mog=%.mf)" 2N/A if [ -f
$(@) ];
then \
2N/A @
print "Skipping dependency generation for $(@F:%.dep=%)" 2N/A# This rule assumes that all links in the $PKGSTAT directories 2N/A# point to valid manifests, and will fail the make run if one 2N/A# does not contain an fmri. 2N/A# We do this in the BEGIN action instead of using pattern matching 2N/A# because we expect the fmri to be at or near the first line of each input 2N/A# file, and this way lets us avoid reading the rest of the file after we 2N/A# We keep track of a failure to locate an fmri, so we can fail the 2N/A# make run, but we still attempt to process each package in the 2N/A# additional useful info. 2N/A# The protolist is used for bfu archive creation, which may be invoked 2N/A# interactively by the user. Both protolist and PKGLISTS targets 2N/A# depend on $(PROC_PKGS), but protolist builds them recursively. 2N/A# To avoid collisions, we insert protolist into the dependency chain 2N/A# here. This has two somewhat subtle benefits: it allows bfu archive 2N/A# creation to work correctly, even when -a was not part of NIGHTLY_OPTIONS, 2N/A# and it ensures that a protolist file here will always correspond to the 2N/A# contents of the processed manifests, which can vary depending on build 2N/A print "Generating $$r $$s package list"; \
2N/A for (i = 1; i < ARGC; i++) { \ 2N/A e = getline f < ARGV[i]; \ 2N/A l = split(f, a, "="); \ 2N/A print "depend fmri=" a[l], \ 2N/A "type=$$(PKGDEP_TYPE)"; \ # rules to validate proto area against manifests, check for safe # file permission modes, and generate a faux proto list # For the check targets, the dependencies on $(PROC_PKGS) is specified # as a subordinate make process in order to suppress output. # This is a convenience target to allow package names to function as # build targets. Generally, using it is only useful when iterating on # development of a manifest. # When processing a manifest, use the basename (without extension) of # the package. When publishing, use the basename with a ".pub" # Other than during manifest development, the preferred usage is to # avoid these targets and override PKGS on the make command line and # use the provided all and install targets. # This is a convenience target to resolve dependencies without publishing # These are convenience targets for cross-platform packaging. If you # want to build any of "the normal" targets for a different # architecture, simply use "arch/target" as your build target. # Since the most common use case for this is "install," the architecture # specific install targets have been further abbreviated to elide "/install." $(SED) -e
"/^# EXPORT DELETE START/,/^# EXPORT DELETE END/d" \
$(SED) -e
"/^# CRYPT DELETE START/,/^# CRYPT DELETE END/d" \