#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Copyright (c) 2006, 2012, Oracle and/or its affiliates. All rights reserved.
#
Mapfiles and versioning in ON
=============================
1.0 Objective of this README
This README describes the engineering practices of creating and updating
visible library interfaces. It describes various kinds of actions that
typically occur as libraries are evolved, and shows how interface
specifications are affected or updated in accordance. It tells you what
you must do as a shared library developer if you:
1. Make interface additions to an existing library
- add a Public interface
- add a Private interface
2. Update an interface in an existing library
- remove an existing interface
- promote a Private interface to Public
- scope a Private interface to local
- move an interface from one library to another
- copy interfaces which are part of the standard to a new or
existing library
3. Introduce a new library
- source directory hierarchy
- creation of the "mapfile-vers" file
- Makefiles
4. Need to specify the mapfile information that allows a library
to be buildable as a stub object:
- enable the stub object feature
- specify ASSERT information for data symbols
-------------------------------------------------------------------------------
2.0 What's a mapfile?
Mapfiles are used to tell the link-editor ("ld") all sorts of things about
how to generate an executable file or a shared object from a collection of
relocatable objects, such as generated by a compiler. For all the gory
details, see the Solaris Linker and Libraries Guide, which can be found
under http://docs.sun.com.
There are two versions of the mapfile language accepted by the link-editor.
Version 1 derives from AT&T System V Release 4 Unix. Version 2 is a newer
syntax specific to Solaris. All mapfiles in the OSnet (ON consolidation) are
required to use version 2 syntax. Note that every mapfile using version 2
syntax must start with the line:
$mapfile_version 2
Here, we are only concerned with specifying externally-visible interfaces
for shared libraries (shared objects) and with specifying their versions
for ABI (Application Binary Interface) purposes. For these purposes, we
only need to deal with a subset of the mapfile language.
There should be a "mapfile-vers" file associated with every shared library
and it should reside in the common source directory for that library, most
often in a "common" directory. This is the usual layout of a library's
top-level directory (usr/src/lib/libwombat):
Makefile amd64/ i386/ sparcv9/
Makefile.com common/ sparc/
The "common" directory contains the source files and other common files
for the library:
The mapfile's name is, by convention, "mapfile-vers" because it is used
for only two purposes: to specify externally-visible interface names while
suppressing visibility of all other names, and to specify their respective
unique version names.
-------------------------------------------------------------------------------
3.0 Contents of mapfile-vers
The structure of mapfile-vers is best explained by an example
(the license notification and copyright notice is omitted here
for brevity):
$mapfile_version 2
STUB_OBJECT;
SYMBOL_VERSION SUNWpublic { # public libwombat symbols
global:
wb_read;
wb_readv;
wb_stat;
wb_write;
wb_writev;
};
SYMBOL_VERSION SUNWprivate { # private libwombat symbols
global:
wb_add;
wb_delete;
wb_search;
local:
*;
};
The STUB_OBJECT directive specifies that this library can be
built as a stub object as well as a normal object. STUB_OBJECT
enables additional tests and sanity checking within the link-editor
(ld) to ensure that the stub object will be a faithful representation
of the object for linking purposes. All libraries in the OSnet
(ON consolidation) must be buildable both ways, so the use of
STUB_OBJECT is not optional for ON mapfiles.
All new public symbols are added to version SUNWpublic.
All new private symbols are added to version SUNWprivate.
Existing Solaris objects may have public versions with numbered
names (e.g. SUNW_1.1). These versions date from older versions of
the OS, which used a different set of versioning rules. All such
versions must be preserved to ensure backward compatibility.
In this case, the SUNWpublic version must inherit all symbols
from the highest numbered SUNW_x.y version. The libwombat mapfile
might then look like:
$mapfile_version 2
STUB_OBJECT;
SYMBOL_VERSION SUNWpublic { # public libwombat symbols
global:
wb_readv;
wb_stat;
wb_writev;
} SUNW_1.1;
SYMBOL_VERSION SUNW_1.1 { # first release of libwombat, Solaris 9
global:
wb_read;
wb_write;
};
SYMBOL_VERSION SUNWprivate { # private libwombat symbols
global:
wb_add;
wb_delete;
wb_search;
local:
*;
};
The two lines in SUNWprivate:
local:
*;
ensure that no symbols other than those listed in the mapfile are
visible to clients of the library. If there is no SUNWprivate,
these two lines should appear in SUNWpublic.
For maintainability, the list of names in each version block should
be sorted in dictionary order (sort -d). Please comply.
The version 2 mapfile language supports a simple mechanism for conditional
input, in which lines in the mapfile apply only to a specific platform or
ELFCLASS (32/64-bit). This mechanism works very much like the #if/#endif
feature of the C preprocessor. For instance, the following mapfile declares
a version SUNWpublic that always exports a symbol foo, and also exports
the symbol bar on 32-bit sparc platforms:
$mapfile_version
SYMBOL_VERSION SUNWpublic {
foo;
$if _sparc && _ELF32
bar;
$endif
};
Conditional input can be used if there are ISA-specific library interfaces
not common to all instances of the library. It is the preferred method for
expressing platform specific items, as long as the differences are simple
(which is almost always the case). For example, see libproc, or, if you
are masochistic, libc or libnsl.
In addition to conditional input, there is a second heavier weight mechanism
for expressing ISA-specific differences. In addition to the common mapfile:
some libraries may have ISA-specific supplemental mapfiles, one in each
of the ISA directories:
The ISA-specific mapfiles look like the common mapfile, except that only
the ISA-specific names appear. The version names are the same as those
in the common mapfile, but only non-empty version instances are present
and no inheritance specification is present. The link-editor reads the
information from the common and ISA-specific mapfiles and merges them
in memory into a single description used to create the resulting object.
ISA-specific mapfiles were used with the version 1 mapfile language, which
lacked conditional input. Their use is rare now, as conditional input is
generally preferred. However, it is important to use conditional input
carefully, or the resulting mapfile can be extremely difficult to read.
-------------------------------------------------------------------------------
4.0 Making interface additions to an existing library
4.1 Adding a Public interface
All new public symbols are assigned to the SUNWpublic version. If the
existing mapfile-vers does not have this version, the engineer adding
the new symbol must add it. If the library contains any existing public
versions of the form SUNW_x.y, then the new SUNWpublic version inherits
from the highest such SUNW_ version. If there are no SUNW_x.y versions, then
the SUNWpublic versions stands alone, and does not inherit.
Incompatible changes to an existing interface within a given library are not
allowed. If an API changes so dramatically as to invalidate dependencies, it
becomes necessary to change the suffix of the shared object from, say, .so.1
to .so.2 and to introduce code to continue to ship the .so.1 version of the
library. This rarely occurs in practice, and is discouraged. A better solution
is to introduce the new functionality under a new name, and retain the old
interface with the old behavior for the benefit of existing applications.
4.2 Adding a Private interface
Private interfaces are the non-ABI interfaces of the library, and are added
to the SUNWprivate version. If this interface happens to be the first
Private interface introduced into the library, the SUNWprivate version must
be created. It inherits nothing and nothing inherits from it.
If the library already has Private interfaces, they may have numbered version
names like SUNWprivate_m.n (due to errors of the past). If so, just use the
highest numbered private version name to version the new interface. There
is no need to introduce a new private version name. Be careful not to use
a lower numbered private version name; doing so can cause runtime errors
(as opposed to load time errors) when running an application with older
versions of the library.
There are libraries in the OSnet consolidation that contain only private
interfaces. In such libraries, the SUNWprivate_m.n may be incremented
to ensure that the programs that depend on them are built and delivered as a
integrated unit. A notable example of this is libld.so (usr/src/cmd/sgs/libld),
which contains the implementation of the link-editor, the public interface to
which is provided by the ld command. When making a modification to the interface
of such a library, you should follow the convention already in place.
4.3 Adding new public interfaces in an update release
Public symbols must appear in the same version in all Solaris releases.
Therefore, when porting a new public interface to an update release,
you must ensure that the symbol is assigned to the same version as
in the release you are porting from. If the library in the update
release already has this version, you simply add the new symbol to it.
If the version does not already exist, you must create it, along with
any missing versions it may inherit from, and then assign the new interface
to the newly created version.
When adding a new public interface to an update, both the mapfiles of the
update release and next marketing release must be consistent. This means
that all new functionality must be first integrated in the marketing gate,
and then ported to the update afterwards.
-------------------------------------------------------------------------------
5.0 How to update an interface in an existing library
5.1 Removing an existing interface
5.1.1 Moving a Public interface
No Public interfaces should ever be removed from any mapfile.
To move an interface from one library to (say) libc, the code has to be
deleted from the library and added to libc, then the mapfile for the
library has to have the interface's entry changed from:
getfoobar;
to:
getfoobar { TYPE = FUNCTION; FILTER = libc.so.1 };
See, for example, libnsl's common/mapfile-vers file.
Follow the rules for adding a new interface for the necessary changes
to libc's mapfile to accommodate the moved interface. In particular,
the new interface must be added to SUNWpublic in the libc mapfile-vers.
To move an entire library into libc, look at what has already been done
for libthread, libaio, and librt.
5.1.2 Removing a Private interface
Deletion of Private interfaces is allowed, but caution should be taken;
it should first be established that the interface is not being used.
To remove a Private interface, simply delete the corresponding entry
for that symbol from the mapfile's SUNWprivate section.
Do not forget to delete these Public or Private interfaces from the library's
header files as well as from the code that implements the interfaces.
5.2 Promoting a Private interface to Public
This is similar to what's done when adding a Public interface. Promoting an
existing Private interface to a Public one only requires a change to the
existing interface definition. Private interfaces have the symbol version name
"SUNWprivate" associated with them. To make the interface a Public one, the
interface must be moved to the SUNWpublic version for the library.
5.3 Scoping a Private interface local
Any interfaces not present in the mapfile-vers file will be scoped local
due to the presence of the
local:
*;
lines discussed earlier. This ensures that such interfaces will not be visible
outside the library. To move an interface from Private to local scope, simply
remove the Private interface from the mapfile-vers file and the header file
to prevent it from being exported. This may require moving the Private
interface into a library-private header file. Scope reduction of Public
interfaces is not allowed without specific ARC review and approval.
For the interface to be used in more than one file within the library, it
should be in a header file that can be included by each file in the library
that uses the interface. For example:
#include "libprivate.h"
-------------------------------------------------------------------------------
6.0 Introducing a new library
6.1 Directories
The normal discipline for introducing a new library in OS/Net is to create a
new subdirectory of /usr/src/lib. The interface definition discipline is to
create a common/mapfile-vers file for the new library. If we were introducing
a new foo library, libfoo, we'd create /usr/src/lib/libfoo containing:
Makefile amd64/ i386/ sparcv9/
Makefile.com common/ sparc/
The common subdirectory would contain the normal source files plus the
mapfile-vers file. See usr/src/lib/README.Makefiles for directions on
how to organize the Makefiles.
6.2 The mapfile
The new common/mapfile-vers file would contain:
$mapfile_version 2
STUB_OBJECT;
SYMBOL_VERSION SUNWpublic {
global:
...
};
SYMBOL_VERSION SUNWprivate {
global:
...
local:
*;
};
If there are no Public interfaces, the SUNWpublic section would be omitted.
If there are no Private interfaces, the SUNWprivate section would be
omitted and the two lines:
local:
*;
would be moved into SUNWpublic.
To decide which interfaces are Public (part of the ABI) and which are Private
(unstable interfaces not intended to be used by third party applications or
unbundled products), the heuristic which works to a first approximation is
that if it has a man page then it's Public. Also, it is really the ARC case
for the new interfaces that prescribes which interfaces are Public and
which are not (hence, which interfaces have man pages and which do not).
For maintainability, the list of names in each version block should
be sorted in dictionary order (sort -d). Please comply.
-------------------------------------------------------------------------------
8.0 Specify Stub Object Details
Stub objects are shared objects, built entirely from their mapfiles,
that are, for the purposes of linking, identical to the actual
shared objects that would result if you built them in the usual
manner. Unlike the real objects:
- Stub objects can be built extremely quickly. Stub objects
for all of usr/src/lib and usr/src/ucblib can be built in
a couple of minutes on a modest desktop system.
- Stub objects are self contained, and are built without requiring
input objects or the existence of any dependency shared objects.
Therefore, all stub objects for the OSnet can be built in parallel,
without ordering or synchronization.
- Real objects can be built using stub objects to stand in for their
real dependencies. Using stub objects in this way, an object,
and the other objects it requires, can be built in parallel, in
any arbitrary order.
Stub objects are a fundamental part of how we build ON, so every library
in the OSnet (ON consolidation) must be buildable as a stub object:
1) Before any objects are built in an ON nightly build, a
stub proto area is established, referenced as $(STUBROOT) in
our makefiles.
2) A stub object for every library is built and installed under
$(STUBROOT).
3) All libraries and executables are linked to the objects found
under $(STUBROOT).
This approach allows every library and command in Solaris to be built
in parallel, and (with a small number of exceptions), without any
explicit ordering or serialization between them.
To enable the possibility of building a library as a stub object, the
library mapfile must specify the STUB_OBJECT directive.
If your library exports only function symbols, then the use of STUB_OBJECT
is all that is required, and you can skip the remainder of this section.
There are many reasons why shared objects should not export global data
directly, and we have long discouraged it. New libraries should never
do this. It is usually easy enough to expose global data via a functional
interface. For instance, Solaris usually exposes the errno global variable
to via the following macro definition found in <errno.h>
extern int *___errno();
#define errno (*(___errno()))
So the prohibition on global data symbols is not the same thing as
prohibiting global data --- we simply recommend that it be exposed
via a functional interface.
Good advice aside, there are two important cases in which you will
need to cope with global data symbols in ON mapfiles:
1) Existing libraries with historical interfaces that must be maintained
2) Third party code, which is out of our control and not subject to
our rules.
Data symbols exported by shared objects built with the STUB_OBJECT
mapfile directive are required to provide mapfile symbol ASSERT directives
for each data symbol, for the following attributes:
- That the symbol is a data symbol.
- The size of the symbol, in bytes.
- The symbol binding, if it is not GLOBAL.
- If the symbol resides in a bss (NOBITS) section in the object.
- If the symbol is an alternative name (ALIAS) for another symbol.
These ASSERT directives serve 2 related purposes:
1) The generation of the stub object is done without access to the symbol
information usually provided with any input relocatable objects.
Instead, the ASSERT directives supply the necessary information
required to generate the correct symbol table entry.
2) When building the normal non-stub version of the object, the
attributes of the real object are compared to the ASSERT values,
and any deviation result in fatal link-time errors.
As a result, the stub and normal versions of the object are guaranteed
to present a completely equivalent linking interface. You can therefore safely
link against the stub, and then run against the non-stub.
The libc mapfile (usr/src/lib/libc/port/mapfile-vers) contains a large
number of historical data symbols, and serves as a good example of how
to write these ASSERT directives. The environ data symbol serves as a
useful example
SYMBOL_VERSION SYSVABI_1.3 {
...
global:
...
environ {
ASSERT {
TYPE=data;
$if __GNUC
SH_ATTR=nobits;
$endif
SIZE=addrsize;
BINDING=weak;
};
};
...
};
Note that it usually suffices to specify TYPE and SIZE --- environ is
a bit more complex than the usual case.
TYPE=data
All data symbols must explicitly assert that they
are expected to be data symbols.
SIZE
All data symbols must assert their size, in bytes. This can
be a decimal, or hexadecimal value. The symbolic name
'addrsize' can also be used, to represent a machine word (pointer)
sized variable. The link-editor automatically substitutes the value
4 for 'addrsize' when building 32-bit objects, and 8 for 64-bit objects.
In addition, SIZE values can specify a repeat count, which is
convenient for describing arrays. All of the following are
valid SIZE values:
Value Meaning
----------------------------------------------------------
SIZE=4 4 bytes
SIZE=0x209 521 bytes
SIZE=addrsize 4 bytes in a 32-bit object,
8 bytes in a 64-bit object.
SIZE=4[4] 16 bytes (4, 4 byte elements)
SIZE=addrsize[3] 12 bytes in a 32-bit object,
24 bytes in a 64-bit object
----------------------------------------------------------
BINDING
Most global symbols have the GLOBAL binding, and the BINDING
assert attribute need not be specified. However, data with WEAK
binding must be explicitly declared.
SH_ATTR
In rare situations, the link-editor will produce different
results when linking against a shared object with global
data, depending on whether the data resides in a regular
section, or a BSS (NOBITS) section. Data that resides in
BSS must therefore contain an assertion to that effect.
This example with the environ symbol is especially interesting,
as the native Oracle Studio compilers do not place environ in
BSS, while gcc does, hence the use of the $if __GNUC to
conditionalize the SH_ATTR attribute.
Another complication involves multiple symbols that all reference the
same physical data address, often with differing global and weak binding.
In order to produce an accurate stub object, the mapfile must include
ALIAS assertions. The lone symbol in libc provides an example of this:
lone { ASSERT { TYPE=data; SIZE=8 } };
_lone {
FLAGS = NODYNSORT;
ASSERT { ALIAS=lone; BINDING=weak };
};
When ALIAS is used, TYPE and SIZE are omitted, as their values are taken
from the source symbol.
-------------------------------------------------------------------------------
9.0 Documentation
For further information, please refer to the following documents:
"Solaris Linker and Libraries Guide", the latest version of
which is available internally from http://linkers.us.oracle.com/
For information on the now-obsolete spec files, used in Solaris releases
7 through 10, see: