OpenBSD Ports: Differences from Other BSD Projects

OpenBSD Ports - Differences from Other BSD Projects [Handbook Index]

Extra Support
Generic Infrastructure Issues
Using make Properly
Fetching Sources
The WRKDIR Infrastructure
Faking Ports
Packaging Tools
Flavors

Extra Support

The porting infrastructure includes several scripts under infrastructure/bin that facilitate the creation of new ports:

check-lib-depends: invoked through make lib-depends-check, to verify shared libraries dependencies.
update-patches: invoked through make update-patches, which should always be used to regenerate patches.
update-plist: invoked through make update-plist. This takes care of most of the finest points of crafting accurate packing-lists. OpenBSD packing-lists are significantly different from those of other BSD projects, in part because the package tools have been completely rewritten.

Check the infrastructure/bin directory for more useful scripts. Most of them have man pages.

Generic Infrastructure Issues

OpenBSD's make(1) supports ${VAR:U} and ${VAR:L} to transform a variable's value into uppercase or lowercase. Accordingly, make tests should be coded in a case-independent way. For example:

.if ${NEED_XXX:L} == "yes"
do stuff if yes
.else
do other stuff
.endif

In theory, all boolean variables recognized by bsd.port.mk should always be defined, so that code like defined(USE_FOO) should not be necessary, and ${USE_FOO:L} != "no" ought to work.

The main bsd.port.mk file has been heavily streamlined and fixed. In particular, it is parallel-make ready. The scripts/{pre,do,post}-* feature has been lost in the process. To replace that feature, invoke the script manually from the Makefile.

Using make Properly

Note that if you invoke make as make VAR=value, the assignment will override whatever value VAR may get from the Makefile. This means that many Makefile patches are not necessary. It is much better to set MAKE_FLAGS correctly, which decreases the maintenance burden.

Fetching Sources

There are two kinds of source archives: DISTFILES and PATCHFILES. OpenBSD processes them in a uniform way and retrieves everything from MASTER_SITES by default. There are no PATCH_SITES nor PATCH_SITES_SUBDIR.

If all files to fetch don't come from the same set of sites, OpenBSD allows the extension filename:0 to filename:9, in which case it will use MASTER_SITES0 to MASTER_SITES9 to retrieve the file.

Some architectures may need specific distfiles. In the past, this has caused trouble where mirroring distfiles was concerned. OpenBSD supports a third set of files: SUPDISTFILES. Those will be considered only for creating checksums and mirroring purposes. Note that SUPDISTFILES may have an overlap with DISTFILES or PATCHFILES. For instance:

DISTFILES=foo-1.0.tgz
.if ${ARCH} == "i386"
DISTFILES+=foo-i386.tgz
.elif ${ARCHI} == "amd64"
DISTFILES+=foo-amd64.tgz
.endif
SUPDISTFILES=foo-i386.tgz foo-amd64.tgz

The `WRKDIR` Infrastructure

We don't want ports that use NO_WRKDIR. All OpenBSD ports must have a work directory. Naming details of those work directories should not be a porter's concerns. If you need to find out about such a name, ask the Makefile:

$ cd that_ports_dir && make show=WRKDIR

This will yield that port's idea of WRKDIR.

The main reason behind this prohibition is that OpenBSD's bsd.port.mk acts like a real Makefile, with dependencies. The fetch stage depends upon the distfiles and patchfiles, and all other stages depend on real files living in the working directory (cookies), so they can't exist without a working directory.

If the DISTFILES extraction is special, set

EXTRACT_ONLY=

and do the extraction in post-extract.

WRKDIR: The port working directory, where it puts its own cookies.
WRKDIST: Subdirectory of WRKDIR where the port actually unpacks. It is also the base directory for patch. Other BSDs currently don't have the WRKDIST/WRKSRC distinction and have only WRKSRC.
WRKSRC: Subdirectory of WRKDIST where the actual source lives.
WRKBUILD: Subdirectory of WRKDIR where the port configure and build will occur. Other BSDs don't have the WRKBUILD/WRKSRC distinction. Programs based on autoconf (mostly) can usually set SEPARATE_BUILD to let the port build happen in a WRKBUILD distinct from WRKSRC.
WRKCONF: Subdirectory of WRKDIR where configure scripts should be run. Defaults to WRKBUILD, which is correct 99% of the time.
WRKINST: Directory where the port will be installed before being packaged (see "faking ports" below).

Note that NO_WRKSUBDIR has been removed: its functionality can be achieved by setting WRKDIST=$(WRKDIR) instead.

Fake Port Installation

Introduction

After a build is complete, other BSDs proceed to install a port, then build a package from the installed port. OpenBSD uses faked installation instead.

An OpenBSD port is configured and built normally (e.g., to install under PREFIX, usually /usr/local).
But it's told to install elsewhere, namely under WRKINST, which is usually a subdirectory of WRKDIR.
Then the false installation is packaged, using the -B option of pkg_create.
Finally, the resulting package can be installed, using pkg_add.

Advantages

For a package builder, it means that most ports don't have to actually be installed, which removes a large number of potential compromises and general nastiness from badly-installed ports. It also allows building several conflicting packages on the same machine. Finally, it allows building a new set of untested packages without hosing a correct installation.
For a port writer, it greatly simplifies the task of finding problems in packing lists, since the fake area of installation is empty before the port gets installed. Also, if a port installs too many files, it is no longer necessary to tweak the port installation: it is enough not to record the extraneous files in the packing list.
For the end user, it improves the quality of packages: since the final port is installed using pkg_add, the end user gets exactly the same software that was prepared on the porter's machine.

How to do it

The targets invoked for make fake are the usual install targets, except for a few differences:

FAKE_FLAGS is used instead of MAKE_FLAGS. By default, FAKE_FLAGS sets DESTDIR=${WRKINST}.
FAKE_TARGET is used instead of INSTALL_TARGET.
The {pre,do,post}-install fragments are invoked with TRUEPREFIX set to $(PREFIX), PREFIX set to $(WRKINST)$(PREFIX), and DESTDIR set to $(WRKINST).

Ports using imake should work as is, since the imake fragments are configured to use DESTDIR. Similarly, recent GNU configure ports should need no change.

Another good technique is a "late binding" trick: configure the ports to use a prefix of $(DESTDIR)/usr/local, so that the resulting Makefile has the following set:

prefix=$(DESTDIR)/usr/local

When the port gets built, since DESTDIR is set to nothing, /usr/local is used. The fake install will put everything into ${WRKINST}/usr/local (e.g., for GNU configure, use CONFIGURE_STYLE= gnu dest).

Pitfalls

Some ports are inconsistent in their DESTDIR processing: most of the port is happy with DESTDIR set, except for one or two offenders. Patch the problem away.
Be careful to distinguish between the actual location where the port is installed, and the location recorded in the configuration files of the package. This is very easy to overlook, but easy to fix using TRUEPREFIX.
Absolute symlinks always need to be tweaked. Luckily, bsd.port.mk will notice problems in that area.
A few ports don't want to leave $(DESTDIR) alone at the configure stage. A post-configure fragment that tweaks all Makefiles to add the DESTDIR is needed.
Very seldom, a port will resist all reasonable attempts to use FAKE. A brute force approach should work: use pre-fake to link or copy everything the port wants to find in the WRKINST area, then perform the install under chroot.

Packaging Tools

The package tools know about quite a few file types and can do a lot of things automatically: in most cases @exec commands or INSTALL scripts are unneeded.

Note that all unneeded scripts should be banned, as they have scalability issues. It is much easier to debug one single package infrastructure than to modify hundreds of scripts to handle new problems. For instance:

@exec ldconfig is not needed, as shared libraries are annotated with @lib libfoo.so.1.0 and ldconfig runs only when needed, and handles chroot gracefully.
@exec install-info is not needed, as info documentation files are annotated with @info file.info. This also takes care of multiple info files, and removes the need for makeinfo --no-split.
fonts get integrated automatically thanks to @font and @fontdir.
New users and groups get handled with @newuser and @newgroup instead of installation scripts. They also get created early enough so that further package extraction can use them.
Most third party database handling is handled through @tag which triggers running tools like update-desktop-database once at the end of installation.
Configuration files are handled through @sample instead of installation scripts.

Refer to pkg_create(1) for more details. In most cases, make update-plist will write a very good approximation of a complete packing-list, and will carry hand-made tweaks from one version to the next.

Flavors

Options have been rationalized as flavors, so that package building can be consistent. A port with options should set FLAVORS to the list of all options that make sense for that port (e.g., FLAVORS=foo bar zoinx), then use FLAVOR to test what options have actually been selected (e.g., FLAVOR=zoinx foo). bsd.port.mk provides some support:

The PKGNAME is tweaked to include dash separated options (e.g., package-foo-zoinx).
The WRKDIR is tweaked so that distinct flavors can be built concurrently without colliding.
Constructs of the form %%flavor%% will trigger the inclusion of PFRAG.flavor.
bsd.port.subdir.mk understands the extension SUBDIR=directory,opt1,opt2 to say "build port in directory with FLAVOR=opt1 opt2."

Checking that a given flavor has been selected is as simple as:

.if ${FLAVOR:Mzoinx}

There is an extra extension, known as MULTI_PACKAGES. Generally speaking, MULTI_PACKAGES and FLAVORS are orthogonal mechanisms. Together, they account for OpenBSD ports tree being somewhat smaller than the other BSDs, as they allow one single port directory to build lots of distinct packages. bsd.port.mk(5) has a full section devoted to FLAVORS AND MULTI_PACKAGES.