Portscout is a neat tool that scans an entire ports tree and then checks the upstream sites for any new releases.

This tool was originally written for FreeBSD and this weekend I hacked it up to work on the OpenBSD ports tree too.

Welcome, OpenBSD!⌗

While adding initial OpenBSD support was surprisingly easy once a few small issues were sorted out (use show=FOO\ BAR instead of chaining -V FOO -V BAR) I actually added some new features to it.

The ports tree ensures that every single port has a list of backup master sites (MASTER_SITE_BACKUP), these would always get added to the list of sites portscout would check for new releases…which is pointless since these are are backup mirrors. Also, there may be other sites that you’d want to skip checking for a number of reasons.

mastersite ignore = *openbsd.org/pub/OpenBSD/distfiles/*

This even supports rudimentary wildcards, thus the above option with blacklist all the default backup sites.

So, that ensured I wouldn’t keep banging on ftp.OpenBSD.org.

However, it would only find about 6000 ports during a scan. Whereas there are more than 8000 ports (minus multi-packages which share the port directory). Turns out, it would only scan the toplevel directories it acquired via make -V SUBDIR in the top level directory.

In order for it to recurse into proper “subcategories” such as x11/gnome or lang/php some extra code was needed based on a simple heuristic. If a directory contains Makefile.inc, scan all the things! And suddenly it would find over 7600 ports.

Outstanding issues/missing features⌗

There are still some small issues/missing features that I’m aware of, and I’d be happy to merge a pull request over at jasperla/portscout which fix these:

  • we could use some awesome OpenBSD-themed templates
  • subdirectories which contain valid ports, but no Makefile.inc are skipped (like audio/festival or devel/jdk)
  • a template for a per-category view would be neat
  • some ports with seemingly valid MASTER_SITES trigger a Can't locate object method "host" via package "URI::_generic" at Portscout/DataSrc/Ports.pm
  • ports with failures during the (re)build phase or with unreachable master sites should be flagged/logged somehow
  • better support for DISTFILES handling

Aside from the above, the current codebase should be tested on FreeBSD again and submitted upstream.

Scan results⌗

A set of results from a scan which ran earlier today is available here