Apache Archiva 1.3.3 released: performance improvements!

If you’re using Archiva for your repository management needs, you should definitely upgrade to the latest release. Download it now!

The 1.3.x series has focused on the biggest offenders in memory usage and performance problems, and Archiva 1.3.3 brings the biggest improvements yet:

• Full scans should take about 1/3rd of the time and consume far less memory
• Removed one-off memory hits at the end of a scan
• File descriptor use during concurrent deployments are better managed

In addition, a new system status page is available for assessing the cause of potential performance issues at runtime, giving better insight into how to tune memory or scanning settings appropriately.

This work is in advance of the upcoming Archiva 1.4 release which has revived the internals more significantly, with further performance improvements and a series of new features.

It’s also worth noting that we dropped support for Archiva 1.1.x and Archiva 1.2.x in November, so there’s no reason left to remain on older versions.

I’d like to thank YourKit, who provided a free license for their profiler, which was of great assistance in tracking down these issues. I’ve used it on occasion for a number of years, and it is one of the easiest tools to use that I’ve ever encountered.

The full set of issues resolved follow:

• [MRM-1097] – Error 500 "too many open files"
• [MRM-1369] – Editing user roles in archiva clobbers continuum redback roles
• [MRM-1396] – Purge task problem : Not enough parts to the path
• [MRM-1421] – Archiva repository purge incorrectly purges based on file timestamps even when the snapshot timestamp is known
• [MRM-1443] – repository statistics collection can cause server to hang
• [MRM-1416] – upgrade to Redback 1.2.5
• [MRM-1439] – improve indexing performance
• [MRM-1440] – system status page
• [MRM-1441] – monitor repository scanning progress
• [MRM-1442] – track time spent in each consumer during a scan, to help diagnose poor scanning performance
• [MRM-1445] – disable referrer check by default

Using a GPG agent for signing Maven releases on Mac OS X

Using the Maven GPG Plugin makes it easy to sign a large number of artifacts when performing a release with Maven.

However, one of the annoying parts is that interactive password entry is not particularly easy, and you often have to put it on the command line or into your settings file in plaintext, which is not very comfortable. The better alternative is to use gpg-agent, and if you’re not using GPG 2.0 this needs to be configured in the POM:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-gpg-plugin</artifactId>
    <version>1.1</version>
    <configuration>
      <useAgent>true</useAgent>
    </configuration>
  </plugin>

GPG will fallback to the same password entry if it is not running or if it is provided via the property, so there’s no particular downside to enabling the option.

The problem on Mac is that the situation with GPG is a little confusing. There seem to be several different projects floating around to provide it.

I’m a big fan of Homebrew, and previously I’d been using that to install GnuPG and gpg-agent. It worked quite well, but the password entry required curses, and also didn’t work in an interactive session – so you’d need to make sure to have entered it manually before starting a release. If it failed mid-release, you’d need to restart the agent. None of this was Homebrew’s fault – I just hadn’t been able to find a native pinentry app for the Mac that stood alone and didn’t crash on use.

After fiddling with it for a while, I decided to try one of the native Mac GPG installers again. This is where it got confusing, since there are several efforts floating around. Others may work, but the one I had success with is the revived GPGMail project. After uninstalling all the packages using Homebrew, I installed GPGTools from the download page. Since this includes a native pinentry application, and pre-configures gpg and the agent to run once on login, it worked much better.

The only catch if you’re trying this is that you need to logout, or in your shell run this until you have:

open /usr/local/libexec/start-gpg-agent.app/
. ~/.gpg-agent-info
export GPG_AGENT_INFO

You can test that it is working outside of Maven by running: gpg -ab (ending with Ctrl-D),

Last chance to register for Training and Receive Maven Book Free

Packt Publishing, the publishers of Apache Maven 2: Effective Implementation, will be present at ApacheCon North America, coming up on November 1 – 5, 2010. They’ll have an exhibitors table with information about their books on Apache projects such as OFBiz, Tomcat, MyFaces, jMeter, Maven, and more.

They’ve generously offered to sponsor free copies of the Maven book for attendees to the Maven training course on November 1. However, this could be extended until today only, so that the correct number can be ordered. Please register now!

One of the attractions of writing for Packt was their involvement in open source, such as through their Open Source Royalties program, and their Open Source Awards. I’m glad to see that they’ll be at ApacheCon.

Apache Maven 3.0 Released: a Few Important Tips

Just short of the 5 year anniversary of the Maven 2.0 release (Oct 19, 2005), Maven 3.0 has shipped today. You can download it from the Maven website. I’ve now been using it for about 6 months for all but one project, by which point it was already quite stable. Luckily it also arrives just in time for ApacheCon, since I have updated my training to cover it! All-in-all it’s a great release – quite a bit faster and in places more predictable.

New Features

For the most part, this is not a feature release, but a performance and architectural release. Much has already been written about underlying technology changes (like switching from Plexus to Guice). The main points of interest for me are:

• parallel builds – build modules in parallel when enabled to utilise multiple cores and get even more performance gains. It’s optional, so take some time to try it out
• improved performance & predictability – while maintaining and documenting compatibility
• improved reactor – behaves more consistently between building multi-module projects and subsets of them
• validation and error reporting – unrecommended and deprecated behaviour is now pointed out and error reporting improved
• improved classloading – extensions and plugins are loaded in a more self-contained fashion to allow more flexibility

There are still some “gotchas”, tips and tricks to take note of, however.

Check the Compatibility Notes

There is a page dedicated to compatibility notes between Maven 2 and Maven 3. This should be considered required reading for anyone making the switch, as it highlights some changes that you may need to adjust your projects or environment for. In most cases there’ll be no major issues, and only some quick fixes. In my opinion, the key ones to keep an eye on:

• Stricter POM validation – many projects will need to quickly tighten up their POMs to get running
• Site plugin – if you’re using this for reporting, you may have work to do to get the same results
• Metadata updates – Maven 3 checks remote repositories less often in most cases. Intermittent remote failures can be cached for a period of time – so check error messages carefully
• Plugin compatibility matrix – check this for any plugins you’re using that might not be updated yet

Watch the Start of the Build

As mentioned above, POM validation is stricter and you may see a few failures on some projects that need updating. A much larger number of projects will probably see warnings about unrecommended behaviour or deprecated features. Watch the start of your builds carefully on the first run of a project for any warnings, and take care of the reported issues as soon as you can.

Make Switching Easy

I highly recommend a script such as the one attached to MNG-2730 to make it easy to switch between Maven versions. This has always been quite useful for adopting new releases, but even more so in this case as you may on rare occasions need to drop back to Maven 2.2.1 for a particular project.

A Few Words

I’ll admit there were long periods of time where I thought this release would never happen. Particular congratulations go to Benjamin for his effort over the last year and a bit to pick it up and methodically drive it home – I’ve been there before and I know that it is full of both fun and frustration!

Implementation of parallel builds was also a big job. I recall Dan hacking away at it last ApacheCon, and from there Kristian put in a huge effort to get a production-ready implementation and work through nasty thread safety issues in some plugins and components. I hope this is something that gains more traction going forward.

It’s also worth acknowledging the guys plugging away at getting the Site plugin infrastructure back in place – Olivier, Hervé, and Dennis in particular.

And congrats to all the Maven developers and contributors that had a hand in this release, and all those that got Maven where it is today. Hopefully more great things to come, and perhaps a little faster next time! 🙂

Register for Training by Oct 13 and Receive Maven Book Free

As I posted earlier this week, I will be running a full day Maven training course at ApacheCon on November 1.

I’ve now been able to confirm that I can offer a free copy of the book Apache Maven 2: Effective Implementation to everyone that completes the course. However, to ensure the right amount can be ordered and shipped in time, you must have registered before October 13.

There are still discounted rates for the training and conference for today, so all the more reason to register as soon as possible!