Apache Archiva 1.3.3 released: performance improvements!

If you’re using Archiva for your repository management needs, you should definitely upgrade to the latest release. Download it now!

The 1.3.x series has focused on the biggest offenders in memory usage and performance problems, and Archiva 1.3.3 brings the biggest improvements yet:

• Full scans should take about 1/3rd of the time and consume far less memory
• Removed one-off memory hits at the end of a scan
• File descriptor use during concurrent deployments are better managed

In addition, a new system status page is available for assessing the cause of potential performance issues at runtime, giving better insight into how to tune memory or scanning settings appropriately.

This work is in advance of the upcoming Archiva 1.4 release which has revived the internals more significantly, with further performance improvements and a series of new features.

It’s also worth noting that we dropped support for Archiva 1.1.x and Archiva 1.2.x in November, so there’s no reason left to remain on older versions.

I’d like to thank YourKit, who provided a free license for their profiler, which was of great assistance in tracking down these issues. I’ve used it on occasion for a number of years, and it is one of the easiest tools to use that I’ve ever encountered.

The full set of issues resolved follow:

• [MRM-1097] – Error 500 "too many open files"
• [MRM-1369] – Editing user roles in archiva clobbers continuum redback roles
• [MRM-1396] – Purge task problem : Not enough parts to the path
• [MRM-1421] – Archiva repository purge incorrectly purges based on file timestamps even when the snapshot timestamp is known
• [MRM-1443] – repository statistics collection can cause server to hang
• [MRM-1416] – upgrade to Redback 1.2.5
• [MRM-1439] – improve indexing performance
• [MRM-1440] – system status page
• [MRM-1441] – monitor repository scanning progress
• [MRM-1442] – track time spent in each consumer during a scan, to help diagnose poor scanning performance
• [MRM-1445] – disable referrer check by default

Apache Maven 3.0 Released: a Few Important Tips

Just short of the 5 year anniversary of the Maven 2.0 release (Oct 19, 2005), Maven 3.0 has shipped today. You can download it from the Maven website. I’ve now been using it for about 6 months for all but one project, by which point it was already quite stable. Luckily it also arrives just in time for ApacheCon, since I have updated my training to cover it! All-in-all it’s a great release – quite a bit faster and in places more predictable.

New Features

For the most part, this is not a feature release, but a performance and architectural release. Much has already been written about underlying technology changes (like switching from Plexus to Guice). The main points of interest for me are:

• parallel builds – build modules in parallel when enabled to utilise multiple cores and get even more performance gains. It’s optional, so take some time to try it out
• improved performance & predictability – while maintaining and documenting compatibility
• improved reactor – behaves more consistently between building multi-module projects and subsets of them
• validation and error reporting – unrecommended and deprecated behaviour is now pointed out and error reporting improved
• improved classloading – extensions and plugins are loaded in a more self-contained fashion to allow more flexibility

There are still some “gotchas”, tips and tricks to take note of, however.

Check the Compatibility Notes

There is a page dedicated to compatibility notes between Maven 2 and Maven 3. This should be considered required reading for anyone making the switch, as it highlights some changes that you may need to adjust your projects or environment for. In most cases there’ll be no major issues, and only some quick fixes. In my opinion, the key ones to keep an eye on:

• Stricter POM validation – many projects will need to quickly tighten up their POMs to get running
• Site plugin – if you’re using this for reporting, you may have work to do to get the same results
• Metadata updates – Maven 3 checks remote repositories less often in most cases. Intermittent remote failures can be cached for a period of time – so check error messages carefully
• Plugin compatibility matrix – check this for any plugins you’re using that might not be updated yet

Watch the Start of the Build

As mentioned above, POM validation is stricter and you may see a few failures on some projects that need updating. A much larger number of projects will probably see warnings about unrecommended behaviour or deprecated features. Watch the start of your builds carefully on the first run of a project for any warnings, and take care of the reported issues as soon as you can.

Make Switching Easy

I highly recommend a script such as the one attached to MNG-2730 to make it easy to switch between Maven versions. This has always been quite useful for adopting new releases, but even more so in this case as you may on rare occasions need to drop back to Maven 2.2.1 for a particular project.

A Few Words

I’ll admit there were long periods of time where I thought this release would never happen. Particular congratulations go to Benjamin for his effort over the last year and a bit to pick it up and methodically drive it home – I’ve been there before and I know that it is full of both fun and frustration!

Implementation of parallel builds was also a big job. I recall Dan hacking away at it last ApacheCon, and from there Kristian put in a huge effort to get a production-ready implementation and work through nasty thread safety issues in some plugins and components. I hope this is something that gains more traction going forward.

It’s also worth acknowledging the guys plugging away at getting the Site plugin infrastructure back in place – Olivier, Hervé, and Dennis in particular.

And congrats to all the Maven developers and contributors that had a hand in this release, and all those that got Maven where it is today. Hopefully more great things to come, and perhaps a little faster next time! 🙂

NPanday 1.2.1 Released

In the midst of the move to Apache that I’ve just written about, Joe has also released NPanday 1.2.1 from Codeplex to wrap up the outstanding issues that were in SVN before we started the migration process.

The main improvements of the release were to dramatically improve the performance of SNAPSHOT resolution, and to support building with Maven 3. In NPanday 1.2, we started to bring the resolution back to the Maven standard APIs to improve compatibility. This release further streamlines the process, and allows you to take advantage of additional performance improvements of the Maven 3 betas.

You can download NPanday 1.2.1 from the Codeplex project, or from the NPanday Maven repository. You can find more information in the installation instructions.

Thanks to those that submitted issue reports and patches for the release!

The full list of changes is reproduced below:

• 13590 Update/Create all three “types” of assembly version during compile when it is not there (AssemblyInfo.cs) initially
• 13758 Resync references may cause timeouts after successive snapshot metadata retrievals
• 13948 Performance Issue when Building projects with many snapshot dependencies
• 13984 NPanday not building resource files correctly
• 11651 Remove for all but the compile and aspx plugins
• 12946 Create Code Coverage Plugin for NPanday
• 13179 Document Workaround: GAC install fails with access denied when UAC is turned on
• 13198 compile-plugin has two deploy phases for ArtifactType.NETPLUGIN
• 13394 Include the steps on how to execute the ITs correctly from NPanday trunk
• 13395 Release process notes is outdated
• 13461 npe building gac artifact
• 13462 add missing dependencies to dotnet poms
• 13477 support jar style custom manifest entries in assemblies
• 13575 Update build to use the NPanday 1.2 release
• 13615 Configure Emma plugin
• 13616 Configure Code Coverage plugin for .Net
• 13635 Snapshot updates are not recognised due to the copies in UAC
• 13652 image missing in Addin docs
• 13655 Fix APT syntax errors on the main site
• 13669 Improve the plugin sites
• 13846 POM files incorrectly updated when including Web References in VS
• 14049 Error message contains unnecessary text if project not supported.
• 14093 Resync Reference doesn’t update SNAPSHOT artifact from local repository that already exist in .references folder
• 14102 duplicate web reference entry in POM when including web reference

TestNG 5.12 Available for Maven Users

Recently, TestNG produced a new release, 5.12. This release is now in the central repository, though under the version 5.12.1 (more on that below).

http://repo1.maven.org/maven2/org/testng/testng/5.12.1/

Upgrading to 5.12.1

If you’ve been using TestNG with Maven, or are familiar with the changes in TestNG 5.12, you’ll notice there is a difference in this version to previous releases. There is no longer a jdk14 and jdk15 variant of the JAR – with the JDK 1.4 javadoc-style annotations no longer supported, there is just the single TestNG JAR. This means that previous versions using ranges will not automatically upgrade, which is expected given the compatibility change.

Where you might previously have had this in your POM:

<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>5.11</version>
  <classifier>jdk15</classifier>
  <scope>test</scope>
</dependency>

You can now simply use this:

<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>5.12.1</version>
  <scope>test</scope>
</dependency>

No other changes should be necessary.

Guice users should be aware that TestNG now depends on Guice 2.0, and bundles the classes in the JAR. Consequently, Guice is listed as a provided dependency – I haven’t noticed this causing any conflict as long as you are using the official Guice 1.0 or 2.0 release, but this is something to look out for in the future.

Getting TestNG to Central

Some will wonder why this still took so long, given the TestNG release was about a month ago.

Firstly, the library had added a dependency on Guice, and removed a dependency on QDox. This required updates to the POM that were not present in the released bundle. As an Ant-built project, the TestNG POM needs to be maintained separately.

Secondly, Cedric very patiently obliged with several requirements to make it easier for Maven releases in future – in particular, the bundle now includes GPG signatures.

Finally, it turned out that there had been a problem in producing the original distribution, leading to a discrepancy between the Maven release and the original release. This is the reason that there is now a version 5.12.1 (which will likewise be on testng.org shortly) that corrects the content and aligns the TestNG distribution with the Maven release of the same JAR.

The upshot of all of this is that, while this required some back-and-forth to get everything right, future releases will now be much quicker to process and the potential is there for further automation.

Are you Using TestNG with Maven?

I’ve been using TestNG with Maven for some time now, finding it particularly powerful for handling functional testing with Selenium. If you’re using TestNG with Maven, what do you think of the current support?

Apache Maven 2.0.11 Released

Just on a year since the Maven 2.0.10 release, Maven 2.0.11 has been announced. This release was to get a number of fixes that had already been applied to that branch out there for those who continue to use Maven 2.0.10 or below.

Note: as mentioned in the announcement, Maven 2.2.1 is still the most current release that all Maven users should upgrade to, as it contains all of these fixes and more – such as great new features like password encryption, reactor build options, and parallel artifact resolution.

As usual, Maven can be downloaded from Apache and its mirrors. The release contains 37 fixes and improvements, which are listed in the release notes.

It’ll be interesting to see what uptake there is on the release. While there were certainly still some testers around when it came up on the list, the poll I ran last July showed that well over half of the readers of this blog had already moved on to newer versions 2.1.0 and above.

It is nearly 4.5 years since the last time I pulled the trigger on a release of Maven itself (that being Maven 2.0). With further releases in 2.0.x now unlikely, it seems I get to bookend the series I put so much effort into. With a possible Maven 2.2.2 (and of course 3.0) on the horizon, it’ll be great to move on and hopefully start to implement some long lingering plans and ideas.