OS X Mountain Lion Mail and Smart Mailbox Syncing

Last night, I was reading the Ars Technica review of Mountain Lion, which included the following quote about iCloud:

Then there’s debugging. Sure, this is all supposed to “just work.” But when it doesn’t, even expert users have very little recourse. To help with cases where a particular device is not showing the data that you expect to see, traditional cloud storage services provide a Web interface to the canonical data store. Even Apple’s own iDisk did this. Currently, iCloud provides no such interface.

That certainly rang true in a bizarre problem for me today. I have two machines that were setup independently and both upgraded to Mountain Lion. I make extensive use of Smart Mailboxes to be able to run through certain lists of mail quickly – one of them includes every open source commits list that I subscribe to. When I went to check that on my second machine yesterday, it was empty – and the Smart Mailbox showed that every rule said “No mailbox selected”. I hadn’t used that mailbox on there since the upgrade, so I figured it was probably an upgrade problem – and went ahead and fixed it.

Then I moved back to my iMac this morning and found it had the same problem. I still had the Macbook open and saw that it was all working there, so I fixed it up on the iMac. Almost instantly, I watched it break on the Macbook.

Well, that’s creepy.

With the help of this post, I discovered that Mail now syncs Smart Mailboxes to iCloud (I suppose using the key-value store, as this is for non-iCloud mail accounts). Inspecting the properties files mentioned, I found that the IMAP URLs for the folders differed between the two machines – one using imap.gmail.com and the other imap.googlemail.com. The sync would merge them, and whichever had the account setup differently would fail.

All I had to do was change the IMAP server so both accounts matched, then edit the smart mailbox (without changing anything and pressing Ok). It would be updated to the new server address and synced, magically repopulating on the other machine.

Syncing Smart Mailboxes is an awesome feature that I was wishing for when setting up the Macbook a few weeks ago – and I hadn’t heard about it prior to today. I think the transparent sync is going to be a great thing as more apps add support for it, but hopefully there’ll be more attention paid to making it clear where it is going to happen, and dealing with conflicts. In this case, it was expected behaviour, but it looked like an awful bug in Mail at first glance.

Hopefully this info will avoid someone facing the same confusion!

Other than that, the upgrade has gone really well. In the past I’ve written long blog posts about what was required to upgrade – this time it is basically identical to Lion. Reinstall Java, reinstall Xcode command line tools, done. Now to start playing with new features…

Automatically Resolving Version Conflicts in Maven POMs When Merging

I’ve recently had a regular task of releasing projects from multiple branches, and then merging branches together. Handling this and any conflicts hasn’t been a big hassle when it is regularly updated, with one notable exception – Maven POM files. When they are merged the version changes on both branches always conflict.

There are ways I’ve got around it to date:

• Subversion’s --accept mine-conflict and similar options can help resolve them quickly if you know ahead of time they are the only conflicts, without additional changes
• If ending up with versions from the wrong branch, follow up with mvn versions:set -DnewVersion=... versions:commit
• Sometimes the right merge tool or IDE will select the right one, or at least make it less repetitive to select the right one for each file

However, I was finding that in projects with a large number of POMs and occasional other conflicts, these were all a little too tedious. I’ve whipped up a basic script that can be used to eliminate the POM-based conflicts first, so that the remaining conflicts are only those you really need to deal with: Automatically resolve conflicts in Maven POMs after a merge — Gist. It doesn’t handle a lot of edge cases, but should work well enough for most uses.

This works pretty well with Git, and can be used with Subversion (if you automatically postpone all the resolve steps with --accept, or use a non-CLI tool to merge).

This is not something I’d want in a standard release/branch workflow – the problem could be avoided by different branching practices so that release commits are not merged. However if someone finds it useful, I’m sure they can improve on the Gist above, or perhaps patch the Maven SCM or Versions plugins to provide the capability more directly.

Apache Maven Training in Vancouver – 1 week left to register

ApacheCon is in Vancouver this year, and now just 1 month away (Nov 7-11). There is still one week left to register for the training sessions, and to qualify for the early bird rate on the conference.

As in previous years, I’ll be offering an up-to-date version of the training course, Apache Maven: Effective Implementation:

This training course is designed to go beyond your current assumptions about Apache Maven and learn how to use it most effectively to manage the build and development process. Whether you are a novice aiming to start on the right foot, or a regular user looking to get more out of Maven and avoid common frustrations, this course will give you the skills you need to apply to your own projects. By working through a series of short exercises applied to a complete sample application, you will learn how to apply common patterns in Maven builds to achieve the desired outcome, while learning best practices and common pitfalls along the way. Topics include installation, Maven fundamentals, working efficiently with multi-module projects, simplifying the POM, the best general purpose plugins that you should know about, integration and functional testing, when (and when not) to use Maven sites and reporting, the role of profiles, snapshots and dependency management, repository management, and performing releases. The content is updated for the latest improvements in Maven 2.2 and Maven 3, and will cater to your preference of development environment. Time is reserved for sharing specific situations that attendees have encountered in existing projects.

The course runs for a full day on Tuesday, 8 November. If you’re planning to attend the training or the conference, you can connect to me on the conference website.

Obligatory OS X Lion Installation Post

A couple of years ago, I wrote up my experiences installing Snow Leopard on my MacBook Pro. With Lion out today, I’m doing the same (either as notes to self, or to help out other poor souls).

This time around I have two machines to install, so the rarely used MacBook is getting updated now, with the iMac on hold for a while so I don’t disrupt work. I’ll be well prepared for that one with both a Time Machine backup and a clone by Carbon Copy Cloner to a different portable drive!

Getting It

In my previous post, you’ll see I had some hassle getting a functional install disc for Snow Leopard. In that light, digital distribution was a welcome change – particularly being able to start downloading it in the middle of the night here as soon as it was out, and having it ready when I woke up. There were a few false starts with the App Store overloaded initially, but once it got purchased the download was fast.

You can check download progress in the “Purchases” tab – though since the store was timing out I found it easier to look in ~/Library/Application Support/AppStore/APPID. Note that this changes after the upgrade to Lion – instead being in a temporary directory (for me, it was $TMPDIR/../C/com.apple.appstore/APPID)

Once it was downloaded, I made sure to copy the install application to a thumb drive, so that I can install it on the desktop later without another hefty download. You’ll want to grab that before you actually install, as it appears to be gone afterwards. While I haven’t tried it yet, there’s plenty of instructions out there for installing from physical media – such as this one (though this isn’t necessary, as you can just run the install app again on the other Snow Leopard machine).

The Install

The install process is very similar to that of Snow Leopard, though I had no problems such as I did that time. It spent its 45 minutes installing, rebooted, and came back to the new login screen.

The Victims

Here’s what I’ve found didn’t work out of the box:

• X-Code command line tools
• Homebrew (by virtue of the above)
• Java
• Java applets
• AUSkey (required Java applets)
• TruePreview (Mail extension to avoid marking as read immediately, will have to live without it for now)
• Skype had some UI glitches related to scrolling (upgrading to 5.2 fixed it)
• The Omnifocus mail integration (re-installed from OmniFocus preferences again)
• The Basics Growl style (text always comes up black, switched to the “Mono” theme from the same site instead)

Parallels went out of their way to email today and say that my old copy of Parallels Desktop 4 won’t work in Lion, though I’m yet to try it. VMWare only refers to version 3.1.3 being fine, but I still seem to be able to run Fusion 2 just fine as a host – though it’s falling behind on guest support. I should be able to keep using that. I’m not inclined to pay for an upgrade, so it might be time to try VirtualBox again.

Some other things that I had previously been using (like MailTags and Mail Act-on) didn’t survive the Snow Leopard upgrade, so that made things simpler. One or two (like Dovecot) I won’t find out until I try again on the iMac.

Xcode Command Line Tools

This has been a problem for a while – each new release you could get a smaller download from Software Update, or the multi-gigabyte one from the Developer Center. Putting the smaller download on multiple machines meant intercepting the download to copy it before the update got erased, though – and it felt like it wouldn’t last forever.

That seems to be the case now – with Xcode 3 not working on Lion, I bit the bullet and got Xcode 4.1 from the App Store. It’s a large download, but the good news is that after “installing”, it just puts an “Install Xcode” application in place, like Lion itself. You can then copy that around to install elsewhere (though I’m yet to try – we’ll see if the App Store still recognises it).

This installs Xcode as well as the command line tools – I’m not sure at this point if there is a leaner option.

The installer has some oddities though. First, it requires that you close iTunes even if it isn’t running – to correct that I had to kill iTunesHelper from the command line. Beyond that, it seems to get stuck at the very end of the progress bar and never ends. I closed the window and everything seems installed, but it hasn’t removed the installer application.

Homebrew

After updating Xcode, brew doctor seemed happy again, so nothing more to do there.

Java

The concern about Java not being installed on Lion seemed to have died down, and it was pretty much a non-event in this case. After installation, I immediately went to a terminal and typed “java”, at which point it prompted to download it for me, and worked just fine after that.

Java Applets

It was a bit unclear at first why, but pages with applets weren’t working in either Safari or Chrome. I found that I had to go to the “Java Preferences” panel and check the box that allows applets, as they had been disabled by default.

Bugs

I’m sure I’ll find several issues as I go along, but for now it has been fairly limited.

• Chrome has the button to go fullscreen, but not the one to go back and permanent scrollbars (this has now been written about)
• Colloquy 2.3 has the same scrollbar problem, and displays all times in UTC. The 2.4 release from http://colloquy.info/downloads/ seems to resolve them but hasn’t been announced yet (but must be close!)
• My original desktop background disappeared on the iMac (fine on the Macbook), replaced by a starfield, and today replaced by a blank blue background. This seems to be a result of setting it from iPhoto ’09 – exporting and setting by right clicking on the JPEG worked

The Verdict

My initial impression of Lion seems to fit with the majority of the observers – a worthwhile upgrade for the price. I haven’t used it enough to really say yet. The feature I’m most looking forward to seeing in practice is the document versioning and application state restoration, but it’s not that useful when it’s only in TextEdit right now.

The UI changes make sense to me so far, though I had to forcefully hide the scrollbars for the reverse swiping to make sense again (the default for the old Macbook trackpad was always on). Honestly the UI changes don’t really excite me a lot either – it overall doesn’t feel too different to Snow Leopard.

The new 3-pane interface and threaded view in Mail is welcome, though otherwise there doesn’t seem to be too much different in this release. I might try iChat over Adium for a while again to compare, since it now supports all those old Yahoo messenger buddies I have.

Aside from that, I’m not going to bother reviewing the features, as plenty have done that so far (the most detailed as always being John Siracusa at Ars Technica).

Hope this helps someone!

Apache Archiva 1.3.3 released: performance improvements!

If you’re using Archiva for your repository management needs, you should definitely upgrade to the latest release. Download it now!

The 1.3.x series has focused on the biggest offenders in memory usage and performance problems, and Archiva 1.3.3 brings the biggest improvements yet:

• Full scans should take about 1/3rd of the time and consume far less memory
• Removed one-off memory hits at the end of a scan
• File descriptor use during concurrent deployments are better managed

In addition, a new system status page is available for assessing the cause of potential performance issues at runtime, giving better insight into how to tune memory or scanning settings appropriately.

This work is in advance of the upcoming Archiva 1.4 release which has revived the internals more significantly, with further performance improvements and a series of new features.

It’s also worth noting that we dropped support for Archiva 1.1.x and Archiva 1.2.x in November, so there’s no reason left to remain on older versions.

I’d like to thank YourKit, who provided a free license for their profiler, which was of great assistance in tracking down these issues. I’ve used it on occasion for a number of years, and it is one of the easiest tools to use that I’ve ever encountered.

The full set of issues resolved follow:

• [MRM-1097] – Error 500 "too many open files"
• [MRM-1369] – Editing user roles in archiva clobbers continuum redback roles
• [MRM-1396] – Purge task problem : Not enough parts to the path
• [MRM-1421] – Archiva repository purge incorrectly purges based on file timestamps even when the snapshot timestamp is known
• [MRM-1443] – repository statistics collection can cause server to hang
• [MRM-1416] – upgrade to Redback 1.2.5
• [MRM-1439] – improve indexing performance
• [MRM-1440] – system status page
• [MRM-1441] – monitor repository scanning progress
• [MRM-1442] – track time spent in each consumer during a scan, to help diagnose poor scanning performance
• [MRM-1445] – disable referrer check by default