Ivy: do we really need more metadata?

I caught wind of a release of Ivy today, that lists among its new features a new repository for project metadata.

Given that they are already using the Maven-developed repository for artifacts at Ibiblio, this struck me as pretty strange. Constructing metadata for over 7000 artifacts by hand is not something I’d want to take on, especially when most of it already exists in the Maven repository. And especially when you are just taking that metadata from the same source.

Let’s compare an example:
http://www.ibiblio.org/maven/commons-cli/poms/commons-cli-1.0.pom
and
http://www.jayasoft.fr/org/ivyrep/apache/commons-cli/ivy-1.0.xml
(the XSL is actually a nice touch to make it browsable).

Now, the Maven descriptor can describe everything the Ivy one does except for who wrote the ivy descriptor (which doesn’t seem all that useful other than for some really basic auditing if it were to be automated). There is a limitation in the POM that some data exists (license) in a super POM not published – but that problem is going to be addressed in the next Maven release, and could certainly be worked around when the metadata is published.

We also have metadata for projects like Hibernate that are not built using Maven – it is a requirement of upload to ibiblio.

The main difference seems to be the listing of the location of repositories that house the artifact. This seems a bit short sighted… do you add a new repository entry to every descriptor in the repository every time you add a mirror?

As a Maven developer, I would have liked to see the Ivy developers approach the Maven project about this and offer to help beef up our metadata, and use it rather than creating a whole new repository.

As a user, I’d prefer to get access to new dependencies as soon as they are released or uploaded, rather than having to submit two different requests (one to ibiblio and one to ivyrep).

Update: Note that I’m not saying anything about Ivy or Maven’s ability to do transitive dependencies – I’m aware that Ivy is doing this and the current release of Maven is not. However there is no reason – that I can see – that Ivy cannot use the existing metadata to implement that feature.

Advertisements

9 responses to “Ivy: do we really need more metadata?

  1. I think you’re missing the point. Ivy uses its metadata on dependencies to build the transitive closure of dependencies for a project. In other words, all I have to say is that I use Hibernate 2.1.8 and it looks up to find out that I need CGLIB 2.0.2, etc.

    Maven may have all of that info, but it’s not using it yet.

  2. Vincent Massol

    Hi Jason,

    What Brett is saying is that metadata for transitive dependencies are already part of artifacts uploaded to the Maven repository (I’m not saying that they are complete but they are getting better every day).

    Thus instead of reinventing new metadata for the same , we could all work together and use the same one.

    Check the *.pom referenced above you’ll see it has dependencies required by the artifact it describes

  3. I am not 100% sure that maven1 metadata is providing everything which is needed to implement transitive dependencies.
    What’s make transitive dependencies very difficult, if not impossible to implement in maven 1 (using v3 POMs) is the way in which project inheritance is implemented and how parent POMs are referenced.
    For example Brett’s sample pom: commons-cli-1.0.pom is affected by this problem. You don’t know where to look for its parent pom and if some dependencies are defined in it (or in parent pom of parent pom etc).

    Michal

  4. Nicola Ken Barozzi

    What Ivy is doing now, is exactly what Maven did with Gump. Gump had and has a very big metadata repository, and Maven decided to rewrite it, despite Gump developers being ready to change the format if needed.

    I’ll put this in the “look who’s talking” department 😉

  5. Nicola, I wasn’t involved in the Maven project at that time so I can’t speak for the motivations. I would suspect that the gump metadata is unversioned and always “latest” would have been a big sticking point.

    Regardless, it is very different to this situation. Maven has a much bigger active participation by users and other projects in maintaining that metadata, and a much larger repository.

  6. Nicola Ken Barozzi

    I would suspect that the gump metadata is
    unversioned and always “latest” would have been
    a big sticking point.

    There is no reason why it cannot be added to the xml. In fact I added it for Centipede, as Gump ignores tags it doesn’t need.

    Regardless, it is very different to this
    situation. Maven has a much bigger active
    participation by users and other projects in
    maintaining that metadata, and a much larger
    repository.

    I don’t see how it’s different (have you looked at Gump metadata BTW?).

    If you think that what happened at that time is ok, then you should not think that what Ivy is doing is unreasonable.

    You say that you don’t know why Ivy cannot use the Maven metadata… I never understood why Maven could not use the Gump metadata.

    At that time I did the wrong thing: I took it personally and got angry about it. Instead, I should have just taken it as a fact of life, got on with it, and implemented what I needed in Gump.

    Darwinism works, but is not nice to see 😉

  7. Brett,

    I understand your point of view concerning ivyrep, you are pointing out the duplication of almost same data between maven repository and ivyrep. I totally agree with you on the point that it would be much better to have only one. The problem is that in my company we needed to have transitive dependencies in ant and some other features right now, and that’s why we developped ivy: because we didn’t find a tool fitting our needs. We then decided to open source it, to help others meeting the same problems as us.

    Then we got some developers asking if there were a repository of ivy files, because they didn’t want to write an ivy file for hibernate, for example, if somebody else have already write one. So we decided to open ivyrep. We didn’t contact you to check if it is possible to put them on ibiblio, because we didn’t think you would agree with that, since ivy concurrence the dependency management of maven. Maybe we were wrong ?

    And why ivy files at all, instead of using poms ? Because maven1 poms are written without transitive dependencies in mind, and thus describe dependencies that are not needed directly by the project. As far as I know, we also lacked a way to describe that a dependency is needed only in a particular context (junit for test only). Maven2 poms seem to give an answer to this with scope (as far as I understand), but ivy also enables to map one “scope” (it’s called configuration in ivy) to another: what is a “runtime” scope in junit maybe a “test” scope in a project using junit, for instance.

    As you see, we needed more metadata than we could find in poms, and we needed it asap, available in ant tasks. That’s why we have ivy and ivy files. And it’s because of our users demands that we opened ivyrep, even if it isn’t the best solution.

    We would really be glad to use poms if we could find all information we need in them. So if you want we can try to share our works. But I’m not sure we really focus on the same thing, ivy being concerned only by dependencies management…

  8. Xavier,

    If you’d like to make suggestions at dev@maven.apache.org, you are more than welcome.

    Other than scope which we now have (and we also map scopes as you’ve said) – what additional information do you need? I was unable to find anything in your metadata, as I outlined in my post.

    I hope you’ll be able to add full support for the Maven2 repository in the future.

  9. Brett,

    I’ve just subscribed to the mailing list, and I’ll see if I can be of any usefulness…

    Concerning metadata we need, our spec is described on our web site, in ivy > reference > ivy files.

    The problem is that maven2 metadata info is still hard to find (you talk about scope mapping, but I haven’t seen it during my last check of maven2 pom spec). But mainly if in pom we can find:
    – a list of artifacts published by the project
    – a list of project dependencies, with for each scope mapping

    and if the scopes can be any string, then I think we could use m2 poms in ivy, and we’ll be really happy to do it ! For sure if we could have only one project metadata repository, it would really be nice for everybody.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s