Working with Maven dependencies not found in public repositories

One of the many pleasant advantages the dependency management of a modern build tool such as Maven or Gradle gives you is that you simply declare the set of dependencies your application needs and the build tool then takes care of downloading them together with their transitive dependencies from a public repository. There’s no need to painstakingly assemble these by hand and later having to hassle with updating them to some newer version. The times when you had to manage a lib directory in your project by hand are luckily long over (anyone remember Ant?).

But what if your project depends on some library that cannot be found in a public repository, either because it is proprietary or because it was never uploaded into any repository in the first place? How can you include such a dependency into the build process anyway? There are basically four options which will be discussed in this article.

  • Use a corporate repository manager such as Nexus or Artifactory
  • Use a dependency with system scope
  • Install a Jar file in your local repository cache
  • Create a local repository in your project

Use a corporate repository manager such as Nexus or Artifactory

The first option should be the most widespread one in a corporate environment and is also the most straightforward one: all dependencies are loaded from a local, company-wide repository manager such as Nexus or Artifactory. This allows for an auditing of the used dependencies and their respective versions and provides an option to disallow certain dependencies. An administrator can upload any dependencies not available from a public repository into the local repository to make them available to all users in the company.

This approach has one disadvantage: when specific dependencies are only available through a corporate repository, a build becomes dependent on this particular environment. A project having such dependencies can only be built when the local repository is online and reachable. This should usually not be a problem if a Maven project exclusively exists inside the company.

Having a local repository manager at one’s disposal is convenient but unfortunately not always an option. If you don’t have a local repository available or if you need your build to be independent from some repository server you still have some more options.

Use a dependency with system scope

The second option is to use Maven’s system scope. You are, however, strongly discouraged to use this approach. System scope is a dependency scope that allows to address a Jar file on the local file system so that this file is used to resolve a dependency instead of contacting a remote repository. As described in the Maven documentation,

Dependencies with the scope system are always available and are not looked up in repository. They are usually used to tell Maven about dependencies which are provided by the JDK or the VM. Thus, system dependencies are especially useful for resolving dependencies on artifacts which are now provided by the JDK, but where available as separate downloads earlier.

This is an example usage of this scope:

<dependency>
    <groupId>javax.sql</groupId>
    <artifactId>jdbc-stdext</artifactId>
    <version>2.0</version>
    <scope>system</scope>
    <systemPath>${java.home}/lib/rt.jar</systemPath>
</dependency>

More importantly, the Maven documentation states that this feature is deprecated! So, if you have ever considered using system scope for one of your dependencies, don’t do it. Since only absolute file paths can be provided with system scope, a build becomes inherently dependent on a particular environment which is an absolute no-go for independent and self-contained software builds. Additionally, this approach is useless when it comes to building assemblies and fat jars, since system-scoped dependencies will not be included into these.

Install a Jar file in your local repository cache

The next option you have when you don’t have a repository manager available is to install the needed dependency into your local repository cache. That’s the one located in ~/.m2. You can install a Jar file myjar-1.0.jar into your local repository with the following command:

$ mvn install:install-file -Dfile=/path/to/myjar-1.0.jar \
      -DgroupId=com.example -DartifactId=myjar \
      -Dversion=1.0 -Dpackaging=jar

This will place the provided file into your local repository where it later can be picked up by Maven during the build process. While this option answers the question how you can include a proprietary dependency into your Maven build, it still has one big disadvantage: using this solution, your builds are no longer self-contained. That is, when any other developer wants to build your project on her machine with a fresh check-out, she first has to install the needed dependency into her own local repository as shown above. So you have to document this prerequisite along with the project, hoping it will be read and understood.

Create a local repository in your project

There is a better solution to this problem, and this is the one that I recommend in any case where you don’t have a corporate repository at your disposal. You can create a new local repository directly inside your Maven project and install all necessary non-public dependencies into this repo. By that, your project remains self-contained and nobody has to jump through hoops to get your project to build. This approach is described in numerous places.

First you create a directory inside your project in which the local repository will reside.

myproject
+- pom.xml
+- src
+- etc
    +- repo

Then you install the required dependency into this repository with the following Maven command:

$ mvn deploy:deploy-file \
      -Durl=file:///path/to/yourproject/etc/repo/ \
      -Dfile=my-non-public-dependency-1.0.jar \
      -DgroupId=com.example \
      -DartifactId=my-non-public-dependency \
      -Dpackaging=jar \
      -Dversion=1.0

Finally, you have to make this repository known to your Maven build by adding it to the pom.xml:

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
...
    <repositories>
        <repository>
            <id>local-repo</id>
            <url>file://${basedir}/etc/repo</url>
        </repository>
    </repositories>
...
</project>

${basedir} is a Maven variable that points to the root directory of the project.

The option to use file://-URLs in a repository definition is what makes this approach possible.

Now you can add the non-public library to the list of dependencies like any other public library using the same Maven coordinates you used when you installed the library to the local repository:

<dependency>
    <groupId>com.example</groupId>
    <artifactId>my-non-public-dependency</artifactId>
    <version>1.0</version>
</dependency>

As a result, your project remains self-contained and every dependency is treated the same, in particular with respect to building assemblies and fat jars.

Short URL for this post: https://wp.me/p4nxik-3aJ
Roland Krüger

About Roland Krüger

Software Engineer at Orientation in Objects GmbH. Find me on Google+, follow me on Twitter.
This entry was posted in Build, config and deploy and tagged , . Bookmark the permalink.

Leave a Reply