This is Gentoo's testing wiki. It is a non-operational environment and its textual content is outdated.

Please visit our production wiki at https://wiki.gentoo.org

Java Developer Guide

From Gentoo Wiki (test)
Jump to:navigation Jump to:search

Before going into details of how Java is handled on Gentoo, please make sure you have read and are aware of the Gentoo Java Packaging Policies and also the Java article. This guide will not cover such, but will cover more specific details on Gentoo Java ebuilds.

Java on Gentoo

This section will give you more insight into how Gentoo handles Java. You should be familiar with the Java User Guide before proceeding.

Virtual machines (VMs)

As discussed in the User Guide, there are several VMs available from portage.

Testing all packages to ensure they build and run with every VM is a huge undertaking, and there simply are not enough resources to guarantee every package will build and run with every VM.

We now maintain a list of "supported virtual machines" for each architecture. These are the VMs we will test the packages against before committing changes to portage. When you emerge a package, it will by default try to use the best "supported virtual machine."

Of course, Gentoo and Linux in general are about choice, so if you prefer a different VM over the "supported virtual machines", you can easily use that VM instead. However, you should also know that bugs reported with one of the non-"supported virtual machine" will get a low priority if it isn't present using a "supported virtual machine".

Configuring which VM to use

You can choose which VM to use on a per-user basis, and which VM to use for the system (ie when running things as root). Both of these are configured using java-config.

A user's current VM is represented by a symlink located at ~/.gentoo/java-config-2/current-user-vm. This symlink points to the JAVA_HOME of the chosen VM. Similarly, the system VM is represented by a symlink at /etc/java-config-2/current-system-vm.

The current VM can also be changed on the fly. This can be accomplished setting the environment GENTOO_VM to contain the name of a VM that java-config knows about.

Java tools

The traditional Java tools, ie, java, javac, javadoc, etc, are all located in /usr/bin. They are actually all symlinks to the run-java-tool script. This script will call the appropriate tool, depending on how the script is invoked, from the current VM. The contents of the GENTOO_VM variable is checked first, then the user VM, and lastly the system VM.

Build-time VM switching

As outlined in the User Guide, and mentioned briefly earlier, the VM will switch at build time to accommodate the needs of the package. The VM to use is first determined by JAVA_PKG_FORCE_VM, then /etc/java-config-2/build/jdk.conf, and lastly the system VM.

Bytecode compatibility

The default behavior of javac is to compile bytecode that will compatible with the current VM version and higher (ie forward compatible). It is possible to specify which VM to compile for to provide the best compatibility.

At build time, the DEPEND and RDEPEND variables will determine what VM to compile for based on virtual/jdk and virtual/jre. Additionally, this can be controlled by the environment variables JAVA_PKG_WANT_SOURCE and JAVA_PKG_WANT_TARGET.

There is a wrapper for javac, ejavac, which will use the appropriate VM's javac, and then specify the appropriate -target and -source. For projects that use ant, the build.xml can be translated to specify the appropriate -target and -source.

Filesystem layout

In general, the directory policies are handled for you by the helper functions in the java-utils-2 eclass.

These functions adhere to the following path name conventions:

  • /usr/share/${PN}-${SLOT}/package.env contains information about the package.
  • .jar files created from source code are installed to /usr/share/${PN}-${SLOT}/lib/
  • .jar pre-built files not compiled by the ebuild are installed to /opt/${PN}-${SLOT}/lib
  • Javadoc documentation is installed to /usr/share/doc/${PF}/html/api/
  • Source archives are installed to /usr/share/${PN}-${SLOT}/source/
  • User-executable scripts are installed to /usr/bin
  • System-wide environment files are in installed to /usr/env.d/java
  • User-specific environment files can be put into ${HOME}/.gentoo/env.d/

Prepare Enviroment

Before getting started there are a few things to prepare in your development environment, such as environment variables to set, and/or packages to merge.

Variables

There are a couple Java specific portage environment variables to be set. They are optional, but the first is recommend to be set by anyone doing Java ebuild development.

JAVA_PKG_STRICT=true
Set this variable in make.conf to enable use of dev-java/java-dep-check. Which will look for and report any unneeded dependencies and/or any bundled dependencies found not part of the package dependencies.
JAVA_PKG_DEBUG=true
Set this variable to enable debugging in Java eclasses

Packages

There are a couple of Java specific packages, produce by Gentoo Developers, that can help with Java ebuild development. They are optional, but the first is recommended to be installed and used anytime you are updating a package and/or library.

dev-java/java-apicheck
This package is used to compare to packages or jars to each other for API changes. It will help you determine if a package needs to be slotted or not. Anytime the API changes to the point of breaking other packages, it needs to be slotted.
app-portage/java-ebuilder
This is an experimental package in development to generate initial ebuilds from Maven pom.xml. It will generate ebuilds for any dependencies that are not bundled. This will not address src_install or anything beyond the most basic ebuild to build and install a jar file for a given package. Since Maven does not handle doing any installation beyond installing to a local Maven repository. It is not possible to generate package specific system installation from app-portage/java-ebuilder.

Writing the ebuild

General guidelines

In addition to standard Gentoo ebuild guidelines, there are a number specific for Java packages:

  • Avoid using bundled .jar files at all costs for source-based packages. Instead, they should use system installed versions with the help of our eclass functions.
  • If you only need the path to installed libraries, you can use java-pkg_getjar(s). Don't call java-config directly, because it will not be recorded as a dependency in the package env.
  • Always provide an easily understandable reason after 'die', so that end-users will provide the maintainers with sensible feedback if the build should fail.
  • Avoid cluttering up the environment by adding environment files to /etc/env.d/. Instead, store your env file in /etc/env.d/java/, and then have user scripts source its environment file when it launches. Otherwise, developers, who regularly override CLASSPATH, CATALINA_HOME and other environment variables, will have problems running regular apps. If you use the launcher it will also automatically source the appropriate env file.
  • Make sure you always compile with correct a source/target. This is important to ensure future and backwards compatibility. If the packages use ant, this can be done for you automatically. See java-ant-2.eclass. If not you will have to patch it to pass $(java-pkg_javac-args) to javac.
  • Do no install jars which contain versions in their filename, ie castor-0.9.7.jar. Use java-pkg_newjar to rename and install versioned jars to not contain the version in their filename.
  • For packages that use Ant to build, try to DEPEND on just dev-java/ant-core when possible instead of dev-java/ant. dev-java/ant-core is automatically added to DEPEND if you inherit java-ant-2. If the package makes use of 'optional' ant tasks, you'll need to DEPEND on dev-java/ant or add WANT_ANT_TASKS="ant-task" before inherit. The latter is of course preferred. see the Ant Guide for details.

USE flags

Gentoo Java ebuilds unlike most others start with a USE flag variable, a Java specific one JAVA_PKG_IUSE. The only thing that can come before this variable in an ebuild is the EAPI variable. You still set the other USE flag for various general USE flags, but there are Java specific USE flags that go in this variable and DO NOT go in the other normal USE flag.


<translate>

About USE flags

For more information regarding USE flags, refer to the USE flags chapter from the Gentoo Development Guide.

Java specific USE flags

There are a few specific common USE flags for Java ebuilds as follows. These use flags do not go in the normal USE variable but go in JAVA_PKG_IUSE instead. Any use flag other than the following would go in the normal USE variable. The JAVA_PKG_IUSE must precede the inherit line in an ebuild.

The USE flags that go in JAVA_PKG_IUSE

  • The doc flag will build API documentation using javadoc.
  • The source flag installs a zip of the source code of a package. This is traditionally used for IDEs to 'attach' source to the libraries that are being use;

</translate>

Java USE flag usage

The JAVA_PKG_IUSE USE flag/variable MUST reside in the ebuild before the inherit line.

If a manual or other extensive documentation is available, it should be installed using the doc USE flag. If the build system can, you should build javadocs also using the doc USE flag. If it does not, you should consider patching it to do so, and provide the patch upstream. HTML documentation should be installed using dohtml and javadocs using java-pkg_dojavadoc.

If you want to go all the way, add the source USE flag that installs the complete source code as a .zip file. Use java-pkg_dosrc for this purpose. This allows IDEs such as Eclipse and NetBeans to do complete source-level debugging. The source USE flag will add app-arch/zip to DEPEND automatically, you do not need to add this dependency.

CODE Example of setting common Java USE flags
JAVA_PKG_IUSE="doc source"

inherit ...

Dependencies

Gentoo Java ebuild dependencies for the most part are just like any other ebuild. There are two common dependencies that will always exist for any Java ebuild, no matter if Java is optional or not. Though if Java is optional, the dependencies would be conditional based on a USE flag, likely java USE flag. There are some packages which may require a JDK at runtime, otherwise most should have a JRE for runtime. When depending on a package, take care that you depend on a sufficiently recent version, and explicitly ensure at building time that the providing package gives you the correct interface, i.e. the correct SLOT[USE].

JDK/JRE

For any package that builds from source, DEPEND should be set to >=virtual/jdk-[minimum-version]. If the package does not build from source and is a Java binary, you can use a JRE instead of a JDK in DEPEND.

All java packages will require a JRE for RDEPEND, and should be set to >=virtual/jre-[minimum-version]. Unless the package requires a JDK at runtime.

Note
The JDK/JRE atom MUST have a version! Do not use just virtual/jdk or virtual/jre in DEPEND or RDEPEND ever!
CODE Required Java dependencies
DEPEND=">=virtual/jdk-1.7"
RDEPEND=">=virtual/jre-1.7"
CODE Required Java dependencies for optional Java support
DEPEND="java? ( >=virtual/jdk-1.7 )"
RDEPEND="java? ( >=virtual/jre-1.7 )"
Note
Never set the JRE version to be less than the JDK version. This causes -target to be less than -source. Which can cause runtime issues, symbols not found, thus should be avoided always! The opposite is safe, though rarely ever required, JRE version greater than JDK version. But that could be used to target newer JRE byte code using older sources.

While it is not required, you will commonly see a dependency variable that is used in both DEPEND and RDEPEND. This variable name is not mandated and is usually CDEPEND or COMMON_DEPEND for anything that will go in both DEPEND and RDEPEND. This is not really Java specific but is very common in most if not all Java ebuilds.

CODE Java dependencies with common ones
CDEPEND="dev-java/xerces:2
        >=dev-java/log4j-1.2.8"
 
DEPEND=">=virtual/jdk-1.7
        ${CDEPEND}"

RDEPEND=">=virtual/jre-1.7
        ${CDEPEND}"
Note
If the sources are in a zip file you will need to add app-arch/unzip to DEPEND.

Slots

For Gentoo Java packages, slots are obligatory for every dependency. Which includes packages with SLOT="0", they must end in :0. Failure to adhere to this can cause issues if a package is later slotted. Portage will pull in the latest version, likely the slotted one, removing the un-slotted version. Which will break usage of the dependency in the Java ebuild.

Versions

For most Gentoo Java packages you will exclude versions from dependencies, unless you need a specific version of a dependency or a newer version than what might be available. For large Java Applications that are split into multiple packages on Gentoo, it is recommended but not required, to add the version of the package to the dependency, ~category/package-${PV}:${SLOT}. This ensures other packages for the same application will all be built using the same version. There could be potential issues mixing pieces of an application with different version. By having all packages depend on the version and slot, it ensures this cannot happen. Upstreams will likely not address or respond to bugs or issues that arise from using pieces with other pieces from different versions. Nor is it something Gentoo developers or users should be wasting time on.

Take the following example using dev-java/netty-buffer which depends on dev-java/netty-common. Both come from the same sources, thus the needed to have dev-java/netty-buffer depend on a specific version of dev-java/netty-common as follows. This would be the depend for dev-java/netty-buffer. The version does not matter, nor does the slot. What matters is we are passing both from the parent package, to its dependency, which is from the same sources.

CODE Example of split package version dependency
CDEPEND="~dev-java/netty-common-${PV}:${SLOT}"
 
DEPEND=">=virtual/jdk-1.7
        ${CDEPEND}"

RDEPEND=">=virtual/jre-1.7
        ${CDEPEND}"

Another example where the slot is different but the version is the same. This example is using www-servers/tomcat and dev-java/tomcat-servlet-api. Due to the nature of servlet-api, and its accompanying virtual java-virtuals/servlet-api, that package gets slotted per servlet-api spec, not the version of the sources. Thus www-servers/tomcat and dev-java/tomcat-servlet-api do not share the same slot, but have the same sources. Which means www-servers/tomcat should use a specific version of dev-java/tomcat-servlet-api.

CODE Example of split package version dependency with different slots
ECJ_SLOT="4.5"
SAPI_SLOT="4.0"

CDEPEND="dev-java/eclipse-ecj:${ECJ_SLOT}
        ~dev-java/tomcat-servlet-api-${PV}:${SAPI_SLOT}"
 
DEPEND=">=virtual/jdk-1.7
        ${CDEPEND}"

RDEPEND=">=virtual/jre-1.7
        ${CDEPEND}"

Preparing sources

Gentoo Java ebuilds do not and should never use the normal src_unpack() and src_prepare() but instead use its own java_prepare(), unless the package has optional java support. There are some cases where you might need to use src_unpack() with java_prepare(), but this is very rare. Within java_prepare(), along with doing the normal patching and other things you might need to do in order to prepare the sources. It is policy and practice to always remove any bundled class and/or jar files that came with the package sources. Which is one of the first things you do for any Java package, regardless of optional Java support or not.

Removing bundled classes and jars

There are a many different ways to go about removing bundled classes and jars such as; rm, find -delete/rm, and more. However these should not be used and rather for standardization use eclass functions for such; java-pkg_clean and JAVA_RM_FILES, and they can be used together.

CODE Removing files with java-pkg_clean
java_prepare() {
     # This does not discriminate,
     # it will remove ALL .class and .jar files
     java-pkg_clean
}
Note
In the future this may be called automatically, and can be overridden by setting JAVA_PKG_NO_CLEAN to any value.
CODE Removing files with JAVA_RM_FILES
java_prepare() {
    # Good for selective removal in the rare event bundled
    # binaries/dependencies must be used
    JAVA_RM_FILES=(
            /path/to/file.class
            /path/to/file.jar
        )
}
CODE Removing files with find and JAVA_RM_FILES
java_prepare() {
    for test_file in $(find . -type f -name '*Test*'.java); do
        JAVA_RM_FILES+=("${test_file}")
    done
}

Replacing removed bundled jars

There are some build systems that require jars to be in certain locations. Per Gentoo Java Packaging Policy and the previous section those jars have now been removed. They need to be replaced using symlinks to system jars to let the build system proceed without further modification. This is mostly a legacy way of doing things, as most build systems will look for jars on the classpath.

CODE Replacing removed bundled jar files with system
java_prepare() {

    # change directory to where the jars you removed were located
    cd lib

    # needed at runtime not just build, will be on the package classpath in package.env
    java-pkg_jarfrom jdom

    # needed at runtime not just build with specific build name, will be on the package classpath in package.env
    java-pkg_jarfrom ant-core ant.jar ant-1.5.4.jar

    # not needed at runtime just build, will NOT be on the package classpath in package.env
    java-pkg_jarfrom --build-only ant-core

}

Modify Build System

There are times where you might need to modify the build system. If a project uses ant as a build system, there can be unwanted targets that are always called. Or targets that will download dependencies outside of portage. Other times there might be unwanted classes or resources on the classpath, or it is missing a classpath that will be set/added to the ebuild.

To avoid unwanted targets you can either remove them entirely from the build.xml file, or you can comment out the targets via XML comments, <!-- •••••• -->. This can be done via one or more patches and/or sed if minimal. Pay close attention to target dependencies, and dependent targets. At times bypassing one unwanted target can bypass other wanted targets. There are times you might need to change a target's dependencies rather than bypassing a target entirely.

Note
When you comment out targets or anything in xml via XML comments, <!-- •••••• -->, those comments will disappear when the xml file is passed through the xml-rewriter. It will remove any comments entirely. This can cause patches to fail if the patch is made after the xml-rewriter has modified the xml file. Make sure to always make patches and sed for raw untouched sources.

Compiling

There are couple common ways to compile full Java packages on Gentoo using either eant or java-pkg-simple eclass functions and variables.

eant

eant is a Gentoo wrapper around ant. One should never invoke ant directly in an ebuild, but instead call eant. Though even calling eant is unnecessary in most packages with ant build systems, and should not be done. Instead eant function can be controlled with global variables in the ebuild. Using such one can omit the entire src_compile() section in most cases.

CODE Using eant Global Variables
inherit java-pkg-2 java-ant-2

...

# Re-write the classpath in any and all build.xmls in the sources
# This should be done in most cases
JAVA_ANT_REWRITE_CLASSPATH="true"

# This is passed to the ANT_TASKS variable, which can be set directly.
# This contains names of Gentoo Java Package(s) including slot if > 0
# There can be more than one task, comma separated, or multi-line no commas
EANT_ANT_TASKS="bnd-3,cpptasks"

# Controls the location of build.xml, with the default being ./build.xml
# Normally do not need to set this, but if its in an abnormal location
# or in a subdirectory, set this to that location
EANT_BUILD_XML="sub/directory/build.xml"

# This tells eant/ant which target to call, default is jar
# If the build target of the package is jar you can omit this variable
EANT_BUILD_TARGET="jars"

# This tells eant/ant which target to call to build javadocs, default javadoc
# If the doc target of the package is javadoc you can omit this variable
EANT_DOC_TARGET="javadocs"

# If the package has a test target set this to the name of that target
EANT_TEST_TARGET="junit"

# This is used to build the classpath for the package
# This contains names of Gentoo Java Package(s) including slot if > 0
# This is only required if you need jars on the classpath
# In some cases where only the jdk is needed, this can be omitted
# Single line comma separated list of package names with slot
# Or if lengthy, multi-line with no commas is preferred!
# Slot format can be either ${PN}:${SLOT} or ${PN}-${SLOT}, 
# except slot -0 is not valid but slot :0 is valid
EANT_GENTOO_CLASSPATH="
        commons-logging:0
        jdom-2
        jgoodies-common:0
        xalan
"

# This is used to add raw jars to the classpath
# Jars either bundled or installed outside portage
# It contains paths to jars, not package names
# Paths can be absolute or relative to the package build system
# Paths are colon separated
EANT_GENTOO_CLASSPATH_EXTRA="lib/bundled.jar:/path/to/system.jar"

# This is used to pass extra arguments to ant
# Or in some case to negate defaults which follow;
# -Dnoget=true -Dmaven.mode.offline=true -Dbuild.sysclasspath=ignore -Dmaven.test.skip=true
# If your packaging something that imports ant classes
# You may need to negate the -Dbuild.sysclasspath=ignore
EANT_EXTRA_ARGS="-Dbuild.sysclasspath=last"

# Set this if you need tools.jar from the JDK itself
# The value does not matter, just has to be set to something
EANT_NEEDS_TOOLS="true"

java-pkg-simple

java-pkg-simple is an eclass that provides a way to build Java packages on Gentoo in a simplistic manner. This is package system independent and should be used for any package that does not come with an Ant build system. Most packages they can be built using java-pkg-simple global variables in the ebuild. Using such one can omit the entire src_compile() section in most cases.

CODE Using java-pkg-simple Global Variables
inherit java-pkg-2 java-pkg-simple

...

# One or more directories with sources in them relative to ${S}
# required all the time
JAVA_SRC_DIR="src/main/java"

# list of java packages to put on the classpath
# required all the time
JAVA_GENTOO_CLASSPATH="xalan:2,log4j:0"

# This is used to add raw jars to the classpath
# Jars either bundled or installed outside portage
# It contains paths to jars, not package names
# Paths can be absolute or relative to the package build system
# Paths are colon separated
JAVA_GENTOO_CLASSPATH_EXTRA="lib/bundled.jar:/path/to/system.jar"

# optional maybe necessary for some
# encoding of source files, used by javac and javadoc
JAVA_ENCODING="ISO-8859-1"

# optional additional arguments to be passed to javac
JAVAC_ARGS="-Xms:64m"

# optional additional arguments to be passed to javadoc
JAVADOC_ARGS="-J-Xmx180m"

Typical examples

Without further ado, here are a few examples:

FILE foo-1.0.ebuildExample of a Java ant ebuild
EAPI=5
JAVA_PKG_IUSE="doc source"

inherit eutils java-pkg-2 java-ant-2

DESCRIPTION="Fictional example ebuild."
HOMEPAGE="http://www.gentoo.org/"
SRC_URI="mirror://gentoo/${P}-src.tar.gz"

LICENSE="Apache-2.0"
SLOT="0"
KEYWORDS="~x86 ~sparc ~ppc ~amd64 ppc64"
IUSE="examples"

CDEPEND="dev-java/xerces:2
        >=dev-java/log4j-1.2.8"

RDEPEND=">=virtual/jre-1.7
        ${CDEPEND}"

DEPEND=">=virtual/jdk-1.7
        ${CDEPEND}"

S=${WORKDIR}/${P}-src

java_prepare() {
        # Remove bundled jars
        java-pkg-clean

        # Replace removed bundled jars with system
        cd "${S}/lib"
        java-pkg_jar-from xerces-2
        # Replace with a version specific jar
        java-pkg_jar-from log4j log4j.jar log4j-1.2.8.jar
}

src_install() {
        java-pkg_newjar target/${P}-dev.jar ${PN}.jar

        use doc && java-pkg_dojavadoc dist/api
        use source && java-pkg_dosrc src/java/org
        use examples && java-pkg_doexamples src/java/examples
}
FILE bar-1.0.ebuildExample of a an ebuild with optional Java support
EAPI=5
inherit eutils java-pkg-opt-2

DESCRIPTION="Fictional example ebuild"
HOMEPAGE="http://www.gentoo.org/"
SRC_URI="mirror://gentoo/${P}.tar.gz"

LICENSE="LGPL-2.1"
SLOT="0"
KEYWORDS="~amd64 ~ia64 ~ppc ~ppc64 ~sparc ~x86"
IUSE="java doc nls"

DEPEND="java? ( >=virtual/jdk-1.7 )"
RDEPEND="java? ( >=virtual/jre-1.7 )"

src_prepare() {
        # Remove bundled jars
        java-pkg-clean

        epatch "${FILESDIR}/${P}.patch"
}

src_compile() {
        local myconf="$(use_enable java)"
        if use java; then
                myconf="${myconf} --with-javac-args=\"$(java-pkg_javac-args)\""
        fi

        econf $(use_enable nls) ${myconf} || die

        emake || die
}

src_install() {
        make install DESTDIR=${D} || die

        if use java; then
                java-pkg_newjar build/java/${P}.jar ${PN}.jar

                if use doc; then
                        java-pkg_dohtml -r doc/java
                fi
        fi
}

Warnings

There are some common warnings that are produced by the java compiler. A few are Gentoo specific and others are general. Some are of concern, and others are not.

Bootstrap class path

Anytime the source is less than the current java version you will see the following warning.

CODE 
[options] bootstrap class path not set in conjunction with -source 1.x
Note
This warning was added in java version 1.7 build 121.

While this is a generic warning, it comes from a Gentoo specific issue. Anytime the source is less than the current java version, a rt.jar from that older version needs to be set as the bootstrap classpath. This is presently not done on Gentoo, thus the warning.

It is safe to ignore, providing you do not try to run the resulting java binaries on a version of java older than that which it was compiled on. That does some what defeat the purpose of using a lower source but sometimes that is necessary for a variety of other reasons.

Retrieved from "https://wikitest.gentoo.org/index.php?title=Java_Developer_Guide&oldid=599024"