Run your Java model

Suggest edits
Documentation > Run

Contents

Making ScalaTask embedding Java code 🔗

OpenMOLE makes the inclusion of your own Java code in a workflow simple. Since the Scala compiler can compile Java code, Java code can be directly fed to the ScalaTask as shown in the ScalaTask example. Additionally, a compiled Java program can be encapsulated in a ScalaTask as a jar file through the libraries parameter as below.

/!\: In this page, there are two example to use a code developed in Java from OpenMOLE. The first one works for simple Java programs and the second one for a more complex program that requires dependencies managed by Maven. For more ambitious developments, it is recommended to consider embedding your code in an OpenMOLE plugin. It is pretty straight forward since they are simple OSGi bundles.

Simple Task for Java 🔗

Let us consider the simple code Hello.java in a directory named hello (to respect Java's package structure):

package hello;

public class Hello {
  public static void run(int arg) {
    System.out.println("Hello from Java! " + arg);
  }
}

We compile the code and generate the JAR file as follows:
mkdir hello
mv Hello.java hello
cd hello
javac Hello.java
cd ..
jar cvf Hello.jar hello

To call this scala code we will use a ScalaTask which is able to call Java code. In order to do that you should first create a folder in the OpenMOLE interface and then upload the Hello.jar file in this folder in OpenMOLE. In the same folder you may then write script (script.oms) which looks like this:
val proto1 = Val[Int]

//Defines the task to perform the hello function
val javaTask = ScalaTask("hello.Hello.run(proto1)") set (
  libraries += workDirectory / "Hello.jar",
  inputs += proto1
)

DirectSampling(
  evaluation = javaTask,
  sampling = proto1 in (0 until 10)
)

The output should look like that (the order in which the lines are printed might be different in your case):
Hello from Java! 0
Hello from Java! 1
Hello from Java! 2
Hello from Java! 3
Hello from Java! 4
Hello from Java! 5
Hello from Java! 6
Hello from Java! 7
Hello from Java! 8
Hello from Java! 9
Hello from Java! 10

In the general case a task is used to compute some output values depending on some input values. To illustrate that, let's consider another Java code:
package hello;

public class Hello {
  public static double[] run(double arg1, double arg2) {
    return double[]{arg1 * 10, arg2 * 10};
  }
}

You can compile it and package as "Hello.jar" in the same manner as the previous example. You can then use the task input variables when calling the Java function, and assign the function result to the task output variable like so:

val arg1 = Val[Double]
val arg2 = Val[Double]
val out1 = Val[Double]
val out2 = Val[Double]

val javaTask = ScalaTask("Array(out1, out2) = hello.Hello.run(arg1, arg2)") set (
  libraries += workDirectory / "Hello.jar",
  inputs += (arg1, arg2),
  outputs += (arg1, arg2, out1, out2)
)

// save the result in a CSV file
val csvHook = AppendToCSVFileHook(workDirectory / "result.csv")

DirectSampling(
  evaluation = javaTask hook csvHook,
  sampling =
    (arg1 in (0.0 to 10.0 by 1.0)) x
    (arg2 in (0.0 until 10.0 by 1.0))
)

This workflow will call the function hello with both arguments varying from 0 to 10 and save the results in a CSV file. Variables arg1 and arg2 are specified in the task output so that they are also written to the CSV file by the hook (see Hooks)

Example of Java code working with files 🔗

When a task needs to access (read from or write to) external files, OpenMOLE needs to know it so it can pass them around to distant computing nodes. Let's consider another "Hello World" code Hello.java. This program reads the content of a file and writes it to another file.

package hello;

import java.io.*;

public class Hello {

  public static void run(int arg, File input, File output) throws IOException {
    //Read the input file
    String content = readFileAsString(input);
    PrintStream myStream = new PrintStream(new FileOutputStream(output));
    try {
      myStream.println(content + "  " + arg);
    } finally {
      myStream.close();
    }
  }

  private static String readFileAsString(File file) throws IOException {
    byte[] buffer = new byte[(int) file.length()];
    BufferedInputStream f = null;
    try {
      f = new BufferedInputStream(new FileInputStream(file));
      f.read(buffer);
    } finally {
      if (f != null) try { f.close(); } catch (IOException ignored) { }
    }
    return new String(buffer);
  }
}

You can compile it and package as "Hello.jar" in the same manner as the previous example. For the program to access external files, give them to the task as inputFile.

val proto1 = Val[Int]
val inputFile = Val[File]
val outputFile = Val[File]

//Defines the scala task as a launcher of the hello executable
val javaTask =
ScalaTask("val outputFile = newFile(); hello.Hello.run(proto1, inputFile, outputFile)") set (
  libraries += workDirectory / "Hello.jar",
  inputs += (proto1, inputFile),
  outputs += (proto1, outputFile),
  inputFile := workDirectory / "input.txt"
)

//Save the output file locally
val copyHook =
  CopyFileHook(
    outputFile,
    workDirectory / "out-${proto1}.txt"
  )

DirectSampling(
  evaluation = javaTask hook copyHook,
  sampling = proto1 in (0 to 10)
)

For this example to work you should create a file named "input.txt" in the work directory of your project.

Building a bundle with Maven 🔗

OpenMOLE can use complex Java libraries as they are well packaged. Maven, that manages Java project life-cycle and dependencies, has a plugin (maven-shade) that can produce such bundles. In order to allow the use of this plugin, you have to configure it in the plugin list of the maven project file. In the <plugins> </plugins> section, you have to add the following code.
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.4.3</version>
    <executions>
        <execution>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <filters>
                    <filter>
                        <artifact>*:*</artifact>
                        <excludes>
                            <exclude>META-INF/*.SF</exclude>
                            <exclude>META-INF/*.DSA</exclude>
                            <exclude>META-INF/*.RSA</exclude>
                        </excludes>
                    </filter>
                </filters>
                <shadedArtifactAttached>true</shadedArtifactAttached>
                <!-- This bit merges the various META-INF/services files. -->
                <transformers>
                    <transformer
                  implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                </transformers>
            </configuration>
        </execution>
    </executions>
</plugin>
Once this code is added, you can produce the bundle by simply executing a build command (with mvn install for example), and it will be created into the target directory. The file is named name_of_youar_artifact-version-shaded.jar. You can now upload it into the OpenMOLE client and use it as described in the tutorial for simple Java code.

Troubleshooting using raster data from GeoTools 🔗

Several libraries, such as GeoTools, use JAI library that embeds a provider mechanism to load the different functions. In order to use it properly, notably for calculation distribution it requires (1) to declare explicitly the use of JAI in the JAva code and (2) to ensure in the OpenMOLE script that all the classes are used in the same class loader. You can refer to a simple project that includes a very simple task that handles raster data in the following GitHub repository

Declaration of JAI 🔗

In order to declare the JAI functions, a code has to be prepared such as following em{initJai()} method that is defined for a TiffReaderTask class.

   public static void initJAI() {
        // Disable mediaLib searching that produces unwanted errors
        // See https://www.java.net/node/666373
        System.setProperty("com.sun.media.jai.disableMediaLib", "true");

        // As the JAI jars are bundled in the geotools plugin, JAI initialization does not work,
        // so we need to perform the tasks described here ("Initialization and automatic loading of registry objects"):
        // http://docs.oracle.com/cd/E17802_01/products/products/java-media/jai/forDevelopers/jai-apidocs/javax/media/jai/OperationRegistry.html
        OperationRegistry registry = JAI.getDefaultInstance().getOperationRegistry();
        if (registry == null) {
            System.out.println("geotools: error in JAI initialization. Cannot access default operation registry");
        } else {
            // Update registry with com.sun.media.jai.imageioimpl.ImageReadWriteSpi (only class listed javax.media.jai.OperationRegistrySpi)
            // it would be safer to parse this file instead, but a JAI update is very unlikely as it has not been modified since 2005
            try {
                new ImageReadWriteSpi().updateRegistry(registry);
            } catch (IllegalArgumentException e) {
                // See #10652: IAE: A descriptor is already registered against the name "ImageRead" under registry mode "rendered"
                System.out.println("GeoTools: error in JAI/ImageReadWriteSpi initialization: "+e.getMessage());
            }

            // Update registry with GeoTools registry file
            try (InputStream in = TiffReaderTask.class.getResourceAsStream("/META-INF/registryFile.jai")) {
                if (in == null) {
                    System.out.println("geotools: error in JAI initialization. Cannot access META-INF/registryFile.jai");
                } else {
                    registry.updateFromStream(in);
                }
            } catch (IOException | IllegalArgumentException e) {
                System.out.println("GeoTools: error in JAI/GeoTools initialization: "+e.getMessage());
            }
        }

        // Manual registering because plugin jar is not on application classpath
        IIORegistry ioRegistry = IIORegistry.getDefaultInstance();
        ClassLoader loader = TiffReaderTask.class.getClassLoader();

        Iterator<Class<?>> categories = ioRegistry.getCategories();
        while (categories.hasNext()) {
            @SuppressWarnings("unchecked")
            Iterator<IIOServiceProvider> riter = ServiceLoader.load((Class<IIOServiceProvider>) categories.next(), loader).iterator();
            while (riter.hasNext()) {
                IIOServiceProvider provider = riter.next();
                System.out.println("Registering " + provider.getClass());
                ioRegistry.registerServiceProvider(provider);
            }
        }
}

This function simply has to be called at the beginning of the task code. For example for the TiffReaderTask :

public static double readGeoTiff(File file) throws IOException {

 System.out.println("initJAI start");
 initJAI();
 System.out.println("initJAI end");
 //Do something ...

Handling the class loader 🔗

When defining the task on the OpenMOLE script, you have to ensure that all JAI classes are managed by the same class loader by using the dedicated function withThreadClassLoader. When using the Java code as a library, it requires some update on the script as follow (for example, for the TiffReaderTask) :

val GeoTifWriter = ScalaTask($tq
  |import fr.openmole.geotools.tiff._
  |withThreadClassLoader(TiffReaderTask.getClassLoader())(TiffWriterTask.write(fileIn, fileOut))".stripMargin
  $tq.stripMargin) set (
  libraries += workDirectory / "test-mupcity-openmole-0.0.1-SNAPSHOT-shaded.jar",
  fileIn :=  workDirectory / "data" / "test.tif",
  fileOut :=  workDirectory / "data" / "out.tif",
  inputs += (fileIn, fileOut),
  outputs += (fileIn, fileOut)
)

The static method getClassLoader(), is a simple code that returns the class loader of the class TiffReaderTask.