Skip to content

Add hdf5-native Maven JARs and SciJava native-lib-loader integration#6356

Draft
matteodg wants to merge 7 commits into
HDFGroup:developfrom
matteodg:6355-add-hdf5-native-maven-artifacts
Draft

Add hdf5-native Maven JARs and SciJava native-lib-loader integration#6356
matteodg wants to merge 7 commits into
HDFGroup:developfrom
matteodg:6355-add-hdf5-native-maven-artifacts

Conversation

@matteodg
Copy link
Copy Markdown

@matteodg matteodg commented Apr 7, 2026

Fixes #6355

I created this with Cursor, as my knowledge of CMake is basically zero: I tried to do it myself, but I was not able. I tested this in Linux x86_64 and Windows x86_64 and it is creating the proper jar and pom.xml file, that I can install in my local Maven repository and depend on them from an example application. This was there is not need of installing HDF5 library or setting up LD_LIBRARY_PATH anymore.

This will be useful for the future of HDFView as well.

@brtnfld
Copy link
Copy Markdown
Collaborator

brtnfld commented Apr 8, 2026

Must Fix:
FFM H5.java static block lacks try-catch around Hdf5NativeLoader call -- NoClassDefFoundError will permanently break H5 if the native-lib-loader JAR is missing

The FFM [H5.java:314] calls Hdf5NativeLoader.loadBundledNativesIfPresent(false) with no try-catch in the static block. If native-lib-loader-2.5.0.jar is missing from the classpath, loading Hdf5NativeLoader will throw NoClassDefFoundError (because it imports org.scijava.nativelib.NativeLoader), which will cause an ExceptionInInitializerError that permanently prevents H5 from being used in that JVM.

The JNI [H5.java:366-374] correctly wraps the call in catch (Throwable err), which handles this. The FFM version does not. The FFM path should have the same protection.

Nice to have:

  • Partial-load when JNI bridge fails but hdf5 succeeds -- works correctly but worth a comment
  • [docs] Transitive native deps (zlib, szip) not bundled -- "zero-install" doesn't hold if HDF5 was built with external filters

Moderate:

  • Duplicate POM profiles across two pom.xml.in files -- The 5 OS-activated profiles (~70 lines each) in pom.xml.in and pom-jni.xml.in) are nearly identical (JNI version adds hdf5-jni-native dependency). This is a lot of duplicated XML that will need to be kept in sync. Not easily avoidable with Maven profiles though.
    HDF5JavaNativeBundles.cmake is include()'d, not add_subdirectory() -- This means all its variables leak into the parent scope. The leading underscores help, but it's still a code smell. Consider using add_subdirectory() or wrapping in a function.

Low Severity

  • Regex in validate-maven-artifacts.sh is getting unwieldy ([line 30]

grep -qE "<artifactId>hdf5-java(-ffm|-jni)?</artifactId>|<artifactId>hdf5-native</artifactId>|<artifactId>hdf5-jni-native</artifactId>|<artifactId>hdf5-java-examples</artifactId>"

Consider using a simpler pattern like hdf5-(java|native|jni-native) or an array-based check.

  • find with -exec cp {} may clobber files -- In ctest.yml and maven-staging.yml, multiple POM files with different names (pom.xml, pom-hdf5-native.xml, etc.) are copied to a flat directory. If there are multiple pom.xml files from different build directories, later ones overwrite earlier ones. This was a pre-existing issue but is now slightly more likely to bite.

  • The _np and _jp variable names in maven-deploy.yml ([lines 112-116] are cryptic. Minor, but NATIVE_POM / JNI_NATIVE_POM would be clearer.

  • RELEASE_PROCESS.md change crams a lot of information into a single bullet point that's now very long and hard to parse.

Verify:

  • Vendored JAR should be checksum-verified against Maven Central; license acknowledged

Copy link
Copy Markdown
Collaborator

@brtnfld brtnfld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Must Fix" needs to be addressed.

@github-project-automation github-project-automation Bot moved this from To be triaged to In progress in HDF5 - TRIAGE & TRACK Apr 8, 2026
@hyoklee hyoklee added the Component - Wrappers C++, Java & Fortran wrappers label Apr 10, 2026
@hyoklee hyoklee added this to the Backlog milestone Apr 10, 2026
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch 8 times, most recently from 4779e1e to f7e31e1 Compare April 27, 2026 21:54
@matteodg matteodg requested a review from brtnfld April 27, 2026 21:58
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch from 2a04368 to cbf65fa Compare April 28, 2026 07:01
@brtnfld
Copy link
Copy Markdown
Collaborator

brtnfld commented May 1, 2026

  • Replace err.printStackTrace() with proper logging

In both java/hdf/hdf5lib/H5.java (FFM) and java/src-jni/hdf/hdf5lib/H5.java (JNI), replace:


catch (Throwable err) {
    err.printStackTrace();
}

with:


catch (Throwable err) {
    log.debug("Bundled HDF5 native library not loaded: " + err.getMessage());
}
  • Remove the unused _HDF5JAVA_NATIVE_POM_DATE variable

In java/cmake/HDF5JavaNativeBundles.cmake:


string (TIMESTAMP _HDF5JAVA_NATIVE_POM_DATE "%Y-%m-%d %H:%M:%S UTC" UTC)
This is computed at configure time but never used anywhere in the file

Nice to have:

  • Add trailing newline to both new POM template files

java/cmake/pom-native.xml.in and java/cmake/pom-jni-native.xml.in both end without a newline (\ No newline at end of file). Add a newline at the end of each.

  • Clarify isHdf5LibraryLoaded dual semantics
    File: java/src-jni/hdf/hdf5lib/H5.java

The problem: isHdf5LibraryLoaded is set in two unrelated places with different meanings:

In the bundled loader block — means "bundled hdf5 loaded"
At H5.H5dont_atexit() — means "hdf5 is confirmed loaded by any means"
A reader maintaining this code later will think the second assignment is a copy-paste error and delete it.

— split the flag's responsibilities):

Replace the two flags with three, each with a single clear meaning:


private static boolean isHdf5LibraryLoaded     = false; // bundled hdf5 load succeeded
private static boolean isHdf5JavaLibraryLoaded = false; // hdf5_java load succeeded
private static boolean isInitialized           = false; // full loadH5Lib() completed

Change the early-exit guard from:

if (isHdf5LibraryLoaded && isHdf5JavaLibraryLoaded)
    return;

to:

if (isInitialized)
    return;

Change the H5.H5dont_atexit() block from:

        try {
            H5.H5dont_atexit();
            isHdf5LibraryLoaded = true;
        }
        catch (HDF5LibraryException e) {
            System.exit(1);
        }

to:

        try {
            H5.H5dont_atexit();
            isInitialized = true;
        }
        catch (HDF5LibraryException e) {
            System.exit(1);
        }

This way isHdf5LibraryLoaded tracks only the bundled load result, isHdf5JavaLibraryLoaded tracks only the JNI bridge load result, and isInitialized guards against redundant re-entry.

Copy link
Copy Markdown
Collaborator

@brtnfld brtnfld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments

@matteodg matteodg marked this pull request as draft May 12, 2026 21:53
matteodg added 2 commits May 12, 2026 23:56
Introduce java/lib/vendored-jars.sha256 and a CI workflow that runs
sha256sum -c against it (no network). Document SLF4J in java/lib/NOTICES.txt.
Add org.scijava:native-lib-loader 2.5.0 under java/lib for consistent
classpath use (examples cache includes the JAR alongside SLF4J). Define
HDF5_JAVA_NATIVE_LIB_LOADER_JAR for downstream CMake consumers.

Add offline SHA-256 verification for java/lib JAR too.
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch from cbf65fa to 02937dc Compare May 12, 2026 22:09
matteodg added 4 commits May 13, 2026 01:49
…a native-lib-loader

Introduce HDF5JavaNativeBundles.cmake to stage shared libraries under
natives/<platform>/ (SciJava native-lib-loader layout), emit manifests
with Implementation-Version, and add pom-native / pom-jni-native
templates. Wire FFM and JNI hdf.hdf5lib builds and install the
native-lib-loader JAR with other Java dependencies.

Register add_subdirectory(java) before printing Maven deploy status so
Java targets and globals exist before the examples subdirectory pulls in
HDF5ExampleCache.cmake.

Add Hdf5NativeLoader (thin wrapper over NativeLoader) and call it from
H5 for FFM and JNI before loading JNI. Declare org.scijava:native-lib-loader
in generated POMs.
Preserves skip/override properties documented in INSTALL.
Include HDF5_JAVA_NATIVE_LIB_LOADER_JAR alongside logging JARs so unit
tests compile and run the same classpath layout as production JARs.
Extend artifact validation for native classifier JARs and adjust
workflows (ctest, maven-staging, maven-deploy) for the new native
bundle layout and dependencies.

Copy POMs from FFM vs JNI CMake output paths instead of matching any
pom.xml under the build tree. ctest uses ACTIVE_PRESET *Maven-FFM*
and fails if both trees contain pom.xml. maven-staging branches on
matrix.implementation and validates exactly one pom.xml in artifacts.
Describe hdf5-native / classpath layout, SciJava loader usage,
Hdf5NativeLoader, and release notes touch-up for published artifacts.

Document that bundled native loading may partially succeed (libhdf5
loaded even if JNI bridge load fails) and clarify that transitive
native dependencies like zlib/szip/plugins are not bundled in the
Maven native JARs.
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch 2 times, most recently from 0eb14bb to 736bd89 Compare May 13, 2026 14:20
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch from 736bd89 to 0bcd2dc Compare May 14, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component - Wrappers C++, Java & Fortran wrappers

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

Publish hdf5-native Maven artifacts (bundled libhdf5 for classpath loading)

3 participants