Add headless mode with software OpenGL rendering for 3D snapshots

[magnesj]

Description

ResInsight can currently only export 3D view snapshots from the GUI application with visible windows and a working GPU/display. The documented headless path (-platform offscreen) works for 2D plots only — projects containing 3D views crash because snapshots go through QOpenGLWidget::grabFramebuffer(), and the offscreen QPA plugin cannot create OpenGL contexts (no WGL backend on Windows, GLX requires an X display on Linux). Console mode (--console) silently ignores all snapshot options.

Goal: ResInsight --headless --project x.rsp --savesnapshots all --snapshotsize 1000 745 [--snapshotfolder <dir>] renders 3D views with software OpenGL (no GPU needed), writes PNGs, and exits — without showing any windows.

Scope: Keep the existing widget-based GUI pipeline (minimal diff). Windows + Linux. --savesnapshots CLI entry point only — no changes to RicfExportSnapshots's GUI gate or the regression test runner.

Strategy

Keep the native QPA platform ("windows"/"xcb"), never show any window, and force software OpenGL. Do not use QT_QPA_PLATFORM=offscreen (no OpenGL support on Windows at all).

Why this works: in Qt 6, QOpenGLWidget::grabFramebuffer() does not require the widget to be shown — it initializes the widget's GL context on an internal QOffscreenSurface, creates the FBO (explicitly handling never-resized hidden widgets), renders via paintGL(), and reads back. The precondition Qt::AA_ShareOpenGLContexts is already set in RiaMain.cpp:125. So the entire existing chain (RicSnapshotAllViewsToFileFeature → Rim3dView::captureSnapshot → caf::Viewer::snapshotImage → grabFramebuffer) works with hidden windows.

Software GL selection:

• Windows: Qt::AA_UseSoftwareOpenGL (set before QApplication construction) makes Qt load opengl32sw.dll (Mesa llvmpipe, already deployed to the build folder by windeployqt).
• Linux: LIBGL_ALWAYS_SOFTWARE=1 forces system Mesa llvmpipe. With no display server at all, stock Qt cannot create a GL context — the documented answer is xvfb-run -a ResInsight --headless ....

Steps

• Register --headless option in ApplicationLibCode/Application/Tools/RiaArgumentParser.cpp (near "console", ~line 73) so the unknown-option check in RiaMain.cpp:152-169 accepts it. Help text: runs without showing windows using software OpenGL rendering; combine with --savesnapshots; on Linux without a display server run under xvfb-run.
• Detect --headless pre-Qt in createApplication() in ApplicationExeCode/RiaMain.cpp:83-101 (same raw-argv scan pattern as --console). If found in GUI mode: set QApplication::setAttribute( Qt::AA_UseSoftwareOpenGL ), on non-Windows qputenv( "LIBGL_ALWAYS_SOFTWARE", "1" ), and call new RiaGuiApplication::enableHeadlessMode() before constructing the application object. Do not touch QT_QPA_PLATFORM or the AA_ShareOpenGLContexts line.
• Add headless state to RiaGuiApplication (.h/.cpp): static void enableHeadlessMode(); static bool isHeadless(); backed by static bool sm_headlessMode = false;
• Guard show/raise/dialog calls in RiaGuiApplication.cpp with isHeadless():
  • initialize() (~:523): disable progress dialogs via caf::ProgressInfoStatic::setEnabled( false ) (cafProgressInfo.h:109)
  • createMainWindow() (:1080): guard m_mainWindow->showWindow(); keep the probe QOpenGLWidget (:1094) — harmless on a hidden parent
  • getOrCreateAndShowMainWindow() (~:1028): guard the show() call
  • getOrCreateAndShowMainPlotWindow() (~:1116): guard the showNormal()/show()/raise()/activateWindow() block
  • onProjectOpened() (~:1361): guard m_mainPlotWindow->show()/raise() and the showPlotWindowOnTop block; keep processEvents(), initializeGuiNewProjectLoaded() and dock-state restore so viewer widgets still get created (RimDockWindowController::updateViewerWidget requires RiaGuiApplication::isRunning(), which stays true)
  • handleArguments() savesnapshots block (~:897-920): guard mainPlotWnd->show()/raise() and mainWnd->show()/raise(); keep both processEvents() calls and the trailing loadWinGeoAndDockToolBarLayout() calls
  • showFormattedTextInMessageBoxOrConsole() (~:1270): headless → print to std::cout and return (mirror RiaConsoleApplication.cpp:421) so --help/bad args don't block on a modal dialog
  • askUserToSaveModifiedProject() (~:278): headless → return true; (main calls closeProject() after EXIT_COMPLETED; a dialog would hang the batch)
  • useShaders() (~:499): no change needed (caf::Viewer::isShadersSupported() returns true unconditionally, no GL access)
• Hidden-widget resize fix in Rim3dView::captureSnapshot() (ApplicationLibCode/ProjectDataModel/Rim3dView.cpp:524-551): hidden widgets defer resize events (Qt::WA_PendingResizeEvent), so after setFixedSize(...) the layout never propagates the new size to the GL widget and grabFramebuffer() would render at a stale size. If !m_viewer->layoutWidget()->isVisible(), deliver a synchronous QResizeEvent to layoutWidget() via QCoreApplication::sendEvent so the layout runs immediately. No change to the visible path.
• Delay-load redirect of opengl32.dll (Windows) — discovered during implementation: VizFwk links statically against system opengl32.dll and calls glGetError/glGetString directly, while Qt::AA_UseSoftwareOpenGL creates the context inside opengl32sw.dll. From system opengl32's view no context is current, and cvf_check_ogl's while (glGetError() != GL_NO_ERROR) loops forever (cvfOpenGL.cpp:172-182 even warns about this). Fix: /DELAYLOAD:opengl32.dll + delayimp on the ResInsight target (ApplicationExeCode/CMakeLists.txt) and a __pfnDliNotifyHook2 hook in RiaMain.cpp that resolves opengl32.dll to opengl32sw.dll when headless. Non-headless runs resolve normally — behavior unchanged.
• Size the viewer's internal FBOs for never-shown widgets in caf::Viewer::snapshotImage() (Fwk/AppFwk/cafViewer/cafViewer.cpp) — discovered during implementation: Qt never calls resizeGL() for a widget that has never been shown (its own grab-FBO gets the right size, but caf::Viewer's offscreen scene FBO stays at its initial 1×1, producing an all-black image). Fix: when !isVisible(), do a first grabFramebuffer() to initialize GL, then send a synchronous QResizeEvent to the viewer (which triggers QOpenGLWidget::resizeEvent → recreateFbos + resizeGL), then grab the final image.
• Log the GL renderer in RicSnapshotAllViewsToFileFeature::exportSnapshotOfViewsIntoFolder (ApplicationLibCode/Commands/ExportCommands/RicSnapshotAllViewsToFileFeature.cpp, after the loop at ~:150): RiaLogging::info with OpenGL version/vendor/renderer from caf::Viewer::contextGroup()->info() (cvfOpenGLContextGroup.h:75). Must be after the loop — with hidden widgets the GL context is first created inside the first grabFramebuffer(). This is the proof the software path (llvmpipe) is in use.
• Build and verify on Windows (ninja from build folder) — verified with TestModels/TEST10K_FLT_LGR_NNC/10KWithWellLog.rsp (one 3D view + two well log plots):
  • --headless --savesnapshots all --snapshotsize 1000 745 → no window, exit 0 in ~2.3 s, 3 PNGs at 1000×745, log shows OpenGL used for snapshots: 3.0 Mesa 11.2.2 / VMware, Inc. / Gallium 0.4 on llvmpipe
  • Same command without --headless (GPU) → 99% of sampled pixels identical (differences are text antialiasing)
  • --headless --help → help text to console, exit 0, no dialog
  • Plot snapshots (QWidget::grab path) written correctly in the same run
• Install opengl32sw.dll in ApplicationExeCode/CMakeLists.txt: the windeployqt post-build step (:394-402) already copies it to the build tree, but install(TARGETS ... RUNTIME_DEPENDENCIES ...) (:405-410) only scans link dependencies and misses dynamically loaded DLLs. Add inside the if(WIN32 AND WINDEPLOYQT_EXECUTABLE) block: install(FILES $<TARGET_FILE_DIR:ResInsight>/opengl32sw.dll DESTINATION ${RESINSIGHT_INSTALL_FOLDER} OPTIONAL). Linux needs nothing shipped (system Mesa).
• Update docs docs/class-diagrams/ResInsightHeadlessSnapshots.txt: replace stale Qt 5 notes with --headless usage for Windows/Linux, xvfb-run -a for display-less Linux, note that -platform offscreen still works for plots-only projects and that the log line confirms llvmpipe.
• Linux verification (CI or Linux box): desktop run shows llvmpipe in log; bare run without DISPLAY fails fast (documented); xvfb-run -a produces PNGs. — verified on Ubuntu 24.04 / Qt 6.11 / Mesa 25.2.8 llvmpipe, see comment below
• Run clang-format on all changed .cpp/.h files before committing.

Risks

• grabFramebuffer() on never-shown widgets: high confidence (stable Qt behavior since 5.4, explicit hidden-widget handling in Qt source). Fallback if it fails: explicit makeCurrent() + render sequence + glReadPixels from caf::Viewer's existing m_offscreenFbo (cafViewer.cpp:214-233) — not implemented now.
• Stray modal dialogs from loaders outside RiaGuiApplication could hang a batch run; known ones are handled, others fixed as encountered.
• llvmpipe renders slowly on large models and output may differ slightly from GPU renders (AA/line widths) — acceptable for batch use.
• Windows service sessions (no interactive desktop) are untested; document as unsupported if they fail.

Links

• Current snapshot chain: RiaGuiApplication.cpp handleArguments savesnapshots block → RicSnapshotAllViewsToFileFeature.cpp → Rim3dView.cpp captureSnapshot → cafViewer.cpp snapshotImage
• Existing headless notes (stale, to be updated): docs/class-diagrams/ResInsightHeadlessSnapshots.txt
• Python headless plot example: headless_plot_export.py
• Qt docs: QOpenGLWidget::grabFramebuffer, Qt::AA_UseSoftwareOpenGL

[magnesj]

Flow of a headless snapshot run. Green nodes are new or modified code; the two deepest fixes are the delay-load redirect (keeps cvf's statically linked GL calls and Qt's context in the same GL implementation) and the double-grab in snapshotImage (sizes the viewer's internal FBOs, which never receive resize events on hidden widgets).

flowchart TD
    A["ResInsight --headless --project x.rsp<br/>--savesnapshots all --snapshotsize W H"] --> B["RiaMain createApplication()<br/>scan raw argv for --headless"]
    B --> C["Set Qt::AA_UseSoftwareOpenGL (Windows: opengl32sw.dll)<br/>Set LIBGL_ALWAYS_SOFTWARE=1 (Linux: Mesa llvmpipe)<br/>RiaGuiApplication::enableHeadlessMode()"]
    C --> D["Delay-load hook (Windows):<br/>opengl32.dll resolves to opengl32sw.dll in headless mode,<br/>so cvf's static GL calls hit the same rasterizer as Qt"]
    D --> E["RiaGuiApplication::initialize()<br/>main windows created but NOT shown<br/>progress dialogs disabled"]
    E --> F["loadProject()<br/>dock machinery creates hidden RiuViewer widgets"]
    F --> G["handleArguments(): savesnapshots<br/>show()/raise() skipped in headless mode"]
    G --> H["RicSnapshotAllViewsToFileFeature<br/>exportSnapshotOfViewsIntoFolder()"]
    H --> I["Rim3dView::captureSnapshot(W, H)<br/>setFixedSize + synchronous QResizeEvent to layoutWidget<br/>(propagates geometry to the hidden GL widget)"]
    I --> J["caf::Viewer::snapshotImage() on hidden widget:<br/>1st grabFramebuffer() initializes GL,<br/>synthetic QResizeEvent sizes the offscreen FBOs via resizeGL(),<br/>2nd grabFramebuffer() renders the final image"]
    J --> K["PNG files written to snapshot folder<br/>log: OpenGL used for snapshots ... llvmpipe"]
    K --> L["EXIT_COMPLETED, app exits"]

    style B fill:#c8e6c9,color:#000
    style C fill:#c8e6c9,color:#000
    style D fill:#c8e6c9,color:#000
    style E fill:#c8e6c9,color:#000
    style G fill:#c8e6c9,color:#000
    style I fill:#c8e6c9,color:#000
    style J fill:#c8e6c9,color:#000
    style K fill:#c8e6c9,color:#000

Loading

[magnesj]

Session summary — implementation complete, verified on Windows

All steps except Linux verification are done on branch headless-3d-snapshots (10 files, ~200 insertions, not yet committed). End-to-end result on Windows: ResInsight.exe --headless --project 10KWithWellLog.rsp --savesnapshots all --snapshotsize 1000 745 exits 0 in ~2.3 s with no visible windows, writes the 3D view and both plot PNGs at the requested size, and logs OpenGL used for snapshots: 3.0 Mesa 11.2.2 / VMware, Inc. / Gallium 0.4 on llvmpipe. A pixel comparison against the same export from the visible GUI with hardware GL shows 99% of sampled pixels identical (differences are text antialiasing).

New insights (things the plan did not predict)

1. Qt::AA_UseSoftwareOpenGL alone deadlocks ResInsight. VizFwk links statically against system opengl32.dll and calls glGetError/glGetString directly, while Qt creates its context inside opengl32sw.dll. From system opengl32's perspective no context is current, and cvf_check_ogl's while (glGetError() != GL_NO_ERROR) loop spins forever — cvfOpenGL.cpp even carries a comment warning about exactly this. Fixed with /DELAYLOAD:opengl32.dll + a __pfnDliNotifyHook2 hook in RiaMain.cpp that resolves opengl32 to opengl32sw.dll only in headless mode. Non-headless runs are unaffected.

2. Hidden widgets never get resizeGL(), leaving caf::Viewer's scene FBO at 1×1. Qt's grabFramebuffer() sizes its own FBO correctly for a never-shown widget, but never delivers a resize, so the viewer's offscreen scene FBO (the one the render sequence actually draws into) stayed at its initial 1×1 — producing a correctly-sized but all-black PNG. Fixed in caf::Viewer::snapshotImage(): when the widget is not visible, do a first grabFramebuffer() to initialize GL, send a synchronous QResizeEvent (drives recreateFbos + resizeGL), then grab the final image. Diagnosed by instrumenting paintGL (fboTex=1x1 while widget size was 500×400) after isolating that llvmpipe+visible rendered fine while llvmpipe+hidden was black.

3. The contextGroup() accessor in cafViewer.h had to move from private to public for the renderer logging.

Remaining work / open questions

• Linux verification (desktop run, no-DISPLAY failure mode, xvfb-run -a) still pending — needs a Linux box or CI.
• Hidden + hardware GL on Windows (headless without the software rasterizer) hits the same glGetError loop inside initializeOpenGLFunctions; not investigated further since headless implies software GL by design. Worth knowing if a --headless-hw variant is ever requested.
• Windows install package should be checked once built: opengl32sw.dll is now installed explicitly via install(FILES ... OPTIONAL) since RUNTIME_DEPENDENCIES cannot see dynamically loaded DLLs.

[magnesj]

Commits

Pull request: #1017

Ref	Description	Date
95de3f9	Add headless mode with software OpenGL rendering for 3D snapshots (branch headless-3d-snapshots)	2026-07-25
69058d5	Move headless rendering setup from RiaMain to RiaMainTools	2026-07-25

[magnesj]

Reopened: the merge of PR #1017 into dev was undone (dev reset to 7788539); the feature lives on branch headless-3d-snapshots, which now has dev merged in (e940bdf).

[magnesj]

Linux verification — done

Verified the headless-3d-snapshots branch (PR #1023, at d422715) on Linux. All three points of the Linux verification checklist item hold, plus the new exit-code handling. Ticking that box.

Environment: WSL2 (kernel 6.18), Ubuntu 24.04, gcc-14 RelWithDebInfo, Qt 6.11.0, Mesa 25.2.8 (llvmpipe, LLVM 20.1.2). Test projects: TestModels/Case_with_10_timesteps/BatchTest.rsp (one 3D view) and TestModels/TEST10K_FLT_LGR_NNC/10KWithWellLog.rsp (one 3D view + two well log plots), with the Windows paths rewritten to local ones.

Results

Scenario	Exit	Result
--headless --savesnapshots all under xvfb-run -a, no DISPLAY	0	3D view PNG written, exactly 1000×745
--headless with a real X display (no xvfb)	0	Works, honours --snapshotsize 800 600
Non-headless baseline under xvfb, same project	0	Byte-identical PNG to the headless run (md5 a690a1e2…)
Well log project (plots + 3D view), headless	0	Both plot PNGs and the 3D view PNG written, all at requested size
--headless --commandFile with exportSnapshots(type=ALL, width=640, height=480)	0	3 PNGs at 640×480 — the RicfExportSnapshots std::expected refactor works
Read-only snapshot folder (forced failure)	2	ERROR: Failed to export 1 of 1 view snapshots, no silent success
-platform offscreen with a 3D view, no display	2	Does not crash; fails cleanly with No snapshot image was captured …
No display server and no xvfb	134	Qt aborts on the missing xcb plugin — fails fast, as documented
ResInsight-tests under xvfb	0	929 tests / 150 suites pass

The rendered 3D image is fully correct — grid geometry, SOIL colouring, fault surfaces, overlay text and the axis cross all present. The software path is confirmed by the new log line:

info: OpenGL used for snapshots: 4.5 (Compatibility Profile) Mesa 25.2.8-0ubuntu0.24.04.2 / Mesa / llvmpipe (LLVM 20.1.2, 256 bits)

stdout logging behaves as intended: present in headless runs, and the non-headless baseline produced a 0-byte stdout log.

Three observations

1. --headless alone never exits. Without --savesnapshots or --commandFile it loads the project and sits in an invisible event loop indefinitely (killed at 90 s). Consistent with the documented "combine with --savesnapshots", but a batch user who forgets the flag gets a hung process with no window and no hint. Worth considering an early error when --headless is given without an exit-producing action.

2. Plot PNGs differ slightly between headless and GUI; 3D views are byte-identical. Compared the well log plots visually: same curves, same data, same 1000×745 size — only legend font metrics differ, shifting the plot area by a few pixels. Cosmetic, from font/DPI resolution on a never-shown window. Relevant only if plot images are ever used as regression baselines.

3. The -platform offscreen doc line is now pessimistic. docs/class-diagrams/ResInsightHeadlessSnapshots.txt says it "crashes if the project contains visible 3D views"; with the new error handling it exits cleanly with code 2 instead. The reason only reaches ~/.resinsight/logs/resinsight.log, not stdout, since the stdout logger is gated on isHeadless(). Softening that line to "fails to render 3D views" would match observed behaviour.

Note on scope

The issue scoped this to the --savesnapshots CLI entry point with "no changes to RicfExportSnapshots's GUI gate or the regression test runner". Making the export functions nodiscard-returning std::expected (f11bc2d) required touching every caller in d422715 — RicfExportSnapshots, RiaRegressionTestRunner, RicAdvancedSnapshotExportFeature and RiuMainWindow. Those are result-handling changes only; no GUI gate or regression-test behaviour was altered.

🤖 Generated with Claude Code