diff --git a/AUDIT_DWABASE_DIELECTRIC.md b/AUDIT_DWABASE_DIELECTRIC.md new file mode 100644 index 00000000..a86a99ea --- /dev/null +++ b/AUDIT_DWABASE_DIELECTRIC.md @@ -0,0 +1,272 @@ +# DwaBase native dielectric specular / IOR audit + +Audit date: 2026-07-16 + +Audited revisions: + +- `moonray`: `e403e8021ca760580d09a9a648a1fd9731a27de2` +- `moonshine`: `5bec7a4be722894ba70f22760d194c862eb67280` + +The worktrees contained pre-existing unrelated changes. This audit changed no production shader or renderer behavior. It added only focused renderer tests. + +## Executive finding + +No Fresnel/IOR defect, double Fresnel, double energy compensation, or additive diffuse/specular error was found in the native `DwaBaseMaterial` path for `metallic=0`, `specular=1`, and `transmission=0`. + +The normal-incidence dielectric response is exact: IOR 1.5 produces `F0=0.04`. The attribute is an absolute material IOR; MoonRay constructs the relative interface from the ray's current medium and correctly swaps incident/transmitted IOR on exit. The default isotropic lobe is Walter-style GGX with separable Smith masking-shadowing. Artist roughness is squared once inside the Cook-Torrance lobe, so `alpha=max(roughness^2, 0.001)` unless normal-map AA or ray roughness clamping increases the input first. + +MoonRay is using an intentional DWA model with two approximations that may differ from other renderers: + +1. a Kelemen/Kulla-Conty directional-albedo compensation lobe, enabled only when input roughness is greater than 0.5; and +2. an approximate, view-dependent `OneMinusRoughFresnel` attenuation for diffuse beneath rough specular, rather than attenuation by the actual integrated directional reflectance of the rough BRDF. + +Neither explains an IOR 1.5 to 1.3 appearance shift. Compensation is absent through roughness 0.5 and raises the normal-direction IOR 1.5 reflectance at roughness 0.8 only from 0.02284 to 0.02498 (9.3%). Changing IOR from 1.5 to 1.3 lowers `F0` and rough-lobe energy by about 57%, a much larger effect. + +## A. Native source path + +1. `moonshine/dso/material/DwaBase/DwaBaseMaterial.cc` + - `collectAttributeKeys()` binds native `specular`, `refractive_index`, `metallic`, metallic colors, `roughness`, anisotropy, transmission, diffuse, coat, normals, and normal-AA controls. + - `DwaBaseMaterial::shade()` calls `resolveParameters()` and then `createLobes()`. +2. `moonshine/lib/material/dwabase/DwaBase.cc` + - `DwaBase::update()` computes which parameter groups are required. + - `resolveSpecularParams()` saturates specular, roughness, and metallic to their documented ranges. + - `resolveParameters()` reads `refractive_index` directly and clamps only to positive epsilon. +3. `moonshine/lib/material/dwabase/DwaBaseLayerable.h` + - `computeMicrofacetRoughness()` normally returns the artist roughness unchanged. Toksvig normal AA may increase it before lobe creation. +4. `moonshine/lib/material/dwabase/DwaBaseLayerable.cc` + - `createLobes()` creates conductor, dielectric reflection/transmission, and diffuse lobes in physical layer order. + - Smooth dielectric uses `MirrorBSDF`; rough isotropic dielectric uses `MicrofacetIsotropicBSDF` with GGX by default and Smith geometry. +5. `moonray/lib/rendering/shading/BsdfBuilder.cc` + - `addComponent(const MicrofacetIsotropicBSDF&)` resolves interface IOR, constructs the native Cook-Torrance lobe, installs exact dielectric Fresnel once, and stages the underlayer attenuator. +6. `moonray/lib/rendering/shading/Ior.h` + - `ShaderIor` resolves incident and transmitted media for entering, exiting, and thin geometry. +7. `moonray/lib/rendering/shading/bsdf/Fresnel.h` + - `DielectricFresnel` implements exact unpolarized dielectric Fresnel. + - `OneMinusRoughFresnel` implements the approximate underlayer attenuation. +8. `moonray/lib/rendering/shading/bsdf/cook_torrance/BsdfCookTorrance.cc` + - Implements Beckmann and GGX Cook-Torrance reflection, Smith masking-shadowing, roughness squaring, sampling, and the compensation-lobe hook. +9. `moonray/lib/rendering/shading/bsdf/cook_torrance/energy_compensation/CookTorranceEnergyCompensation.cc` + - Implements the Kelemen/Kulla-Conty compensation term using tabulated directional albedo. + +The scalar C++ and vector ISPC implementations follow the same structure. The numerical audit exercised the C++ lobe directly. + +## B. Exact formulas and conventions + +### Parameters + +- `refractive_index`: absolute material IOR, default 1.5. It is not an F0 or relative-IOR input. +- `specular`: scalar weight in `[0,1]`, default 1. The documentation intends binary 0/1 use for physical plausibility. The value becomes the dielectric Fresnel weight. +- dielectric specular color: none. `metallic_color` and `metallic_edge_color` affect only the conductor branch. `transmission_color` tints transmission, not ordinary dielectric reflection. Iridescence is a separate optional modifier. +- `roughness`: perceptual roughness in `[0,1]`, default 0.5. +- `metallic`: default 0. At zero, no conductor lobe is created. At nonzero values DwaBase uses its own layered conductor/dielectric semantics; this is not Autodesk Standard Surface's documented linear statistical metalness mixture. That does not affect the audited `metallic=0` case. + +### Interface IOR and Fresnel + +For ordinary front-facing geometry in air: + +`eta_i = 1`, `eta_t = refractive_index`. + +On exit, MoonRay swaps these. Thin geometry is always treated as viewed from outside. The exact dielectric implementation applies Snell's law, total internal reflection, and the average of squared parallel/perpendicular polarization amplitudes. At normal incidence it reduces to: + +`F0 = ((eta_t - eta_i) / (eta_t + eta_i))^2`. + +Thus from air: + +`F0 = ((ior - 1) / (ior + 1))^2`. + +The specular weight is applied once inside the Fresnel object. There is no second IOR-level scale. + +### GGX microfacet BRDF + +For input roughness `r`: + +`alpha = max(r^2, 0.001)`. + +Normal-map Toksvig AA and the ray's minimum-roughness clamp can increase `r` before this conversion. There is no other ordinary parameter remap. The sampling distribution is widened slightly at grazing view angles to reduce variance; evaluation remains at the original alpha. + +The reflection lobe is: + +`f_r(wo, wi) = F(h.wi) D_GGX(h) G1(wo) G1(wi) / (4 |n.wo| |n.wi|)` + +with: + +`D_GGX = alpha^2 / [pi cos(theta_h)^4 (alpha^2 + tan(theta_h)^2)^2]` + +and separable Smith: + +`G1(mu) = 2 / [1 + sqrt(1 + alpha^2 (1-mu^2)/mu^2)]`. + +### Multiple-scattering compensation + +This is not a Heitz random-walk multiple-scattering BSDF. It is the Kelemen 2001 compensation lobe as presented by Kulla and Conty in 2017, using precomputed single-scatter directional albedo `E(mu,r)`: + +`f_ms = [(1-E_o)(1-E_i)/(1-E_avg)] * [F_avg^2 E_avg / (1-F_avg(1-E_avg))] / pi`. + +For entering dielectrics, MoonRay approximates: + +`F_avg = (eta-1) / (4.08567 + 1.00071 eta)`. + +The implementation computes a sampling mixture weight: + +`w_ms = max(0, r-0.5)`. + +It also uses `w_ms > 0` as the gate for evaluating the compensation term. Consequently compensation is exactly absent at roughness 0.1, 0.25, and 0.5, and active at 0.8 in the requested matrix. It is not applied twice. + +### Diffuse beneath specular + +All relevant lobes are added with `BSDFBUILDER_PHYSICAL`, meaning they receive attenuation from earlier layers and attenuate later layers. Diffuse is not simply added at full strength. + +For a rough dielectric over diffuse, the builder wraps diffuse with `OneMinusRoughFresnel`. With weighted Fresnel `F_w` and `mu_o=n.wo`: + +`t = 1 - (1-r^2)^3` + +`A(mu_o,r) = lerp(1-F_w(mu_o), 1-F_w(1), t)`. + +The diffuse lobe is multiplied by `A`. This accounts for Fresnel and specular weight, but it is only a view-direction approximation to rough directional albedo. It is not full layered transport and is not the exact `1-reflectance(rough_BRDF)` formula specified by Autodesk Standard Surface. The approximation can also break strict BSDF reciprocity for the combined underlayer, although the GGX specular lobe itself is reciprocal. + +## C. Smooth dielectric Fresnel results + +The new test calls the native exact `DielectricFresnel` directly at normal incidence, from air, with unit weight and no other lobe: + +| IOR | expected F0 | MoonRay result | +|---:|---:|---:| +| 1.0 | 0 | 0 | +| 1.1 | 0.0022676 | 0.0022676 | +| 1.3 | 0.0170132 | 0.0170132 | +| 1.5 | 0.0400000 | 0.0400000 | +| 2.0 | 0.1111111 | 0.1111111 | + +All RGB channels pass at absolute tolerance `1e-7`. + +## D. Rough dielectric numerical results + +Definitions: + +- `Rdir`: directional-hemispherical specular reflectance for `wo=n`. +- `Rhh`: cosine-weighted hemispherical average of directional specular reflectance. +- `peak`: peak normal-view BRDF value in `sr^-1`. +- `HWHM`: polar half-maximum angular radius of the normal-view lobe. +- `Tdir` and `Thh`: total specular plus a unit-albedo Lambert diffuse substrate using MoonRay's actual rough-Fresnel underlayer attenuation. This is a white-furnace stress case. + +The integrator uses the native GGX lobe and its native sampler with a deterministic low-discrepancy sequence. Values are raw linear. The last few digits are numerical estimates. + +| r | IOR | Rdir | Rhh | peak | HWHM deg | Tdir | Thh | +|---:|---:|---:|---:|---:|---:|---:|---:| +| 0.10 | 1.0 | 0 | 0 | 0 | 0 | 1.00000 | 1.00000 | +| 0.10 | 1.3 | 0.01701 | 0.06062 | 13.6035 | 0.735 | 1.00000 | 1.00079 | +| 0.10 | 1.5 | 0.04000 | 0.09125 | 31.9833 | 0.735 | 1.00000 | 1.00101 | +| 0.10 | 2.0 | 0.11110 | 0.16008 | 88.8425 | 0.735 | 0.99999 | 1.00095 | +| 0.25 | 1.0 | 0 | 0 | 0 | 0 | 1.00000 | 1.00000 | +| 0.25 | 1.3 | 0.01697 | 0.05375 | 0.34663 | 4.632 | 0.99995 | 1.00038 | +| 0.25 | 1.5 | 0.03987 | 0.08377 | 0.81497 | 4.632 | 0.99987 | 1.00110 | +| 0.25 | 2.0 | 0.11069 | 0.15232 | 2.26381 | 4.632 | 0.99957 | 1.00043 | +| 0.50 | 1.0 | 0 | 0 | 0 | 0 | 1.00000 | 1.00000 | +| 0.50 | 1.3 | 0.01584 | 0.03338 | 0.02166 | 20.136 | 0.99883 | 0.99775 | +| 0.50 | 1.5 | 0.03705 | 0.05846 | 0.05093 | 20.133 | 0.99705 | 0.99661 | +| 0.50 | 2.0 | 0.10225 | 0.12079 | 0.14147 | 20.129 | 0.99114 | 0.98881 | +| 0.80 | 1.0 | 0 | 0 | 0 | 0 | 1.00000 | 1.00000 | +| 0.80 | 1.3 | 0.01066 | 0.01658 | 0.00506 | >=90 | 0.99365 | 0.99751 | +| 0.80 | 1.5 | 0.02498 | 0.03324 | 0.01077 | >=90 | 0.98498 | 0.99083 | +| 0.80 | 2.0 | 0.06988 | 0.08023 | 0.02679 | >=90 | 0.95877 | 0.96681 | + +The tiny maximum `Thh` overshoot is 0.11%. It is within the scale of the numerical integration and MoonRay's documented approximate rough-Fresnel underlayer attenuation. There is no material energy excess remotely large enough to explain a 1.5-to-1.3 appearance shift. At medium/high roughness the approximation is mildly energy-losing rather than energy-adding. + +Compensation impact at roughness 0.8: + +| IOR | Rdir single | Rdir compensated | increase | Rhh single | Rhh compensated | increase | +|---:|---:|---:|---:|---:|---:|---:| +| 1.3 | 0.00985 | 0.01066 | 8.3% | 0.01583 | 0.01658 | 4.8% | +| 1.5 | 0.02284 | 0.02498 | 9.3% | 0.03124 | 0.03324 | 6.4% | +| 2.0 | 0.06244 | 0.06988 | 11.9% | 0.07321 | 0.08023 | 9.6% | + +## E. Why IOR 1.5 becomes silver-like faster than IOR 1.3 + +Within MoonRay itself, the effect is the expected Fresnel ratio, not an anomalous remap: + +- `F0(1.5) / F0(1.3) = 0.04 / 0.0170132 = 2.351`. +- At roughness 0.5, `Rdir(1.5) / Rdir(1.3) = 2.339`. +- At roughness 0.8, the same ratio is 2.342. + +Lowering IOR from 1.5 to 1.3 therefore removes about 57% of the specular energy at both smooth and rough settings. The visual match around 1.3 is consistent with another setup having a lower effective specular weight/F0, but MoonRay is not internally converting 1.5 to an excessive Fresnel value. + +Broad rough reflection can look pale because it carries achromatic environment illumination over a wider solid angle while the substrate is colored. The numerical data show that MoonRay broadens and lowers the peak as expected; it does not increase normal-direction integrated reflectance with roughness. At IOR 1.5, `Rdir` falls from 0.04000 at roughness 0.1 to 0.02498 at 0.8. + +The remaining cross-render difference must be isolated at closure level. Relevant non-equivalences include exact rough-BRDF implementation/compensation, diffuse attenuation strategy, base/specular weights and colors, normal AA, ray roughness clamping, and which incident radiance is visible to the specular lobe. “Same-looking” controls are not sufficient evidence of equal closures. + +## F. Mathematical validity + +The dielectric Fresnel, GGX NDF, and separable Smith masking-shadowing implementation are mathematically valid standard microfacet components. The compensation formula is a recognized physically motivated approximation to missing multiple scattering. + +The full Dwa layer stack is not an exact layered BSDF. Its view-only rough-Fresnel underlayer attenuation is an approximation and can deviate slightly from exact directional-albedo conservation and reciprocity. + +## G. Physical plausibility + +The audited ordinary dielectric is physically plausible. It has correct F0, correct medium-side handling, no meaningful white-furnace energy excess, and sensible roughness trends. The small total-energy deviations are characteristic of the approximate layer combination, not evidence of a grossly nonphysical silver boost. + +## H. Compatibility with Karma / Arnold / Standard Surface + +The current Dwa model is not specified to be numerically appearance-compatible with Karma or Arnold. + +DreamWorks' 2017 primary source describes the DWA Physical Materials as a DreamWorks parameter-blended uber-material. It says Refractive used anisotropic Cook-Torrance/Beckmann with a non-mappable IOR default of 1.5, and Solid Dielectric shared that specular model. Current public source defaults to GGX, showing that the internal DWA model evolved, not that it became Autodesk Standard Surface. + +Autodesk Standard Surface also specifies GGX and squared perceptual roughness, but its layer equation attenuates the substrate by `1 - specular * specular_color * reflectance(specular_brdf)`, where `reflectance` is the rough BRDF's directional albedo. DwaBase instead uses `OneMinusRoughFresnel`. Standard Surface also exposes dielectric `specular_color`, a separate base weight, and a documented statistical metalness mix that DwaBase does not share. + +MaterialX's PBS specification permits renderer-dependent microfacet implementations and says implementations are expected to preserve rough-surface energy, commonly using multiple-scattering compensation. That is not a numerical appearance guarantee. Arnold documentation identifies its Standard Surface implementation and Fresnel behavior, but Arnold's closure source is not available here for a direct numerical source comparison. + +Primary references: + +- [OpenMoonRay DWA family documentation](https://docs.openmoonray.org/user-reference/scene-objects/materials/dwa/) +- [Physically Based Shading at DreamWorks Animation, Xie and Lanz, 2017](https://blog.selfshadow.com/publications/s2017-shading-course/dreamworks/s2017_pbs_dreamworks_notes.pdf) +- [Autodesk Standard Surface specification](https://autodesk.github.io/standard-surface/) +- [MaterialX PBS specification](https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/documents/Specification/MaterialX.PBRSpec.md) +- [Arnold Standard Surface documentation](https://help.autodesk.com/cloudhelp/ENU/AR-Core/files/ac-shading/ac-surface-shaders/arnold_user_guide_ac_surface_shaders_ac_standard_surface_html.html) +- [Walter et al., Microfacet Models for Refraction through Rough Surfaces, 2007](https://diglib.eg.org/bitstream/handle/10.2312/EGWR.EGSR07.195-206/195-206.pdf?sequence=1) +- [Kulla and Conty, Revisiting Physically Based Shading at Imageworks, 2017](https://blog.selfshadow.com/publications/s2017-shading-course/imageworks/s2017_pbs_imageworks_slides.pdf) + +## I. Bug classification + +### Confirmed bug + +None in the requested native ordinary-dielectric path. + +### Likely bug + +None established by source trace or numerical tests. + +### Intentional model difference + +Confirmed: DWA's Kelemen/Kulla-Conty compensation gate and `OneMinusRoughFresnel` underlayer approximation differ from other possible GGX/layering implementations. + +### Parameter convention mismatch + +Rejected for IOR and isotropic roughness: DwaBase uses absolute IOR and `alpha=roughness^2`, matching the stated conventions of Autodesk Standard Surface for these two controls. A broader material-parameter mismatch remains possible because DwaBase does not have Standard Surface's exact base/specular-color/metalness semantics. + +### Inconclusive + +The exact source of the reported MoonRay-vs-Karma/Arnold image difference remains unproven until those renderers' isolated closures are sampled numerically under identical incident radiance. Their screenshots alone cannot distinguish closure response from material weights, lighting visibility, or integrator behavior. + +One unrelated edge-case code smell should receive a separate test: `BsdfBuilder.cc` constructs the diffuse underlayer attenuator using `component.getTransmissionRoughness()`. This equals reflection roughness in the audited default path, but may differ when independent transmission roughness is enabled even if transmission weight is zero. It cannot explain the reported default case and is not classified as a bug here. + +## J. Recommended next action + +Do not change production behavior, default IOR, or roughness mapping. + +1. Land or review the focused native numerical tests added by this audit. +2. Build a renderer-neutral closure probe that records `f(wo,wi)`, directional albedo, and white-furnace totals from MoonRay, Arnold, and Karma for the same GGX alpha and exact F0, bypassing their artist materials. +3. In the existing comparison scene, verify numerically rather than visually: + - raw base/diffuse weight and color; + - specular weight and color; + - actual alpha after every remap/normal-AA/min-roughness operation; + - direct and indirect incident radiance visible to the specular lobe; + - light-linking/specular visibility; + - front-face state and current medium IOR. +4. If the closure probe shows a MoonRay-only excess, isolate it by toggling only `mFavg` compensation and then substituting exact directional-albedo diffuse attenuation in a test harness. Do not alter production until one term reproduces the measured delta. +5. If a defect is demonstrated, patch only that term and add a legacy-render impact note. Any compensation or underlayer change will affect existing rough dielectric and conductor renders, especially roughness above 0.5. + +## Tests added + +- `moonray/tests/lib/rendering/shading/TestDielectricFresnel.h` +- `moonray/tests/lib/rendering/shading/TestDielectricFresnel.cc` +- registered in `moonray/tests/lib/rendering/shading/CMakeLists.txt` + +The test executable builds successfully and reports `OK (8)`. The new suite directly tests exact normal-incidence Fresnel and the requested roughness/IOR matrix, including compensated versus single-scatter energy, peak, width, and unit-diffuse totals. diff --git a/CMakeLists.txt b/CMakeLists.txt index a9b83bd7..7c718068 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -82,6 +82,32 @@ add_subdirectory(moonray) add_subdirectory(rats) add_subdirectory(scripts) +if(IsDarwinPlatform) + install(CODE [[ + set(_omr_codesign_root "$ENV{DESTDIR}${CMAKE_INSTALL_PREFIX}") + execute_process( + COMMAND /bin/bash -c " + set -e + root=\"$1\" + command -v codesign >/dev/null 2>&1 || exit 0 + for dir in \"$root/bin\" \"$root/lib\" \"$root/plugin\"; do + [ -d \"$dir\" ] || continue + find \"$dir\" -type f \\( -perm -111 -o -name '*.dylib' -o -name '*.so' \\) -print0 + done | while IFS= read -r -d '' file; do + if /usr/bin/file \"$file\" | /usr/bin/grep -q 'Mach-O'; then + /usr/bin/codesign --force --sign - \"$file\" + fi + done + " + _ "${_omr_codesign_root}" + RESULT_VARIABLE _omr_codesign_result + ) + if(NOT _omr_codesign_result EQUAL 0) + message(FATAL_ERROR "macOS ad-hoc codesign of installed OpenMoonRay runtime failed") + endif() + ]]) +endif() + if(GLD STREQUAL "$ENV{STUDIO}") install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/SDKScript DESTINATION . @@ -90,4 +116,3 @@ if(GLD STREQUAL "$ENV{STUDIO}") GROUP_READ GROUP_WRITE WORLD_READ) endif() - diff --git a/CMakeMacOSPresets.json b/CMakeMacOSPresets.json index f13d05ef..1b91eb09 100644 --- a/CMakeMacOSPresets.json +++ b/CMakeMacOSPresets.json @@ -5,9 +5,10 @@ "name": "macos-environment", "hidden": true, "environment": { - "DEPS_ROOT" : "${sourceParentDir}/installs", - "BUILD_DIR" : "${sourceParentDir}/build", + "DEPS_ROOT" : "/Applications/MoonRay/installs", + "BUILD_DIR" : "/Applications/MoonRay/build", "Boost_ROOT" : "$env{DEPS_ROOT}", + "TBB_ROOT": "$env{DEPS_ROOT}", "Libuuid_ROOT" : "$env{DEPS_ROOT}", "CppUnit_ROOT" : "$env{DEPS_ROOT}", "ISPC" : "$env{DEPS_ROOT}/bin/ispc", @@ -22,7 +23,7 @@ "PXR_INCLUDE_DIRS" : "$env{DEPS_ROOT}/include", "Random123_ROOT" : "$env{DEPS_ROOT}", "ZLIB_ROOT" : "$env{DEPS_ROOT}", - "OIIO_PYTHON" : "$env{DEPS_ROOT}/lib/python3.9/site-packages" + "OIIO_PYTHON" : "$env{DEPS_ROOT}/lib/python3.11/site-packages" }, "cacheVariables": { "CMAKE_PREFIX_PATH": "$env{DEPS_ROOT}" @@ -33,16 +34,21 @@ "hidden": true, "inherits": "macos-environment", "environment": { - "PREFIX_PXR" : "${sourceParentDir}/building/macOS/pxr-houdini", - "HOUDINI_INSTALL_DIR" : "/Applications/Houdini/Houdini20.0.751", + "PREFIX_PXR" : "/Applications/MoonRay/building/macOS/pxr-houdini", + "HOUDINI_INSTALL_DIR" : "/Applications/Houdini/Houdini20.5.584", "PXR_LIB_PREFIX" : "$env{HOUDINI_INSTALL_DIR}/Frameworks/Houdini.framework/Versions/Current/Libraries", "PXR_INCLUDE_PREFIX" : "$env{HOUDINI_INSTALL_DIR}/Frameworks/Houdini.framework/Versions/Current/Resources/toolkit/include", "PXR_DEPS_PREFIX" : "$env{DEPS_ROOT}", - "PXR_BOOST_PYTHON_LIB" : "$env{HOUDINI_INSTALL_DIR}/Frameworks/Houdini.framework/Versions/Current/Libraries/libhboost_python39-mt-a64.dylib" + "PXR_BOOST_PYTHON_LIB" : "$env{HOUDINI_INSTALL_DIR}/Frameworks/Houdini.framework/Versions/Current/Libraries/libhboost_python311.dylib" }, "cacheVariables": { "CMAKE_PREFIX_PATH": "$env{DEPS_ROOT};$env{PREFIX_PXR}", - "MOONRAY_USE_HOUDINI": "TRUE" + "MOONRAY_USE_HOUDINI": "TRUE", + "BOOST_PYTHON_COMPONENT_NAME": "python311", + "Python_ROOT_DIR": "$env{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.11", + "Python_EXECUTABLE": "$env{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.11/bin/python3.11", + "Python_LIBRARY": "$env{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib", + "Python_INCLUDE_DIR": "$env{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.11/include/python3.11" } }, { @@ -52,7 +58,7 @@ "binaryDir": "$env{BUILD_DIR}", "cacheVariables": { "CMAKE_BUILD_TYPE": "Release", - "CMAKE_INSTALL_PREFIX": "${sourceParentDir}/installs/openmoonray" + "CMAKE_INSTALL_PREFIX": "/Applications/MoonRay/installs/openmoonray" }, "generator": "Xcode" }, @@ -63,7 +69,7 @@ "binaryDir": "$env{BUILD_DIR}", "cacheVariables": { "CMAKE_BUILD_TYPE": "Release", - "CMAKE_INSTALL_PREFIX": "${sourceParentDir}/installs/openmoonray" + "CMAKE_INSTALL_PREFIX": "/Applications/MoonRay/installs/openmoonray" }, "generator": "Xcode" } diff --git a/building/Rocky9/CMakeLists.txt b/building/Rocky9/CMakeLists.txt index ed551a42..ab8a52bc 100644 --- a/building/Rocky9/CMakeLists.txt +++ b/building/Rocky9/CMakeLists.txt @@ -92,7 +92,7 @@ set(CHAIN embree) ExternalProject_Add(OpenColorIO GIT_REPOSITORY https://github.com/AcademySoftwareFoundation/OpenColorIO - GIT_TAG d1ef24cbf8021f642e42f45f50c2cd4cc77141ce # v2.2.1 + GIT_TAG b35fb73b8674176098d3084deaa9da3d111d72d1 # v2.3.0 INSTALL_DIR ${InstallRoot} CMAKE_ARGS -DOCIO_BUILD_APPS=OFF @@ -205,4 +205,3 @@ ExternalProject_Add(OptiXHeaders DEPENDS ${CHAIN} ) set(CHAIN OptiXHeaders) - diff --git a/building/macOS/CMakeLists.txt b/building/macOS/CMakeLists.txt index 12ef7ce3..0163c6e7 100644 --- a/building/macOS/CMakeLists.txt +++ b/building/macOS/CMakeLists.txt @@ -23,8 +23,17 @@ set(THIS_DIR ${CMAKE_CURRENT_SOURCE_DIR}) # User configurable settings set(InstallRoot ${rootSrcDir}/../../../../installs CACHE FILEPATH "Install root for dependencies") -set(PythonVer 3.9.6 CACHE STRING "Python version (n.m), e.g. 3.6") +set(PythonVer 3.11 CACHE STRING "Python version (n.m), e.g. 3.6") set(PythonRoot /usr CACHE FILEPATH "Location of Python install") +if(DEFINED ENV{HOUDINI_INSTALL_DIR}) + set(_defaultHoudiniInstallDir "$ENV{HOUDINI_INSTALL_DIR}") +elseif(DEFINED ENV{HFS}) + file(REAL_PATH "$ENV{HFS}/../../../../.." _defaultHoudiniInstallDir) +else() + set(_defaultHoudiniInstallDir "/Applications/Houdini/Houdini20.5.584") +endif() +set(HOUDINI_INSTALL_DIR "${_defaultHoudiniInstallDir}" CACHE PATH "Houdini install root") +set(HoudiniPythonRoot "${HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/${PythonVer}") ExternalProject_Add(Blosc GIT_REPOSITORY https://github.com/Blosc/c-blosc @@ -39,11 +48,11 @@ ExternalProject_Add(Blosc set(CHAIN Blosc) ExternalProject_Add(Boost - URL https://sourceforge.net/projects/boost/files/boost/1.78.0/boost_1_78_0.tar.gz + URL https://sourceforge.net/projects/boost/files/boost/1.82.0/boost_1_82_0.tar.gz UPDATE_COMMAND ./bootstrap.sh --prefix=${InstallRoot} --with-toolset=clang linkflags="-arch ${BOOST_ARCH}" CONFIGURE_COMMAND "" - PATCH_COMMAND patch -p3 -N < ${THIS_DIR}/Boost.patch - BUILD_COMMAND ./b2 install ${JOBS_ARG} --user-config=${THIS_DIR}/user-config.jam --with_python variant=release link=shared threading=multi define=BOOST_UNORDERED_HAVE_PIECEWISE_CONSTRUCT=0 + PATCH_COMMAND patch -p3 -N < ${THIS_DIR}/Boost.patch || true + BUILD_COMMAND ./b2 install ${JOBS_ARG} --user-config=${THIS_DIR}/user-config.jam --with-python --with-iostreams --with-filesystem --with-system --with-thread --with-chrono --with-date_time --with-atomic --with-program_options --with-regex python=${PythonVer} variant=release link=shared threading=multi architecture=arm address-model=64 define=BOOST_UNORDERED_HAVE_PIECEWISE_CONSTRUCT=0 BUILD_IN_SOURCE 1 INSTALL_COMMAND "" DEPENDS ${CHAIN} @@ -146,9 +155,10 @@ ExternalProject_Add(OpenVDB set(CHAIN OpenVDB) ExternalProject_Add(Log4CPlus - GIT_REPOSITORY https://github.com/log4cplus/log4cplus - GIT_TAG REL_2_0_5 + URL https://github.com/log4cplus/log4cplus/releases/download/REL_2_0_5/log4cplus-2.0.5.tar.xz PATCH_COMMAND patch -p1 -N < ${THIS_DIR}/log4plus-limit-threads.patch || true + COMMAND find . -name "Makefile.in" -exec touch {} + + COMMAND touch aclocal.m4 configure config.h.in BUILD_IN_SOURCE 1 CONFIGURE_COMMAND ./configure --prefix ${InstallRoot} BUILD_COMMAND make @@ -215,7 +225,7 @@ set(CHAIN embree) ExternalProject_Add(OpenColorIO GIT_REPOSITORY https://github.com/AcademySoftwareFoundation/OpenColorIO - GIT_TAG 056b7b0cb0d087961e9dba75104820e44faf52a1 # v2.0.2 + GIT_TAG b35fb73b8674176098d3084deaa9da3d111d72d1 # v2.3.0 BUILD_COMMAND make ${JOBS_ARG} CMAKE_ARGS -DCMAKE_PREFIX_PATH:PATH=${InstallRoot} @@ -332,11 +342,11 @@ if(NOT NO_USD) -DPXR_ENABLE_PYTHON_SUPPORT=ON -DPXR_USE_PYTHON_3=ON -DPXR_USE_DEBUG_PYTHON=OFF - -DPYTHON_EXECUTABLE=/usr/bin/python3 - -DPYTHON_LIBRARY=/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/libpython3.9.dylib - -DPYTHON_INCLUDE_DIR=/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers + -DPYTHON_EXECUTABLE=${HoudiniPythonRoot}/bin/python${PythonVer} + -DPYTHON_LIBRARY=${HoudiniPythonRoot}/lib/libpython${PythonVer}.dylib + -DPYTHON_INCLUDE_DIR=${HoudiniPythonRoot}/include/python${PythonVer} -DPYTHON_VERSION_MAJOR=3 - -DPYTHON_VERSION_MINOR=9 + -DPYTHON_VERSION_MINOR=11 -DTBB_USE_DEBUG_BUILD=OFF -DPXR_BUILD_TESTS=OFF -DPXR_BUILD_EXAMPLES=OFF @@ -421,4 +431,3 @@ ExternalProject_Add(GLFW DEPENDS ${CHAIN} ) set(CHAIN GLFW) - diff --git a/building/macOS/pxr-houdini/pxrConfig.cmake b/building/macOS/pxr-houdini/pxrConfig.cmake index 28237a87..a639b2d7 100644 --- a/building/macOS/pxr-houdini/pxrConfig.cmake +++ b/building/macOS/pxr-houdini/pxrConfig.cmake @@ -2,7 +2,7 @@ # Modified to use the version of USD with headers in $env{PXR_INCLUDE_PREFIX} # and libs in $env{PXR_LIB_PREFIX}, with dependencies installed to $env{PXR_DEPS_PREFIX} # $env{PXR_BOOST_PYTHON_LIB} must be set to the location of the boost_python so used by USD -# (Houdini names it libhboost_python39...) +# (Houdini names it libhboost_python311...) # Defines the following variables: # PXR_MAJOR_VERSION - Major version number. @@ -17,9 +17,9 @@ get_filename_component(PXR_CMAKE_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH) set(PXR_MAJOR_VERSION "0") -set(PXR_MINOR_VERSION "22") +set(PXR_MINOR_VERSION "25") set(PXR_PATCH_VERSION "5") -set(PXR_VERSION "2205") +set(PXR_VERSION "2505") include("${PXR_CMAKE_DIR}/pxrTargets.cmake") set(libs "arch;tf;gf;js;trace;work;plug;vt;ar;kind;sdf;ndr;sdr;pcp;usd;usdGeom;usdVol;usdMedia;usdShade;usdLux;usdRender;usdHydra;usdRi;usdSkel;usdUI;usdUtils;usdPhysics;garch;hf;hio;cameraUtil;pxOsd;glf;hgi;hgiGL;hgiInterop;hd;hdSt;hdx;usdImaging;usdImagingGL;usdRiImaging;usdSkelImaging;usdVolImaging;usdAppUtils;usdviewq") diff --git a/building/macOS/pxr-houdini/pxrTargets-release.cmake b/building/macOS/pxr-houdini/pxrTargets-release.cmake index ae3a89da..485bf292 100644 --- a/building/macOS/pxr-houdini/pxrTargets-release.cmake +++ b/building/macOS/pxr-houdini/pxrTargets-release.cmake @@ -421,12 +421,12 @@ list(APPEND _IMPORT_CHECK_FILES_FOR_usdImagingGL "${_LIB_PREFIX}/libpxr_usdImagi # Import target "usdRiImaging" for configuration "Release" set_property(TARGET usdRiImaging APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE) set_target_properties(usdRiImaging PROPERTIES - IMPORTED_LOCATION_RELEASE "${_LIB_PREFIX}/libpxr_usdRiImaging.dylib" - IMPORTED_SONAME_RELEASE "libpxr_usdRiImaging.dylib" + IMPORTED_LOCATION_RELEASE "${_LIB_PREFIX}/libpxr_usdRiPxrImaging.dylib" + IMPORTED_SONAME_RELEASE "libpxr_usdRiPxrImaging.dylib" ) list(APPEND _IMPORT_CHECK_TARGETS usdRiImaging ) -list(APPEND _IMPORT_CHECK_FILES_FOR_usdRiImaging "${_LIB_PREFIX}/libpxr_usdRiImaging.dylib" ) +list(APPEND _IMPORT_CHECK_FILES_FOR_usdRiImaging "${_LIB_PREFIX}/libpxr_usdRiPxrImaging.dylib" ) # Import target "usdSkelImaging" for configuration "Release" set_property(TARGET usdSkelImaging APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE) diff --git a/building/macOS/pxr-houdini/pxrTargets.cmake b/building/macOS/pxr-houdini/pxrTargets.cmake index f3cd6feb..af4b3d45 100644 --- a/building/macOS/pxr-houdini/pxrTargets.cmake +++ b/building/macOS/pxr-houdini/pxrTargets.cmake @@ -3,7 +3,7 @@ # Modified to use the version of USD with headers in $env{PXR_INCLUDE_PREFIX} # and libs in $env{PXR_LIB_PREFIX}, with dependencies installed to $env{PXR_DEPS_PREFIX} # $env{PXR_BOOST_PYTHON_LIB} must be set to the location of the boost_python so used by USD -# (Houdini names it libhboost_python39...) +# (Houdini names it libhboost_python311...) if("${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION}" LESS 2.6) message(FATAL_ERROR "CMake >= 2.6.0 required") @@ -17,8 +17,8 @@ cmake_policy(VERSION 2.6...3.21) # Commands may need to know the format version. set(CMAKE_IMPORT_FILE_VERSION 1) -SET(HPYTHONLIB $ENV{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib) -SET(HPYTHONINC $ENV{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.9/include/python3.9) +SET(HPYTHONLIB $ENV{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib) +SET(HPYTHONINC $ENV{HOUDINI_INSTALL_DIR}/Frameworks/Python.framework/Versions/3.11/include/python3.11) # Protect against multiple inclusion, which would fail when already imported targets are added once more. set(_targetsDefined) diff --git a/building/macOS/user-config.jam b/building/macOS/user-config.jam index f163655e..05edcf8e 100644 --- a/building/macOS/user-config.jam +++ b/building/macOS/user-config.jam @@ -1,6 +1,6 @@ -using python : 3.9 - : "/usr/bin/python3" - : "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers" - : "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib" +using python : 3.11 + : "/Applications/Houdini/Houdini20.5.584/Frameworks/Houdini.framework/Versions/Current/Resources/bin/hython" + : "/Applications/Houdini/Houdini20.5.584/Frameworks/Python.framework/Versions/3.11/include/python3.11" + : "/Applications/Houdini/Houdini20.5.584/Frameworks/Python.framework/Versions/3.11/lib" ; using clang : : : -"arch arm64" "-arch arm64" ; diff --git a/docs/Houdini21_macOS_PR_notes.md b/docs/Houdini21_macOS_PR_notes.md new file mode 100644 index 00000000..cbd184f8 --- /dev/null +++ b/docs/Houdini21_macOS_PR_notes.md @@ -0,0 +1,42 @@ +# PR Notes: Houdini 21 macOS Compatibility + +## Reviewer Summary +Adds macOS/Houdini 21 compatibility updates required to build and install OpenMoonRay against Houdini 21 using Houdini-provided USD/PXR and Python 3.11. This includes Boost 1.82 alignment, dependency fetch/build fixes, Houdini USD library naming compatibility, and source-driven Houdini plugin payload installation. This branch validates build/install/selectability in Solaris, but does not resolve native DWA material authoring/render behavior in Houdini 21. + +## Scope +- Build/dependency/install compatibility for Houdini 21 on macOS. +- No intended look/scene/rendering behavior change claim. +- No claim that native DWA Solaris authoring is fixed. + +## Validated vs Not Resolved +| Area | Status | Notes | +|---|---|---| +| Dependency build | Validated | Build path completes with current compatibility adjustments. | +| Main configure/build | Validated | Houdini-target integration build path succeeds. | +| Install | Validated | Install path completes and plugin payload is install-driven. | +| Solaris renderer selectability | Validated | MoonRay selectable in Solaris viewport. | +| Native DWA material influence in Material Library / mtlx subnet | Not resolved | Native DWA nodes do not reliably drive visible render changes in simple test. | +| Light live-update behavior (`spread` example) | Not resolved | Manual IPR update required in observed tests. | +| Dome light with texture | Not resolved | Not working in current tested branch state. | + +## Temporary Dependency Note +This branch may temporarily depend on a `moonray_dcc_plugins` fork/branch for Houdini 21 Python 3.11 payload alignment. +Current local kickoff pointer: +- branch: `houdini21-py311-kickoff` +- base commit: `eda4e57` (before additional fork commits) + +Current intent for that fork is minimal: +- install-driven `python3.11libs` payload support +- required Houdini plugin payload continuity (`otls`, `soho`, `toolbar`, integration files) +- no broad OTL/authoring semantic rewrite + +This fork kickoff is not fully stable as a native DWA workflow fix. Native DWA material behavior remains unresolved. + +## Runtime Observation Captured During Simple DWA Test +- `{dispatcherExit} Message Dispatcher [libcomputation_progmcrt.dylib] : exiting : reason is 'socket was disconnected'` +- `{clientSocketError} SocketPeer::receive: Bad file descriptor` +- `signal ... 15` + +## Reviewer Guidance +- Evaluate this PR primarily as compatibility/build/install integration work. +- Treat native DWA Solaris authoring/runtime issues as follow-up work outside this PR closure. diff --git a/docs/Houdini21_macOS_changes.md b/docs/Houdini21_macOS_changes.md new file mode 100644 index 00000000..2f3254f0 --- /dev/null +++ b/docs/Houdini21_macOS_changes.md @@ -0,0 +1,78 @@ +# Houdini 21 macOS Compatibility Changes + +## Summary +Adds macOS/Houdini 21 compatibility updates required to build and install OpenMoonRay against Houdini 21 using Houdini-provided USD/PXR and Python 3.11. This includes Boost 1.82 alignment, dependency fetch/build fixes, Houdini USD library naming compatibility, and source-driven Houdini plugin payload installation. This branch validates build/install/selectability in Solaris, but does not resolve native DWA material authoring/render behavior in Houdini 21. + +Compatibility and integration improved in this branch. Native DWA authoring/runtime behavior in Solaris remains unresolved. + +## Tested Environment +- macOS Tahoe +- Xcode 26.0.1 class toolchain flow +- Houdini 21 series validation target: +- `21.0.680` (original PR target) +- `21.0.671` (current local validation path) + +## Build/Dependency Changes +- Boost dependency alignment moved to `1.82.0`. +- Boost patch command made tolerant of already-applied state (`patch ... || true`). +- `log4cplus` dependency fetch moved from git tag checkout to release tarball: +- `https://github.com/log4cplus/log4cplus/releases/download/REL_2_0_5/log4cplus-2.0.5.tar.xz` +- `log4cplus` reproducible build workaround added before configure: +- `touch Makefile.in` +- `touch aclocal.m4 configure config.h.in` + +These are build reproducibility fixes, not renderer/material behavior fixes. + +## Python 3.11 Alignment +- Houdini 21 Python 3.11 include/lib/executable alignment was applied for USD/PXR integration in this branch. +- `building/macOS/user-config.jam` was moved from Python 3.9 config to Python 3.11 config for this compatibility track. +- Current pathing is validated for local Houdini 21 installs and should be parameterized before upstream hardening. + +## Houdini USD/PXR Pathing +- Houdini USD integration uses Houdini-provided PXR paths and library resolution. +- Compatibility mapping included: +- `libpxr_usdRiImaging.dylib` -> `libpxr_usdRiPxrImaging.dylib` +- PXR metadata compatibility updates for Houdini 21 USD packaging were applied in `pxr-houdini` CMake metadata. + +## Install Payload Changes +Source-driven install path includes Houdini plugin payload required for this branch validation: +- `otls` +- `soho` +- `python3.11libs` +- `toolbar` (where present in plugin payload layout) +- integration files under Houdini plugin tree + +Manual post-install copying is not the intended validation path for this branch. + +## Validation Results +- Dependency build: pass +- Main configure/build: pass +- Install: pass +- MoonRay renderer selectable in Solaris viewport: pass + +## DWA Material Test Status +Status: unresolved in current build. + +Observed behavior in Solaris Material Library / mtlx subnet: +- Native DWA nodes are visible/creatable, but do not reliably drive visible render changes in the simple test. +- Control path through `mtlxstandard_surface` can influence render, but this is not a native DWA workflow fix. + +Observed runtime/disconnect signal during simple DWA tests: +- `{dispatcherExit} Message Dispatcher [libcomputation_progmcrt.dylib] : exiting : reason is 'socket was disconnected'` +- `{clientSocketError} SocketPeer::receive: Bad file descriptor` +- `signal ... 15` + +## Known Unresolved Issues +- Native DWA material authoring/render influence in Solaris remains unresolved for this branch. +- Some light parameter edits (for example `spread`) require manual IPR refresh and do not always live-update automatically. +- Dome light textured workflow is currently not working in this tested branch state. + +## Notes For Reviewers +- This branch should be reviewed as a Houdini 21 macOS build/install compatibility update. +- Do not treat native DWA node visibility/selectability as evidence that native DWA material workflow is fixed. +- No claim is made that MaterialX/native DWA Solaris authoring is production-ready in this branch. + +## Future Cleanup / Upstreaming Notes +- Parameterize local Houdini/Python paths before upstream merge hardening. +- Keep compatibility/build fixes separated from shader authoring/runtime behavior work. +- Track native DWA Solaris authoring/runtime closure as a follow-up effort. diff --git a/moonray/hydra/hdMoonray b/moonray/hydra/hdMoonray index 6366b993..deab1844 160000 --- a/moonray/hydra/hdMoonray +++ b/moonray/hydra/hdMoonray @@ -1 +1 @@ -Subproject commit 6366b993e276708ee4289e236fccef47f3b89d30 +Subproject commit deab18442404369697a2a21e034dd5fb27add96d diff --git a/moonray/hydra/moonray_sdr_plugins b/moonray/hydra/moonray_sdr_plugins index 77a0ed6d..3c7f9b65 160000 --- a/moonray/hydra/moonray_sdr_plugins +++ b/moonray/hydra/moonray_sdr_plugins @@ -1 +1 @@ -Subproject commit 77a0ed6dd3ff9389d46d0ca9e3471a56335b875a +Subproject commit 3c7f9b65df9fb8f05f037e2a9cb2aa625e434113 diff --git a/moonray/moonray b/moonray/moonray index 2343dde0..e58cc783 160000 --- a/moonray/moonray +++ b/moonray/moonray @@ -1 +1 @@ -Subproject commit 2343dde0bae2f70141b1004371282405e2d82215 +Subproject commit e58cc783616722d598f588f02670c4f31678b882 diff --git a/moonray/moonray_dcc_plugins b/moonray/moonray_dcc_plugins index eda4e578..ece65c71 160000 --- a/moonray/moonray_dcc_plugins +++ b/moonray/moonray_dcc_plugins @@ -1 +1 @@ -Subproject commit eda4e578a2adee2cd7478035e927e4b69c4f91bd +Subproject commit ece65c71e0a8dd1ba25fff05d977b66cc4f5fbe8 diff --git a/moonray/moonshine b/moonray/moonshine index 32a6e649..ee3c84c1 160000 --- a/moonray/moonshine +++ b/moonray/moonshine @@ -1 +1 @@ -Subproject commit 32a6e64967677008bddf05ea2fc5fe5533c0f47b +Subproject commit ee3c84c1d1ca2ea8e3bb60309718bfba9a4ab5a7 diff --git a/moonray/moonshine_usd b/moonray/moonshine_usd index d3452328..495000e6 160000 --- a/moonray/moonshine_usd +++ b/moonray/moonshine_usd @@ -1 +1 @@ -Subproject commit d3452328c52fa60d891b4d5d7fbd18a14d9ba632 +Subproject commit 495000e6fe8e90084c9eb622c7a054958ddf9be2 diff --git a/moonray/render_profile_viewer b/moonray/render_profile_viewer index 2e68fc1a..5b183d40 160000 --- a/moonray/render_profile_viewer +++ b/moonray/render_profile_viewer @@ -1 +1 @@ -Subproject commit 2e68fc1aa618b545646ac7efb091fff38f347e1f +Subproject commit 5b183d40e45ca111092398b65114a045ae26e3b7 diff --git a/moonray/scene_rdl2 b/moonray/scene_rdl2 index b1309584..b7aa377e 160000 --- a/moonray/scene_rdl2 +++ b/moonray/scene_rdl2 @@ -1 +1 @@ -Subproject commit b1309584b7340bc665fdbf0c2519e69b41b3a9f3 +Subproject commit b7aa377ee59b6743806f9d3ba88f4e93cd91f317 diff --git a/scripts/Rocky9/setupHoudini.sh b/scripts/Rocky9/setupHoudini.sh index c5b09444..0e959bea 100644 --- a/scripts/Rocky9/setupHoudini.sh +++ b/scripts/Rocky9/setupHoudini.sh @@ -1,6 +1,6 @@ omr_install_dir=/opt/MoonRay/installs/openmoonray -# save/restore PYTHONPATH, since Houdini seems to reject python3.9 +# save/restore PYTHONPATH, since Houdini runtime can be sensitive to non-Houdini site-packages OLDPP=${PYTHONPATH} source ${omr_install_dir}/scripts/setup.sh export PYTHONPATH=${OLDPP} diff --git a/scripts/macOS/setupHoudini.sh b/scripts/macOS/setupHoudini.sh index 68ac25fb..8db1be0d 100644 --- a/scripts/macOS/setupHoudini.sh +++ b/scripts/macOS/setupHoudini.sh @@ -1,14 +1,112 @@ omr_install_dir=/Applications/MoonRay/installs/openmoonray +houdini_install_dir=${HOUDINI_INSTALL_DIR:-/Applications/Houdini/Houdini20.5.584} +houdini_fallback="${houdini_install_dir}/Frameworks/Houdini.framework/Versions/Current/Resources/houdini" -# save/restore PYTHONPATH, since Houdini seems to reject python3.9 +# save/restore PYTHONPATH, since Houdini runtime can be sensitive to non-Houdini site-packages OLDPP=${PYTHONPATH} -source ${omr_install_dir}/scripts/setup.sh +if [ -f "${omr_install_dir}/scripts/setup.sh" ]; then + source "${omr_install_dir}/scripts/setup.sh" +fi export PYTHONPATH=${OLDPP} -export REL=${omr_install_dir} -export RDL2_DSO_PATH=${omr_install_dir}/rdl2dso.proxy:${omr_install_dir}/rdl2dso -export MOONRAY_CLASS_PATH=${omr_install_dir}/shader_json -export ARRAS_SESSION_PATH=${omr_install_dir}/sessions -PXR_PLUGINPATH_NAME=${omr_install_dir}/plugin/pxr -#export HOUDINI_DSO_ERROR=5 -export HOUDINI_PATH=/Applications/Houdini/Houdini20.0.751/Frameworks/Houdini.framework/Versions/Current/Resources/houdini:${omr_install_dir}/houdini/:${omr_install_dir}/plugin/houdini +export REL="${omr_install_dir}" +export RDL2_DSO_PATH="${omr_install_dir}/rdl2dso.proxy:${omr_install_dir}/rdl2dso" +export MOONRAY_CLASS_PATH="${omr_install_dir}/shader_json" +export ARRAS_SESSION_PATH="${omr_install_dir}/sessions" + +prepend_unique_path() { + local add_path="$1" + local current="${2:-}" + case ":${current}:" in + *":${add_path}:"*) echo "${current}" ;; + *) + if [ -n "${current}" ]; then + echo "${add_path}:${current}" + else + echo "${add_path}" + fi + ;; + esac +} + +prepend_existing_path() { + local add_path="$1" + local current="${2:-}" + if [ -d "${add_path}" ]; then + prepend_unique_path "${add_path}" "${current}" + else + echo "${current}" + fi +} + +resolve_ocio_package_value() { + local raw_value="$1" + local value="${raw_value}" + + case "${value}" in + \${OCIO-*}) + value="${value#\$\{OCIO-}" + value="${value%\}}" + ;; + esac + + value="${value//\$HOME/${HOME}}" + case "${value}" in + "~/"*) value="${HOME}/${value#~/}" ;; + esac + echo "${value}" +} + +set_ocio_from_houdini_packages() { + if [ -n "${OCIO:-}" ]; then + return + fi + + local package_dirs="${HOUDINI_PACKAGE_DIR:-${HOME}/Library/Preferences/houdini/20.5/packages:${houdini_install_dir}/Frameworks/Houdini.framework/Versions/Current/Resources/packages}" + local package_dir + while IFS= read -r package_dir; do + if [ ! -d "${package_dir}" ]; then + continue + fi + local package_file + while IFS= read -r package_file; do + if [ ! -f "${package_file}" ] || ! grep -q '"OCIO"' "${package_file}"; then + continue + fi + local raw_value + raw_value="$(sed -nE 's/.*"OCIO"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/p' "${package_file}" | head -1)" + if [ -z "${raw_value}" ]; then + continue + fi + local candidate + candidate="$(resolve_ocio_package_value "${raw_value}")" + if [ -f "${candidate}" ]; then + export OCIO="$(realpath "${candidate}")" + return + fi + done < <(find "${package_dir}" -maxdepth 1 -type f -name '*.json' 2>/dev/null) + done < <(printf '%s\n' "${package_dirs}" | tr ':' '\n') +} + +# Houdini loads package files itself, but standalone MoonRay tools launched from +# this shell do not. Mirror a package-authored OCIO default without overriding an +# explicitly supplied OCIO so husk/hd_usd2rdl use the same color config as Houdini. +set_ocio_from_houdini_packages + +# Preserve any existing USD plugin search path while guaranteeing MoonRay plugin location is present. +export PXR_PLUGINPATH_NAME="$(prepend_unique_path "${omr_install_dir}/plugin/pxr" "${PXR_PLUGINPATH_NAME}")" +export PXR_PLUGIN_PATH="$(prepend_unique_path "${omr_install_dir}/plugin/pxr" "${PXR_PLUGIN_PATH}")" +export PXR_PLUGINPATH_NAME="${PXR_PLUGINPATH_NAME%:}" +export PXR_PLUGIN_PATH="${PXR_PLUGIN_PATH%:}" +export PYTHONPATH="$(prepend_existing_path "${omr_install_dir}/lib/python" "${PYTHONPATH}")" + +# Prefer layering MoonRay onto an existing Houdini env (from houdini_setup). +# If that wasn't sourced yet, fall back to the configured Houdini resources path. +if [ -n "${HOUDINI_PATH}" ]; then + export HOUDINI_PATH="$(prepend_existing_path "${omr_install_dir}/plugin/houdini" "${HOUDINI_PATH}")" + export HOUDINI_PATH="$(prepend_existing_path "${omr_install_dir}/houdini" "${HOUDINI_PATH}")" +else + export HOUDINI_PATH="${houdini_fallback}:&" + export HOUDINI_PATH="$(prepend_existing_path "${omr_install_dir}/plugin/houdini" "${HOUDINI_PATH}")" + export HOUDINI_PATH="$(prepend_existing_path "${omr_install_dir}/houdini" "${HOUDINI_PATH}")" +fi diff --git a/scripts/setup.sh b/scripts/setup.sh index 999594db..d1124efe 100644 --- a/scripts/setup.sh +++ b/scripts/setup.sh @@ -8,10 +8,19 @@ omr_root="$(realpath ${sourcedir}/..)" # Walk up to find the top-level install dir where the dependencies are installed install_root=${omr_root} -while [ "$(basename ${install_root})" != "installs" ] +while [ "$(basename ${install_root})" != "installs" ] && [ "${install_root}" != "/" ] do install_root=$(dirname ${install_root}) done +if [ "$(basename ${install_root})" != "installs" ]; then + sibling_installs="$(dirname "$(dirname "${omr_root}")")/installs" + if [ -d "${sibling_installs}" ]; then + install_root="$(realpath "${sibling_installs}")" + else + echo "Warning: could not find parent installs directory for ${omr_root}; using ${omr_root}" + install_root=${omr_root} + fi +fi echo "Found install root at ${install_root}" echo "Setting up release in ${omr_root}" @@ -19,8 +28,35 @@ echo "Setting up release in ${omr_root}" # NB required for Arras to function (it needs to find execComp) export PATH=${omr_root}/bin:${PATH} -# need python modules for the USD interface and for the RATS tests -export PYTHONPATH=${install_root}/lib/python:/usr/local/lib/python:${install_root}/lib64/python3.9/site-packages/:${PYTHONPATH} +prepend_unique_path() { + add_path="$1" + current="${2:-}" + case ":${current}:" in + *":${add_path}:"*) echo "${current}" ;; + *) + if [ -n "${current}" ]; then + echo "${add_path}:${current}" + else + echo "${add_path}" + fi + ;; + esac +} + +prepend_existing_path() { + add_path="$1" + current="${2:-}" + if [ -d "${add_path}" ]; then + prepend_unique_path "${add_path}" "${current}" + else + echo "${current}" + fi +} + +# Need Python modules for the USD interface and for the RATS tests. Only add +# paths that exist so source-tree setup does not inject stale system locations. +export PYTHONPATH="$(prepend_existing_path "${omr_root}/lib/python" "${PYTHONPATH}")" +export PYTHONPATH="$(prepend_existing_path "${omr_root}/lib64/python3.11/site-packages" "${PYTHONPATH}")" # tell moonray where to find dsos @@ -43,7 +79,11 @@ export PXR_PLUGIN_PATH=${omr_root}/plugin/pxr:${PXR_PLUGIN_PATH} # for legacy DW # create shader descriptions if they don't exist if [ ! -d "${omr_root}/shader_json" ] then - echo "Building shader descriptions..." - ${omr_root}/bin/rdl2_json_exporter --out ${omr_root}/shader_json/ --sparse - echo "...done" + if [ -x "${omr_root}/bin/rdl2_json_exporter" ]; then + echo "Building shader descriptions..." + ${omr_root}/bin/rdl2_json_exporter --out ${omr_root}/shader_json/ --sparse + echo "...done" + else + echo "Warning: ${omr_root}/bin/rdl2_json_exporter not found; shader descriptions were not generated" + fi fi