[for reference] Add Support for Image Planes (as a concrete schema type) by nxkb · Pull Request #1 · LumaPictures/USD

nxkb · 2023-09-25T22:44:21Z

Description of Change(s)

This is a full feature reference Pull Request (used in production at Luma) to help guide development of the ultimate Image Plane support in the USD core.
usd-proposal-pr
link to wg draft

Adds ImagePlane prim type, as a textured plane parented to the camera that will display in usdview to support VFX plate workflows.

implementation author: @sirpalee

I have verified that all unit tests pass with the proposed changes

I have submitted a signed Contributor License Agreement

…ured surface.

meshula · 2023-10-28T18:16:07Z

@@ -0,0 +1,25 @@
+-- glslfx version 0.1


is the introduction of imaging/lib/hdSt/shaders/... intentional?

meshula · 2023-10-28T18:18:26Z

        work
        ${Boost_PYTHON_LIBRARY}
-        ${TBB_tbb_LIBRARY}
+        ${OIIO_LIBRARIES}


OpenUSD 23.11 has OpenEXR support built in, so it shouldn't be necessary to directly link OIIO or OpenEXR

exr does not support 8-16 etc formats

Sure, to be more clear about my comment, we won't be able to add a dependency on OpenImageIO to usdGeom. Loading textures is a function of the Hio library, and Hio already has a plugin to support OpenImageIO reading and writing.

meshula · 2023-10-28T18:35:53Z

+        doc = "Rotation value for the image plane. Specified in degrees."
+    )
+    int2 coverage = (0, 0) (
+        doc = "Coverage of the image in pixels. 0 is equal to the image's size."


we typically use coverage to mean geometric coverage of a texel. OpenEXR uses the word window ~ perhaps windowSize and windowOrigin would work here?

I think he means protection (0.05) or padding +0.05 (same as overscan 1.05)

meshula · 2023-10-28T18:36:39Z

+        doc = "Alpha Gain"
+    )
+    float depth = 100.0 (
+        doc = "Depth"


Does depth mean distance from the local origin, in scene units? I wonder if the word distance might be the right choice?

meshula · 2023-10-28T18:37:45Z

+        doc = "Fitting mode for the image plane."
+    )
+    float2 offset = (0.0, 0.0) (
+        doc = "Image plane offset in inches."


How do imageCenter, and offset relate? How do inches relate to scene units?

meshula · 2023-10-28T18:40:02Z

+        doc = "3D offset for the image plane."
+    )
+    float2 size = (-1.0, -1.0) (
+        doc = "Image plane size in inches. Less than zero means equal to the camera aperture size."


if we know the camera parameters, and the distance of the image plane from the camera, wouldn't we want to specify this in NDC? I'm not sure why inches is the choice here.

units should be inherited - imperial or metric. Would call this Physical Dimension,
In practical cameras, sensor are defined in mm, with 4 digits of precision being the tiniest photosite known (0.7 nm on some samsung phone sensor).
Not clear what negative value reference is? Not a fan of semantic associated to sign...

meshula · 2023-10-28T18:41:15Z

+        doc = "Using frame extension"
+    )
+    int frameOffset = 0 (
+        doc = "Frame offset"


I realize this is necessary since USD hasn't got support for multiframe textures. I have a feeling this should be addressed upstream. In terms of this proposal, this is probably the right choice, but we should bookmark the problem for further discussion.

meshula · 2023-10-28T18:41:30Z

+        doc = "Coverage origin in pixels. Clamped to -image's size and image's size."
+    )
+    bool useFrameExtension = false (
+        doc = "Using frame extension"


What is a frame extension? Perhaps a little more doc.

meshula · 2023-10-28T18:42:55Z

+        doc = "Depth"
+    )
+    float squeezeCorrection = 1.0 (
+        doc = "Squeeze Correction"


Might we name this pixelAspectRatio? squeeze is fine, but I'm biased to OpenEXR conventions :)

-pedantic:

squeeze correction?

theory/convention is you have "Anamorphic Squeeze" = Optical Compression (lens thing, how image is projected on sensor). A camera lens is described as 1.33X, 2X anamorphic... not 0.5, Yes?
As in, to account at lens distortion level.

and PAR- Pixel Aspect Ratio (a sampling thing, what is a pixel?...) which in contemporary cameras is always 1.0 (only legacy video formats, NTSC | PAL had non-square pixels). Yes?
For example a squeezed SBS stereo is not anamorphic, is squeezed when recording video (storage aspect ratio versus display aspect ratio).
And something to deal with I guess for a background image that is not square pixels if you overlay render over.
[e.g: In "scope" the desqueeze is performed by the projector lens- reversing the camera lens anamorphic]

In practice at fileIn level: Anamorphic Stretch = Anamorphic Desqueeze = 1/Anamorphic_Squeeze = Pixel Aspect Ratio = is all the same transform.
There might be a restoration project somewhere in the universe having to deal with scope telecined in an NTSC or PAL context, but not a fan of supporting legacy complexity.

the oval shape bokeh "effect" as if it was shot anamorphic (as in some shots in Toy Story 4)
vs

putting a background image on the "film back" (sensor crop dimension) of Maya does have lens squeeze ratio as parameter, I forget if people still have to compensate with overscan so the back plate matches?

I see the expression Anamorphic Desqueeze as most used to describe stretching image to fit the desired image aspect ratio.

I'd be fine with a naming that includes the word "anamorphic" and clarity in the rest of the name around whether one is to multiply or divide in order to undistort an image.

meshula · 2023-10-28T18:44:11Z

+        doc = "Height"
+    )
+    float alphaGain = 1.0 (
+        doc = "Alpha Gain"


alpha gain could use an explanation.

+1 - same question, not a thing

meshula · 2023-10-28T18:47:37Z

+        doc = "The frame offset for animated textures."
+    )
+    token fit = "best" (
+        allowedTokens = ["fill", "best", "horizontal", "vertical", "to size"]


best and to size should be described. I can't guess what they are without reading the code....

In ASC-FDL template, best was ignored as an option as it's not the same meaning in different apps... Also what does fill mean?

My understanding from previous discussion is that fill means "stretch the image arbitrarily such that it fills the rectangle".

meshula · 2023-10-28T18:47:49Z

+        doc = "Asset path to the file containg the image data."
+    )
+    double frame = 0.0 (
+        doc = "The frame offset for animated textures."


frameOffset might be more descriptive

meshula · 2023-10-28T18:48:50Z

+        doc = "Precached frame count"
+    )
+    float width = 0.0 (
+        doc = "Width"


width of the image plane in NDC, same question for height?

meshula · 2023-10-28T18:49:27Z

@@ -0,0 +1,368 @@
+//
+// Copyright 2016 Pixar


year of introduction would be 2023 (wherever a new file is introduced)

meshula · 2023-10-28T18:57:12Z

This is looking pretty good. It's pretty obvious, I imagine, from my questions/comments that I'm trying to get a mental picture of the model. I'm wondering if a diagram might be created that shows a camera frustum, the image plane, and how all the values modify the display of the image plane? That would really help me understand all the moving parts :)

chadrik · 2023-11-21T22:18:54Z

I love that this PR is getting some traction right now, but the authors of this PR are no longer at Luma, and nor am I, so I can't speak to whether there are resources at Luma to work on this. I would encourage others to carry this forward and make a proposal to the USD community.

nxkb · 2023-11-22T03:10:56Z

You can carry this forward by contributing to the latest proposal or adapting this PR to match that proposal.
Here is a link to the proposal as far as we got: PixarAnimationStudios/OpenUSD-proposals#22

sirpalee added 2 commits September 19, 2023 18:13

[usdView] Loading default shaders for usdview and adding a plain text…

2076be5

…ured surface.

[USD] Support for Image Planes in Maya / Hydra.

779f47f

meshula reviewed Oct 28, 2023

View reviewed changes

Denise-Yang mentioned this pull request Feb 26, 2026

Adding Back Plate Support for USD PixarAnimationStudios/OpenUSD-proposals#104

Merged

1 task done

Conversation

nxkb commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of Change(s)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

revisionfx Nov 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

revisionfx Nov 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meshula Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meshula Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meshula Oct 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

meshula commented Oct 28, 2023

Uh oh!

chadrik commented Nov 21, 2023

Uh oh!

nxkb commented Nov 22, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

nxkb commented Sep 25, 2023 •

edited

Loading

revisionfx Nov 21, 2023 •

edited

Loading

revisionfx Nov 21, 2023 •

edited

Loading

meshula Oct 28, 2023 •

edited

Loading

meshula Oct 28, 2023 •

edited

Loading

meshula Oct 28, 2023 •

edited

Loading