19 replies to this topic

[Kristof]

Proposing a method to realistically insert synthetic objects into existing photographs without requiring access to the scene or any additional scene measurements. With a single image and a small amount of annotation, this method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene. Demonstrating in a user study that synthetic images produced by this method are confusable with real scenes, even for people who believe they are good at telling the difference. Further, our study shows that our method is competitive with other insertion methods while requiring less scene information. The method also collected new illumination and reflectance datasets; renderings produced by the system compare well to ground truth. System has applications in the movie and gaming industry, as well as home decorating and user content creation, among others.

http://kevinkarsch.c...tions/sa11.html

[Tilt]

yay! We'll be out of work next month

[Kristof]

Hehe, d'you think so? I also like their approach for matching the camera. Pretty intuitive.

[Persoonaje]

I'll stand on one leg for 1 month, or work free for 2 monhts if so needed to get my hands on this

[Tilt]

Kristof, on 17 November 2011 - 10:31 AM, said:

Hehe, d'you think so? I also like their approach for matching the camera. Pretty intuitive.

indeed. when I made my projection tutorial I wished there was a way to orbit the cam around a corner of the box that I had already aligned.

[Kristof]

Speaking of projection, did you try Gimpel3D?

Quote

Gimpel3D converts single images or frame sequences into stereoscopic 3D using a combination of traditional approaches and a proprietary projective modeling system.

The user works in true proportional space where the scene can be viewed from any location. The scene is edited geometrically in space using tools specifically designed to work with the perspective projection of the image.

This creates a virtual workspace that is intuitive for the user and proportionally accurate.

http://www.gimpel3d.com/


I haven't tried it myself yet but hope to soon. It's free now too and apparently this guy is willing to collaborate, so how about it Eyeon? Seriously. Don't get sidetracked by the stereoscopic features.

[Galvenon]

Wow, this thing looks unbelievable. There has to be some catch!
But it doesn't seem to be a product yet, is it? looks like just some sort of study. So no telling when it's coming to market. Eyeon should just buy the technology and put it in Fusion, before the Foundry gets it. LOL.

[ChadCapeland]

But what is the technology? Break down the process in the paper, and what you have is a bunch of small little tools that all probably should be in Fusion for their own sake. Camera extraction, billboard making, light extraction, retinex, etc.

[Kristof]

I learned a new word today: retinex. Love it And you are right on the money when you write "what you have is a bunch of small little tools that all probably should be in Fusion for their own sake."
Always interesting to read comments and have some more insight from people who understand what's going on underneath the hood of these things.

It's a finished product, Galvenon. They present feedback from users in their presentation, I believe. Probably not available commercially. Not now anyway.

[Tilt]

I think just a couple of small improvements to the 3D viewer and its widgets would allow you to align projections as easily as in the video. So far you can:

• orbit a perspective view around an object by pressing O (edit: it's "C")
  
• rotate an image plane or shape around its pivot

These are current shortcomings (in my opinion)

• can't orbit a camera view like a perspective view
  
• can't rotate in camera or world space, just in the object's local coordinate space.
  
• picking a new pivot moves the object because its translation and rotation values aren't adjusted

So what would we need to be able to align projectors to geometry (or image planes to a projection) a bit better?

• a switch/toolbar button that allows you to orbit a camera view like it's a perspective view instead of the current mode
  
• a way to place the invisible pivot for this orbit motion anywhere, not just at the center of an object. What about... if you press and hold "O" for more than a few milliseconds, the cursor turns into a position picker that allows you to define a pivot anywhere on an object's surface (not in empty space, but that wouldn't be necessary). It's like you would pick a position pass. With this, you could easily put an image plane in front of a camera, define one corner of a room/building as the pivot and orbit the projection around this point until everything lines up.
  
• To rotate image planes that are receiving a projection in a similar fashion, there would have to be a way to move the pivot on the plane's surface without affecting the object's position in space. It could also be a toolbar toggle button that, when enabled, modifies translation and rotation accordingly when you use the position picker for a new pivot.

[Kristof]

Agree totally! Although I have to press F to center and hover around an object in the 3D viewer. O doesn't do anything or am I missing something?

A "parent in place" feature would be very handy also. Let's say you have a 3D shape that matches the dimensions of something in frame and you know the focal length and set it accordingly. Then you just rotate, scale (on all three axis though!) and move the object so that it matches its equivalent in the footage, link your camera to the object while Fu is applying the necessary calculations automatically (parent in place) so in world space the camera keeps its initial position. Now reset everything of the 3D shape that needs to be reset and finally unlink the camera once more time, but again, maintaining the same position in world space.

Also nice to have if you have a bunch of 3D transforms and you want to get rid of them, but apply all those transformation to the original object up the pipeline.

These are all pretty basic things.

More advanced would be a feature that will allow you to calculate the proper focal length and transformation based on known dimensions and angles of an object in view.

[Kristof]

Like so:

You load an object that matches the one in the image.

 Clipboard01.png   162.37K   90 downloads

You drag vertices around to match the image in the background and once you're done you click on a button to get a solve.

 Clipboard02.png   191.43K   84 downloads

Sorry for the small size of these pictures and strange way to include them. Too lazy to upload them somewhere. I can fix a bigger batch next week if needs be.

That's an old plugin for Lightwave called CamFit. Great thing is that you can zoom in on the image / viewport and accurately re-position a vertex. That's a critical feature--the zooming.

[Tilt]

it's "C". sorry, my bad. F is "fit", while C will move the invisible center or rotation (for perspective view only) to the selected object.

You can consolidate upstream transforms using the CoordTransform modifier! But it only works for translation (wish there was a solution for rotation/scale as well). You could probably write a tool script that automatically adds such a modifier that looks at the upstream transforms and then uses it to "bake" the chain of parents.

edit: fixed rotation/translation typo. I need some sleep ;-)

Edited by Tilt, 18 November 2011 - 12:49 PM.

[Kristof]

Aha! C it is. How did you find that one out? Looked for an overview of keyboard shortcuts that work in the 3D viewer but didn't find anything.
So is this a typical case of RTFM?

Thanks for the modifier tip. I'll check it out.

[Tilt]

Kristof, on 19 November 2011 - 03:25 AM, said:

Aha! C it is. How did you find that one out? Looked for an overview of keyboard shortcuts that work in the 3D viewer but didn't find anything.
So is this a typical case of RTFM?

Thanks for the modifier tip. I'll check it out.

It's mentioned here:
http://www.vfxpedia....Material_Viewer