Compositing video footage into a 3d scene?

Blender Asked by EthanEast on January 6, 2022

I am trying to composite some footage of a person that has been chroma keyed into a scene of a dark room with flickering lights. My problem is that I don’t even know how I would get close to making it seem somewhat realistic.

How would the lights affect the person if they aren’t actually a 3D model, just a flat video? For reference, this video is basically what I am trying to replicate, minus the robotic camera movements.

Video Link

One Answer

It will depend on a number of things, not the least of which is your footage device. If your footage device can capture a log encoding, there’s a trivial way to get from the camera code value encoding to the reflectance level and colours that the camera saw.

In order, you will loosely need to consider the following:

  1. Illumination of the scene. Any CGI will need to have the illumination / emissions of the scene pushed back into the CGI context. HDRIs or proper camera footage captured from the location can help with this, with further work.
  2. Tracking. If the camera moves at all, it will require tracking.
  3. Distortion. The camera will have a degree of lens distortion that will need to be removed, then reapplied.
  4. Proxy geometry. For certain emission effects and shadows, a loose idea of the scene will be required in terms of geometry.
  5. Proper footage. Log encodings that capture scene reflectance is more or less mandatory here.
  6. Decent camera rendering transform. The CGI and the footage will need to be composited under a reasonable camera rendering transform to make the effect as seamless as possible within the limitations of your toolset.

Paul Chambers has done a fantastic series of blog posts that leads down this rabbit hole, and is well worth researching. From using loose proxy geometry to exploring how scene referred lighting ratios work, his posts are tremendously insightful. There is a wealth of information hidden away over there.

Answered by troy_s on January 6, 2022

Add your own answers!

Related Questions

ColorRamp following musgrave islands

1  Asked on December 16, 2021 by eric-brochu


Choppy video even with ‘Lossless’ settings?

0  Asked on December 16, 2021 by user101850


How would I go about making this fuselage?

1  Asked on December 16, 2021 by captainrob34


Importing Maya Files into Blender

0  Asked on December 16, 2021 by ufo-alien


How to add weight to your textures

1  Asked on December 16, 2021 by yash-sasso


Metaball Active Element Panel is missing

1  Asked on December 14, 2021 by barry-neilsen


Saving startup file does not effect startup

2  Asked on December 14, 2021 by truecameltype


Denoising options in latest 2.91 nightly builds

1  Asked on December 14, 2021 by garyo


Asymmetrical Armature Pose on Object with Mirror Modifier

0  Asked on December 14, 2021 by rob-proctor


How do you texture a remeshed particle system?

0  Asked on December 14, 2021 by kerbacho


Ask a Question

Get help from others!

© 2023 All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP