Compositing video footage into a 3d scene?

Blender Asked by EthanEast on January 6, 2022

I am trying to composite some footage of a person that has been chroma keyed into a scene of a dark room with flickering lights. My problem is that I don’t even know how I would get close to making it seem somewhat realistic.

How would the lights affect the person if they aren’t actually a 3D model, just a flat video? For reference, this video is basically what I am trying to replicate, minus the robotic camera movements.

Video Link

One Answer

It will depend on a number of things, not the least of which is your footage device. If your footage device can capture a log encoding, there’s a trivial way to get from the camera code value encoding to the reflectance level and colours that the camera saw.

In order, you will loosely need to consider the following:

  1. Illumination of the scene. Any CGI will need to have the illumination / emissions of the scene pushed back into the CGI context. HDRIs or proper camera footage captured from the location can help with this, with further work.
  2. Tracking. If the camera moves at all, it will require tracking.
  3. Distortion. The camera will have a degree of lens distortion that will need to be removed, then reapplied.
  4. Proxy geometry. For certain emission effects and shadows, a loose idea of the scene will be required in terms of geometry.
  5. Proper footage. Log encodings that capture scene reflectance is more or less mandatory here.
  6. Decent camera rendering transform. The CGI and the footage will need to be composited under a reasonable camera rendering transform to make the effect as seamless as possible within the limitations of your toolset.

Paul Chambers has done a fantastic series of blog posts that leads down this rabbit hole, and is well worth researching. From using loose proxy geometry to exploring how scene referred lighting ratios work, his posts are tremendously insightful. There is a wealth of information hidden away over there.

Answered by troy_s on January 6, 2022

Add your own answers!

Related Questions

Blender 2.8 no shadow lighting

1  Asked on August 23, 2020 by user77887


UV Unwrapping makes a mess and misses faces

1  Asked on August 19, 2020 by ilmu011


Blend character animation without blending rotation

0  Asked on August 19, 2020 by gage-benham


Bone to mesh – convert

1  Asked on August 16, 2020


Opening a file from Version 2.83 in Version 2.49

1  Asked on August 15, 2020 by hopper


Principled BSDF Glass Issue

1  Asked on August 14, 2020 by hexoral333


Why is my material rendering black?

0  Asked on August 12, 2020 by rocketsurgeon


AVI Composite Export

1  Asked on July 30, 2020 by lechinor


Eevee Normal pass: remap negative values

1  Asked on July 29, 2020 by sergey-kritskiy


Move existing hole on the flat model

1  Asked on July 28, 2020 by lustic


Ask a Question

Get help from others!

© 2023 All rights reserved. Sites we Love: PCI Database, MenuIva, UKBizDB, Menu Kuliner, Sharing RPP, SolveDir