Then you say to yourself: "ok, I know that this set of pixels or object I have contained in the tracking area will still be visible in the next frame, and the next one thereafter and a few more. But it may have moved slightly. And I think it will be nearby the current location. It will be located nearby, within a search area. So we define two items: the position and size of a tracked area (the area of interest), whcih is the inner box (red), as well as a larger search area around it, the outer box (cyan). The motion tracker is then tasked to go to the next frame, and see if it can determine where within the search area the tracked area of interest has moved to.
And it does that again and again, for each subsequent frame. For each frame it has the new current position of the tracked square, and a surrounding search region. And off it goes, looking for the same pixels in the new region, frame by frame.
Once the search is completed, i.e. once it has tracked the object across its movement through all frames, you've essentially gotten a motion path out of it, a path that tells you where that initial area of interest is moving to, frame by frame.
All this being said, now that we understand how a basic motion tracker works, we can imagine that quite a bit of computation already needs to go into tracking the position of one single area of interest. Now how about tracking a set of multiple points, perhaps for rotoscoping?
PD Pro lets you do that too, laying a set of motion trackers along a curve that you draw to select a more complex area of interest. Each tracker is tasked with following the movemnt of its designated small area, and they all stay connected through the curve. The resulting curve can change position and shape, defining a tracking matte.
Example: tracking the motion of a cat's eye by laying several motion trackers around its perimeter. This can be used to place another 'object' on top of it, such as monocle, or perhaps to change the color.
Now multiply this effort by thousands.
Motion Estimation in PD Pro Howler
Imagine you shot video of a monkey that's eating lunch, and you wanted to let him chew on it in slow motion. Some parts like the jaw are going to move fast. The hands may move too, feeding new leaves to the mouth on a regular basis. Much of the rest may not move much at all, but it still can. Perhaps there's a slight breeze and the aspen leaves are rustling. Or you just forgot to use a tripod and the camera's stabilizer. Things are bound to be moving.
Imagine now that you placed a motion tracker on all parts that move: the hand, the jaw, the twitching ears, the sratching foot, the leaves, ... and what's that, another monkey, a bird, a grass hopper? Every pixel is likely to move between frames. Are we going to track every pixel? Are we going to place thousands of motion trackers across the image, trying to determine where each and every small area of interest in thouse thousands of trackers is moving? You betcha!
A grid is laid down over the entire image. Each grid cell contains a small square area of interest. By default, it is 12 by 12 pixels in size, but you can change it: you can make it coarser or finer. That 12x12 square is the tracked area of the motion tracker. And there are hundreds, or even thousands of them, depending on your frame's dimensions. For example, a 1280x720 video frame can pack more than 106 of them horizontally, and 60 vertically, i.e. well over 6000 little squares over the entire frame.
Now imagine tracking the motion of each and every one of these 12x12 'tiles'. We call them squarelets. Where did they move to on the next frame? That's what we want to know. Because once we know the answer to that, we can then morph each squarelet from one to the next frame, in any number of desired frames inbetween.
Very simply stated, the Motion Prediction Module will analyze the video in an attempt to predict which way the various parts of the images in it are moving.
PD Pro Howler lets you adjust the size of these grid, and of course the size of the search region that the trackers are allowed to explore in their quest for the search of every tile's next position. But there's more: Imagine you have changing lighting conditions, and between several frames you kno that some pixels may be flickering a bit or changing color somewhat randonly. This could make it hard to successfully track some tiles in the grid. In order to diminish this problem we can reach out beyond the next frame, using the information of two more frames ahead, and two frames back prior to the current one. We can use an average of 5 pixels to determine what it is we're really interested in tracking. And why wait there? Let's do an average of 15 frames for each tracked pixel. Of course, this comes at an amazing cost of computing time, but it's the right thing to do, when dealing with noisy video.
There are other parameters under your control. If there is a lot of noise in the video, larger Fudge factors may be recommended. Blending between the squarelets may be suggested most cases.
Details of the user interface and how to operate this feature can be fond at thebest3d.com in the PD Pro Howler documentation. (look for version 7).
One place where this all works really well is with a panoramic swipe, for example if you trun the camera from left to right. All squarelets or tiles pretty much move the same way, e.g. to the left. Here's an example with a 3D rendered scene in Carrara:
Now multiply this effort by thousands.
