Motion Prompting: Controlling Video Generation with Motion Trajectories

Gallery: "Jello" Motion Transfer

Here we show more Motion Transfer results using a video of "jello" as the source motion. Click the buttons above to see examples using motion from other source videos. Note that some videos, in particular the ones with leaves or sand, are of slightly lower quality due to video compression.

In the source video a finger taps the jello. As a result, the tracks in the upper right are occluded briefly. Interestingly, our model has learned about occlusions and in most cases will generate something that accounts for the observed occlusions, such as leaves, clouds, or some distortion.