Google’s Autoflip Can Intelligently Crop Videos on the Fly to Fit Any Aspect Ratio

Google has released an clear - source tool , Autoflip , that could make bad cropping a thing of the past by intelligently reframing video to correctly check alternate expression ratios .

In ablog spot , Google ’s AI squad wrote that footage shot for television and desktop computers normally comes in a 16:9 or 4:3 data formatting , but with mobile devicesnow outpace TVin terms of video recording consumption , the footage is often displayed in a way that looks odd to the oddment - user . Fixing this problem typically requires “ video curators to manually key out salient capacity on each frame , track their passage from skeletal system - to - frame , and adjust harvest regions accordingly throughout the video , ” soaking up time and exertion that could be better spent on other work .

Autoflip aim to fix that with a model that applies TV stabilizer - esque techniques to keep the camera focus on what ’s authoritative in the footage . Using “ ML - enabled object signal detection and track technologies to intelligently understand video capacity ” built on the MediaPipe framework , Google ’s team wrote , it ’s capable to line up the frame of a picture on the fly front .

Gif: (Google AI)

What ’s more , Autoflip automatically conform between scenes by identifying “ changes in the composition that intend scene change to insulate scenes for processing , ” allot to the caller . Finally , it dissect each vista to determine whether it should employ a inactive frame or tracking mode .

This is pretty neat and offers obvious advantages over motionless cropping of videos , though it ’s probably well suited to things like news footage and Snapchat television than movies and TV shows ( where being able to view an entire shot is more important ) .

For a more proficient explanation of how all this works , the Google AI teamexplained the various technology involvedin its web log place . The undertaking ’s source code is alsoavailable to view on Github , along with educational activity on how to take it for a spin .

Gif: