MIT’s New Camera Drone System Uses AI to Frame a Shot

Before drones, the stunning aerial shots in Hollywood movies were accomplished by a small crew in a helicopter. Even with the arrival of drones, crew members were still needed to plan the shots with painstaking precision and operate the drones. But researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have engineered a new autonomous drone system that would take the crew out of the picture. It’s another example of the seemingly limitless applications of drone technology (and AI) that internet of things app developers rave about.

With CSAIL’s so-called “real-time motion planning for aerial videography,” a director can dial in the specs for a shot and trust the drones to capture the footage needed. Filmmakers input the subject they want filmed and how they want it framed, and the drones adjust in real-time to keep the subjects in the shot, all while avoiding collisions. The system achieves this by estimating the speeds of moving objects in the environment 50 times per second, which allows it to anticipate potential obstacles and change trajectory as needed. This level of control is an innovation that should really inspire IoT app developers.

The system may not be ready for elaborate, large-scale productions quite yet, but what MIT’s team has accomplished so far is very promising. In tests, the researchers were able to keep three subjects in frame at a time. Mobile app developers might not that there are drones on the market with similar object recognition and tracking features, but these consumer models lack the level of control required for professional filmmaking. Although cinematographers and camera operators aren’t in danger of losing their jobs just yet, MIT’s new rig is a step toward changing the way films are shot. Their new drone system will be demoed at the International Conference on Robotics and Automation later this month.