Physical Analytics - Intelligent drones


Description: The goal of this project is development a testing platform for cognitive on-drone computation systems using as a surrogate the game of ping pong. For this we are integrating various hardware and software technologies such as computer vision, energy-efficient computation, 3D object tracking, motion dynamics and  game intelligence into a drone.

Intelligent drone prototype.

In the current prototype we are using a 3D Robotics IRIS drone system, Raspberry PIs for on-board computation, Microsoft Kinect cameras for the computer vision system.  One of the goal of this research is for the drone to carry out its applications regardless of the connectivity. This requirement poses significant constraints on the system design, which includes trading off between computational efficiency (in order to be low-power), payload,  and agility of the drone.  Please see the demo video below for our current prototype:

In the current prototype version, we have one drone playing against a robo-pong machine. We employ two Kinect cameras as the computer vision system that determine the drone position and generate a "synthetic GPS" signal. The Kinects cameras are connected to the ground-based PC running on MATLAB. We developed the codes to perform the object detection and 3D co-location. The location information of both the drone and the ping pong are then continuously fed to the drone as it plays the ping-pong game.  

Planned square path via waypoints using a synthetic GPS -see also https://youtu.be/7YmhK2V0YNs

We also developed ping-pong motion tracking system by studying its basic flight dynamics. For this purpose, we built a small wind tunnel to study the ping pong drag in the presence of laminar air flow. In this experiment we extracted the drag coefficient CD of a ping-pong  ball as it "travels" through air (see also reference).

Ping-pong flight dynamics


Recent publications:

Press:

  • Tech Times,  "IBM Research Team Programs Drone to Play Table Tennis"
  • Le Pais (Spain), ""El dron que juega al ping pong llegará muy lejos" (The drone-playing ping-pong will go far)
  • GIZMODO, "It won't be long before drones start winning Olympic gold medals in table tennis"
  • SiliconRepublic, "Robots could soon demolish humans at table tennis"
  • Mentalfloss, "IBM Researchers Built a Drone That Can Play Ping Pong"
  • Dronepunks, "This Drone Plays Table Tennis Like A Champ"
  • ShortList,  "A drone playing table tennis is the weirdest thing you'll watch today"
  • Trumpiot, "IBM Programmed IRIS+ 3DR Drones For Ping Pong"
  • LeptiDrone, "Quand les drones jouent au ping pong"
  • Middronepro, "¿Hasta donde llegaran los drones? la idea detrás del dron que juega al ping pong"
  • Borg, "IBMの研究チームが開発した「卓球ができるドローン」が話題に"
  • Egg-life, "UAV to play table tennis because of fear of being suspended or beaten yet"
  • TeamUSA, "IBM Research Team Programs Drone To Play Table Tennis"

Movies & Demos: 

Team:

Wang Zhou (hardware & software developer, currently at Northwestern Univ.)
Dhruv Nair (hardware and computer vision system, currently at Columbia Univ.)
Theodore van Kessel (system design)
Oki Gunawan (electronics, computer vision system)
Hendrik F. Hamann (project manager)
Various funding and technical support by Pradip Bose, Sharathchandra Pankanti and Chung-Ching Lin.