Jonathan H. Connell  Jonathan H. Connell photo         

contact information

computer vision, biometrics, robotics, natural language, learning, AI
Thomas J. Watson Research Center, Yorktown Heights, NY USA



I am a Research Staff Member at IBM's T.J. Watson Research Center in Hawthorne, New York (north of NYC, near White Plains). I graduated from MIT in 1989 with a Ph.D. in Artificial Intelligence and have been here ever since.

I have written books on robots, robot learning, and biometrics. I was an Associate Editor for IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) from 1995-1999, and co-taught a undergraduate seminar course on man-machine interaction (Spring Semester 2001) and another on "The Quest for AI" (Spring Semester 2006) as part of the Cognitive Science Program in the Department of Psychology at Vassar College. See my CV for more details.

At IBM I have done work on mobile robot navigation, machine learning, wearable computers, and vegetable recognition. More recently I have been involved with computer vision for video search, biometric identification, and advanced user interfaces. See the web site for the Exploratory Computer Vision Group for more details on some of these.

A while back I worked on an IBM Far Reaching Research (FRR) project called "Language and Robots" in which a robot can learn new nouns and verbs from a human using a combination of speech and gesture. This same technology could also to help customize and extend conventional user interfaces (charts).

Currently I am pursuing the ALIA cognitive architecture. This is a symbolic rule-based system which is able to take advice based on spoken natural language. It can be used for issuing commands, teaching new procedures, and explaining how to investigate situations. A careful separation was made between the core reasoning engine and the physical grounding routines so that many different types of systems (even non-robotic) can take advantage of the architecture's capabilities.

I am generally interested in robotics, vision, natural language, and complete AI systems.

Some of my robotics and AI papers, as well as videos:

Verbal Programming of Robot Behaviors, J. Connell, 2019.
>>Don't grab video

Teaching Perception, J. Connell, 2019.
>>Tiger video

An Extensible Language Interface for Robot Manipulation, J. Connell. E. Marcheret, S. Pankanti, M. Kudoh, and R. Nishiyama, Proc. Artificial General Intelligence Conf. (AGI-12), LNAI 7716, pp. 21-30, December 2012.
>> ELI video

Structured Differential Learning for Automatic Threshold Setting, J. Connell and B. Herta, IBM Computer Science Research Report RC 25144, December 2010.

Four Paths to AI, J. Connell and K. Livingston, AGI-08 Conf. on Artificial General Intelligence, March 2008.

Fusing Animals and Humans, J. Connell, AGI-08 Conf. on Artificial General Intelligence, March 2008.

Visual Rotation Detection and Estimation for Mobile Robot Navigation, M. Albert and J. Connell, Proc. of the 2004 IEEE Int. Conf. on Robotics and Automation (ICRA-04), pp. 4247-4252, April 2004.

Extensible Shallow Parsing for Semantic Nets, J. Connell, IBM Computer Science Research Report RC 22339, January 2001.

Beer on the Brain, J. Connell, Proc. of the 2000 AAAI Spring Symposium, My Dinner with R2D2: Natural Dialogues with Practical Robotics Devices, pp. 25-26, March 2000.

Get Me That Screwdriver! Developing a Sensory-Action Vocabulary for Fetch-and-Carry Tasks, J. Connell, IBM Computer Science Research Report RC 19473, March 1994.

Extending the Navigation Metaphor to Other Domains, J. Connell, Proc. of the 1992 AAAI Fall Symposium, pp. 29-35, November 1992.

SSS: A Hybrid Architecture Applied to Robot Navigation, J. Connell, Proc. of the 1992 IEEE Int. Conf. on Robotics and Automation (ICRA-92), pp. 2719-2724, 1992.
>> TJ video

Automatic Programming of Behavior-based Robots using Reinforcement Learning, S. Mahadevan and J. Connell, Artificial Intelligence, vol. 55, Nos. 2-3, pp. 311-365, June 1992.
>> Obelix video

Neural Control of a Mobile Robot, J. Connell, Expanded version of an article that appeared in Popular Electronics, August 1991.

Designing Behavior-based Robots, J. Connell, Expanded version of a paper that appeared in the SPIE-91 Conf. on Mobile Robots VI, Vol. 1613, pp. 34-45, 1991.

Cooperative Control of a Semi-Autonomous Mobile Robot, J. Connell and P. Viola, Proc. of the 1990 IEEE Int. Conf. on Robotics and Automation (ICRA-90), pp. 1118-1121, 1990.
>> Mr. Ed video

A Colony Architecture for an Artificial Creature, J. Connell, Ph.D thesis, MIT AI Lab 1989, available as MIT AITR-1151 also in book form as Minimalist Mobile Robotics, Academic Press, 1990.
>> Herbert video, navigation video, reaching video

Misc videos:

Surgical tray layout

CEL person scanner

Student robot project 

Kinect 2 driver for Windows 7:

This DLL will efficiently extract BGR images and 16 bit depth images from multiple Kinect 2's. It uses the libusbk driver and needs some other DLLs compiled with Visual Studio 2015: freenect2, glfw3, libusb-1.0, and turbojpeg. The final DLL itself (below) is known to work with VS2008.