Cognitive Technologies       

links

Takashi Ando photo photo photoNathaniel  (Nat) Mills photo photoJeff Shearer photoJames H. (Jim) Stathis photo Alan C. Thomas photo Ernest Y. (Ernest Y) Wu photo

Cognitive Technologies - overview


In a world where the amount of data being created is increasing exponentially, the quest to use data in a productive way is the driving force for cognitive computing – systems that can “learn at scale, reason with purpose, and interact with humans naturally” [1].

Deep Neural Networks (DNNs)

In recent years, deep neural networks (DNN) have demonstrated significant business impact in large scale analysis and classification tasks such as speech recognition, visual object detection, pattern extraction. Training of large DNNs, however, is time consuming and computationally intensive task. Human brain can outperform modern processors on such task thanks to its massively parallel architecture connecting myriad low-power computing elements (neurons) and adaptive memory elements (synapses).

Non-volatile Memory (NVM)

Dense crossbar arrays of non-volatile memory (NVM) devices, where NVM elements represent synaptic weights, is one possible path for implementing neuromorphic computing systems [2].

G. W. Burr et al., ADVANCES IN PHYSICS: X, 2016
VOL. 2, NO. 1, 89–124

http://dx.doi.org/10.1080/23746149.2016.1259585

Resistive Processing Unit (RPU)

Researchers in IBM proposed a concept of resistive processing unit (RPU) devices [3] that can potentially accelerate DNN training by orders of magnitude while using much less power and defined specs for ideal devices. The RPU device can store and update the weight values locally thus minimizing data movement during training and allowing to fully exploit the locality and the parallelism of the training algorithm. The ideal RPU requires highly symmetric switching behaviors for positive and negative updates for synaptic weights. Such none-volatile memory does not exist today and IBM Research is exploring new materials and devices to attain the ideal device.

 

Gokmen and Y. Vlasov, Frontiers in Neuroscience 10 (2016).

Resistive RAM (ReRAM)

Resistive RAM (ReRAM) is a two-terminal non-volatile memory that switches between low and high resistance states in response to electrical bias. This is achieved by connecting and disrupting a current conducting filament formed within an ReRAM cell. ReRAM devices can be categorized into Conductive Bridging RAM (CBRAM) and Oxide ReRAM. The filament forming elements are metal atoms (e.g. Cu, Ag) for CBRAM and oxygen vacancies in metal-oxides (e.g. HfOx, TaOx, TiOx) for Oxide ReRAM. Both types of ReRAM do not require high temperature processing and they are compatible with CMOS back-end-of-line (BEOL). The high compatibility with conventional CMOS manufacturing makes ReRAM a strong candidate for artificial synapse in neural networks for cognitive computing.

 

 

[1] IBM whitepaper, “Computing, cognition, and the future of knowing"
https://ibm.biz/BdHErb


[2] G. W. Burr et al., ADVANCES IN PHYSICS: X, 2016
VOL. 2, NO. 1, 89–124
http://dx.doi.org/10.1080/23746149.2016.1259585


[3] T. Gokmen and Y. Vlasov,
Frontiers in Neuroscience 10 (2016).