Increasing need for semiconductors suitable for AI processing
High-speed parallel computing, the limits of the von Neumann architecture
PIM semiconductors, memory-centric computing, etc. are needed


As the value of data utilization increases, the demand for high-speed data processing and computing capabilities is growing. According to the 'Data Age 2025' white paper published by market research firm IDC in 2018, the total size of data worldwide is expected to increase by an average of 61% per year to 175 ZB (zetabytes) in 2025.

IDC predicted that during the same period, IoT devices will produce more than 50% of all data, about 50% of all data will be stored and processed in the public cloud, and about 30% of all data will be consumed in real time, such as in autonomous vehicles.

On the 1st, ETRI's Intelligence Policy Research Lab emphasized in the white paper 'ETRI AI Implementation Strategy 2: Strengthening AI Semiconductor and Computing System Technology Competitiveness' that "data types and utilization methods are diversifying" and "in order to respond to this, it is necessary to advance data processing and calculation in the data processing chain that connects cloud computing, edge computing, and devices."


Why we need semiconductors optimized for AI processing and computing
There are many difficulties in achieving advanced data processing and calculations using existing semiconductor and computing technologies. The white paper points out that semiconductor and computing technologies have two structural limitations.

The first is the physical limit of semiconductor integration represented by Moore's Law. As transistor integration increases, it becomes more difficult to overcome limitations such as heat generation and interference between devices.

The second is that the Von-Neumann method, which is the basis of the existing computer system structure, is not efficient for high-speed parallel operations suitable for AI processing. The Von-Neumann structure, which is based on serial processing from the main memory, central processing unit, and input/output devices, causes a data bottleneck problem when performing high-speed parallel operations.

Parallel processing requires data movement between processors and storage devices, and between each storage layer. As this movement increases, the data movement speed, rather than the CPU processing speed, affects computing performance and energy consumption. The white paper explains that high-performance computing technology is needed to solve this problem, and in the long term, a transformational computing system material, structure, and computational model are needed.

In terms of technological development and market and industrial competitiveness, the importance of AI semiconductors and computing technology is growing day by day. This is because AI technology is being rapidly applied to various industries. Stable and efficient AI implementation depends on semiconductor and computing technology, which leads to industrial development and, furthermore, to increased national competitiveness.


AI Accelerator vs PIM Semiconductor
AI semiconductors are semiconductors optimized for AI data processing, such as high-speed parallel operations. Current system semiconductors have low efficiency, such as power consumption, in implementing AI, so dedicated semiconductors are required. The white paper pointed out that research is being conducted in two directions to solve the two problems above.

The first direction is the development of AI accelerators, or AI-specific processors, that support computational patterns specialized for AI algorithm processing. AI-specific processors implement parallel circuits between the processor and memory to improve parallel processing performance, delay time, and power efficiency, and are divided into learning and inference types depending on the purpose.

The AI-only processor market is dominated by NVIDIA. NVIDIA GPUs account for 97% of AI accelerators used in the four major cloud services (Amazon, Microsoft, Google, and Alibaba). Google has been developing the TPU (Tensor Processing Unit), which has adopted an AI algorithm-only acceleration structure, since 2016, and is responding to NVIDIA by applying the TPU with a performance of 100PF (petaflops) to its cloud.

The second direction is the development of HBM (High Bandwidth Memory)-based PIM (Processing in Memory) processors that package memory and processor on the same chip.

HBM-based PIM processors overcome the inefficiency of data movement from memory to processor by co-packaging the processor with memory chips or stacked memory cells. This reduces bottleneck problems caused by bandwidth differences between CPU and memory and energy consumption caused by data movement. Samsung Electronics and SK Hynix are focusing on HBM development, while Micron is focusing on HMC (Hybrid Memory Cube) technology.


Memory-centric computing, a computing structure suitable for AI processing
The white paper defines AI computing systems as high-performance computers that produce, process, and utilize large amounts of data at ultra-high speeds. AI technology requires more and higher-resolution training data as it implements higher accuracy, which entails an exponential increase in computational volume.

Processor-centric computing with the von Neumann architecture is not suitable for large-scale parallel processing such as AI applications because it is more affected by data movement speed than CPU processing performance when processing large amounts of data.

Most supercomputers are not energy efficient. The white paper cited the example of Summit, the world’s top supercomputer, which performs 148,000 trillion calculations per second and consumes up to 13 megawatts of power, enough to power 8,000 U.S. homes simultaneously.

Recent high-performance computing research is adopting memory-centric computing to overcome energy efficiency issues. Memory-centric computing is a computing model that reorganizes the computer architecture around memory rather than processors to minimize data movement between processors.

It is characterized by connecting multiple memory nodes with a high-speed fabric to form a huge shared memory pool, which allows multiple computing nodes to process data in parallel, thereby improving information processing performance.

The white paper stated that active cooperation and alliances between companies are underway to develop technologies and create an ecosystem for computing architectures, system SW, etc. that utilize high-speed interconnect and non-volatile memory technologies, which are the foundation of memory-centric computing.

Since 2016, the 'CCIX (Cache Coherent Interconnect for Accelerators)' consortium centered around AMD and Xilinx, 'Gen-Z' centered around Dell EMC and HPE, and 'Open CAPI' consortium centered around IBM have been working to establish industry standards for next-generation interconnects for memory-centric computing.

In March 2019, Intel launched the Compute Express Link (CXL) consortium and announced CXL 1.0, an interconnect technology that supports memory sharing between accelerators and CPUs at high bandwidth.


Neuromorphic, a semiconductor that mimics the human brain
Neuromorphic semiconductors are non-von Neumann semiconductors that use Spiking Neural Network (SNN) technology to mimic the neuron-synapse structure that functions within the biological brain. The core contains several electronic components, including transistors and memory. Some of the core components act as neurons in the brain, and the memory semiconductors act as synapses that connect neurons.

Neuromorphic semiconductors, which can process large amounts of data with low power, have high integration capacity, can learn like the human brain, and have high computational performance. Their performance is similar to that of existing deep learning methods, and their power efficiency is also high. Therefore, they are suitable for mobile systems with limited power resources.

However, software-based neuromorphic systems mathematically define the functions of neurons and synapses, and the actual computations are performed by existing computer systems, which ultimately limits performance, learning time, and power consumption. To solve this, hardware-based neuromorphic semiconductors are needed.

Research on neuromorphic semiconductors that mimic the human brain has been conducted as a national R&D project since the mid-2000s, mainly in Europe and the United States. The EU has been conducting a large-scale original research project on the human brain called the Human Brain Project (HBP) with an investment of 1 billion euros since 2013, and the United States has also established the BRAIN Initiative since 2013 and is promoting extensive technological development.

IBM has been participating in the SyNAPSE project led by DARPA, an affiliate of the U.S. Department of Defense, since 2008 and developed a neuromorphic chip called 'TrueNorth' in 2014. The chip can classify images at 1,200 to 2,600 frames per second with just 1/10,000th the power consumption of conventional processors.

In 2012, 'SpiNNaker', developed mainly by the University of Manchester in the UK, is a massively parallel processing neuromorphic computer that can model spiking neural networks in real time and can simulate 1 billion neurons. In 2019, Intel announced 'Loihi', a spiking neuromorphic chip capable of learning. This chip proved to have learning and execution performance 1 million times faster than DNN (Deep Neural Network).


AI semiconductors, communication with industries utilizing AI is essential
We have entered an era where the speed of AI calculations is linked to a company’s competitiveness. Accordingly, demand for semiconductors optimized for AI processing is increasing.

The industry sees parallel processing of data as a more effective way to perform AI processing than serial processing. Accordingly, major companies and organizations are focusing on developing processors that perform multiple operations with many cores rather than processors that perform high-level operations with a small number of cores.

We are also working to develop memory-centric semiconductor and computing architectures that simplify data movement by placing where data is stored and where it is processed as close as possible. The development of neuromorphic semiconductors that mimic the human brain is also ongoing, and research on quantum computing that goes beyond the existing computing units is being conducted around the world.

Our government has also been increasing investment and policies in this field, such as the 'National Artificial Intelligence Strategy (19.12)', 'AI Semiconductor Industry Development Strategy (20.10)', 'National Supercomputing Leading Project (20.03)', and 'Quantum Computing Technology Development Project Promotion Plan (19.01)', in order to secure technological competitiveness in the AI era.

In his keynote speech at the Arm DevSummit 2020 last November, Furiosa AI CEO Baek Jun-ho said, “In order for an AI semiconductor company to grow, the company itself must secure world-class AI semiconductors and the software stack that goes on top of those semiconductors.”

He added, “We also need support from industries that use AI semiconductors.” This means that we need to create demand-oriented AI semiconductors that are optimized for manufacturing AI, finance AI, automotive AI, and bio AI, beyond semiconductors optimized for AI processing.

The AI semiconductor industry is a technology-intensive industry where intellectual property (IP) and specialized design capabilities are important, but it is still an early market with no dominant player. It is a time when not only various industries’ efforts to dominate the market, but also government support for inter-industry communication is needed.