Friday, February 23, 2024
HomeAI Realization of overwhelmingly low latency|EdgeCortix opens up the Edge AI field!

[Edge AI 2022] Realization of overwhelmingly low latency|EdgeCortix opens up the Edge AI field!

Edge AI

Table of contents

  • Company Profile
    • History of growth
    • About the representative
  • About the SAKURA AI coprocessor
    • How did you achieve low latency?
    • Demand for AI coprocessors is growing, but there are few companies handling them
  • Trends in Edge AI
    • Solutions currently offered by EdgeCortix
  • Future prospects

Company Profile

In July 2019, EdgeCortix was established in Singapore as a fabless semiconductor design company, and in September of the same year, the Tokyo R&D Headquarters was established. Many of our engineers are currently based in our Tokyo office, where we do most of our development.

EdgeCortix was developed with a novel idea of ​​taking a software-first approach while designing a reconfigurable processor dedicated to edge AI from the ground up using a patented technology called “co-design of hardware and software”.

Working on It also primarily targets advanced computer vision applications, using software IP on existing processors such as FPGAs and custom ASIC designs for defense, security, aerospace, smart cities, Industry 4.0, autonomous driving, and robotics. We are actively developing the rapidly growing AI semiconductor field.

History of growth

Our founding philosophy was to take a software-first approach and design from the ground up a processor architecture specifically for artificial intelligence (AI). We call this method “software and hardware cooperative search”.

This “cooperative search” technology was patented as a basic component of our products. In this way, we design the processor (hardware) architecture while giving top priority to software robustness.

This is in contrast to the past decades of processor design, where new hardware chips have been brought to market first and software, including compilers, has been left behind.

EdgeCortix overcomes the fundamental mismatch between current-generation AI/machine learning software, or deep neural networks, and off-the-shelf hardware processors such as CPUs and GPUs to achieve a 10x or greater advantage in performance and energy efficiency. to “bring near-cloud-level performance to all forms of devices at the edge, deliver extraordinary energy efficiency and processing speed, and dramatically reduce operating costs without sacrificing accuracy.” Advancing our core mission.

And now it is possible to deliver near cloud-level performance directly to infrastructure endpoints such as edge devices, vehicles, cameras, robotic systems, and drones. In other words, data curation and decision making can be done at the same time with low power consumption, and we can provide an environmentally friendly and highly efficient solution.

About the representative

Dr. Sakyasinga Dasgupta is the founder and CEO of the EdgeCortix group of companies.

An artificial intelligence (AI) and machine learning researcher, engineer, and entrepreneur with over a decade of experience in developing cutting-edge AI research from idea stage to scalable products in a variety of industries.

After leading teams at global companies such as Microsoft, IBM Research / IBM Japan, and national research institutes such as RIKEN and the Max Planck Institute in Germany, he has worked in semiconductor technology, robots/autonomous vehicles, and fins in Japan and Singapore.

Involved in establishing and leading technology departments using lean startup in the tech field, then brain-inspired computing, robotics, CV (computer vision), edge AI acceleration on semiconductors, wearable devices, IoT , finance and healthcare. After more than 10 years of research and development in various fields such as machine learning, in 2019, he founded EdgeCortix, a fabless semiconductor design company focused on enabling highly efficient edge intelligence.

Prior to founding EdgeCortix, he studied entrepreneurship at the MIT Sloan School of Management and received a PhD in Complex Systems Physics from the Max Planck Institute in Germany. He holds more than 15 patents worldwide and his research has been cited more than 800 times.

About the SAKURA AI coprocessor

 

EdgeCortix SAKURA is a TSMC 12nm FinFET coprocessor (accelerator) that delivers industry-leading computational efficiency and latency for edge artificial intelligence (AI) inference.

SAKURA is equipped with EdgeCortix’s proprietary Dynamic Neural Accelerator® (DNA) IP , a neural processing engine with a built-in run-time reconfigurable data path that connects all computational engines, on a single core, delivering 40 TOPS of computation. achieve performance.

DNA enables simultaneous execution of multiple deep neural network models with ultra-low latency while maintaining excellent TOPS on the SAKURA edge AI coprocessor, a first for EdgeCortix.

This unique property is key to improving the processing speed, energy efficiency and longevity of SoCs (system-on-chips), providing customers with superior TCO ( total cost of ownership ). SAKURA edge AI processors are optimized for streaming and high-resolution data inference.

SAKURA has the following hardware features

  • Up to 40 TOPS (single chip) / 200 TOPS (multichip)
  • PCI-e device TDP @10W~15W
  • Typical model power consumption ~5W
  • 2×64 LPDDR4x – 16GB
  • PCIe Gen 3 – Up to 16GB/s bandwidth
  • Available in two form factors: dual M.2 card, half-height/half-length PCIe card

How did you achieve low latency?

SAKURA employs DNA IP and MERA software stack optimized for low latency . DNA IP leverages a unique reconfigurable datapath to achieve maximum utilization for low batch size workloads , which is key for edge devices .

Many current neural networks have a structure in which the dimensionality of the operators changes from the upper layer to the lower layer. Therefore, flexibility is needed to efficiently parallelize processing from one layer to another, or between different neural networks.

SAKURA uses a patented reconfigurable datapath to coarse-tune the computational processing of the neural network to the IP computational units to take advantage of the most efficient parallelization dimension of each operator and to ensure that the computational units are turned on. It allows optimal utilization of chip memory bandwidth without stalling.

Low-level optimization and scheduling algorithms within the MERA software coordinate this process with the hardware architecture, statically planning SAKURA’s computation order and allocating resources to increase available compute utilization. , the combination of these techniques can achieve significantly lower latency and efficiency compared to GPUs and other AI chips.

In addition, SAKURA, an edge AI coprocessor, specializes in industries that reform and recreate through artificial intelligence (AI) reasoning centered on advanced computer vision (CV).

Enabling computers to see and understand the physical world is one of the most important steps along with next-generation innovations in AI.

Our main use cases using SAKURA are in the following business areas.

Intelligent transportation system/self-driving car

Intelligent Subsystems Powering Autonomous Vehicles Like LiDAR, Not Just Automobiles

Commercial Drones & Robots (UAV, UGV, UUV)

Defense/Security

Military drones, airborne or satellite-based cameras, signal processing capabilities, smart AR/VR capabilities

Smart Manufacturing / Robotics

Specialized in manufacturing inspection and quality assurance

EdgeCortix’s solution can improve the accuracy of product quality inspection while increasing manufacturing speed (due to low-latency solution)

Warehouse robots, autonomous forklifts

Smart cities (including aspects such as security and surveillance, traffic management, and waste management)

Advanced traffic monitoring and traffic management, parking management, security systems/surveillance, waste management/surveillance

Smart retail (including brick-and-mortar intelligence with cameras)

Visitor analysis based on smart cameras, smart inventory management, automation of kiosk terminals

Demand for AI coprocessors is growing, but there are few companies handling them

Motivated by the huge market size of edge AI, there are other companies that are targeting dedicated solutions for edge AI acceleration in terms of semiconductor products for AI, but we are the most dominant in the edge AI ​​market. We believe that we should pay attention to the current state of GPUs, which are processors.

The reason is that many companies are only throwing more GPUs at the table to enable compute power when deploying machine learning inference, and are quiet about the power, throughput and cost inefficiencies of their solutions.

The current situation is that it is enduring We believe that the reason for this situation is that the edge AI ecosystem is still in the development stage, and that the approach to solving the problem is not well known.

EdgeCortix’s goal is to provide customers with a total solution for Edge AI and a “go-to-market” strategic message, and our products have outstanding performance, power efficiency, software robustness, and various It is to effectively spread awareness that it will be a clue to problem solving and to realize it.

Features of EdgeCortix include the following two points, which are methods of providing value to end users.

First one:

As mentioned earlier, EdgeCortix was founded on the idea of ​​applying a software-first approach with a unique “software and hardware co-design”.

This approach with the open source MERA compiler and software stack allows customers to run their machine learning applications on the heterogeneous hardware environment they already have.

Second:

Our unique neural processing engine (Dynamic Neural Accelerator – DNA) uses a patented “runtime reconfigurable” technology, and by combining with our MERA compiler, our unique SAKURA accelerator SoC, third Any hardware platform, such as a party FPGA or other custom-designed ASIC, can achieve both high throughput and low latency at lower power (by maximizing the efficient use of the processor’s computational resources).

Will be This enables EdgeCortix to leverage software and hardware to provide an objective counterpoint to GPUs (industry-leading power efficiency – inferences/sec/watt) in terms of performance, energy efficiency and TCO. can do.

Trends in Edge AI

When it comes to market opportunities, many leading analysts say the edge will “bring the next wave of digital transformation.”

Analysts such as IDC predict that the “edge” will grow rapidly even from a global perspective, with the global edge computing market TAM reaching $250 billion over the next two years.

(“New IDC Spending Guide Forecasts Double-Digit Growth for Investments in Edge Computing,” January 2022) According to an analysis by Valuates Reports, the edge AI hardware market will grow at a CAGR of 18.8% by 2030. growth rate) to reach more than $38 billion.

In addition, according to a report by Allied Market Research, the combined market size of edge AI hardware and software in Japan is expected to grow from $8 billion in 2020 to $40.8 billion in 2030.

According to Omdia Research (May 2020), computer vision market revenue is expected to reach $33.5 billion by 2025. Deep learning is the technology of choice, driving demand for new chipsets and software for computer vision applications.

Regardless of how you assess the market opportunity analysis, the market is booming, impacting multiple businesses across consumer electronics, manufacturing, defense & security, robotics, and automotive, and growing at an extraordinary rate. . In both cases, they are embedding AI-powered solutions into their respective end devices and edge applications.

Solutions currently offered by EdgeCortix

EdgeCortix offers solutions through three main product lines.

Dynamic Neural Accelerator®️(DNA)

DNA is EdgeCortix’s proprietary neural processing engine with a patented “runtime reconfigurable datapath” that connects all compute engines within a single core.

DNA is a heterogeneous solution, our custom ASICs and third-party based FPGA solutions (assuming AMD-Xilinx and Intel) run multiple deep neural network models simultaneously while maintaining excellent TOPS utilization. It allows you to run with ultra-low latency.

This run-time reconfigurable data path is what drives TOPS and is the key contributor to SoC speed, energy efficiency, improved longevity, and improved total cost of ownership (TCO).

DNA IP is specifically optimized for inference with streaming (Batch-1) and high-resolution data (such as real-time video and signal processing).

MERA™️

MERA is a compiler and software toolkit that enables graph compilation and inference of deep neural networks using our proprietary neural processing engine (aka DNA).

With built-in support for open source frameworks such as Apache TVM and MLIR, MERA provides the tools, APIs , and tools needed to deploy pre-trained deep neural networks after a simple calibration and quantization step. Provides code generator and runtime.

MERA supports quantizing models directly in deep learning frameworks such as Pytorch and TensorflowLite.

The point of MERA is to enable software engineers and data scientists to leverage the value of neural network models without having to deal with the underlying semiconductors.

The MERA frontend has been open sourced by EdgeCortix and is available for use.

SAKURA™️

SAKURA is a TSMC 12nm FinFET coprocessor (accelerator) ASIC that delivers industry-leading computational efficiency and latency for edge AI ( artificial intelligence ) inference.

SAKURA’s first model features a 40TOPS single-core Dynamic Neural Accelerator® (DNA) IP. EdgeCortix’s proprietary neural processing engine with built-in DNA, a run-time reconfigurable datapath connecting all compute engines.

SAKURA offers very low power consumption, especially when compared to GPUs, with a TDP of less than 15W, and less than 5W for main inference runs.

DNA (discussed above) enables the new SAKURA AI coprocessor to run multiple deep neural network models simultaneously, with ultra-low latency while maintaining outstanding TOPS.

This unique property greatly contributes to system-on-chip processing speed, energy efficiency and longevity, resulting in excellent TCO (Total Cost of Ownership).

Future prospects

We see our future potential due to a combination of two major factors:

  1. Abundant Growth Opportunities for Edge AI (as mentioned above)
  2. EdgeCortix’s truly differentiating business benefits:
  • Ability to provide power-saving, low-latency, and low-cost ML inference solutions that operate within customers’ existing heterogeneous environments
  • Robust and flexible software stack that has been open-sourced and easily evaluated and developed by the global community of machine learning and AI engineers
  • The EdgeCortix solution is not just a plan, it is not a concept or a set of roadmap items, nor is it a “beta”. The solution is real, has been benchmarked against competitors, put into production, and can be used to solve customer pain points in today’s broad edge AI market.

We are now focused on tapping into that growing market, reaching our target audience and offering our solutions in form factors that meet the needs of our customers and prospects.

The real question for edge AI inference is performance in a heterogeneous environment:

Software engineers create models using  various ML frameworks (PyTorch, TensorFlow , ONNX, etc.). The hardware solution is complex and can have different types of host processors (think ARM, Intel, AMD x86, RISC-V, etc.).

Combined with a host, today’s AI acceleration has the potential to be achieved on a variety of hard-to-program hardware platforms, from FPGAs (think AMD-Xilinx and Intel) and GPUs to new domain-specific ASICs.

EdgeCortix has worked with end users in several major business areas to date. Our software and processor IP (MERA + DNA) have been deployed on 3rd party hardware like FPGAs (AMD-Xilinx/Intel) and microprocessors (ASICs), and in Q3 2022, our recently announced the industry’s top-class high-efficiency edge AI coprocessor, “SAKURA,” according to the customer’s environment.

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Recent Posts

Most Popular

Recent Comments