Co-Optimization of Models and Hardware for Neural Network Inference in Variable-Parallelism Processor

Hardware-software co-design is crucial in the design of artificial intelligence (AI) inference accelerators. This study supports the development of a variable-parallelism reconfigurable architecture designed to achieve high-efficiency and low-power computation for diverse AI models. We provide software-side design support to achieve high accuracy with limited computational resources. We propose an algorithm that automatically explores hardware configurations by selecting a layer-wise computation flow to achieve high inference accuracy with limited computational resources, which is realized via software-based emulation. The proposed method is employed to reduce weights by 75% through post-training quantization while limiting the Top-1 accuracy degradation to about 3% for image classification on ImageNet-1k with convolutional neural network models such as VGG16. Furthermore, we verify that the proposed exploration method can efficiently identify both the optimal computation flow and minimal hardware requirements required to maintain accuracy.