Deploying artificial intelligence (AI) at the edge (EdgeAI) offers significant advantages in latency and privacy, but modern AI's computational demands conflict with the strict power and memory constraints of edge devices. These ultra-low-power (ULP) systems are increasingly heterogeneous, combining general-purpose central processing units (CPUs) with specialized processing elements (PEs) to meet performance demands. However, traditional real-time operating systems (RTOS) are ill-equipped to manage this complexity.
This dissertation confronts this challenge with a holistic three-pillar strategy that co-optimizes the entire EdgeAI application lifecycle: (1) system-level scheduling, (2) quiescent non-processing state power management, and (3) active-task orchestration.
First, we challenge static scheduling conventions by introducing a lightweight dynamic scheduler for RTOS. Counter-intuitively, this approach not only reduces missed deadlines by up to 60% but also lowers scheduler timing and memory overhead by up to 34.7 % and 74.7 %, respectively, boosting system throughput on event-driven workloads.
Second, to optimize energy during idle periods, we propose a novel analytical model for selecting optimal operational frequencies. By holistically integrating application requirements with platform-specific overheads like frequency switching time, a factor previously overlooked, this method reduces total application energy by up to 58% compared to state-of-the-art techniques.
Third, for managing active deep neural network (DNN) tasks, we present MEDEA (Manager for Energy-efficient DNNs on hEterogeneous ULP Architectures). MEDEA is the first manager for ULP systems to integrate kernel-level dynamic voltage and frequency scaling (DVFS), memory-aware tiling, and fine-grained scheduling, formulating the problem as a multiple-choice knapsack problem (MCKP). On a custom RISC-V platform, MEDEA achieves energy reductions of up to 38% for a Transformer model. Our analysis also reveals a critical efficiency crossover, where the optimal PE changes with operating voltage, underscoring the necessity of a unified co-optimization approach.
Collectively, these validated solutions provide a comprehensive framework to enable the next generation of efficient and intelligent edge systems, offering a clear pathway to deploy sophisticated AI on severely constrained hardware.
