Applied Science, Faculty of
Electrical and Computer Engineering, Department of
DSpace
UBCV
Maheshe, Joseph
Master of Applied Science - MASc
University of British Columbia
The end of Moore's Law and Dennard Scaling has shifted computing systems toward specialized accelerators to increase performance and lower power-per-operation. Accelerators are special-purpose hardware optimized to perform specific functions within general-purpose computing systems, and modern SoCs combine many general-purpose processors with a growing number of accelerators. Accelerators achieve higher performance and power efficiency by prioritizing specialization and heterogeneity over generality and homogeneity. When specialization reduces the cost of processing low enough, accelerators become memory-limited. High-performance accelerators typically access memory directly or through caches to circumvent throughput bottlenecks from accessing data only through the processor's register file. However, direct access to shared memory introduces coherence and consistency issues, both challenging aspects of modern processor design. As accelerators proliferate in shared-memory heterogeneous SoCs, ensuring coherence and consistency in each accelerator adds significant design complexity. This thesis proposes a novel shared load-store unit (LSU) to address these challenges collectively for all accelerators. The shared LSU specifically targets instruction-based accelerators integrated into the processor's instruction set architecture. Sharing an LSU consolidates LSU logic, buffering, multiple memory access ports, and coherence and consistency logic into a single unit. It can also enable automatic dependence checking, leading to improved performance. The shared LSU aims to alleviate the burden on designers who typically create accelerator-specific LSUs. Its platform-independent design is written in standard SystemVerilog HDL and uses open-source communication protocols to facilitate hardware reuse. It also offers programmers a simplified programming model featuring custom, accelerator-based, fine-grain fences. A case study demonstrates a shared LSU that replaces individual vector LSUs in high-performance soft RISC-V vector cores integrated into a single scalar core. Each vector core enables simplified multithreading of single-threaded programs using contexts. Many of these vector cores form a novel multicore and multithreaded vector architecture that further enhances performance. The shared LSU seamlessly integrates with the vector cores in an SoC implemented on the AMD ZCU104 evaluation kit. In this system, a single-threaded program using two vector cores and two contexts achieves a 2.66x speedup on rgba2luma, effectively hiding memory latency and pipeline stalls.
Shared Load-Store Unit forInstruction-based AcceleratorsCase Study with Soft Multicore and MultithreadedRISC-V Vector ProcessorsbyJoseph MahesheB.Eng. Electrical Engineering, Polytechnique Montréal, 2022A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF APPLIED SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Electrical and Computer Engineering)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)December 2024© Joseph Maheshe 2024The following individuals certify that they have read, and recommend tothe Faculty of Graduate and Postdoctoral Studies for acceptance, the thesisentitled:Shared Load-Store Unit for Instruction-based Accelerators: CaseStudy with Soft Multicore and Multithreaded RISC-V VectorProcessorssubmitted by Joseph Maheshe in partial fulfillment of the requirements forthe degree of Master of Applied Science in Electrical and ComputerEngineering.Examining Committee:Guy Lemieux, Professor, Electrical and Computer Engineering, UBCSupervisorSteve Wilton, Professor, Electrical and Computer Engineering, UBCSupervisory Committee MemberPrashant Nair, Professor, Electrical and Computer Engineering, UBCSupervisory Committee MemberiiAbstractThe end of Moore’s Law and Dennard Scaling has shifted computing systemstoward specialized accelerators to increase performance and lower power-per-operation. Accelerators are special-purpose hardware optimized to performspecific functions within general-purpose computing systems, and modernSoCs combine many general-purpose processors with a growing number ofaccelerators. Accelerators achieve higher performance and power efficiency byprioritizing specialization and heterogeneity over generality and homogeneity.When specialization reduces the cost of processing low enough, accelera-tors become memory-limited. High-performance accelerators typically accessmemory directly or through caches to circumvent throughput bottlenecksfrom accessing data only through the processor’s register file. However, directaccess to shared memory introduces coherence and consistency issues, bothchallenging aspects of modern processor design. As accelerators proliferatein shared-memory heterogeneous SoCs, ensuring coherence and consistencyin each accelerator adds significant design complexity. This thesis proposes anovel shared load-store unit (LSU) to address these challenges collectively forall accelerators. The shared LSU specifically targets instruction-based acceler-ators integrated into the processor’s instruction set architecture. Sharing anLSU consolidates LSU logic, buffering, multiple memory access ports, and co-herence and consistency logic into a single unit. It can also enable automaticdependence checking, leading to improved performance. The shared LSU aimsto alleviate the burden on designers who typically create accelerator-specificLSUs. Its platform-independent design is written in standard SystemVerilogHDL and uses open-source communication protocols to facilitate hardwarereuse. It also offers programmers a simplified programming model featuringcustom, accelerator-based, fine-grain fences. A case study demonstrates ashared LSU that replaces individual vector LSUs in high-performance softRISC-V vector cores integrated into a single scalar core. Each vector coreenables simplified multithreading of single-threaded programs using contexts.Many of these vector cores form a novel multicore and multithreaded vectorarchitecture that further enhances performance. The shared LSU seamlesslyiiiAbstractintegrates with the vector cores in an SoC implemented on the AMD ZCU104evaluation kit. In this system, a single-threaded program using two vectorcores and two contexts achieves a 2.66× speedup on rgba2luma, effectivelyhiding memory latency and pipeline stalls.ivLay SummaryGeneral-purpose computing systems, or CPUs, are everywhere, ranging fromsimple embedded processors in calculators to powerful processors in datacenters. For decades, computing performance and efficiency improved throughprocessor scaling. However, as scaling has largely ended, computing systemshave shifted toward specialized accelerators. One such example is the graphicsprocessing unit (GPU), widely used in graphics and deep learning, whichoffers orders of magnitude improvements over CPUs. Modern applicationslike deep learning demand substantial data throughput, and acceleratorstypically access memory directly to meet these needs. Yet, direct memoryaccess introduces unique design challenges that each accelerator must address.This thesis proposes a shared unit to address these challenges collectively forall accelerators, aiming to provide high performance, easy programmability,and reuse. A case study with multiple GPU-like accelerators shows that thisshared unit integrates seamlessly and provides high performance.vPrefaceThis thesis represents the original and independent work of Joseph Mahesheunder the guidance of Guy Lemieux. Portions of Chapter 4 are included in amanuscript submitted for publication [1], and portions of Chapter 3 appear ina manuscript in preparation for submission. The author has heavily modifiedthe original Saturn-V vector unit, designed by Caroline White [2], to integrateinto this thesis. Key modifications include replacing Saturn-V’s dedicatedload-store unit with a more capable shared load-store unit and updating itscoprocessor interface to the CX logic interface. Further changes to Saturn-V’smicroarchitecture improve performance. These include full AXI-MM supportwith burst aligned and unaligned transfers, non-blocking vector loads, earlyretirement of vector stores, multi-ported RAM design of the vector registerfile enabling concurrent reads of computes and stores and concurrent writesof computes and loads, and parallel execution of configuration, compute, load,and store instructions.viTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiList of Listings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiList of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . xivAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Accelerator Era . . . . . . . . . . . . . . . . . . . . . 11.1.2 Coherence and Consistency Issues . . . . . . . . . . . 31.2 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1 Accelerators and Memory Access . . . . . . . . . . . . . . . 72.1.1 Memory Consistency Models . . . . . . . . . . . . . . 72.1.2 Accelerator Cache-coherence Models . . . . . . . . . 8viiTable of Contents2.2 RISC-V Instruction Set Architecture . . . . . . . . . . . . . 92.3 RISC-V Memory Model . . . . . . . . . . . . . . . . . . . . 92.4 RISC-V Custom Extension . . . . . . . . . . . . . . . . . . . 112.4.1 Draft RISC-V Composable Custom Extensions Specifi-cation . . . . . . . . . . . . . . . . . . . . . . . . . . 112.5 RISC-V Vector Extension . . . . . . . . . . . . . . . . . . . 152.5.1 Evolution of Vector Processors to RISC-V Vectors . . 152.5.2 RISC-V Vector Processors . . . . . . . . . . . . . . . 172.5.3 Vector Coprocessor Interfaces . . . . . . . . . . . . . . 192.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 202.6.1 Common Memory Units . . . . . . . . . . . . . . . . . 202.6.2 Multithreaded Vector Architectures . . . . . . . . . . 213 Shared Load-Store Unit . . . . . . . . . . . . . . . . . . . . . 223.1 The Need for a Shared Unit . . . . . . . . . . . . . . . . . . 223.2 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.3 Coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.4 Shared LSU Architecture . . . . . . . . . . . . . . . . . . . . 273.4.1 Interface with External Memory . . . . . . . . . . . . 273.4.2 Interface with Accelerators . . . . . . . . . . . . . . . 283.4.3 Load-Store Queue . . . . . . . . . . . . . . . . . . . . 303.4.4 Invalidation Unit . . . . . . . . . . . . . . . . . . . . 324 Multicore × Multithreaded Vector Processors . . . . . . . 334.1 Multithreaded×Multicore Vector Architecture as High-PerformanceSolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2 Modifications to Saturn-V . . . . . . . . . . . . . . . . . . . 354.3 MXM-RVV Architecture . . . . . . . . . . . . . . . . . . . . 384.3.1 Decoding and Dispatching of Vector Instructions . . 394.3.2 Latency Hiding . . . . . . . . . . . . . . . . . . . . . 394.4 Integration with the Shared LSU . . . . . . . . . . . . . . . 414.5 Programming . . . . . . . . . . . . . . . . . . . . . . . . . . 435 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.1 Architecture Parameters . . . . . . . . . . . . . . . . . . . . 475.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 475.2.1 Comparison to Prior Work . . . . . . . . . . . . . . . 485.2.2 Detailed Per-Benchmark Performance . . . . . . . . . 515.2.3 Trade-Offs Between Vector Length and Multiple Con-texts . . . . . . . . . . . . . . . . . . . . . . . . . . . 54viiiTable of Contents5.3 Area Breakdown and Clock Frequency . . . . . . . . . . . . 555.4 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.5 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . 586 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 606.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 60Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62ixList of Tables3.1 Resolution of data races between a scalar processor and anaccelerator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.2 Resolution of data races between accelerators. . . . . . . . . . 265.1 Benchmark vector instruction count per output row. . . . . . 495.2 Benchmark performance on MXM-RVV+LSU SoC with 1 vec-tor unit × 1 state context and Vicuna. . . . . . . . . . . . . . 515.3 Benchmark performance on MXM-RVV+LSU SoC. . . . . . . 535.4 Benchmark performance on MXM-RVV+LSU SoC with 1 vec-tor unit × n state contexts × VLEN. . . . . . . . . . . . . . . 555.5 Area breakdown for 2× n MXM-RVV+LSU SoC (64-bit exe-cution and memory widths, VLEN=1024). . . . . . . . . . . . 565.6 Maximum frequency for m× n MXM-RVV+LSU SoC (64-bitexecution and memory widths, VLEN=1024). . . . . . . . . . 565.7 Scalability of the shared LSU with 64-entry LSQ and increasingvector units. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58xList of Figures1.1 Heterogeneous SoC with scalar processor (CPU), image process-ing unit (IPU), vector processing unit (VPU) and cryptographyunit (CRYPTO). . . . . . . . . . . . . . . . . . . . . . . . . . 22.1 FENCE instruction encoding. . . . . . . . . . . . . . . . . . . 112.2 CX custom instruction formats. . . . . . . . . . . . . . . . . . 133.1 Differing SoCs showing CPU and RVV cores accessing memory. 233.2 CX R-type instruction encoding. . . . . . . . . . . . . . . . . 243.3 Block diagram of the shared LSU from the perspective of eachaccelerator. ACC, AXI, and INV signal groups connect to theaccelerator, memory, and scalar cache, respectively. . . . . . . 273.4 Space-time diagrams with two accelerators, the shared LSU,and external memory that illustrates communication protocolsfor fence, read, and write transactions. Each interface is repre-sented with a distinct color: red-orange for lookup-req,green for lookup-resp, blue for allocate-req, pink forallocate-resp, gold for read-req, violet for read-data,cyan for write-req, and orange for write-data. . . . . 313.5 Conflict detection in the LSQ. Each allocated entry includes avalid bit (v), memory access type (r/w, 0 for read and 1 forwrite), and address range (ba-ea) from base address (ba) toend address (ea). . . . . . . . . . . . . . . . . . . . . . . . . . 324.1 SoC with one CVA5 CPU and two M-RVV processors sharingLSU for memory access. . . . . . . . . . . . . . . . . . . . . . 344.2 Internal Saturn-V micro-architecture. Taken from White [2]. . 374.3 Information flow for executing a vector instruction in MXM-RVV. 384.4 Block diagram of a single multithreaded M-RVV core. . . . . 404.5 State diagram for lookup and allocate interfaces in MXM-RVV. 42xiList of Figures5.1 Scalability of the shared LSU with 64-entry LSQ and increasingvector units. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57xiiList of Listings1 Example instruction stream with potential coherence and con-sistency issues in a shared-memory system. . . . . . . . . . . 42 Example instruction stream with resolved coherence and con-sistency issues in a shared-memory system. . . . . . . . . . . 253 Example instruction stream with dependent vector instructions. 344 Vectorized RGBA to LUMA conversion using CIA’s ABI andAPI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Vectorized RGBA to LUMA conversion using double buffering. 46xiiiList of AbbreviationsABI Application Binary InterfaceAGU Address Generation UnitAI Artificial IntelligenceALU Arithmetic Logic UnitAPI Application Programming InterfaceASIC Application-Specific Integrated CircuitAXI Advanced eXtensible InterfaceAXI-MM AXI Memory Mapped InterfaceAXI-S AXI Stream InterfaceBRAM Block RAMCAM Content-Addressable MemoryCC Cycle CountCFU Custom Function UnitCFU-LI CFU Logic InterfaceCIA Composable Instructions for AcceleratorsCMO Cache Management OperationxivList of AbbreviationsCPU Central Processing UnitCSR Control/Status RegisterCV-X-IF Core-V eXtension InterfaceCX Composable eXtension. The interface contract of a com-posable custom extension consisting of a set of customfunction (CF) instructions and CX custom CSRs (CXCSRs) and their behavior.CXU CX UnitCXU-LI CXU Logic InterfaceDLP Data-Level ParallelismDMA Direct Memory AccessDNN Deep Neural NetworkDRAM Dynamic RAMDSP Digital Signal ProcessingFF Flip-FlopFPGA Field-Programmable Gate ArrayGPU Graphics Processing UnitHPC High-Performance ComputingIC Instruction CountILP Instruction-Level ParallelismIMT Interleaved MultithreadingxvList of AbbreviationsI/O Input/OutputIOMMU Input–Output Memory Management UnitIPC Instructions Per CycleIPU Image Processing UnitISA Instruction Set ArchitectureL1 First-Level CacheL2 Second-Level CacheLLC Last-Level CacheLMUL Vector Register Group MultiplierLSQ Load-Store QueueLSU Load-Store UnitLUT LookUp TableMIG Memory Interface GeneratorML Machine LearningMSHR Miss Status/Handler RegisterMU Memory UnitM-RVV Multithreaded RISC-V VectorsMXM-RVV Multicore and Multithreaded RISC-V VectorsOBI Open Bus InterfaceOVI Open Vector InterfacexviList of AbbreviationsRAM Random Access MemoryRF Register FileRoCC Rocket Custom Coprocessor InterfaceRVV RISC-V Vector ExtensionRVWMO RISC-V Weak Memory Ordering consistency modelRVTSO RISC-V Total Store Ordering consistency modelSEW Vector Selected Element WidthSIMD Single Instruction, Multiple DataSRAM Static RAMSVP Soft Vector ProcessorSoC System on a ChipTLP Thread-Level ParallelismURAM UltraRAM, a BRAM with larger capacityVCIX Vector Coprocessor Interface eXtensionVCSR Vector CSRVIQ Vector Instruction QueueVLEN Vector Register Length (bits)VLMAX Maximum Vector Length (elements)VPU Vector Processing UnitVRF Vector Register FilexviiAcknowledgementsI would like to extend my deepest gratitude to my supervisor, Guy Lemieux,for his guidance throughout my master’s. Not only did he introduce me tothe problem but also taught me the importance of approaching research withplanning, critical analysis, and integrity. I am also grateful to the membersof my research group for their camaraderie and valuable input — thank you,Brandon!xviiiDedicationTo my parents.xixChapter 1IntroductionTo meet future performance and power efficiency, processors now incorporatespecialized instructions executed by hardware accelerators. These instruction-based accelerators require direct memory access to achieve their performancegoals. The following introductory chapter begins by highlighting the need forefficient data transfers in modern accelerator-based computing systems andthe motivation behind this thesis (section 1.1). Then, section 1.2 presentschallenges these accelerators face in shared-memory systems. The thesisstatement, research approach, and contributions are presented in section 1.3.Finally, section 1.4 outlines the organization of the remainder of this thesis.1.1 MotivationThis section traces the evolution of computing systems toward acceleratorsand examines coherence and consistency issues that may arise when theseaccelerators access memory. Solutions to these issues motivate this thesis.1.1.1 Accelerator EraWith the end of Moore’s Law and Dennard Scaling, architects must find moreefficient ways to exploit parallelism [3]. This has shifted computing systemsaway from traditional approaches like instruction-level parallelism (ILP),which extracts parallelism from a single instruction stream, and thread-levelparallelism (TLP), executing independent threads or contexts concurrentlyacross multiple cores. Power dissipation concerns have also ushered in thedark silicon era [4], where idle cores are turned off to prevent overheating.Recently, there has been a significant shift toward accelerators. Acceleratorsare special-purpose hardware optimized to perform specific functions withingeneral-purpose computing systems [5]. By prioritizing specialization andheterogeneity over generality and homogeneity, accelerators achieve higherperformance and power efficiency [6].11.1. MotivationThe adoption of accelerators has led to various classifications. Cascaval etal. [5] identify three types of accelerators based on their architecture: fixed-function, programmable, and reconfigurable. Fixed-function architectures,typically implemented using ASIC technology, are tailored for specific tasks togain substantial performance and power benefits. Programmable architectures,like GPUs, allow more flexibility while providing customization throughsoftware. Reconfigurable architectures, such as overlays, offer the highestflexibility, with the architecture and the software being definable.Accelerators can also be classified based on their invocation model [5].Instruction-level invocation refers to instruction-based accelerators integratedinto the processor’s instruction set architecture (ISA). Command packetinvocation applies to accelerators connected as I/O devices. Task-level is acoarse-grained invocation where programmable accelerators receive work listsfrom software. Workload invocation offloads entire applications to workload-specialized appliances, typically standalone systems. This thesis considersinstruction-based accelerators.In today’s accelerator-based computing systems, data-intensive applications,like machine learning and computer vision, rely on efficient data transfers.These applications achieve peak performance when they saturate availablememory bandwidth. Accessing data only through the processor’s registerfile can result in bottlenecks. Hence, high-performance accelerators caninclude internal states to circumvent register file bottlenecks. Additionally,they can directly access memory to circumvent data throughput bottlenecks;direct access may go through the caches or bypass the cache hierarchy tomain memory. Figure 1.1 illustrates a heterogeneous system with one scalarprocessor and three accelerators (image processing, vector, and cryptographyunits), all accessing shared memory through a memory interconnect.Figure 1.1: Heterogeneous SoC with scalar processor (CPU), image processingunit (IPU), vector processing unit (VPU) and cryptography unit (CRYPTO).21.1. Motivation1.1.2 Coherence and Consistency IssuesWhen accelerators directly access shared memory by address, cache coherenceand memory consistency issues may arise. Both of these are challengingaspects of modern processor design. A memory system is said to be coherentif any read of a data item returns the most recently written value of that dataitem [7]. However, caching shared data can violate coherence by introducinga global and a local state. The global state is defined primarily by the mainmemory, while the local state is the individual caches, which can be private[7]. It then becomes possible for processors and accelerators to see differentvalues for the same memory location.While coherence defines the behavior of reads and writes to the same location,consistency specifies the ordering of reads and writes to differing locations [7].Memory operations are assumed to execute one at a time in the order the pro-gram specifies [8]. However, to improve performance, an implementation doesnot need to strictly maintain that order among all memory operations. Thememory consistency problem then arises from memory reordering considera-tions. Processor architectures have adopted one or more memory consistencymodels to indicate memory reorderings that implementations can perform[8, 9].Listing 1 shows a dynamic instruction stream with potential coherence andconsistency issues. The stream consists of memory load and store instructions,where scalar memory operations execute on a scalar processor, and vectorones are offloaded to a vector accelerator. The stream begins with a scalarstore to address a1, followed by a vector store to a2. Next, a scalar loadreads a2, and a vector load reads a1. Coherence problems may occur if thevector accelerator reads a1 before the scalar processor’s cache writes it backto memory. Also, if the vector store to a2 does not update the scalar cache,the scalar load may read old or stale data.Consistency issues can arise if the scalar processor and the vector acceleratoruse store buffers and hold back writes to memory, thereby allowing disjointloads to proceed ahead of store operations. This may lead to a data race ifloads read from memory before or after store buffers complete writing theirvalues to memory. For example, if the scalar load reading a2 picks up thevalue written by the vector store to a2, one might naturally assume thatthe vector load from a1 must also pick up the scalar store to a1. However,with a scalar store buffer, the scalar store to a1 may be held back and notbe observable by the vector load from a1. Consistency models define the31.2. Problem////////////////////////////////////////////////////////////////////// scalar processor // vector accelerator //////////////////////////////////////////////////////////////////////...1 sw x1,0(a1) // scalar store2 vse32.v v1,(a2) // vector store3 lw x2,0(a2) // scalar load4 vle32.v v2,(a1) // vector load...Listing 1: Example instruction stream with potential coherence and consis-tency issues in a shared-memory system.types of reorderings that must be enforced by hardware so they can be reliedupon by software. Outside of these enforced orderings, a data race mayoccur. Software must resolve these data races with explicit synchronization(fence) operations [10]. The consistency problem raises the question of whena processor should observe updates from another processor.1.2 ProblemAs instruction-based accelerators become more prevalent in shared-memoryheterogeneous systems, ensuring coherence and consistency in each acceleratoradds significant design challenges as accelerators’ direct memory access (DMA)and load-store units must operate in diverse environments, including variouson-chip protocols, programming models, and host processors [11]. Theincreasing complexity of these units in modern SoCs impedes their reuse.Accelerators face other challenges when accessing memory. They may needto support data alignment for efficient memory access and handle widthmismatches in memory interfaces. Besides, they may need to add their cachesor manage complex requirements like address translation to maintain a unifiedvirtual memory space between host processors and accelerators. While thisthesis does not solve these challenges, they remain to be addressed.1.3 ApproachA shared load-store unit (LSU) can address these challenges, particularlycoherence and consistency addressed in this thesis, by handling them oncefor all accelerators. The primary requirement for such a shared unit is highperformance, determining whether the benefits justify the implementation41.3. Approachcost and programming complexity. However, performance must not come atthe expense of programmability. Thus, ease of programming is the secondrequirement. The shared unit should offer an intuitive programming modeland require minimal changes to existing software. The third requirement ishardware reuse through open-source protocols and parametrization, as theshared unit will integrate into heterogeneous SoCs.This thesis proposes the design of a shared LSU for high-performanceinstruction-based accelerators in shared-memory heterogeneous systems. Thisdesign aims to alleviate the burden on designers who typically create LSUstailored for accelerators. Additionally, it offers programmers a simplifiedprogramming model with range fences to ensure consistency. The solution isnon-obvious, as it carefully identifies which aspects of accelerator LSUs tofactor out. The key insight is that multiple accelerators can share much of thelogic for burst accesses (e.g., tensors, strides, indexed accesses). Also, central-izing memory access enables fine-grain dependence checking, preventing dataraces, and allows more reorderings, which leads to improved performance.To simplify its design in this thesis, the shared LSU is limited to unit-stridedata transfers and non-coherent accelerators that bypass a write-through cachehierarchy to access memory. The write-through policy simplifies coherence byensuring caches and external memory are up-to-date, eliminating the need toflush the accelerator data region before execution, as required in a write-backpolicy. Consequently, the shared LSU only needs to invalidate address rangesmodified by the accelerators to enforce coherence. It ensures consistencythrough an internal load-store queue that performs automatic dependencechecking and provides range fences. The accelerators accessing the sharedLSU are each implemented as a CX unit (CXU), a reusable hardware moduleimplementing a Composable Extension (CX) — an ISA contract that specifiesthe behavior of a set of custom instructions [12].To meet the vision for hardware reuse, the shared LSU is written in standardSystemVerilog HDL and uses open-source protocols for communication: theCX logic interface between the host processor and the accelerator, the AXIStream interface between the accelerator and the shared LSU, and the AXIMemory Mapped interface between the shared LSU and memory.This thesis uses soft vector processors (SVPs) as accelerators for the case study.SVPs are FPGA overlays that simplify programming for developers with littleor no hardware experience. They exploit data-level parallelism (DLP), wherea single instruction operates on multiple data elements, making them highlysuitable for FPGA accelerations. SVPs facilitate the rapid development of51.4. Organizationhigh-performance solutions [13–15]. Furthermore, the abundance of blockRAMs in modern FPGAs [16] enhances their capabilities [17]. This thesisuses these block RAMs to also enable simplified multithreading within eachSVP. Combining data-level and thread-level parallelism further improvesperformance by hiding memory latency and reducing pipeline stalls.This thesis removes load-store units from each SVP to use the shared LSUunit. Each SVP is a RISC-V vector core, with many forming a Multicoreand Multithreaded RISC-V Vector architecture referred to as MXM-RVV(read “Maximum RVV”) throughout the thesis.Overall, the major contributions of this thesis are:1. A shared LSU design for high-performance instruction-based accel-erators in shared-memory heterogeneous systems with write-throughprimary caches.2. An AXI Stream-based interface for instruction-based accelerators touse the shared LSU.3. A simplified programming model with range fences for accelerators andhost processors to ensure consistency.4. A multicore and multithreaded RISC-V vector architecture as high-performance soft vector processors demonstrating the shared LSU.1.4 OrganizationChapter 2 provides background on memory access in accelerators, RISC-Vmemory model, RISC-V custom and vector extensions, and RISC-V vectorcoprocessor interfaces. It also covers related work on common memory unitsand multithreaded vector architectures. Chapter 3 outlines the features anddesign of the shared LSU unit. Chapter 4 details the MXM-RVV architecture,its integration with the shared LSU and programming model. Chapter 5presents benchmark and implementation results of the MXM-RVV+LSU SoCand its verification. Finally, Chapter 6 concludes and proposes future work.6Chapter 2BackgroundThis thesis uses soft RISC-V multithreaded vector processors as instruction-based accelerators for a case study of the shared load-store unit (LSU). Thischapter provides a background on accelerator memory access in section 2.1,the RISC-V instruction set architecture (ISA) in section 2.2, the RISC-Vmemory model in section 2.3, RISC-V Custom Composable Extensions insection 2.4, and the RISC-V vector extension in section 2.5. Finally, itdiscusses related work on common memory units and multithreaded vectorarchitectures in section 2.6.2.1 Accelerators and Memory AccessThis section reviews memory consistency models in section 2.1.1 and accel-erator cache-coherence models in section 2.1.2 to address consistency andcoherence issues, respectively, that may arise when accelerators access sharedmemory by address.2.1.1 Memory Consistency ModelsMemory consistency models span from weak to strong models. Strongmodels provide simpler programming models but at the cost of imposingmore restrictions on hardware optimizations, which limits performance [8].The most straightforward model of such a model is sequential consistency.Sequential consistency requires the result of any execution to be the sameas if the operations of all the processors were executed in some sequentialorder, and the operations of each processor appear in this sequence in theorder specified by its program [8].To improve performance, weak memory models allow hardware implemen-tation flexibility to hide memory latency at the expense of a more complexprogramming model [8]. The weak programming model distinguishes betweendata and synchronization operations, or fences [8]. Weak memory models72.1. Accelerators and Memory Accessonly ensure consistency at synchronization points: implementations can freelyreorder data operations between pairs of synchronization operations. A signif-icant drawback of weak models is that they expose programmers to low-levelmemory reorderings.As processors offload computations to accelerators, preserving the correctordering of memory accesses is critical. Instruction-based accelerators, inparticular, retain the notion of fetching instructions, which conveys thesequential order of requests at the memory interface needed to address consis-tency [18]. In contrast, spatial accelerators based on dataflow architectureslose that order, further complicating the consistency problem [18]. Hence,this thesis focuses on instruction-based accelerators, such as the RISC-Vvector processor, whose vector instruction set includes memory load andstore instructions. The RISC-V base ISA also provides simple fences forsynchronization, described later in section 2.3.2.1.2 Accelerator Cache-coherence ModelsGiri et al. [19] identify three accelerator cache-coherence models: fully coher-ent, LLC-coherent, and non-coherent. In fully coherent models, acceleratorshave private caches and implement cache coherence protocols similar to thosein processor caches. With a coherent interconnect, processors and acceler-ators can exchange cache lines directly. Since the cache hierarchy handlescoherent accesses, this model completely hides the coherence issue from theprogrammer. However, this comes at the cost of a more complex design.LLC-coherent accelerators send DMA requests to the shared last-level cache(LLC). This model is efficient when DMA requests have a high LLC hitrate. However, this approach requires special treatment of the acceleratordata region in private scalar caches, as they must flush the region before theaccelerator execution. Alternatively, accelerators can interface through othercache levels [20], such as the first-level cache (L1) to eliminate the need forflushing. A limitation of these models is that the LLC’s memory interfacelimits bandwidth. In addition, they require breaking down data transfers tofit them into cache lines.Non-coherent accelerators bypass the cache hierarchy to access memory, whichrequires flushing the accelerator data region in the cache before the acceleratorexecution. This model is efficient for high-throughput accelerators, allowingthem to operate on more than cache lines. Non-coherent accelerators havesimpler designs as they eliminate the need for additional hardware to maintain82.2. RISC-V Instruction Set Architecturecoherence. However, they require programmers to manage synchronizationand flushing between the accelerator and the processor [21], which can bechallenging.2.2 RISC-V Instruction Set ArchitectureThis section overviews the RISC-V Instruction Set Architecture (ISA), cover-ing its modular design, opcode space, and control/status registers (CSRs).The RISC-V ISA [22] is an open and modular ISA initially developed in 2010at the University of California, Berkeley. Since then, it has gained extensivetool support, including GCC and LLVM cross-compilers, a software ISAsimulator, an ISA verification suite, and Linux.RISC-V is defined as a base integer ISA, denoted as 'I', that can be augmentedwith optional extensions. The 'I' extension includes integer computationalinstructions, loads, stores, and control-flow instructions. It is characterizedby the number of integer registers and the width of the integer registers,XLEN (either 32 or 64).To satisfy most computing needs, RISC-V provides a set of standard exten-sions: integer multiply/divide (M), atomic operations (A), single-precisionfloating-point (F), double-precision floating-point (D), and compressed in-structions (C). These standard extensions (IMAFDC) are combined with thebase ISA and termed 'GC' . Beyond 'GC' , standard extensions, such as the'V' vector extension, tend to be more domain-specific for a restricted classof applications. Additionally, RISC-V reserves space in the ISA for futurestandard extensions and user-defined custom extensions for accelerators.RISC-V defines both instructions and CSRs; the latter are primarily used bythe operating system to control instruction and system behavior. RISC-V’s32-bit opcode space is divided into 28 major opcode blocks, where the 2 leastsignificant bits are fixed to 11, and the next 5 bits represent the major blocknumber. Each block corresponds to a major instruction group, e.g., load:00000 is for integer loads, and store: 01000 is for integer stores.2.3 RISC-V Memory ModelThis section explains essential concepts of the RISC-V memory model, whichensures consistency across different hardware implementations. It also de-scribes the FENCE instruction required in this model and explains the92.3. RISC-V Memory ModelRISC-V rules on coherence. Additional details are available in Chapter 17 ofthe RISC-V Unprivileged Specification [22].A RISC-V hardware implementation can include one or more RISC-V-compatible processing cores, fixed-function accelerators, various memorystructures, I/O devices, and an interconnect structure to allow the compo-nents to communicate [22]. In RISC-V, a ‘core’ refers to a component withan independent instruction fetch unit [22]. A RISC-V core may incorporatespecialized instruction-set extensions or a coprocessor. It may also supportmultiple RISC-V-compatible hardware threads, or ‘harts’, each representinga copy of the user register state and a program counter.Memory accesses by a hart may appear out of order to other harts oragents accessing the same memory (e.g., accelerators). RISC-V defines the‘RISC-V Weak Memory Ordering’ (RVWMO) model to constrain reorderings.Under RVWMO, code on a single hart executes in order from its perspective.However, memory instructions may appear out of order to other harts oragents. To ensure order across different harts or agents, the software mayneed explicit synchronization using the FENCE instruction RISC-V providesunder the major opcode misc-mem: 00011.The FENCE instruction enforces ordering of device I/O and memory accessesas viewed by other RISC-V harts, external devices, or coprocessors. It canorder any combination of I/O device input (I), I/O device output (O), hartmemory read (R), and hart memory write (W) operations with any othercombination of the same operations; the RISC-V execution environmentdetermines which instructions should operate as device I/O rather thanstandard memory reads and writes. The FENCE instruction prevents anyoperation in a successor (S) set from being observed by another RISC-V hartor external device before any operation in a predecessor (P) set.Figure 2.1 shows the encoding of the FENCE instruction. The fields PI,PO, PR, and PW represent I/O device input, I/O device output, hart memoryread, and hart memory write in the predecessor set. In contrast, SI, SO, SR,and SW represent corresponding operations in the successor set. The fencemode field fm specifies the semantics of the FENCE instruction: fm=0000denotes RVWMO’s FENCE, fm=1000 corresponds to the FENCE for theRISC-V Total Store Ordering (RVTSO) memory model, which is defined asa delta from RVWMO, and other fm values are reserved for future use. Thefunct field identifies the FENCE instruction within the misc-mem opcode.The fields rs1 and rd are reserved for potential future extensions, such as102.4. RISC-V Custom Extension31 28 27 26 25 24 23 22 21 20 19 15 14 12 11 7 6 0fmPIPOPRPWSISOSRSWrs1 funct rd misc-memFigure 2.1: FENCE instruction encoding.finer-grain memory disambiguation. The lack of such disambiguation canresult in excessive stalls for memory access to disjoint address regions.RVWMO and RVTSO memory models do not constrain microarchitecturesfrom employing advanced speculation or optimization techniques to enhanceperformance. These models impose no requirements on cache hierarchies orcoherence protocols at all. Instead, they define the observable behaviors thatsoftware must adhere to. Microarchitectures can adopt any pipeline design,coherent or non-coherent cache hierarchy, or on-chip interconnect as long asthe implementation complies with the rules of the memory model.2.4 RISC-V Custom ExtensionThis section outlines RISC-V’s support for custom extensions and reviewsthe Draft RISC-V Composable Custom Extensions specification [12] used toimplement the instruction-based accelerators in this thesis.RISC-V is designed to be flexible and extensible to integrate custom accel-erators, such as vector accelerators [23], into a host core. It allocates fourmajor opcodes for custom extensions within its 32-bit base instruction format:custom-0: 00010, custom-1: 01010, custom-2: 10110, and custom-3:11110. This continuous opcode space is especially appealing to FPGA users.2.4.1 Draft RISC-V Composable Custom ExtensionsSpecificationThis section reviews the Draft RISC-V Composable Custom Extensions speci-fication [12], highlighting its core features and logic interface. It also overviewsthe software programming model of the Composable Instructions for Accel-erators specification [1], which builds upon the Draft RISC-V ComposableCustom Extensions specification.OverviewCustom instruction extensions are unregulated by RISC-V. Differing orga-nizations may author software libraries that use custom extensions or cores112.4. RISC-V Custom Extensionthat implement these extensions, leading to composability issues. Conflictscan then arise between composable extensions in custom opcodes, customCSRs, or implementations, impairing software and hardware reuse.The Draft RISC-V Composable Custom Extensions specification defines aframework to support decentralized and cooperative reuse of custom instruc-tion and CSR spaces. This framework enables the practical reuse of multiple,independently authored composable extensions (CXs), CX libraries, and CXunits (CXUs) while remaining compatible with legacy custom extensions.The Draft CX specification forms the basis for the Composable Instructionsfor Accelerators (CIA) specification, which further provides standard, modernOS-level mechanisms of accelerator-agnostic protection/isolation of state,access control, virtualization, and context save/restore.The Draft CX specification describes a method for rapidly switching customopcode and CSR spaces by writing to the mcx_selector custom machine-levelCSR a selector, specified by an 8-bit accelerator unit number, cxu_id, andan 8-bit state context number, state_id. A set of custom instructions andCSRs that adhere to the specification is called a CX Extension (CXE), whichmay possess internal state (e.g., a register file). The 8-bit state_id allows formultiple contexts (or copies) of this state. CX’s multi-context state featureenables simplified vector multithreading of single-threaded programs in theinstruction-based accelerators used in this thesis. The multi-unit feature addsmulticore capability to enhance the performance of these accelerators further.In CX, the custom opcode space is the aggregate of all four major customopcode blocks, custom-0 through custom-2. Each block is assigned aninstruction format as shown in figure 2.2. Each accelerator instruction isplaced in the major block of its format, constraining the type of instructionsCX can support, e.g., no branch or large-immediate instructions.R-type and I-type follow the standard RISC-V formats of the same name. Thefunct field provides up to 1024 R-type instructions or 8 I-type instructions.The F-type is a new ‘flexible’ format. It is similar to the R-type but does notuse the destination register field, leaving it custom. The F25-type form ofinstruction issues an arbitrary 25-bit custom instruction. However, the CIAspecification modifies the flexible type encodings to introduce an optionalinternal register file, denoted as Z, which the accelerator can access.Additional details in the specification include limitations on opcode encodings,a logic interface, and functionality of custom instructions and CSRs. Aprimary limitation of CX is that it supports only compute-oriented custom122.4. RISC-V Custom Extension31 25 24 20 19 15 14 12 11 7 6 0funct[9:3] rs2 rs1funct[2:0] rd custom-0R-typeimm[11:0] rs1funct[2:0] rd custom-1I-typefunct[9:3] rs2 rs1funct[2:0]custom custom-2F-typecustom custom-2F25-typeFigure 2.2: CX custom instruction formats.instructions without direct memory access. This thesis addresses this byintroducing memory access in a way that could extend the specification.CX Logic InterfaceThe Draft CX Specification defines a logic interface, CXU-LI, which is aset of common hardware logic signals that enable the composition of CPUsand CXUs. This interface includes a request channel, cx_request, to theaccelerator and a response channel, cx_response, to the CPU. For eachrequest, there is exactly one response.The CXU-LI interface has four feature levels: L0: combinational; L1: fixedlatency; L2: variable latency; L3: reordering. Each feature level adds morerequest and response signals, module ports, and behaviors to the feature levelbelow it. This thesis uses the highest level CXU-L3 to maximize performance.In CXU-L3, the CXU computes the request function and returns a responsein a later clock cycle, using the ready/valid protocol for handshaking requestsand responses. CXU-L3 also incorporates a request-response ID for therequester to correlate responses received to requests sent.Request Channel The request channel consists of six signal groups:req_insn, req_data0, req_data1, req_cxu, req_state, and req_id. WhenCX is enabled, the request channel forwards the 32-bit custom instructionvia req_insn for further decoding and execution within the accelerator.Along with the instruction, the request channel sends any required scalar32-bit operands from source registers rs1 and rs2 through req_data0 andreq_data1, respectively.The request channel also sends an 8-bit mcx_selector.cxu_id and an 8-bit*.state_id via req_cxu and req_state, respectively. The cxu_id identifiesa specific accelerator instance and is used to route the data through an132.4. RISC-V Custom Extensionintermediary CX switch. If necessary, the switch arbitrates and forwardsrequests to the corresponding accelerator and later arbitrates and forwardsresponses back to the CPU. The state_id identifies a state context withinthe accelerator. For stateless accelerators, req_state is set to zero.Each request carries a unique ID, req_id, for correlating responses with theiroriginal requests. An out-of-order completion CPU can supply the request’sdestination register into that ID and rely on it when the response returns.Response Channel The response channel provides a 32-bit result throughresp_data. This channel also includes a response ID, resp_id, which matchesthe original request ID, and a status signal, resp_status, to report accumu-lated errors in the cx_status custom user-level CSR.CIA ABI and ABIThis section provides a brief overview of CIA’s uniform software programmingmodel, as defined by its ABI and API [1, 24]. Further details are available inFreiberger’s thesis [25].The CIA ABI ensures the correct composition of independently authored CXlibraries and legacy (i.e., traditional) custom extension libraries under CXmultiplexing. It follows a caller-save convention, requiring the CX library tore-select the CX after each function call as the legacy mode is selected priorto calling an ‘ordinary’ function, returning, or stack unwinding.The CIA API defines four main functions: open, close, select, and status.Accessing an accelerator in a software thread begins with opening it to obtaina selector, which the thread uses later to close the accelerator. A selector issimilar to a file descriptor and indicates a specific accelerator unit, U, and aspecific state context within that unit, S. A selector can then be described asa pair of values, <U,S>. ‘Open’ and ‘close’ system calls handle the discoveryand allocation of resources within the operating system. The ‘open’ call mayfail when resources are fully committed or not present. The ‘close’ call, or aprocess exit, returns these resources.A hart can rapidly change its currently selected <U,S> pair by using the‘select’ call, which writes the new pair to the mcx_selector machine-levelCSR directly in a bare-metal environment (like in this thesis) or indirectly inan OS/privileged environment. Once selected, the hart directs all subsequentcustom instructions to that accelerator’s state context.142.5. RISC-V Vector ExtensionThe ‘status’ call waits until all of the hart’s issued custom instructions arecomplete, then clears the cx_status custom user-level CSR and returns itslast content.The CIA specification distinguishes between physical and logical state contexts.A physical state context, simply referred to as a state context in this thesis,represents an internal state that a CXE may possess. Stateful acceleratorshave one or more physical state contexts, and opening an accelerator multipletimes produces multiple open selectors. Virtualization of a physical statecontext produces multiple logical state contexts.When a thread opens an accelerator, it must specify a virtualization typeindicating the desired time-sharing policy of the physical state context acrosslogical state contexts and processes. For instance, if the thread requests novirtualization, the ‘open’ call will request exclusive access to the physicalstate context, returning an invalid selector if it fails to get exclusive access.A thread may use multiple accelerators to maximize performance whenmultiple copies of the same accelerator exist, such as in this thesis withmultiple soft vector processors. In this case, the ‘open’ call balances physicalstate context allocations across all available accelerators, so repeated requestsallocate one physical state context from each matching accelerator instancebefore assigning another context from the same unit.2.5 RISC-V Vector ExtensionThis section traces the evolution of vector processors, leading to the RISC-VVector extension, and reviews FPGA and ASIC-based implementations ofthat vector extension. It also reviews coprocessor interfaces for connectingthese vector processors to host cores.2.5.1 Evolution of Vector Processors to RISC-V VectorsEarly Vector Processors Early supercomputers in the late 1970s, such asCRAY-1 [26], were vector processors. However, in the 1990s, the “Attack ofthe Killer Micros” predicted by Brooks [27] came true as superscalar micro-processors began to overtake ‘big iron’ vector processors in performance andcost efficiency, leading to the decline of vector processors in high-performancecomputing (HPC). However, Asanovic [28] hinted at a possible resurgence,anticipating that future applications would contain an abundance of paral-lelism. Espasa et al. later demonstrated that advanced techniques such as152.5. RISC-V Vector Extensiondecoupling [29] — which allows data movement to bypass arithmetic opera-tions — multithreading [30], and out-of-order execution [31], could furtherboost vector performance.As vector processors declined in HPC, the VIRAM ASIC implementation[32] in the early 2000s illustrated the effectiveness of vector processors inaccelerating embedded multimedia tasks. Using the VIRAM ISA, VIPERS[33] and VESPA [34] processors showed that soft vector architectures could beefficiently implemented on FPGAs, achieving significant speedups. Additionalhigh-performance soft vector processors soon followed, including BlueVec [14]and MXP [35]. Meanwhile, in the ASIC space, Lee et al. [23] introduced theHwacha vector architecture, a single-lane decoupled vector pipeline basedon the open RISC-V ISA, with its instruction set implemented as a customextension. Hwacha resembles traditional CRAY vector pipelines more closelythan SIMD units like SSE, AVX, or NEON. Hwacha’s design ultimatelyinspired the development of the RISC-V Vector Extension (RVV) [36], whichwas initially proposed in 2015 and ratified as RVV 1.0 after six years ofdevelopment.RISC-V Vectors RVV provides general support for data-parallel executionwithin RISC-V’s 32-bit instruction encoding space. It adds over 400 instruc-tions under one new major opcode, op-v: 10101 for vector arithmetic andconfiguration, and two existing major opcodes: load-fp: 00001 for vectorloads and store-fp: 01001 for vector stores. RVV’s vector memory in-structions support unit-stride, constant-stride, and indexed (scatter/gather)addressing modes that follow RVWMO at the instruction level. RVV’s vectormemory instructions support unit-stride, constant-stride, and indexed (scat-ter/gather) addressing modes that follow the RVWMO weak memory modelat the instruction level. A key feature of RVV is its vector-length agnosticmodel compared to fixed-width SIMDs, which are less efficient [37]. Thisfeature is enabled through the vset{i}vl{i} configuration instructions.RVV adds 32 vector registers, named v0 to v31, each storing VLEN bits.It also introduces seven unprivileged CSRs, including vl for vector lengthand vtype for vector type. The vl register specifies the number of elementsto update in each vector instruction result. The vtype register indicatesthe organization of elements in each vector register through the selectedelement width (SEW) setting and how to group multiple vector registersusing the length multiplier (LMUL) setting. These settings together determineVLMAX = LMUL ·VLEN/SEW, the maximum number of elements a single162.5. RISC-V Vector Extensionvector instruction can process. Additionally, the vtype register specifies howto handle masked-off elements and elements past the current vector length.2.5.2 RISC-V Vector ProcessorsThis section reviews prominent FPGA and ASIC-based RVV implementationsand their interaction with memory. Several other implementations are startingto appear in the literature and on open-source platforms [38–44]. Also,companies such as Andes [45, 46], SiFive [47, 48], and Semidynamics [49]have developed commercial processors with RVV. However, not enoughinformation is publicly available to determine the details of the commercialimplementations.FPGA-based ImplementationsVicuna Vicuna [50] is one of the earliest FPGA-based RVV 1.0 processors.It is a 32-bit integer vector coprocessor using the in-order Core-V Ibex orCV32E40X processors as scalar cores. After decoding a vector instruction,Vicuna signals the host core to wait for a scalar result or continue executinginstructions. If a vector instruction produces no scalar result and only writesto a vector register or memory, the host core can proceed with furtherinstructions in parallel with the coprocessor. However, if a scalar result isrequired, the host core stalls until the coprocessor completes the instruction.Vicuna and its scalar core share a data cache to access memory, which alsoaddresses coherence. The data cache uses a replacement policy that prioritizesaccesses from the vector core. Vicuna aims for timing predictability, withtiming variability arising only from vector loads and stores, whose executiontimes depend on the state of the data cache. It guarantees in-order memoryaccess by delaying access following a cache miss in the host core until pendingvector loads and stores are complete, leading to excessive stalls.Saturn-V Saturn-V [2] is another FPGA-based RVV processor. It imple-ments the RVV-Lite ISA, a subset of the RVV 1.0 ISA that discards abouthalf of the 400 vector instructions: instructions with floating-point operations,64-bit element sizes, ternary operands, and some with mixed-size operands.Saturn-V constrains the ratio SEW/LMUL to 8 to simplify the register filedesign, resulting in a VLMAX of VLEN/8.Saturn-V attaches to the in-order VexRiscv [51] processor as a CustomFunction Unit (CFU) [52] extension through a CFU port. When VexRiscv172.5. RISC-V Vector Extensiondecodes a vector instruction, it dispatches the instruction to Saturn-V via theCFU port, along with any required scalar operands. If a response is required,the scalar core waits for it. VexRiscv has a write-through data cache and onecustom instruction to invalidate the cache entirely.A significant limitation of the CFU framework is its lack of direct memoryaccess. Saturn-V addresses this by modifying the CFU port to enable memoryaccess. Another limitation is that the CFU stalls the CPU pipeline until itproduces a result, creating a bottleneck during the execution of long-latencyinstructions, such as vector instructions. However, this approach eliminatesconsistency issues by ensuring in-order memory accesses from the vectorprocessor and the in-order scalar host. Saturn-V does not address coherence.ASIC-based ImplementationsAra and Ara2 Ara [53] is among the first vector processors to implementRVV. Designed in ASIC, Ara is compliant with RVV 0.5, while Ara2 [54]adds support for RVV 1.0 and multicore capabilities. Ara2 uses the in-orderCore-V CVA6 [55] processor, known initially as Ariane, as its scalar host.CVA6 dispatches vector instructions to Ara2 and waits for acknowledgmentand results before committing. Ara2 follows a fire-and-forget dispatch [56],acknowledging CVA6 immediately when there is no result to return.Ara2 is non-coherent and supports store-load and store-store memory disam-biguation between vector and scalar instructions. It handles consistency andcoherence in four ways. First, CVA6’s L1 data cache has a write-throughpolicy (CVA6 has no L2+ caches). Second, when the vector unit performs avector store, it invalidates the corresponding cache lines in the L1 data cache.Third, CVA6 issues scalar loads only if no vector stores are in flight and scalarstores only if no vector loads or stores are in flight. Fourth, CVA6 dispatchesvector instructions to Ara2 only when they reach the top of the scoreboardand all previous instructions are complete. This dispatch mechanism im-plicitly preserves memory ordering for vector stores following scalar loadssince CVA6 commits scalar loads only after their results are returned frommemory [57]. To serialize scalar stores followed by vector memory operationsand account for CVA6’s store buffer, Ara2 adds explicit logic to issue vectorloads or stores only if no scalar stores are pending. Although this approachsolves coherence and consistency issues, it may lead to excessive stalls, e.g.,by preventing the reordering of accesses to disjoint address regions.182.5. RISC-V Vector ExtensionMore recent vector processors, such as Quark [56] and Sparq [58], build onAra2 to optimize sub-byte quantized DNN inference. MSPARQ [59] uses theOpenPiton [60] manycore framework to scale Sparq to a multicore processor.SPEED SPEED [61] is designed for efficient multi-precision DNN inference.It adds custom vector instructions to support DNN operations with precisionsfrom 4-bit to 16-bit. SPEED is tightly coupled to a RISC-V scalar core andcan access external memory. However, whether the scalar core includes acache or how SPEED manages coherence and consistency is not stated.T1 T1 (Torrent-1) [62] is a RISC-V Vector implementation inspired bythe Cray X1 vector machine, targeting lane-based microarchitectures withintensive chaining support and SRAM-based vector register files (VRFs). Ituses the in-order Rocket processor [20] as its scalar host and includes essentialvector machine features like lanes, chaining, and a large LSU.A key feature of the LSU is its support for multiple memory banks, eachwith three Miss Status/Handler Registers (MSHRs) for outstanding memoryinstructions. Also, T1 supports instruction-level out-of-order vector loadsand stores to maximize the high memory bandwidth that the vector cacheprovides. However, T1 does not include support for coherence.Saturn Saturn [63] is a parameterized RVV 1.0-compliant vector unit. Itimplements a compact short-vector-length vector microarchitecture suitablefor deployment in a DSP-optimized or area-efficient general-purpose core.Saturn supports Rocket and Shuttle scalar cores through the Chipyard [64]SoC design framework. Further documentation on Saturn is yet to be released.2.5.3 Vector Coprocessor InterfacesThis section reviews logic interfaces to attach vector coprocessors to hostcores. The term ‘coprocessor’ in RISC-V refers to a unit attached to a core,primarily sequenced by a RISC-V instruction stream [22]. It can containadditional architectural state and instruction-set extensions and operate withlimited autonomy relative to the main RISC-V instruction stream.The RISC-V community has developed several open-source logic interfacesto attach coprocessors to host cores, some of which are used to attach vectorcoprocessors. The Rocket Custom Coprocessor Interface (RoCC) [20] is onesuch example that decouples the Rocket core from attached coprocessors.Many coprocessors have been implemented using RoCC, including the Hwacha192.6. Related Workvector unit. RoCC allows a coprocessor to share the host core’s data cacheand address translation unit. Also, it provides a direct connection to theouter memory system through the Tilelink [65, 66] coherent interconnect.The Core-V eXtension interface (CV-X-IF) [67] from the OpenHW Groupaims to extend processors with instructions implemented in coprocessors.The CV-X-IF interface supports both standard and custom instructions andcustom CSRs. Ara and Vicuna use CV-X-IF to connect to their in-orderCore-V host cores, with modifications to enable memory access.The custom function unit (CFU) Playground [52] defines a logic interface,CFU-LI, for connecting processors to application-specific hardware acceler-ators. Initially used by VexRiscv, it allows up to 1024 custom instructionswithout support for memory access. Saturn-V uses CFU-LI to attach toVexRiscv, modifying the interface to enable memory access. The CFU-LIinterface inspired the creation of the Draft CX Specification’s logic inter-face, CXU-LI. In this thesis, CXU-LI connects Saturn-V to the out-of-orderprocessor Core-V CVA5 [68], adding support for memory access.Other logic interfaces include SiFive’s Vector Coprocessor Interface eXtension(VCIX) [69] and Semidynamics’ Open Vector Interface (OVI) [70] for customvector instructions. The VCIX interface extends application processors withcustom coprocessors and variable-latency arithmetic units. Coprocessorinstructions use the custom-2 major opcode, with encoding and operandpatterns similar to standard RISC-V vector instructions.Semidynamics uses the OVI interface to decouple an attached ‘Vector Unit’from a RISC-V core. The host core manages vector memory accesses andcontrols CSRs, while the Vector Unit handles vector computations.2.6 Related WorkThis section reviews common memory units, such as the shared LSU, thatcentralize memory access. It also covers multithreaded vector architectures,as the instruction-based accelerators in the case study consist of multiple softRISC-V vector processors, each featuring simplified multithreading.2.6.1 Common Memory UnitsManaging latency and complex memory access patterns is essential to improv-ing performance in modern accelerator-like computing systems. Providingdirect memory access is one way to provide such performance. iDMA [11] is a202.6. Related Workmodular and configurable open-source DMA architecture consisting of threeindependently customizable components: the front-end implements controlplane binding to the SoC; the mid-end accelerates complex data transferpatterns such as multi-dimensional transfers, scattering, or gathering; andthe back-end interfaces with AXI and OBI on-chip communication protocols.iDMA achieves up to a 15.8× speedup in high-performance systems withonly a 1% area increase compared to a base system without DMA.CoRAM [71] is an FPGA memory architecture that serves as a portable bridgebetween distributed computation kernels and external memory interfaces. Itprovides a virtualized memory environment for hardware kernels to simplifydevelopment and improve application portability and scalability by separatingthe compute kernel and memory fetch stages.2.6.2 Multithreaded Vector ArchitecturesAt the end of the 1990s, Espasa and Valero [30] showed multithreadingin vector processors could significantly improve throughput and maximizeresource utilization. Using a trace-driven approach, they simulated a selectionof Perfect Club [72] and SPECfp92 [73] programs. They compared executiontimes on a conventional vector architecture with a single memory port anda multithreaded vector architecture. They focused on optimizing memoryport usage, which they identified as the most expensive resource in typicalvector computers, and analyzed the overhead of duplicating the vector registerfile. They showed that multithreading could achieve over 1.4× speedup forrealistic memory latencies, with 95% utilization of the single memory port.A more recent example is the T13 core from the family of Klessydra-T proces-sors [74], a RISC-V core that supports the RV32IMA instruction set and addscustom mathematical vector instructions. Klessydra-T13 implements inter-leaved multithreading (IMT) in a 4-stage-pipeline processor. IMT alternatesinstructions from different threads, with Klessydra-T13 interleaving threehardware threads (harts). Each hart has its own register file, program counter,and CSR unit, while a hardware context counter switches between threads tofetch instructions. Implemented as an FPGA softcore, Klessydra-T13 benefitsfrom thread-level and data-level parallelisms to accelerate compute-intensivetasks like convolution, matrix multiplication, and fast Fourier transform.Modern GPUs are also multithreaded vector architectures as they interleavewarps in a fine-grain manner [75].None of these architectures is a multithreaded RISC-V vector architecture.21Chapter 3Shared Load-Store UnitThis chapter presents the design of a novel shared LSU for high-performance,instruction-based accelerators in shared-memory heterogeneous systems. Theshared LSU targets non-coherent accelerators that bypass the processor’swrite-through cache hierarchy to access memory.The shared LSU features a load-store queue (LSQ) that performs automaticmemory disambiguation and enables accelerator-based range fences to ensureconsistency. It also includes a cache invalidation unit to handle coherence. Thedesign is platform-independent and uses open-source communication protocolsto facilitate hardware reuse, with AXI Stream interfaces for connecting toaccelerators and an AXI Memory Mapped interface for connecting to externalmemory. Currently, the shared LSU only supports unit-stride data transfers.The remainder of this chapter is organized as follows. Section 3.1 discussesthe need for a shared unit. Section 3.2 presents the memory model the sharedLSU uses for consistency. Section 3.3 describes how the shared LSU handlescoherence, and section 3.4 details the design of the shared LSU.3.1 The Need for a Shared UnitThis section explains the rationale for sharing a LSU accross accelerators.The need for a shared unit stems from four key observations:1. Considerable logic within an accelerator’s load-store unit can be sharedacross multiple accelerators, especially those requiring burst accesses(e.g., tensors, strides, or indexed accesses like those supported by RVV).2. High performance demands maximum overlap of operations and auto-matic dependence checking while ensuring coherence and consistency.3. Mismatches in memory access widths between the accelerator andmemory interface may require shifting or realignment.223.1. The Need for a Shared Unit4. Complex requirements like address translation services (e.g., IOMMU)may be needed.A shared LSU can address these challenges by handling them once for allaccelerators. Sharing an LSU consolidates LSU logic, buffering, multiplememory access ports, and coherence and consistency logic into a single unit.Figure 3.1 illustrates two differing SoCs with CPU and RVV cores accessingmemory. In figure 3.1a, the RVV core accesses memory through its dedicatedvector LSU. In contrast, figure 3.1b shows a modified RVV that accessesmemory via an external shared LSU, as proposed in this thesis. The benefitis that other accelerators in the SoC can reuse this shared LSU.(a) RVV accessing memory through its dedicated vector LSU.(b) RVV accessing memory through an external shared LSU.Figure 3.1: Differing SoCs showing CPU and RVV cores accessing memory.233.2. Consistency3.2 ConsistencyThis section outlines the memory model the shared LSU uses to addressconsistency and describes the accelerator-based fences essential to this model.This thesis uses RVV processors, implemented as CXUs, to serve as instruction-based accelerators and demonstrate the shared LSU. For consistency, RISC-Vdefines the RVWMO weak memory model, which RVV follows at the instruc-tion level [22]. Under RVWMO, memory operations of a hart appear in orderfrom the perspective of that same hart but may appear out-of-order fromthe perspective of other harts (see section 2.3). RISC-V also introduces theFENCE instruction to order device I/O and memory accesses as viewed byother harts and external devices or accelerators; the execution environmentdetermines which instructions should operate as device I/O rather thanstandard memory reads and writes. However, the FENCE instruction doesnot support finer-grain memory disambiguation, as RISC-V leaves its designopen to future extensions. This limitation can lead to excessive stalls whenaccessing disjoint address regions.This thesis ignores RVV memory ordering semantics. Instead, it requiresprogrammers to insert finer-grain accelerator-based custom FENCE instruc-tions and standard RISC-V FENCE instructions, treating accelerator loadsand stores as device input and output operations. This model providesgreater flexibility for hardware implementation and enhances performance, asmemory accesses between fences can operate in parallel. Also, the finer-grainmemory disambiguation of accelerator accesses reduces the excessive stalls.The accelerator-based custom range fences (cx_fence) use CX’s R-typeformat shown in figure 3.2. The rs1 and rs2 register fields specify subsequentmemory operations’ base and end addresses, respectively, and the 10-bitfunct field indicates the memory access type: 1008 for a scalar load, 1009 fora scalar store, 1010 for an accelerator load, and 1011 for an accelerator store.31 25 24 20 19 15 14 12 11 7 6 0funct[9:3] rs2 rs1funct[2:0] rd custom-0Figure 3.2: CX R-type instruction encoding.Table 3.1 presents the resolution of data races between a scalar processorand an accelerator by inserting standard RISC-V FENCE instructions oraccelerator-based custom FENCE instructions between conflicting memoryoperations. Table 3.1 shows various combinations of loads and stores with243.2. ConsistencyTable 3.1: Resolution of data races between a scalar processor and an accel-erator.Scalar \ Acc. Load StoreLoadCase 1: Scalar → Acc.No data race.Case 2: Acc. → ScalarNo data race.Case 1: Scalar → Acc.Insert fence r, o.Case 2: Acc. → ScalarInsert cx_fence 1008.StoreCase 1: Scalar → Acc.Insert fence w, i.Case 2: Acc. → ScalarInsert cx_fence 1009.Case 1: Scalar → Acc.Insert fence w, o.Case 2: Acc. → ScalarInsert cx_fence 1009.////////////////////////////////////////////////////////////////////// scalar processor // vector accelerator //////////////////////////////////////////////////////////////////////...1 sw x1,0(a1) // scalar store2 fence w,i // standard fence3 vse32.v v1,(a2) // vector store4 cx_fence 1008,a2,a2 // custom fence5 lw x2,0(a2) // scalar load6 vle32.v v2,(a1) // vector load...Listing 2: Example instruction stream with resolved coherence and consistencyissues in a shared-memory system.a happened-before relation (→). A data race occurs when two memoryoperations access the same memory location, with at least one being a writeoperation. In listing 2, necessary standard and custom fences are insertedto resolve data races encountered in listing 1. Another example of using thecustom fence instruction can be to ensure that a vector store is completebefore subsequent dependent scalar loads that verify the vector store data.For data races between accelerators, the resolution is to insert cx_fence1010 if the subsequent memory operation is an accelerator load or cx_fence1011 if it is an accelerator store, as shown in table 3.2. These two fences donot stall the scalar processor since they only synchronize accelerators.253.3. CoherenceTable 3.2: Resolution of data races between accelerators.Acc. 1 \ Acc. 2 Load StoreLoadCase 1: Acc. 1 → Acc. 2No data race.Case 2: Acc. 2 → Acc. 1No data race.Case 1: Acc. 1 → Acc. 2Insert cx_fence 1011.Case 2: Acc. 2 → Acc. 1Insert cx_fence 1010.StoreCase 1: Acc. 1 → Acc. 2Insert cx_fence 1010.Case 2: Acc. 2 → Acc. 1Insert cx_fence 1011.Case 1: Acc. 1 → Acc. 2Insert cx_fence 1011.Case 2: Acc. 2 → Acc. 1Insert cx_fence 1011.3.3 CoherenceThis section describes how the shared LSU manages coherence issues thatmay arise when accessing shared memory.The shared LSU is designed for non-coherent accelerators that bypass thecache hierarchy to access shared memory in heterogeneous systems. Manyaccelerators are non-coherent [76] to reduce hardware complexity, cost, andpower consumption while potentially offering superior performance and achiev-ing high throughput. However, the disadvantage of non-coherence is that thesoftware must explicitly synchronize between accelerators and processors by in-serting cache flushing and synchronization instructions. This synchronizationprocess is error-prone and, when done conservatively, impacts performance.This thesis considers systems with scalar write-through caches to simplify thedesign and implements Ara’s hardware coherence mechanism [53]. This mech-anism performs coherence actions only when necessary, without programmerintervention. In non-coherent models, the software must flush the acceleratordata region before execution. For instance, in listing 1, a vector load followinga scalar store may read old data from memory before the scalar cache writesback, requiring flushing before executing the vector load. The write-throughpolicy eliminates this need since caches and external memory are up-to-date.However, when an accelerator externally updates memory, the processor maysee stale data. In listing 1, a scalar load following a vector store may readstale data from the scalar cache. To address this, the shared LSU invalidatescorresponding cache lines in the scalar cache with each accelerator store.263.4. Shared LSU Architecture3.4 Shared LSU ArchitectureThis section details the design of the shared Load-Store Unit (LSU), describingits interface to external memory in section 3.4.1, interface to acceleratorsin section 3.4.2, internal LSQ in section 3.4.3, and invalidation unit insection 3.4.4. Figure 3.3 illustrates the architecture of the shared LSU fromthe perspective of each accelerator, with ACC, AXI, and INV signal groupsconnecting to the accelerator, memory, and scalar cache, respectively.Figure 3.3: Block diagram of the shared LSU from the perspective of eachaccelerator. ACC, AXI, and INV signal groups connect to the accelerator,memory, and scalar cache, respectively.3.4.1 Interface with External MemoryThis section provides an overview of the AXI Memory Mapped Interface(AXI-MM) [77] that the shared LSU uses to communicate with memory.AXI-MM is a high-performance, non-coherent on-chip memory protocolwidely used in system design. It is burst-based and features five independentchannels for memory operations: two for reads (read address AR and readdata R channels) and three for writes (write address AW, write data W, andwrite response B channels). Each channel uses the READY/VALID handshake.273.4. Shared LSU ArchitectureThe address channels, AR and AW, carry control information for addressgeneration and burst logic, including the burst address, length, size, type, andtransaction ID. The transaction ID uniquely identifies each transaction. Theburst address specifies the address of the first transfer in a transaction, whilethe burst length indicates the exact number of transfers within a burst. Theburst type and size determine the address calculation for each transfer in theburst. Although AXI-MM supports various burst types, the shared LSU onlyimplements incrementing bursts for unit-stride accesses. The shared LSUbegins each burst transfer by driving control information in these channels.In response to a request in the read address channel, the read data channelR steers data from memory to the shared LSU, which then forwards it tothe requesting accelerator. The write data channel W transfers data fromthe shared LSU, received from the requesting accelerator, to memory; thesequence of write data must match the control information in the writeaddress channel. Additionally, read and write data channels include a LASTsignal to indicate the final data item in a transaction.AXI-MM also provides response signaling for read and write transactions,which includes the transaction ID. For read operations, the response is in theread data channel, while for writes, it is in the write response channel. Theshared LSU uses this signaling to retire accelerator loads and stores whenthey are complete and to route read data back to the requesting accelerator.3.4.2 Interface with AcceleratorsThis section describes how the shared LSU connects with accelerators throughthe AXI Stream (AXI-S) [78] protocol.To facilitate hardware reuse in heterogeneous SoCs, the shared LSU uses thestandard AXI-S protocol to communicate with accelerators. AXI-S providesflow-controlled unidirectional data exchange between a transmitter and areceiver, with the transmitter sending TDATA bits per cycle. A transfer occurswhenever the transmitter asserts the TVALID signal and the receiver assertsthe TREADY signal. AXI-S has additional optional features such as TSTRB,which specifies valid byte lanes, and TID, a stream identifier used for routing.The shared LSU connects to each accelerator through eight independent andconcurrent AXI-S interfaces, requiring fair round-robin arbitration withineach stream set across accelerators. These AXI-S interfaces are:283.4. Shared LSU Architecture• allocate-request, allocate-response• lookup-request, lookup-response• read-request, read-data• write-request, write-dataThe allocate-request interface carries an entry, in TDATA, to allocate inthe shared LSU’s LSQ, described in section 3.4.3. If the LSQ has spaceavailable, the allocate-response interface returns a unique tracking ID inTDATA, used to identify the memory transaction. The LSQ later uses that IDto deallocate the corresponding entry when the transaction is complete.The lookup-request interface checks if an entry in TDATA conflicts withpreviously allocated entries in the LSQ. In response, the lookup-responseinterface provides a 1-bit result in TDATA, indicating whether a conflict exists.The lookup interfaces allow for the implementation of range fences and enabledisjoint memory operations to proceed without excessive stalls.The read-request and write-request interfaces generate control informa-tion for AXI-MM’s read and write address channels using a memory requestpacket in TDATA, containing the base and end addresses, burst size, andtracking ID. After arbitration, the shared LSU combines the packet’s trackingID with the selected accelerator’s ID to form AXI-MM’s request ID. Thepacket’s base address and burst size determine AXI-MM’s burst address andsize, respectively. The shared LSU calculates AXI-MM’s burst length fromthe packet’s base address, end address, and burst size. It also sets AXI-MM’sburst type to incrementing mode for unit-stride accesses.The read-data and write-data interfaces use TDATA to transmit data in asingle burst between accelerators and memory. They use TLAST to signalmultiple elements. The write-data interface steers AXI-MM’s write datafrom accelerators, while the read-data interface routes AXI-MM’s read datato accelerators. For routing, the shared LSU identifies the proper acceleratorread-data interface by extracting its ID from AXI-MM’s response ID. Italso extracts the tracking ID from this response, which it forwards to theaccelerator in TID for further processing.Figure 3.4 illustrates with distributed systems space-time diagrams the sharedLSU’s communication protocols for the fence, read, and write transactions.It shows the interactions between two accelerators, shared LSU, and externalmemory. A fence transaction only uses lookup interfaces to query the LSQfor conflicts with previously allocated entries. After the query, read and write293.4. Shared LSU Architecturetransactions can allocate entries in the LSQ using allocate interfaces. Then,the read-request and write-request interfaces send control informationto memory via the shared LSU. A write transaction also sends write datathrough the write-data interface. A read transaction is complete whenexternal memory returns data to the accelerators via the read-data interface.3.4.3 Load-Store QueueThis section describes the load-store queue (LSQ) the shared LSU uses totrack memory operations and perform dependence checking. It is also used toenable accelerator-based custom fences. This section also presents key LSQoperations: entry allocation, deallocation, and conflict detection.To handle memory disambiguation and detect true dependencies, the sharedLSU includes an LSQ, similar to those in traditional out-of-order proces-sors, which the accelerators can access. Typically, an LSQ uses a content-addressable memory (CAM) for simultaneous associative searches of memorydependencies [79]. However, the shared LSU’s LSQ uses a CAM-like archi-tecture that searches for range overlaps instead of exact matches. As inassociative caches, this architecture includes a valid bit to manage entryallocation and deallocation. Each entry consists of a 1-bit memory accesstype (0 for read, 1 for write) and 32-bit base and end addresses representingthe unit-stride access range. The CAM-like architecture is register-basedand implemented with flip-flops. It consists of N entries, where N is adesign parameter, and each entry is 65 bits wide (access type, base, and endaddresses). Deep register-based CAMs typically demand significant resources[80], as they require registers, comparators, and address decoders. Therefore,the LSQ’s CAM-like architecture should limit the number of entries.The primary LSQ operations are entry allocation, deallocation, and conflictdetection. For allocation, the LSQ uses a priority encoder, which scans thevalid bits to identify the first available ID. Once located, the LSQ storesthe entry and sets the valid bit to mark it as occupied. It then returns theID to the shared LSU, where it acts as a tracking ID. For deallocation, theshared LSU extracts the tracking ID from the AXI-MM ID in the read dataor write response channel and signals the LSQ to clear the valid bit for thatID, thereby freeing the entry. A conflict in the LSQ occurs when the addressrange of a query entry overlaps with that of an allocated entry, and at leastone of the entries involves a write operation. Figure 3.5 illustrates the conflictdetection logic in a LSQ of depth 4.303.4. Shared LSU Architecture(a) Fence transaction.(b) Read transaction.(c) Write transaction.Figure 3.4: Space-time diagrams with two accelerators, the shared LSU,and external memory that illustrates communication protocols for fence,read, and write transactions. Each interface is represented with a distinctcolor: red-orange for lookup-req, green for lookup-resp, blue forallocate-req, pink for allocate-resp, gold for read-req, violetfor read-data, cyan for write-req, and orange for write-data.313.4. Shared LSU ArchitectureFigure 3.5: Conflict detection in the LSQ. Each allocated entry includesa valid bit (v), memory access type (r/w, 0 for read and 1 for write), andaddress range (ba-ea) from base address (ba) to end address (ea).3.4.4 Invalidation UnitThis section describes the invalidation unit the shared LSU requires tomaintain coherence with the processor’s write-through cache.As the shared LSU bypasses the cache hierarchy to access memory directly, itmust invalidate addresses it modifies to prevent the scalar cache from readingstale data. After arbitrating the AXI-S write-request interfaces and usingcontrol information from the write address channel AW, the invalidate unitissues a range invalidation for addresses between the base and end addresses,incrementing each address by one. The shared LSU holds back write data bydeasserting its ready signal on the interface until the invalidation is complete.The shared LSU interfaces with the scalar cache using a 32-bit address, a validbit, and an acknowledgment bit. Additionally, the invalidate unit provides aready signal to indicate when it is available to process new requests.32Chapter 4Multicore × MultithreadedVector ProcessorsThis chapter presents the design of high-performance soft RISC-V vector coresthat use the proposed shared LSU for memory access, replacing individualload-store units. Each core supports multithreading, with many forminga Multicore and Multithreaded RISC-V Vector architecture called MXM-RVV. Implemented as CX units, the vector cores leverage CX’s multi-contextstate feature for simplified multithreading of single-threaded programs. Themulti-unit feature adds multicore capability, further enhancing performance.MXM-RVV consists of N multithreaded M-RVV vector cores connected tothe scalar host via CXU-LI and an intermediary 1-to-N CX switch. It usesthe 32-bit out-of-order completion processor CVA5 as the scalar host tomaximize performance and uses the in-order Saturn-V vector engine as thebase for each vector core. Figure 4.1 illustrates an SoC with one CVA5 CPUcore and two M-RVV vector cores accessing memory through the shared LSU.This chapter is organized as follows. Section 4.1 explains how a multithreadedand multicore architecture like MXM-RVV enables a high performance solu-tion. Section 4.2 outlines the modifications made to Saturn-V for integrationinto MXM-RVV. Section 4.3 details the architecture of MXM-RVV. Section4.4 explains how MXM-RVV integrates with the shared LSU for memoryaccess and disambiguation and addresses consistency and coherence. Finally,section 4.5 presents MXM-RVV’s simplified programming model, which usesthe CIA ABI and API.334.1. Multithreaded × Multicore Vector Architecture as High-Performance SolutionFigure 4.1: SoC with one CVA5 CPU and two M-RVV processors sharingLSU for memory access.4.1 Multithreaded × Multicore VectorArchitecture as High-Performance SolutionThis section explains how a multithreaded and multicore architecture likeMXM-RVV enables high performance.Parallelism is one of the most important methods for improving performance[7]. For high performance, vector architectures exploit data-level parallelism(DLP), where a single instruction operates on multiple data elements simul-taneously. However, dependencies and hazards limit how much parallelismcan be exploited [7]. Data dependency, in particular, arises when the sourceof one instruction is dependent on the result of another. Listing 3 shows adynamic instruction stream with dependent vector instructions. For example,the vector multiply depends on the vector load via the vector register v4....vle32.v v4,(a5) // vector loadvmul.vx v8,v4,a1 // vector multiplyvadd.vx v12,v8,a2 // vector addvse32.v v12,(a4) // vector store...Listing 3: Example instruction stream with dependent vector instructions.344.2. Modifications to Saturn-VWhen two instructions are dependent, they are not parallel and must beexecuted in order, although partial overlap is possible, such as with chaining[26] in vector architectures. In chaining, a vector register is not a single entitybut groups of individual registers, enabling pipeline forwarding of each group[81]. This mechanism allows dependent vector instructions to execute early.Yet, this comes at the cost of a more complex design with forwarding paths.With fine-grain multithreading, independent vector instructions from differentcontexts can be executed between dependent instructions to break depen-dency chains and fill pipeline bubbles. On a multicore architecture, theseindependent instructions can run concurrently across multiple cores, furtherenhancing performance. Despite these benefits, programming a multithreadedand multicore architecture is often challenging. MXM-RVV introduces asimplified programming model, described later in section 4.5, that uses asingle scalar thread.4.2 Modifications to Saturn-VMXM-RVV uses Saturn-V as its base vector core, and this section outlinesthe modifications made to Saturn-V for integration. The main modificationsare removing Saturn-V’s vector load-store unit to use the shared LSU and up-dating its CFU-LI interface to CXU-LI. However, this thesis further modifiesSaturn-V’s original microarchitecture to enhance performance.Figure 4.2 illustrates Saturn-V’s original microarchitecture [2]. Three addressgeneration units (AGUs) connect to the ALU to generate source addressesAa and Ab and the destination address Ac, based on the current SEW setting(VSEW). The direct memory access (DMA) engine includes two additionalAGUs for vector load (VLD) and vector store (VST) operations. The vectorregister file (VRegFile) supports two vector reads and one vector write percycle, with data Da, Db, and Dc corresponding to these operations; the byteenable signal, Be, specifies the bytes written to the vector register file.Saturn-V’s original design supports only AXI-MM Lite for memory access.Unlike full AXI-MM, AXI-MM Lite lacks burst mode, resulting in highermemory latency as it sends each data packet individually. This thesis addsfull AXI-MM support with burst transfers, reducing latency and area byremoving the address increment logic. The modified microarchitecture alsohandles unaligned transfers, aligning the burst address and increasing theburst length accordingly. It adds a read data aligner to shift the data asneeded and a write data aligner to set the valid byte lanes.354.2. Modifications to Saturn-VThis thesis adds a read port for vector stores and a write port for vector loadsto Saturn-V’s vector register file. The vector register file is now implementedas a multi-ported RAM, following Abdelhadi and Lemieux’s XOR-basedapproach [82] and adding support for byte enables. This modification enablesconcurrent reads of vector ALU operands and store data and concurrent writesof vector ALU results and load data, thereby improving performance. Tosupport this, address generation for the source register vs3 (in vector stores)is decoupled from vs1 (in computes). Address generation for the destinationregister vd is also handled separately for vector loads and computes.Saturn-V does not support chaining and executes vector ALU, configuration,load, and store instructions within a single pipeline. This thesis splits thepipeline into four units, enabling a single-instruction issue with parallelexecution of these instruction groups. An additional unit manages the in-order retirement of these instruction groups that may complete out-of-order.This thesis modifies the behavior of vector loads and stores to hide latency.In Saturn-V, vector loads stall the entire vector pipeline because the memoryport can only handle one transaction at a time. This thesis makes vector loadsnon-blocking and changes the conditions for retiring vector stores. Section4.3.2 provides further details on these modifications.Finally, this thesis adds performance counters to measure vector ALU utiliza-tion and pipeline stall cycles from structural or data hazards, with custominstructions to read the counters. It also fixes the functionality of some vectorinstructions and rewrites their implementations for clarity. However, thesemodifications do not impact performance.364.2. Modifications to Saturn-VFigure 4.2: Internal Saturn-V micro-architecture. Taken from White [2].374.3. MXM-RVV Architecture4.3 MXM-RVV ArchitectureThis section presents MXM-RVV’s architecture, which comprises multipleM-RVV vector processors, each implemented as a CXU. Figure 4.3 illustratesthe information flow for executing a vector instruction in MXM-RVV. AnM-RVV processor is a loosely coupled accelerator connected to the CPUvia CXU-LI (see section 2.4.1). It has direct access to memory through theshared LSU, while the CPU accesses memory through its memory unit (MU).The rest of this section is organized as follows. Section 4.3.1 describes thedecoding of vector instructions in the scalar CPU and their dispatch to M-RVV. Section 4.3.2 outlines how M-RVV hides latency using parallel vectorinstruction queues (VIQs), non-blocking vector loads, and early retirementvector stores. It also describes how M-RVV schedules vector instructionswithin these queues.Figure 4.3: Information flow for executing a vector instruction in MXM-RVV.384.3. MXM-RVV Architecture4.3.1 Decoding and Dispatching of Vector InstructionsIn CX, most of the instruction set remains fixed except for a predeterminedblock of opcodes (custom opcode block) that are dynamically ‘bank switched’between accelerators at runtime. In addition to the custom opcode block,this thesis issues all RISC-V vector opcodes to the CXU-LI interface andtreats them as additional custom opcodes. This approach enables addingmultiple vector accelerators with multiple state contexts to the scalar hostwithout extensively redesigning the vector ISA.Once a vector instruction is decoded and confirmed as non-speculative on thescalar host, the host dispatches it to the M-RVV core through the CXU-LIinterface and any required scalar operands. If a response is required, thescalar host waits for it; otherwise, the vector instruction is retired.Among other signals described in section 2.4.1, the CXU-LI interface sendsan 8-bit cxu_id and an 8-bit state_id, both read from the mcx_selectorcustom machine-level CSR (see figure 4.3). The 8-bit cxu_idmultiplexes up to256 CXUs, where each CXU represents a replicated vector unit. Replicating anentire vector unit enables thread-level parallelism among vector instructions,similar to a chip multiprocessor offering parallelism among scalar instructions.The 8-bit state_id provides up to 256 independent state contexts, essentiallyreplicating all state. Each state context consists of the vector register file(VRF) and two vector CSRs (VCSRs): vl and vtype; the remaining fivevector CSRs are not yet supported. M-RVV accesses the correct context inthe vector register file by concatenating vector register numbers with thestate_id bits. Furthermore, a software thread running on the host candispatch instructions separately to each M-RVV state context. These issuedinstructions are stored in parallel vector instruction queues (VIQs) of theM-RVV cores (see figure 4.3), allowing them to execute concurrently withthe issuing hart. These queues are referred to as microthreads.4.3.2 Latency HidingM-RVV modifies the behavior of Saturn-V’s vector loads and stores and addsparallel vector instruction queues to reduce stalls. Since instructions fromdifferent state contexts, i.e., different instruction queues, are independent, theM-RVV core dynamically schedules instructions to fill any pipeline bubbles.This scheduling reduces data dependency and memory latency stalls, providingMXM-RVV with the benefits of vector multicore, vector multithreading, and394.3. MXM-RVV Architecturevector prefetching, all under the control of a single program thread. Figure4.4 shows the detailed architecture of a single M-RVV vector core.Figure 4.4: Block diagram of a single multithreaded M-RVV core.Vector Loads and StoresM-RVV supports non-blocking vector loads, and its vector stores retire assoon as store data is written into a store buffer. This design allows subsequentindependent vector compute instructions to be executed. A current limitationis that each state context supports only one outstanding load or store, thoughmultiple memory operations can run concurrently across different contexts.M-RVV implements non-blocking vector loads by splitting them into twomicro-instructions: load-issue and load-commit. Load-issue initiates mem-ory read requests through the shared LSU and later copies data to a loadbuffer, while load-commit transfers data from the load buffer to the vectorregister file. Load-issue sets a ‘use’ bit in a scoreboard bitmap dedicated tovector load instructions. The use bit gets cleared upon the retirement of thecorresponding load-commit, which is also the retirement of the original load.A separate instruction queue tracks load-commit instructions (see figure4.4). When a vector load is in the main instruction queue, it is a load-issueinstruction; once it reaches the head of this queue, it executes and movesto the load-commit queue. M-RVV issues the load-commit once load data404.4. Integration with the Shared LSUarrives, thereby retiring the original vector load. By removing load-issueoperations from the main queue, M-RVV allows subsequent instructions tobypass the load-commit unless a data dependency exists, making vector loadsnon-blocking. When there is data dependency, the scheduling can selectinstructions from other contexts, if there are any, to hide latency.SchedulingM-RVV uses a round-robin scheme to select instructions that are ready toexecute, skipping empty queues. The round-robin arbiters choose one fromthe main queues and one from the load-commit queues. A priority arbiterthen chooses a load-commit instruction over the other instruction in the mainqueue to retire loads and resolve data dependencies as soon as possible. Theround-robin scheduling also helps interleave independent vector instructions,which improves pipeline utilization and reduces data dependency stalls. Incontrast, in chained vector processors, Espasa and Valero [30] use a priorityscheme that favors chaining between vector instructions whenever possible.4.4 Integration with the Shared LSUThis section describes how MXM-RVV integrates with the shared LSU formemory access and disambiguation and explains how MXM-RVV handlescoherence and consistency issues that may arise in the shared LSU.The space-time diagrams in figure 3.4 illustrate that fence transactions onlyuse the lookup interfaces to query the shared LSU’s LSQ for conflicts withpreviously allocated entries. In addition, the read and write transactions usethe allocate interfaces to allocate entries in the LSQ. M-RVV implementsthese space-time diagrams for lookup and allocation in its sequencer usingthe state machine shown in figure 4.5, where vector memory instructions arereferred to as cx_mem instructions.For memory access, M-RVV pre-decodes the cx_mem instruction to extractthe base address and the memory access type (0 for read and 1 for write)and extracts the vector length from vector configuration instructions tocompute the end address from the base address. M-RVV uses these addressesand the memory access type to create an LSQ entry consisting of the baseand end addresses and the memory access type. It then polls the LSQ toresolve conflicts via the lookup interfaces, ensuring stalls occur only due totrue data dependencies. After polling, it allocates the LSQ entry for thevector memory operation through the allocate interfaces; the lookup and414.4. Integration with the Shared LSUFigure 4.5: State diagram for lookup and allocate interfaces in MXM-RVV.424.5. Programmingallocate interfaces are decoupled since the former are also used for customfences. In response to an allocation, the LSQ generates a tracking ID usedto initiate a memory request through the read-request or write-requestinterface. M-RVV later steers load data via the read-data interface using thetracking ID or forwards vector store data through the write-data interface tocomplete the write transaction. Note that initiating a memory write requesttriggers invalidations to the scalar write-through cache using the shared LSU.This approach guarantees coherence as M-RVV processors are non-coherentaccelerators that bypass the cache hierarchy (see section 3.3).To ensure consistency, MXM-RVV requires programmers to insert standardRISC-V FENCE instructions and accelerator-based custom fence instructionscx_fence (see section 3.2). The scalar host decodes standard fences toextract the device input and output status. If the device input status is setin the successor set, the scalar host delays dispatching vector loads until allscalar stores are complete; and if the device output status is set, it delaysvector stores until all scalar loads and stores are complete. Additionally,the scalar host decodes custom fences before forwarding them to M-RVV.If the subsequent memory operation in the custom fence is a scalar loador store, the host’s load-store unit is stalled by deasserting its ready signaluntil the custom fence retires. This retirement occurs after querying theLSQ until conflicts with previously allocated entries are resolved using thelookup interfaces and the base and end addresses and memory access typeextracted from the instruction encoding. Within MXM-RVV’s cores, memorydisambiguation of cx_mem instructions is implicit, though it can be madeexplicit if needed by also using cx_fence.4.5 ProgrammingThe MXM-RVV programming model follows the CIA ABI and API (seesection 2.4.1). This section provides an example of using this programmingmodel with the vectorized rgba2luma microbenchmark, which targets theMXM-RVV architecture and requires minimal changes to the original code.Additionally, it compares this to programming with double buffering.To fully exploit the MXM-RVV architecture, the software must partition tasksinto independent microthreads that can run concurrently. Listing 4 showsthe vectorized rgba2luma microbenchmark, converting 32-bit RGBA pixelsto 8-bit luminance pixels. This microbenchmark operates on a 128 × 128matrix and vectorizes across rows, where a row of bytes fills a vector register434.5. Programmingwithout stripmining. It microthreads across rows in an interleaved fashion,threading by vector units and state contexts with minimal code changes.Note that the vsetvli instruction does not update the register file with x0as its destination register. This design allows MXM-RVV to proceed andretire vector instructions without waiting for the scalar write-back, preventingpotential bottlenecks. Besides, the code shows the use of the standard fenceafter initializing the input and the custom fence before verifying the output.For comparison, listing 5 shows the prefetch code, which manually unrollsthe loop body once and uses additional vector registers to double buffer,i.e., prefetch into the vector register file. Double buffering leverages non-blocking vector loads for dynamic instruction scheduling. However, themanual unrolling is laborious and adds significant register pressure, moreso in a multiple buffering implementation, which increases code densitylinearly. In contrast, using the CIA API with multiple contexts simplifiesthe code. Furthermore, the same source code can run different MXM-RVVconfigurations of vector units and state contexts.444.5. Programming#include <ci.h> // CIA APIconst int N=2; // number of vector unitsconst int P=2; // state contexts per unitconst int T=(N*P); // total number of contextsconst int IMG_H=128; // image heightconst int IMG_W=128; // image widthvoid rgba2luma( uint8_t *luma, uint32_t *rgba,cia_sel_t *selectors, int num_microthreads ) {assert( IMG_W <= 4*RV_VLEN/32 ); // simplifying assumptionfor( int i=0; i < IMG_H/num_microthreads; ++i ) {for( int t=0; t<num_microthreads; ++t ) {/////////////////////////////////////////////////////// distribute row evenly across all cores X threadscia_select( selectors[t] );/////////////////////////////////////////////////////// set vector length and typeasm volatile( "vsetvli x0, %[REQ_VL], e16, m2, ta, mu":: [REQ_VL] "r" (IMG_W) );/////////////////////////////////////////////////////// fetch rowasm volatile( "vle32.v v4, (%0)":: "r"(rgba) );/////////////////////////////////////////////////////// compute on row// hidden: 15 vector insns. to process row r as follows// {R8[r],G8[r],B8[r],discard} = RGBA32[r]// LUMA16[r] = 66*R8[r] + 129*G8[r] + 25*B8[r] + 128// LUMA8[r] = LUMA16[r] >> 8rgba += IMG_W;luma += IMG_W;}}cia_select( CIA_LEGACY ); // CIA ABI rule}int benchmark( uint8_t *luma, uint32_t *rgba ) {asm volatile ("fence w, i");cia_sel_t selectors[T];for ( int i=0; i<T; ++i )selectors[i] = cia_open( MRVV_CIA_UUID, CIA_NO_VIRT, -1 );rgba2luma( luma, rgba, selectors, T );cia_select( selectors[0] );cx_fence_scalar_read( &luma[0], (char*)(&luma[IMG_H*IMG_W])-1 );for ( int i=0; i<T; ++i )cia_close( selectors[i] );return 0;}Listing 4: Vectorized RGBA to LUMA conversion using CIA’s ABI and API.454.5. Programmingvoid rgba2luma( uint8_t *luma, uint32_t *rgba,const uint32_t IMG_W, const uint32_t IMG_H ) {assert( IMG_W <= 4*RV_VLEN/32 ); // simplifying assumption 1assert( IMG_H&1 == 0 ); // simplifying assumption 2/////////////////////////////////////////////////////// set vector length and typeasm volatile( "vsetvli x0, %[REQ_VL], e16, m2, ta, mu":: [REQ_VL] "r" (IMG_W) );/////////////////////////////////////////////////////// prolog: prefetch row 1 to prime the loopasm volatile( "vle32.v v4, (%0)":: "r"(rgba) );for( uint32_t i = 0, vl; i < IMG_H; i += 2 ) {/////////////////////////////////////////////////////// set vector length and typeasm volatile( "vsetvli x0, %[REQ_VL], e16, m2, ta, mu":: [REQ_VL] "r" (IMG_W) );/////////////////////////////////////////////////////// prefetch row 2asm volatile( "vle32.v v24, (%0)":: "r"(rgba + IMG_W) );/////////////////////////////////////////////////////// compute on row 1// hidden: 15 vector instructions operating on row 1// {R8[j],G8[j],B8[j],discard} = RGBA32[j]// LUMA16[j] = 66*R8[j] + 129*G8[j] + 25*B8[j] + 128// LUMA8[j] = LUMA16[j] >> 8rgba += IMG_W;luma += IMG_W;/////////////////////////////////////////////////////// set vector length and typeasm volatile( "vsetvli x0, %[REQ_VL], e16, m2, ta, mu":: [REQ_VL] "r" (IMG_W) );/////////////////////////////////////////////////////// prefetch row 1asm volatile ( "vle32.v v4, (%0)":: "r"(rgba + IMG_W) );/////////////////////////////////////////////////////// compute on row 2 (duplicated code, similar to row 1)// hidden: 15 vector instructions operating on row 2// {R8[j],G8[j],B8[j],discard} = RGBA32[j]// LUMA16[j] = 66*R8[j] + 129*G8[j] + 25*B8[j] + 128// LUMA8[j] = LUMA16[j] >> 8rgba += IMG_W;luma += IMG_W;}}Listing 5: Vectorized RGBA to LUMA conversion using double buffering.46Chapter 5EvaluationThis chapter evaluates the implementation of the MXM-RVV+LSU SoC,which integrates multiple soft RISC-V multithreaded vector processors asinstruction-based accelerators with an external LSU that manages sharedmemory accesses for these accelerators. The chapter is organized as follows:section 5.1 defines the SoC architecture parameters, section 5.2 assesses theperformance of the MXM-RVV+LSU SoC using benchmarks, and section5.3 provides a detailed area breakdown and maximum clock frequency ofthe system. Section 5.4 discusses the scalability of the shared LSU. Finally,section 5.5 covers MXM-RVV+LSU SoC’s verification.5.1 Architecture ParametersThis section specifies MXM-RVV+LSU SoC’s architecture parameters.The MXM-RVV+LSU SoC is implemented on the AMD ZCU104 evaluationkit with 512M × 64b DDR4 SODIMM. Its 32-bit scalar host processor, Core-V CVA5, includes a 16 kB instruction cache and 64 kB write-through datacache. MXM-RVV’s main instruction queues have a depth of 64, while theload queues have a depth of 1. Saturn-V, MXM-RVV’s base vector core, isconfigured with a 64-bit width for execution and memory. Saturn-V constrainsthe ratio SEW/LMUL to 8, which results in a VLMAX of VLEN/8. Theshared LSU’s LSQ has a depth of 64.5.2 PerformanceThis section evaluates the performance of the MXM-RVV+LSU SoC, compar-ing its performance to Vicuna’s in section 5.2.1, detailing its per-benchmarkperformance in section 5.2.2, and discussing trade-offs when adding morecontexts in resource-constrained systems in section 5.2.3.475.2. PerformancePerformance is evaluated on the MXM-RVV+LSU SoC and Vicuna usingthree microbenchmarks in a bare-metal environment. These microbenchmarksinclude rgba2luma and median, developed in-house, and jacobi, adaptedfrom the RiVEC benchmark suite [83]. The rgba2luma benchmark converts32-bit RGBA pixels to 8-bit luminance pixels, jacobi performs five-pointstencil computations to solve Laplace’s differential equation on a squaredomain, and median uses a sorting network-based architecture for 3 × 3median filtering on 8-bit pixels. All the benchmarks operate on a 128× 128matrix, parallelizing computation across rows. The jacobi and medianbenchmarks, in particular, use input padding, resulting in a 130× 130 inputsize. In MXM-RVV, VLEN is set to 1024, which results in a VLMAX of1024/8 = 128 that allows rows of bytes to fill vector registers without requiringstripmining. All the benchmarks are written in C, using inline assembly forvector instructions, and cross-compiled with RISC-V GCC 13.2.0. TargetingCVA5 in the MXM-RVV+LSU SoC requires, in addition, Picolibc 1.7.3, alightweight C standard library designed for small embedded systems withlimited RAM.The benchmarks use configuration instructions, unit-stride loads and stores,basic vector integer arithmetic instructions, and some vector mask and permu-tation instructions. In addition, jacobi and rgba2luma use integer multiplyand shifting instructions. The jacobi benchmark adapts necessary floating-point operations to fixed-point as Saturn-V does not include floating-pointinstructions. Table 5.1 shows the vector instructions used across bench-marks and their respective counts. For each instruction, the SEW/LMULconfiguration is represented as eXmY , where X denotes SEW and Y , LMUL.5.2.1 Comparison to Prior WorkThis section compares Vicuna’s benchmark performance to MXM-RVV’sunder similar configurations.Vicuna provides utilities to compile applications with RISC-V GCC, simulatewith Verilator, and run on FPGA boards. It connects to the 32-bit Core-VIbex scalar processor and accesses memory through the scalar cache. Vicunauses on-chip BRAM for main memory, while Saturn-V and MXM-RVVuse external DRAM. BRAM provides lower latency and timing-predictablememory accesses, resulting in similar simulation and board execution results.Vicuna allows extensive parameterization with independent configuration ofexecution pipeline layout, execution pipeline width, memory interface width,485.2. PerformanceTable 5.1: Benchmark vector instruction count per output row.Benchmark Instruction Counte8m1 e16m2 e32m4rgba2luma 2 9 4A. ALU 1 9 3vadd 0 3 0vand 0 3 0vmul 0 3 0vnsrl 1 0 3B. Memory 1 0 1vle 0 0 1vse 1 0 0C. Configuration 1 1 0vsetvl 1 1 0jacobi 0 0 10A. ALU 0 0 8vadd 0 0 3vmv.v 0 0 2vslide1 0 0 2vsrl 0 0 1B. Memory 0 0 2vle 0 0 1vse 0 0 1C. Configuration 0 0 0vsetvl 0 0 0median 100 0 0A. ALU 98 0 0vmaxu 28 0 0vminu 28 0 0vmv.v 40 0 0vslide1 2 0 0B. Memory 2 0 0vle 1 0 0vse 1 0 0C. Configuration 0 0 0vsetvl 0 0 0495.2. Performancevector length, and vector register file read ports. For a fair comparison,Vicuna’s configuration closely matches Saturn-V’s. Vicuna’s pipeline layoutcontains two concurrent pipelines for ALU and LSU. This layout allowsmemory instructions to execute independently of compute instructions. Theexecution pipeline and memory interface widths are set to 64. The vector reg-ister file provides a 1024-bit vector length and three read ports for concurrentreads of vector ALU operands and store data. Unlike Saturn-V’s register file,which includes two write ports for concurrent writes of vector ALU resultsand load data, Vicuna’s register file only provides one. Furthermore, thecaches are disabled for both Vicuna and MXM-RVV.Table 5.2 presents performance results for unoptimized vector code andprefetching vector code that uses software double-buffering. Table 5.2 com-pares the average cycle count across ten independent runs on the MXM-RVV+LSU SoC’s board, configured with one vector unit and one statecontext, and Vicuna under hardware simulation. The baseline is the unopti-mized, non-prefetching vector code on MXM-RVV.On Vicuna, only rgba2luma outperforms the MXM-RVV vector baseline, withthe prefetching code achieving a 1.34× speedup and the non-prefetching code,similar to the 1 × 1 MXM-RVV configuration, providing a 1.29× speedup.In contrast, the jacobi and median benchmarks show no speedup, primarilydue to inefficient handling of unaligned loads and stores. When the baseaddress is unaligned, Vicuna converts unit-stride memory operations intoconstant-stride operations with a stride of 1, resulting in one transactionper element. Removing this conversion reduces the number of kilocycles inthe jacobi benchmark to 71.0 (1.01×) for the prefetching code and 71.1(1.01×) for the non-prefetching code. In the median benchmark, the numberof kilocycles decreases to 241.2 (1.00×) for the prefetching code and 243.6(0.99×) for the non-prefetching code. The remaining performance loss stemsfrom pipeline stalls and data hazards within Vicuna. Overall, under similarconfigurations, MXM-RVV using slower external DRAM is competitive withVicuna using on-chip BRAM. Additionally, scalar instructions can execute inparallel with longer-latency vector instructions on both Vicuna and MXM-RVV, minimizing the impact of differing scalar cores on performance. Thenext section shows how adding state contexts and vector units can furtherenhance MXM-RVV’s performance.505.2. PerformanceTable 5.2: Benchmark performance on MXM-RVV+LSU SoC with 1 vectorunit × 1 state context and Vicuna.Benchmark Config. kCycles (Speedup)MXM-RVV Vicunargba2luma unoptimized 92.9 (1.00×) 72.1 (1.29×)prefetch 70.3 (1.32×) 69.1 (1.34×)jacobi unoptimized 72.0 (1.00×) 76.3 (0.94×)prefetch 70.6 (1.02×) 77.7 (0.93×)median unoptimized 240.8 (1.00×) 265.5 (0.91×)prefetch 231.1 (1.04×) 244.7 (0.98×)5.2.2 Detailed Per-Benchmark PerformanceThis section compares MXM-RVV+LSU SoC’s benchmark performance acrossdifferent configurations of vector units and state contexts and shows how theseconfigurations further enhance performance. It also presents the performanceof two scalar-only versions (with cold and warm caches) and two vectorversions: an ideal version, which hides all memory latency, and a prefetchingversion with software double-buffering.Table 5.3 reports performance results of scalar code, ideal vector code, vectorcode using prefetching (double-buffering), and vector code on various MXM-RVV configurations, shown as m vector units × n state contexts. Scalarcode runs with a cold and warm (with 100 iterations) cache, while vectorconfigurations run with a cold cache. The baseline is the unoptimized vectorcode running on the 1×1 MXM-RVV configuration. For other configurations,the vector code is microthreaded across rows in an interleaved manner. Table5.3 compares the average cycle count and vector ALU utilization acrossten independent runs. Vector ALU utilization refers to the ratio of theALU’s busy cycles to its total cycles (busy+idle), with 100% ideal utilizationindicating fully hidden memory latency.The ideal cycle count on a single Saturn-V vector core is calculated as follows:CCideal = rows×∑X∈{8,16,32,64}row sizeexecution widthX·AICe{X}m{X/8} (5.1)In equation 5.1, CC represents the cycle count, and AIC denotes the in-struction count for ALU instructions, as detailed in table 5.1. For all thebenchmarks, the number of rows and the row size are 128 elements, while the515.2. Performancevector execution width on Saturn-V is 64 bits. The ideal cycle count includesonly ALU instruction cycles, assuming fully hidden memory latency andomitting cycles for configuration instructions, which execute in a single cycle.This ideal cycle count sums ALU instruction cycles because Saturn-V doesnot support chaining and has a single ALU pipeline, which is the bottleneck.For the rgba2luma benchmark, the scalar code achieves 0.80 instructionsper cycle (IPC) with a cold cache and 0.99 with a warm cache. The vectorbaseline is faster than the cold scalar code by 5.0×. On a single vectorcore, ideal performance indicates a 1.46× speedup over the baseline, whileprefetching only offers a 1.32× speedup. Using four contexts in MXM-RVV,the vector code reaches a 1.36× speedup. With two vector cores in MXM-RVV, it achieves a 2.66× speedup with four and eight contexts compared tothe baseline. Note that ALU utilization increases as performance improvesin vector configurations.In jacobi, similar trends emerge. The vector baseline is nearly 5.0× fasterthan the cold scalar code with an IPC of 0.58. The warm scalar code achievesa 0.89 IPC. On a single vector core, ideal performance indicates a 1.10×speedup, while prefetching provides a 1.02× speedup. With four contextsin MXM-RVV, vector code performance approaches ideal, reaching a 1.07×speedup. Configurations with two vector cores nearly double the performanceof those with one core as they achieve up to 2.04× speedup over the baseline.The median benchmark benefits the most from vectorization, with the vectorbaseline improving on the scalar code by 23×; cold and warm scalar codesachieve an IPC of 0.6. The benchmark uses a sorting network with lessdata-dependent control flow, making it suitable for vectorization. Idealperformance on a single vector core indicates a 1.20× speedup over thebaseline. However, prefetching offers only a marginal 1.04× speedup. Thismarginal improvement is due to ALU data hazards, as the benchmark requiresover 90 ALU instructions per output row. MXM-RVV reduces these hazardsand memory latency by interleaving instructions across multiple contexts; thevector code achieves a 1.19× speedup with four contexts. With two vectorcores, MXM-RVV reaches up to 2.33× speedup.In summary, the vector code significantly improves over the scalar code asit exploits data-level parallelism. For rgba2luma, jacobi and median, thevector code using software double buffering reaches 0.90×, 0.93×, and 0.87×,respectively, of an ideal performance on a single vector core. Using two con-texts in MXM-RVV (equivalent to hardware double buffering) improves theseratios to 0.91×, 0.93×, and 0.97×. With four contexts, these ratios increase525.2. PerformanceTable 5.3: Benchmark performance on MXM-RVV+LSU SoC.Benchmark Config. kCycles (Speedup) % Util.rgba2lumascalar (c) 468.9 (0.20×) 0scalar (w) 380.1 (0.24×) 0ideal 63.5 (1.46×) 100prefetch 70.3 (1.32×) 94.21 × 1 92.9 (1.00×) 71.41 × 2 69.6 (1.33×) 95.61 × 4 68.3 (1.36×) 97.41 × 8 71.3 (1.30×) 93.32 × 1 47.0 (1.98×) 71.4, 71.52 × 2 35.4 (2.62×) 95.2, 95.42 × 4 35.0 (2.66×) 96.7, 96.82 × 8 34.9 (2.66×) 97.7, 96.9jacobiscalar (c) 341.9 (0.21×) 0scalar (w) 222.4 (0.32×) 0ideal 65.5 (1.10×) 100prefetch 70.6 (1.02×) 93.51 × 1 72.0 (1.00×) 92.31 × 2 70.3 (1.02×) 94.61 × 4 67.4 (1.07×) 98.81 × 8 70.5 (1.02×) 94.22 × 1 37.1 (1.94×) 91.7, 91.32 × 2 36.3 (1.98×) 93.3, 93.32 × 4 35.2 (2.04×) 96.5, 96.22 × 8 36.0 (2.00×) 94.8, 94.6medianscalar (c) 5729.8 (0.04×) 0scalar (w) 5689.8 (0.04×) 0ideal 200.7 (1.20×) 100prefetch 231.1 (1.04×) 87.01 × 1 240.8 (1.00×) 83.71 × 2 207.5 (1.16×) 97.21 × 4 202.7 (1.19×) 99.61 × 8 203.3 (1.18×) 99.42 × 1 121.7 (1.98×) 83.5, 83.32 × 2 105.4 (2.29×) 96.6, 96.52 × 4 103.6 (2.33×) 98.5, 98.42 × 8 104.9 (2.30×) 97.6, 97.5535.2. Performancefurther to 0.93×, 0.97×, and 0.99×. Beyond four contexts, performance andALU utilization plateau as structural conflicts to commit load instructionsarise. Furthermore, with two vector cores, MXM-RVV nearly doubles theperformance of single vector unit configurations. All these configurations runthe same source code with minimal changes to the original code.5.2.3 Trade-Offs Between Vector Length and MultipleContextsThis section discusses trade-offs between vector length and multiple contextsin systems with a constrained number of states.MXM-RVV introduces additional states with each context, including vectorinstruction queues, vector register files, and vector CSRs. In a resource-constrained system, the number of states can remain constant by dividingVLEN by n when adding n contexts.Table 5.4 presents MXM-RVV’s benchmark performance with 1 vector unit× n contexts and VLEN set to 1024/n. For rgba2luma, using four contextsand a VLEN of 256 achieves the highest speedup of 1.27× over the baseline.In median, the highest speedup, 1.13×, results from the configuration withtwo contexts and a VLEN of 512. jacobi starts with a high ALU utilizationof 92.3% using a single context and a VLEN of 1024. However, adding morecontexts reduces performance and utilization. Except for jacobi, addingcontexts proves more beneficial than using long vectors. Long vectors increasememory latency, which can degrade performance. Short vectors with multiplecontexts retain some advantages of long vectors while effectively hidingmemory latency. Moreover, Saturn-V and MXM-RVV cannot adjust theSEW/LMUL ratio like other RVV cores, which may be beneficial on jacobito recover lost performance in resource-constrained systems.545.3. Area Breakdown and Clock FrequencyTable 5.4: Benchmark performance on MXM-RVV+LSU SoC with 1 vectorunit × n state contexts × VLEN.Benchmark Config. kCycles (Speedup) % Util.rgba2luma1 × 1 × 1024 92.9 (1.00×) 71.41 × 2 × 512 73.1 (1.27×) 91.31 × 4 × 256 73.0 (1.27×) 92.21 × 8 × 128 76.2 (1.22×) 89.7jacobi1 × 1 × 1024 72.0 (1.00×) 92.31 × 2 × 512 73.7 (0.98×) 90.31 × 4 × 256 84.3 (0.85×) 78.81 × 8 × 128 167.2 (0.43×) 39.5median1 × 1 × 1024 240.8 (1.00×) 83.71 × 2 × 512 213.4 (1.13×) 94.61 × 4 × 256 225.2 (1.07×) 89.71 × 8 × 128 424.1 (0.57×) 47.65.3 Area Breakdown and Clock FrequencyThis section details the area breakdown and maximum clock frequency of theMXM-RVV+LSU SoC.Table 5.5 summarizes the area breakdown of primary blocks in the MXM-RVV+LSU SoC, configured with 2 vector units × n contexts and VLEN set to1024. In MXM-RVV, doubling the number of contexts increases the numberof lookup tables (LUTs) and flip-flops (FFs), though not proportionally.However, the block RAMs (BRAMs) implementing vector register files doubleas the configuration scales from one to two contexts; beyond four contexts,the implementation switches to larger ultraRAMs (URAMs) to reduce BRAMpressure. The shared LSU uses up to 5k LUTs and 6k FFs, primarily for theLSQ, while the CVA5 CPU requires 3k LUTs, 2k FFs, and 29 BRAMs forits caches. Among all blocks, the AXI interconnect and memory interfacegenerator (MIG) consume the most resources: the interconnect requires 11kLUTs, 8k FFs, and 48 BRAMs, and the MIG uses 15k LUTs, 21k FFs, and 26BRAMs. The overall system, consisting of primary blocks 2× 8 A, B, C, D,and E, consumes about 61k LUTs out of 230k available, 55k FFs out of 460k,135 BRAMs out of 312, 16 URAMs out of 96, and 23 DSPs out of 1728.Table 5.6 reports the maximum clock frequency for various MXM-RVVconfigurations, shown as m vector units × n state contexts. Across all con-figurations, the MXM-RVV+LSU SoC can achieve a maximum frequency ofabout 200MHz on the AMD ZCU104 evaluation kit with 16 nm programmable555.4. ScalabilityTable 5.5: Area breakdown for 2×n MXM-RVV+LSU SoC (64-bit executionand memory widths, VLEN=1024).Block LUT FF BRAM URAM DSPA. MXM-RVV2 × 1 6286, 6392 5036, 5028 8, 8 0, 0 8, 82 × 2 7435, 7562 6319, 6309 16, 16 0, 0 8, 82 × 4 9660, 9782 7135, 7137 0, 0 8, 8 8, 82 × 8 13 239, 13 320 9099, 9088 0, 0 8, 8 8, 8B. LSU 5253 5934 0 0 0B.1. LSQ 2710 4252 0 0 0C. CVA5 2996 2185 29 0 4D. AXI Interconnect 11 010 8094 48 0 0E. DDR4 MIG 14 932 21 023 25.5 0 3Overall System 60 750 55 423 134.5 16 23ZCU104 230 400 460 800 312 96 1728Table 5.6: Maximum frequency for m × n MXM-RVV+LSU SoC (64-bitexecution and memory widths, VLEN=1024).Config. Max Freq. (MHz)1 × 1 214.51 × 2 208.21 × 4 202.11 × 8 206.52 × 1 205.02 × 2 210.72 × 4 212.32 × 8 207.8logic. For comparison, the fastest Vicuna configuration runs at 100MHz ona Xilinx 7 Series FPGA using 28 nm technology.5.4 ScalabilityThis section evaluates the scalability of the shared LSU.The scalability of the shared LSU is evaluated by increasing vector unitsconnected to it, with an LSQ depth of 64. Table 5.7 presents FF and LUTcounts of the shared LSU after synthesis for configurations ranging fromtwo to 128 vector units. Figure 5.1 illustrates regression lines for both FFand LUT counts, each achieving a coefficient of determination R2 = 1.00,indicating linear growth and hinting at the scalability of the shared LSU.565.4. Scalability0 20 40 60 80 100 120No. of Vector Units5000750010000125001500017500200002250025000FFdata pointsy = 151.87x + 5009.82, R² = 1.00(a) FF count.0 20 40 60 80 100 120No. of Vector Units1000020000300004000050000LUTdata pointsy = 369.63x + 5358.51, R² = 1.00(b) LUT count.Figure 5.1: Scalability of the shared LSU with 64-entry LSQ and increasingvector units.575.5. VerificationTable 5.7: Scalability of the shared LSU with 64-entry LSQ and increasingvector units.No. of Vector Units LUT FF2 5271 60794 5693 68318 6229 831216 7327 11 28132 9928 17 20264 14 768 29 032128 24 427 52 6585.5 VerificationThis section describes the verification of the MXM-RVV+LSU SoC.MXM-RVV+LSU is verified through integration and unit tests simulatedwith Verilator 5.004. Integration testing uses the benchmarks presented inthis chapter. For unit testing, a repository, adapted from Ara’s RISC-Vtests [84], is maintained. The repository tests each vector instruction in aseparate file, covering masking and various configurations of SEW and LMUL.The vector instructions tested include vector logic operations (vand, vor,vxor), integer arithmetic operations (vadd, vmax, vmin, vmul, vsll, vsrl),permutation operations (vslide1down, vslide1up), unit-stride loads andstores (vle, vse), and configuration instructions (vsetvl). Additional testingis required before relying upon the instructions not verified yet.58Chapter 6ConclusionThis chapter concludes the thesis, summarizing key findings in section 6.1,outlining limitations in section 6.2, and suggesting future work in section 6.3.6.1 SummaryThis section provides a summary of this thesis and its main findings.Modern SoCs are shared-memory heterogeneous systems, combining general-purpose processor cores with a growing number of specialized acceleratorswhose performance and efficiency rely on efficient memory systems. Modernapplications like deep learning are data-intensive, and accelerators typicallyaccess shared memory directly by address to circumvent data throughputbottlenecks. However, direct access to shared memory introduces coherenceand consistency issues, and addressing these in each accelerator adds signif-icant design complexity. Moreover, accelerator memory units may requirefeatures like automatic dependence checking for performance, data alignmentfor efficient access, or address translation services for a unified virtual memoryspace. This thesis proposed a novel shared LSU that centralizes memoryaccess and can address these challenges collectively for all accelerators.The primary requirements for this shared LSU included hardware reuse,ease of programming, and high performance. The shared LSU facilitateshardware reuse with a platform-independent design written in standardSystemVerilog HDL and using standard communication protocols: AXIStream interfaces connect the shared LSU to accelerators, and an AXIMemory Mapped interface connects it to external memory. The shared LSUoffers a simplified programming model with standard RISC-V fences andaccelerator-based, custom, finer-grain fences to ensure consistency, allowingmemory operations between fences to reorder freely. For coherence, theshared LSU handles it with a hardware mechanism that performs coherenceactions only when necessary, without programmer intervention. To assess596.2. Limitationsperformance, this thesis integrated the shared LSU with a novel multicoreand multithreaded soft RISC-V vector architecture, implementing each vectorcore as a CX unit for simplified multithreading and multicore capabilities.The shared LSU seamlessly integrates with the vector cores in an SoCimplemented on the AMD ZCU104 evaluation kit. In this system, a single-threaded program running on a single vector core nears ideal performance by0.93× on the rgba2luma benchmark, effectively hiding memory latency andpipeline stalls. Using two vector cores further improves performance by 2×,all with minimal changes to software.6.2 LimitationsThis section outlines the current limitations of the shared LSU.The shared LSU specifically targets instruction-based accelerators integratedinto the processor’s instruction set architecture. Instruction-based acceleratorsretain the notion of fetching and decoding (memory) instructions needed toaddress consistency and coherence. The shared LSU is designed for systemswith scalar write-through caches, eliminating the need to flush the acceleratordata region before the accelerator execution, as required in write-back cachesystems. It operates in a bare-metal environment with a 32-bit physicaladdress space and does not support virtual memory. Finally, the shared LSUonly provides unit-stride memory accesses and does not yet handle morecomplex memory access patterns, such as strided or indexed accesses.6.3 Future WorkThis section suggests future work on the shared LSU and MXM-RVV.Future work on the shared LSU will include integrating accelerators besidesvector processors. Supporting other accelerators will require adding AXIStream interfaces to connect them to the shared LSU, and more than twoaccelerators will be necessary to assess the shared LSU’s scalability. Also,supporting other memory interfaces, such as Wishbone, could facilitateintegrating the shared LSU with more SoCs. Table 5.5 shows that theregister-based LSQ consumes most of the resources in the shared LSU, soswitching to a RAM-based design, if possible, could improve the sharedLSU area. The shared LSU currently uses round-robin arbitration withineach stream set across accelerators. Alternative arbitration schemes, such606.3. Future Workas priority arbitration, could be considered to prioritize critical accelerators.Also, to address scenarios where some accelerators flood the shared LSU withrequests, causing others to starve, the arbitration scheme could be adaptedto penalize accelerators flooding the shared LSU, ensuring requests fromstarved accelerators are accepted. To address the scalability of the sharedLSU and reduce congestion and contention due to all accelerators accessing asingle unit, shared LSUs could be distributed to form a network, each servinga limited group of accelerators. Furthermore, the shared LSU will requirea memory interface wider than 64 bits to increase its throughput. Finally,implementing an IOMMU in the shared LSU will enable address translation.For MXM-RVV, future work will involve implementing missing layers inSaturn-V and fully testing existing ones. Additionally, more powerful bench-marks will be required to stress the MXM-RVV architecture. Finally, addingsupport for other RISC-V hosts can be explored, which will involve modifyingthem to add the CXU-LI port without requiring changes to MXM-RVV.61Bibliography[1] Brandon Freiberger, Joseph Maheshe, Jan Gray, and Guy G.F. Lemieux.CIA: Breaking the Opcode Bottleneck with Composable Instructions forAccelerators. Submitted for publication.[2] Caroline White. Design and implementation of RVV-Lite: a layeredapproach to the official RISC-V vector ISA. Master’s thesis, Universityof British Columbia, 2023.[3] John L. Hennessy and David A. Patterson. A new golden age for computerarchitecture. Communications of the ACM, 62(2):48–60, January 2019.[4] Hadi Esmaeilzadeh, Emily Blem, Renée St. Amant, Karthikeyan Sankar-alingam, and Doug Burger. Dark silicon and the end of multicore scaling.In 2011 38th Annual International Symposium on Computer Architecture(ISCA), pages 365–376, June 2011.[5] C. Cascaval, S. Chatterjee, H. Franke, K. J. Gildea, and P. Pattnaik. Ataxonomy of accelerator architectures and their programming models.IBM Journal of Research and Development, 54(5), September 2010.[6] Biagio Peccerillo, Mirco Mannino, Andrea Mondelli, and Sandro Bar-tolini. A survey on hardware accelerators: Taxonomy, trends, challenges,and perspectives. Journal of Systems Architecture, 129(C), August 2022.[7] John L. Hennessy, David A. Patterson, and Krste Asanović. ComputerArchitecture: A Quantitative Approach. Morgan Kaufmann/Elsevier,5th edition, 2012.[8] Kourosh Gharachorloo. Memory Consistency Models for Shared-MemoryMultiprocessors. PhD thesis, Standford University, 1996.[9] ARM. ARM Architecture Reference Manual ARMv7-A andARMv7-R edition. https://developer.arm.com/documentation/ddi0406/cb/Appendixes/Barrier-Litmus-Tests/Introduction/Overview-of-memory-consistency?lang=en, September 2024.62Bibliography[10] Sarita V. Adve, Mark D. Hill, Barton P. Miller, and Robert H. B. Netzer.Detecting data races on weak memory systems. SIGARCH ComputerArchitecture News, 19(3):234–243, April 1991.[11] Thomas Benz, Michael Rogenmoser, Paul Scheffler, Samuel Riedel,Alessandro Ottaviano, Andreas Kurth, Torsten Hoefler, and Luca Benini.A High-Performance, Energy-Efficient Modular DMA Engine Architec-ture. IEEE Transactions on Computers, 73(1):263–277, January 2024.[12] Jan Gray, Guy G.F. Lemieux, et al. Draft Proposed RISC-V ComposableCustom Extensions Specification. https://github.com/grayresearch/CX/blob/main/spec/spec.pdf, April 2024.[13] Aaron Severance and Guy Lemieux. VENICE: A compact vector pro-cessor for FPGA applications. In 2012 International Conference onField-Programmable Technology, pages 261–268, December 2012.[14] Matthew Naylor, Paul J Fox, A Theodore Markettos, and Simon WMoore. Managing the FPGA memory wall: Custom computing orvector processing? In 2013 23rd International Conference on Fieldprogrammable Logic and Applications, pages 1–6, September 2013.[15] Aaron Severance, Joe Edwards, Hossein Omidian, and Guy Lemieux.Soft vector processors with streaming pipelines. In Proceedings of the2014 ACM/SIGDA international symposium on Field-programmable gatearrays, pages 117–126. ACM, February 2014.[16] Ian Kuon, Russell Tessier, and Jonathan Rose. FPGA Architecture:Survey and Challenges. Foundations and Trends® in Electronic DesignAutomation, 2(2):135–253, 2008.[17] Jason Yu and Guy Lemieux. A Case for Soft Vector Processors in FPGAs.In 2007 International Conference on Field-Programmable Technology,pages 341–344, December 2007.[18] Lana Josipovic, Philip Brisk, and Paolo Ienne. An Out-of-Order Load-Store Queue for Spatial Computing. In 2017 IEEE 25th Annual Interna-tional Symposium on Field-Programmable Custom Computing Machines(FCCM), pages 134–134, April 2017.[19] Davide Giri, Paolo Mantovani, and Luca P. Carloni. Accelerators andCoherence: An SoC Perspective. IEEE Micro, 38(6):36–45, November2018.63Bibliography[20] Krste Asanović, Rimas Avizienis, Jonathan Bachrach, Scott Beamer,David Biancolin, Christopher Celio, Henry Cook, Daniel Dabbelt, JohnHauser, Adam Izraelevitz, Sagar Karandikar, Ben Keller, Donggyu Kim,John Koenig, Yunsup Lee, Eric Love, Martin Maas, Albert Magyar,Howard Mao, Miquel Moreto, Albert Ou, David A. Patterson, BrianRichards, Colin Schmidt, Stephen Twigg, Huy Vo, and Andrew Water-man. The Rocket Chip Generator. Technical Report UCB/EECS-2016-17,University of California, Berkeley, Apr 2016.[21] May Young. Dynamic race detection for non-coherent accelerators. Mas-ter’s thesis, University of British Columbia, 2020.[22] RISC-V Foundation. The RISC-V Instruction Set Manual Volume I:Unprivileged Architecture, December 2019.[23] Yunsup Lee, Andrew Waterman, Rimas Avizienis, Henry Cook, ChenSun, Vladimir Stojanović, and Krste Asanović. A 45nm 1.3GHz 16.7double-precision GFLOPS/W RISC-V processor with vector accelerators.In ESSCIRC 2014 - 40th European Solid State Circuits Conference(ESSCIRC), pages 199–202, September 2014.[24] Brandon Freiberger, Jan Gray, and Guy Lemieux. A Simple Plan: AnAPI and ABI for Managing Multiple Distinct Sets of Custom Extensions.In RISC-V Summit North America, October 2024.[25] Brandon Freiberger. An API and ABI for Managing Multiple DistinctSets of Custom Instructions. Master’s thesis, University of BritishColumbia, 2024.[26] Richard M. Russell. The CRAY-1 computer system. Communicationsof the ACM, 21(1):63–72, January 1978.[27] Eugene Brooks. The attack of the killer micros. Teraflop ComputingPanel, Supercomputing’89, 1989.[28] Krste Asanovic. Vector microprocessors. PhD thesis, University ofCalifornia, Berkeley, 1998.[29] R. Espasa and M. Valero. Decoupled vector architectures. In Proceed-ings. Second International Symposium on High-Performance ComputerArchitecture, pages 281–290. IEEE Comput. Soc. Press, 1996.64Bibliography[30] R. Espasa and M. Valero. Multithreaded vector architectures. In Pro-ceedings Third International Symposium on High-Performance ComputerArchitecture, pages 237–248, February 1997.[31] R. Espasa, M. Valero, and J.E. Smith. Out-of-order vector architectures.In Proceedings of 30th Annual International Symposium on Microarchi-tecture, pages 160–170. IEEE Comput. Soc, 1997.[32] Christoforos Kozyrakis. Scalable vector media-processors for embeddedsystems. PhD thesis, University of California, Berkeley, 2002.[33] Jason Yu, Guy Lemieux, and Christpher Eagleston. Vector processingas a soft-core CPU accelerator. In Proceedings of the 16th InternationalACM/SIGDA Symposium on Field-Programmable Gate Arrays, pages222–232. ACM, February 2008.[34] Peter Yiannacouras, J. Gregory Steffan, and Jonathan Rose. VESPA:portable, scalable, and flexible FPGA-based vector processors. In Pro-ceedings of the 2008 International Conference on Compilers, Architecturesand Synthesis for Embedded Systems, pages 61–70. ACM, October 2008.[35] Aaron Severance and Guy G.F. Lemieux. Embedded supercomputing inFPGAs with the VectorBlox MXP Matrix Processor. In 2013 Interna-tional Conference on Hardware/Software Codesign and System Synthesis(CODES+ISSS), pages 1–10. IEEE, September 2013.[36] RISC-V International. RISC-V "V" Vector Extension, 2021.[37] David Patterson and Andrew Waterman. SIMD Instruc-tions Considered Harmful. https://www.sigarch.org/simd-instructions-considered-harmful/, September 2017.[38] Imad Al Assir, Mohamad El Iskandarani, Hadi Rayan Al Sandid, andMazen A. R. Saghir. Arrow: A RISC-V Vector Accelerator for MachineLearning Inference. arXiv, July 2021.[39] V. Naveen Chander and Kuruvilla Varghese. A Soft RISC-V VectorProcessor for Edge-AI. In 2022 35th International Conference on VLSIDesign and 2022 21st International Conference on Embedded Systems(VLSID), pages 263–268, February 2022.[40] Francesco Minervini, Oscar Palomar, Osman Unsal, Enrico Reggiani,Josue Quiroga, Joan Marimon, Carlos Rojas, Roger Figueras, Abra-ham Ruiz, Alberto Gonzalez, Jonnatan Mendoza, Ivan Vargas, César65BibliographyHernandez, Joan Cabre, Lina Khoirunisya, Mustapha Bouhali, JulianPavon, Francesc Moll, Mauro Olivieri, Mario Kovac, Mate Kovac, LeonDragic, Mateo Valero, and Adrian Cristal. Vitruvius+: An Area-EfficientRISC-V Decoupled Vector Coprocessor for High Performance ComputingApplications. ACM Transactions on Architecture and Code Optimization,20(2):1–25, June 2023.[41] Intel Labs. IntelLabs/riscv-vector. https://github.com/IntelLabs/riscv-vector, October 2024.[42] Reconfigurable Computing Lab. I2SRV32-V-v1. https://github.com/RClabiisc/I2SRV32-V-v1, October 2024.[43] Martin Riis. RISC-V-Vector-Processor. https://github.com/martinriis/RISC-V-Vector-Processor, October 2024.[44] Nikola Kovacevic. RISC-V-vector-processor. https://github.com/Nikola2444/RISC-V-vector-processor, October 2024.[45] Andes Technology. RISC-V:NX27V. http://www.andestech.com/en/products-solutions/andescore-processors/riscv-nx27v/, Septem-ber 2024.[46] Andes Technology. RISC-V: AX45MPV. https://www.andestech.com/en/products-solutions/andescore-processors/riscv-ax45mpv/,September 2024.[47] SiFive. SiFive Performance P270. https://www.sifive.com/cores/performance-p270, September 2024.[48] SiFive. SiFive Intelligence X280. https://www.sifive.com/cores/intelligence-x280, September 2024.[49] Semidynamics. Semidynamics Vector Unit - Only 100% customisableRISC-V Vector Unit. https://semidynamics.com/en/technology/vector-unit, September 2024.[50] Michaela D. Platzer and Peter P. Puschner. Vicuna: A Timing-Predictable RISC-V Vector Coprocessor for Scalable Parallel Computa-tion. In Euromicro Conference on Real-Time Systems, July 2021.[51] SpinalHDL. VexRiscv. https://github.com/SpinalHDL/VexRiscv, Oc-tober 2024.66Bibliography[52] Shvetank Prakash, Tim Callahan, Joseph Bushagour, Colby Banbury,Alan V. Green, Pete Warden, Tim Ansell, and Vijay Janapa Reddi.CFU Playground: Full-Stack Open-Source Framework for Tiny MachineLearning (TinyML) Acceleration on FPGAs. In 2023 IEEE InternationalSymposium on Performance Analysis of Systems and Software (ISPASS),page 157–167. IEEE, April 2023.[53] Matheus Cavalcante, Fabian Schuiki, Florian Zaruba, Michael Schaffner,and Luca Benini. Ara: A 1-GHz+ Scalable and Energy-Efficient RISC-VVector Processor With Multiprecision Floating-Point Support in 22-nmFD-SOI. IEEE Transactions on Very Large Scale Integration (VLSI)Systems, 28(2):530–543, February 2020.[54] Matteo Perotti, Matheus Cavalcante, Renzo Andri, Lukas Cavigelli, andLuca Benini. Ara2: Exploring Single- and Multi-Core Vector Processingwith an Efficient RVV 1.0 Compliant Open-Source Processor. IEEETransactions on Computers, 73(7):1822–1836, July 2024.[55] F. Zaruba and L. Benini. The Cost of Application-Class Processing:Energy and Performance Analysis of a Linux-Ready 1.7-GHz 64-BitRISC-V Core in 22-nm FDSOI Technology. IEEE Transactions on VeryLarge Scale Integration (VLSI) Systems, 27(11):2629–2640, November2019.[56] MohammadHossein AskariHemmat, Théo Dupuis, Yoan Fournier, NizarEl Zarif, Matheus Cavalcante, Matteo Perotti, Frank Gürkaynak, LucaBenini, François Leduc-Primeau, Yvon Savaria, and Jean-Pierre David.Quark: An Integer RISC-V Vector Processor for Sub-Byte QuantizedDNN Inference. In 2023 IEEE International Symposium on Circuits andSystems (ISCAS), pages 1–5, May 2023.[57] Matteo Perotti and Frank Gürkaynak. Personal Communication, January2025.[58] Théo Dupuis, Yoan Fournier, MohammadHossein AskariHemmat,Nizar El Zarif, François Leduc-Primeau, Jean Pierre David, and YvonSavaria. Sparq: A Custom RISC-V Vector Processor for Efficient Sub-Byte Quantized Inference. In 2023 21st IEEE Interregional NEWCASConference (NEWCAS), pages 1–5, June 2023.[59] Elisabeth Humblet, Théo Dupuis, Yoan Fournier, MohammadHosseinAskariHemmat, François Leduc-Primeau, Jean Pierre David, and Yvon67BibliographySavaria. MSPARQ: A RISC-V Vector Processor Array Optimized forLow-Resolution Neural Networks. In 2024 IEEE 67th InternationalMidwest Symposium on Circuits and Systems (MWSCAS), pages 464–468, August 2024.[60] Jonathan Balkind, Michael McKeown, Yaosheng Fu, Tri Nguyen, YanqiZhou, Alexey Lavrov, Mohammad Shahrad, Adi Fuchs, Samuel Payne,Xiaohua Liang, Matthew Matl, and David Wentzlaff. OpenPiton: AnOpen Source Manycore Research Framework. In Proceedings of theTwenty-First International Conference on Architectural Support for Pro-gramming Languages and Operating Systems, pages 217–232, AtlantaGeorgia USA, March 2016. ACM.[61] Chuanning Wang, Chao Fang, Xiao Wu, Zhongfeng Wang, and Jun Lin.A Scalable RISC-V Vector Processor Enabling Efficient Multi-PrecisionDNN Inference. In 2024 IEEE International Symposium on Circuits andSystems (ISCAS), pages 1–5, May 2024.[62] CHIPS Alliance. T1. https://github.com/chipsalliance/t1, Octo-ber 2024.[63] Jerry Zhao. The Saturn Vector Unit: Design of a Fully CompliantOpen-Source RISC-V Vector Unit. In 2024 10th Open Source DigitalDesign Conference (ORCONF), September 2024.[64] Alon Amid, David Biancolin, Abraham Gonzalez, Daniel Grubb, SagarKarandikar, Harrison Liew, Albert Magyar, Howard Mao, Albert Ou,Nathan Pemberton, Paul Rigge, Colin Schmidt, John Wright, JerryZhao, Yakun Sophia Shao, Krste Asanović, and Borivoje Nikolić. Chip-yard: Integrated Design, Simulation, and Implementation Frameworkfor Custom SoCs. IEEE Micro, 40(4):10–21, July 2020.[65] Henry Cook. Productive Design of Extensible On-Chip Memory Hierar-chies. PhD thesis, University of California, Berkeley, 2016.[66] SiFive. SiFive TileLink Specification 1.9.3. https://www.sifive.com/document-file/tilelink-spec-1.9.3, October 2024.[67] OpenHW Group. OpenHW Group Specification: Core-V eXtensioninterface (CV-X-IF) - Development — Core-V eXtension interface (CV-X-IF) v1.0.0-rc.3-dev.3 documentation. https://docs.openhwgroup.org/projects/openhw-group-core-v-xif/en/latest/, September 2024.68Bibliography[68] Eric Matthews, Zavier Aguila, and Lesley Shannon. Evaluating thePerformance Efficiency of a Soft-Processor, Variable-Length, Parallel-Execution-Unit Architecture for FPGAs Using the RISC-V ISA. In 2018IEEE 26th Annual International Symposium on Field-ProgrammableCustom Computing Machines (FCCM), pages 1–8, April 2018.[69] SiFive. SiFive Vector Coprocessor Interface (VCIX) SoftwareSpecification 1.1. https://www.sifive.com/document-file/sifive-vector-coprocessor-interface-vcix-software, Octo-ber 2024.[70] Semidynamics. Semidynamics - OVI technology. https://semidynamics.com/en/technology/ovi, September 2024.[71] Eric S. Chung, James C. Hoe, and Ken Mai. CoRAM: an in-fabricmemory architecture for FPGA-based computing. In Proceedings ofthe 19th ACM/SIGDA International Symposium on Field-PogrammableGate Arrays, pages 97–106. ACM, February 2011.[72] M. Berry, D. Chen, P. Koss, D. Kuck, S. Lo, Y. Pang, L. Pointer,R. Roloff, A. Sameh, E. Clementi, S. Chin, D. Schneider, G. Fox,P. Messina, D. Walker, C. Hsiung, J. Schwarzmeier, K. Lue, S. Orszag,F. Seidl, O. Johnson, R. Goodrum, and J. Martin. The Perfect ClubBenchmarks: Effective Performance Evaluation of Supercomputers. TheInternational Journal of Supercomputing Applications, 3(3):5–40, 1989.[73] R. Giladi and N. Ahitav. SPEC as a performance evaluation measure.Computer, 28(8):33–42, August 1995.[74] Abdallah Cheikh, Stefano Sordillo, Antonio Mastrandrea, FrancescoMenichelli, Giuseppe Scotti, and Mauro Olivieri. Klessydra-T: DesigningVector Coprocessors for Multithreaded Edge-Computing Cores. IEEEMicro, 41(2):64–71, March 2021. Conference Name: IEEE Micro.[75] Onur Mutlu. Lecture notes in Fined-Grained Multithreading. ETHZürich, Fall 2020.[76] May Young, Alan J. Hu, and Guy G. F. Lemieux. Cache Abstractionfor Data Race Detection in Heterogeneous Systems with Non-coherentAccelerators. ACM Transactions on Embedded Computing Systems,22(1):1–25, January 2023.[77] ARM. AMBA AXI Protocol Specification. https://developer.arm.com/documentation/ihi0022/k/?lang=en, September 2024.69Bibliography[78] ARM. AMBA AXI-Stream Protocol Specification. https://developer.arm.com/documentation/ihi0051/b/?lang=en, September 2024.[79] I. Park, Chong Liang Ooi, and T.N. Vijaykumar. Reducing design com-plexity of the load/store queue. In Proceedings. 36th Annual IEEE/ACMInternational Symposium on Microarchitecture, 2003. MICRO-36., pages411–422, December 2003.[80] Ameer M.S. Abdelhadi and Guy G.F. Lemieux. Deep and narrow binarycontent-addressable memories using FPGA-based BRAMs. In 2014International Conference on Field-Programmable Technology (FPT),pages 318–321, December 2014.[81] David Patterson. Lecture notes in Vector Processing. University ofCalifornia, Berkeley, Fall 1996.[82] Ameer M.S. Abdelhadi and Guy G.F. Lemieux. Modular multi-portedSRAM-based memories. In Proceedings of the 2014 ACM/SIGDA Inter-national Symposium on Field-Programmable Gate Arrays, pages 35–44.ACM, February 2014.[83] Cristóbal Ramírez, César Alejandro Hernández, Oscar Palomar, OsmanUnsal, Marco Antonio Ramírez, and Adrián Cristal. A RISC-V Simulatorand Benchmark Suite for Designing and Evaluating Vector Architectures.ACM Transactions on Architecture and Code Optimization, 17(4):1–30,December 2020.[84] UBC ORCA Lab. RVV Tests. https://github.com/UBC-ORCA/rvv-tests, September 2024.70
Thesis/Dissertation
eng
Electrical and Computer Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Lemieux, Guy
Shared load-store unit for instruction-based accelerators : case study with soft multicore and multithreaded RISC-V vector processors
Text