C4 Bus

You have designed the single-cycle processor NPC, and you understand how devices work. However, we previously let the simulation environment provide the functionality of the devices. Now it's time to implement the hardware logic that lets NPC communicate with other devices.

In a computer, modules do not work independently; data exchange is required between different modules: between the CPU and the memory controller, between the memory controller and the memory chips, between the instruction fetch unit and the decode unit, and so on. Communication between these modules must be conducted through a set of agreed-upon protocols. The same is true for software. In the DiffTest tool that we use, NEMU needs to communicate with Spike, and NPC also needs to communicate with NEMU to implement the corresponding functions.

In a broad sense, a bus is a communication system used to transfer data between different modules. We will now focus on the hardware and introduce the bus in a narrower sense, i.e. the communication protocols between hardware modules.

Bus - Communication Protocols Between Hardware Modules

We will start with communication between modules inside the processor to understand the general organization of bus protocols.

The Simplest Bus

NPC contains the instruction fetch unit IFU and the instruction decode unit IDU, and these two units need to communicate to transfer instructions from the IFU to the IDU.

+-----+           +-----+
| IFU | inst ---> | IDU |
+-----+           +-----+

In this simple scenario, there is an implied bus protocol: the module that initiates communication is generally called the master, and the module that responds to communication is called the slave. In the simple interaction scenario above, the IFU, as the master, sends information to the IDU, which is the slave, with the information being the current instruction. As a single-cycle processor, obviously, their communication actually implies the following agreement:

Every cycle, the master sends valid information to the slave.
Once the master sends valid information, the slave can receive it immediately.

Asynchronous Bus

However, if the IFU cannot guarantee to fetch an instruction every cycle, then the IDU needs to wait for the IFU to complete the instruction fetch. In this case, it is necessary to add a valid signal to the communication content to indicate when the IFU sends a valid instruction to the IDU. The communication protocol also needs to be updated as follows:

Information is considered valid only when the valid signal is valid.
Once the master sends valid information, it is considered received by the slave immediately.

+-----+ inst  ---> +-----+
| IFU | valid ---> | IDU |
+-----+            +-----+

So, how do we prevent the IDU from executing invalid instructions? Recalling the state machine model of the processor, we just need to keep the processor's state unchanged when the instruction is invalid. At the circuit level, the state is the sequential logic components; therefore, we just need to set the write enable of the sequential logic components to invalid when the instruction is invalid.

Further, suppose some instructions are too complex to decode, and the IDU needs multiple cycles to decode a single instruction. Under this assumption, when the IFU successfully fetches an instruction, the IDU may not have finished decoding the previous instruction. At this time, the IFU should wait for the IDU to complete the current decoding work before sending the next instruction to the IDU. To implement the function of making the IFU wait, it is necessary to add a ready signal to the communication content to indicate when the IDU can receive the next instruction. The communication protocol also needs to be updated as follows:

Information is considered valid only when the valid signal is valid.
The information sent by the master is considered received by the slave only when the ready signal is valid.

+-----+ inst  ---> +-----+
| IFU | valid ---> | IDU |
+-----+ <--- ready +-----+

This actually refers to an asynchronous bus, where it is unpredictable when communication between the two modules will occur; it only happens when both valid and ready are valid. Both signals being valid, also known as a "handshake," indicates that the master and slave have reached a consensus on the successful transmission of information. Clearly, this communication protocol is more flexible, allowing the master and slave to decide when to send or receive messages based on their own work situations. When the valid signal is valid while the ready signal is invalid, the IFU needs to wait for the IDU to be ready. At this time, the IFU should buffer the data to avoid losing it.

The RTL Implementation of an Asynchronous Bus

The asynchronous bus determines when to communicate through a handshake protocol. At the RTL level, it mainly consists of two parts: interface signals and communication logic.

Interface signals are the signals that need to be transmitted between modules, such as the inst, valid, and ready signals in the example above. Chisel provides a Decoupled template, which comes with valid and ready, and asynchronous bus interfaces can be easily implemented through meta-programming:

class Message extends Bundle {
  val inst = Output(UInt(32.W))
}

class IFU extends Module {
  val io = IO(new Bundle { val out = Decoupled(new Message) })
  // ...
}
class IDU extends Module {
  val io = IO(new Bundle { val in = Flipped(Decoupled(new Message)) })
  // ...
}

When more information needs to be transmitted, it is also very convenient to add signals. This is the benefit of abstraction!

 class Message extends Bundle {
   val inst = Output(UInt(32.W))
 +  val pc = Output(UInt(32.W))
 }

Next is the communication logic, i.e. how to implement the bus protocol above with circuits. The protocol is an agreement between the master and the slave, and what really needs to be implemented in the circuit is the behavior of both parties while adhering to this agreement. That is, both parties enter different states based on the different conditions of the handshake signals, thereby taking different actions. This is the state machine! For the master, we can easily draw its state transition diagram:

   +-+ valid = 0
   | v         valid = 1
1. idle ----------------> 2. wait_ready <-+
   ^                          |      |    | ready = 0
   +--------------------------+      +----+
              ready = 1

Specifically:

Initially in the idle state, set valid to invalid
1. If there is no need to send a message, remain in the idle state
2. If a message needs to be sent, set valid to valid, enter the wait_ready state, and wait for the slave to be ready
In the wait_ready state, simultaneously detect the slave's ready signal
1. If the ready signal is valid, the handshake is successful, return to the idle state
2. If the ready signal is invalid, continue to stay in the wait_ready state and wait

With the state transition diagram, we can easily write the corresponding RTL code. Here is a simple Chisel code snippet for your reference:

class IFU extends Module {
  val io = IO(new Bundle { val out = Decoupled(new Message) })

  val s_idle :: s_wait_ready :: Nil = Enum(2)
  val state = RegInit(s_idle)
  state := MuxLookup(state, s_idle)(List(
    s_idle       -> Mux(io.out.valid, s_wait_ready, s_idle),
    s_wait_ready -> Mux(io.out.ready, s_idle, s_wait_ready)
  ))

  io.out.valid := need to send an instruction
  // ...
}

Implement IDU Communication in RTL

After understanding the implementation process of IFU communication, try to analyze and draw the state transition diagram of IDU, and write the RTL code for IDU communication. This is just a bus exercise; you don't need to modify the IDU code in your NPC at the moment.

When we fully implement the communication logic of both IFU and IDU, the above bus protocol will also be effectively implemented.

Processor Design from a Bus Perspective

We can reconsider the processor design itself from a bus perspective. Assuming the processor consists of 4 modules that need to communicate with each other: IFU sends instructions to IDU, IDU sends the decoded results to EXU, EXU sends the computation results to WBU.

+-----+ inst  ---> +-----+  ...  ---> +-----+  ...  ---> +-----+
| IFU | valid ---> | IDU | valid ---> | EXU | valid ---> | WBU |
+-----+ <--- ready +-----+ <--- ready +-----+ <--- ready +-----+

From this perspective, different microarchitectures of processors essentially represent varying communication protocols between modules:

For a single-cycle processor, it is essentially every message sent by the upstream being valid each cycle, while the downstream remains ready to receive new messages.
For a multi-cycle processor, messages are invalid when the upstream module is idle, and new messages are not accepted when the downstream module is busy; the IFU fetches the next instruction only after receiving the completion signal from WBU.
- Unlike traditional textbooks, this is a message-controlled distributed multi-cycle processor, where the "distributed" aspect lies in: whether two modules can communicate with each other solely depends on their states, independent of other modules.

Multi-cycle Processor in Textbooks

In a multi-cycle processor, an instruction is divided into different stages and executed in different cycles, hence each instruction takes multiple clock cycles to complete:

In the 1st cycle, the IFU fetches the instruction and passes it to the IDU
In the 2nd cycle, the IDU sends the decoded result to the EXU
In the 3rd cycle, the EXU passes the computation result to the WBU
In the 4th cycle, the WBU writes back the result to the register file
In the next cycle, the IFU fetches the next instruction...

For a pipelined processor, the IFU can continuously fetch instructions, and each module attempts to send messages downstream every cycle as soon as it completes processing any message.
For an out-of-order execution processor, it can be seen as a minor extension of the pipeline: each downstream module has a queue, and the upstream module only needs to send messages to the queue without worrying about the state of the downstream queue.

Usually, centralized control requires a global controller responsible for collecting the states of all modules and then using these states to control the next steps of each module. The multi-cycle processor typically introduced in traditional textbooks is often a centralized multi-cycle processor based on a large state machine. It collects the states of each module through a global state machine to control communication between each module and determine what the next state should be. For example, if the IFU completes instruction fetching in the current cycle, then in the next cycle it should enter the decoding state and control the IDU to perform decoding.

                   +--------------+
   +-------------> |  Controller  | <--------------+
   |               +--------------+                |
   |                ^            ^                 |
   v                v            v                 v
+-----+  inst   +-----+   ...   +-----+   ...   +-----+
| IFU | ------> | IDU | ------> | EXU | ------> | WBU |
+-----+         +-----+         +-----+         +-----+

In textbooks, usually only a few instructions are used to introduce the basic principles of multi-cycle state machines, which is not a big problem. However, the scalability of centralized control is relatively low. As the number of modules and the complexity increase, the design of the controller becomes more and more complex:

With an increase in the number of instructions, the types of instructions also increase. The centralized state machine needs to consider all stages of execution for each type of instruction.
Inserting a new stage in this processor would require a redesign of the controller.
In real processors, the working time of each module may vary:
- Instruction fetching in the IFU may have delays.
- The IDU may complete decoding in just one cycle.
- In the EXU, different instructions may have varying execution times:
  - Integer arithmetic instructions in RVI typically complete calculations in one cycle.
  - Division usually takes a very long time.
  - Multiplication might be faster but could still take several cycles.
  - Memory access instructions have unpredictable wait times.
The execution of some instructions may trigger exceptions, and interrupts can arrive at any time.

In such complex scenarios, considering the combination of different states for each module, making unified decisions becomes very challenging.

In the distributed control mentioned earlier, each module's behavior depends only on its own state and the state of downstream modules, so each module can work independently. For example, in an out-of-order execution processor, the upstream module can continue working until the queue in the downstream module is full. In distributed control, it is very easy to insert a new module; you only need to modify the interface implementation of its upstream and downstream modules. Therefore, distributed control offers better scalability.

By adopting this handshake-based distributed control, the design of processors with different microarchitectures can be unified. Furthermore, out-of-order execution processors are inherently distributed control systems, because out-of-order execution processors have many modules and many states for each module, and various different events can arrive at any time (such as interrupts arriving, pipeline stalls, etc.). If centralized control were used, it would be nearly impossible to ensure that the controller makes correct decisions when different events occur in each module.

Benefits of Buses in System Design

You should be able to appreciate one of the benefits of buses in system design: by dividing modules and enabling communication between them, the overall complexity of system design and maintenance is reduced. A centralized controller needs to communicate with every module, making its design and maintenance the most difficult; by decomposing global communication into interactions between upstream and downstream modules through bus protocols, the need for a centralized controller is eliminated, thereby reducing the complexity of the entire processor design.

In fact, many examples in the field of computing use message passing to reduce system complexity, such as microkernels in operating systems, MPI programming frameworks in distributed systems, client-server models in software architecture, and even the entire internet communicates through network packets.

While processor design deals with hardware, design patterns are not solely a hardware issue, and we can still draw useful experiences from the software domain to help improve processor design.

Using the function abstraction and meta-programming features in Chisel, we can unify the design patterns of processors with different microarchitectures and easily "upgrade" the processor microarchitecture.

class NPC extends Module {
  val io = // ...

  val ifu = Module(new IFU)
  val idu = Module(new IDU)
  val exu = Module(new EXU)
  val wbu = Module(new WBU)

  StageConnect(ifu.io.out, idu.io.in)
  StageConnect(idu.io.out, exu.io.in)
  StageConnect(exu.io.out, wbu.io.in)
  // ...
}

object StageConnect {
  def apply[T <: Data](left: DecoupledIO[T], right: DecoupledIO[T]) = {
    val arch = "single"
    // To illustrate the concept of abstraction, some details have been omitted in this code.
    if      (arch == "single")   { right.bits := left.bits }
    else if (arch == "multi")    { right <> left }
    else if (arch == "pipeline") { right <> RegEnable(left, left.fire) }
    else if (arch == "ooo")      { right <> Queue(left, 16) }
  }
}

NPC Refactoring

Try to refactor the NPC using the bus concept above. Although this is not mandatory, if you are developing with Chisel, we strongly recommend that you do the refactoring; this will also prepare you for the upcoming SoC integration and future pipeline implementation.

If you are developing in Verilog, you might find the refactoring work somewhat tedious. However, we want to emphasize that design patterns and RTL implementation are different layers. Despite potential difficulties, we encourage you to think about how to implement the correct design patterns above through Verilog.

System Bus

Apart from the interconnection of modules within the processor as mentioned above, how the processor connects to memory and devices is also crucial, since a real processor cannot work independently without memory and peripherals. The bus that connects the processor with memory and devices is typically referred to as the system bus. Below, we will use the connection between the processor and memory as an example to introduce how the system bus should be designed.

Communication Requirements of the System Bus

Previously, NPC accessed memory through the DPI-C mechanism and interfaces such as pmem_read() provided by the simulation environment. This process has no read latency; the read data can be returned in the current cycle upon receiving the read request. But this is only to facilitate the implementation of a single-cycle processor; in reality, no such memory device exists.

Compared with the DPI-C mechanism, in a real chip, the biggest difference in the processor's access to memory is that the access process has latency. In other words, from issuing a read request to receiving the data replied by the memory, the processor needs to wait for several cycles. This requires the processor to additionally implement the following mechanisms:

It needs to recognize when the data replied by the memory arrives.
Before the data replied by the memory arrives, the processor needs to wait, and only after the reply data arrives can the processor continue executing the current instruction. This means the processor will no longer satisfy the property of "executing one instruction per cycle".
Furthermore, the processor's instruction fetch operation does not need to be performed every cycle. For example, it should not fetch instructions while waiting for the memory's reply; therefore, the memory needs to be informed when instruction fetching actually occurs.

To solve these problems, in real chips, the processor and memory communicate through an asynchronous bus mechanism: the two communicating parties interact according to some agreed-upon convention, thereby achieving correct information transfer.

However, real chips usually adopt industrial-grade bus protocols similar to AXI, which have many details; for example, the AXI bus protocol has about more than 30 signals. This means that if a real chip is to communicate with devices, in principle the processor needs to implement these bus protocols. To reduce your burden, we will first start with a simplest custom bus protocol called SimpleBus.

Accessing Read-Only Memory

Since reading data out of memory is the most basic need, we first consider how the processor completes read operations through the system bus. Assume the processor specification is fixed as Nx32, i.e., the memory contains N words, each word being 32 bits. Also assume the memory's read latency is fixed at 1 cycle. This is actually a synchronous memory, meaning the delay from receiving a read request to returning data is fixed, which is exactly the access characteristic of SRAM.

If we do not consider write operations, a read-only memory (ROM, Read-Only Memory) is enough. To read data out of the ROM, the corresponding bus only needs two signals, namely address and data, whose bit widths are log2(N) and 32, respectively.

+-----+ raddr[log2(N)-1:0] ---> +-----+
| CPU | <---        rdata[31:0] | MEM |
+-----+                         +-----+

Its communication protocol is:

The master (CPU) sends the read address raddr to the slave (MEM).
In the next cycle, the slave replies the data rdata to the master.
The above actions occur every cycle.

Evaluate the Clock Frequency and Program Performance of the Single-Cycle NPC

Before further modifying the NPC, try to evaluate the current NPC's clock frequency using the yosys-sta project you used in the pre-learning phase. However, before the evaluation, you need to do the following:

First run the microbench test at the train scale and record the number of cycles required for its completion.
In RTL, comment out the code that calls pmem_read() and pmem_write() through DPI-C, and then instantiate one memory for instruction fetch and one for memory access, respectively. To maintain the single-cycle property, the instantiated memory needs to return read data in the current cycle, so we can implement it through flip-flops just like the register file. If you are using Verilog, you can directly instantiate the RegisterFile module; of course, you need to connect the ports correctly. To unify the test results, we agree to instantiate memories of size 256x32b, i.e., 1KB, with two such memories instantiated in total, for a total size of 2KB.

The reason we modify it this way is that a single-cycle NPC requires completing the full lifecycle of an instruction every cycle, so it cannot be connected to any real-world memory; it can only be evaluated together with two register-file-like memories. After the modification, you can evaluate the clock frequency of the single-cycle NPC.

Based on the evaluated clock frequency and the number of cycles for microbench execution recorded earlier, you can estimate how long the future NPC will take to run microbench. Note that this is not simulation time, but the time for the program to run assuming the NPC runs at the clock frequency above. For example, a certain version of yzh's NPC has a clock frequency of 51.491MHz on the nangate45 process provided by default by the yosys-sta project, so microbench can be calculated to take 3.870s to run, but the simulation took 19.148s.

Of course, this estimation is not actually accurate, and can even be said to be very optimistic:

This single-cycle NPC is still far from the tape-out configuration; for example, when we modified the memory just now, we actually ignored all the I/O-related parts.
The clock frequency above is the post-synthesis frequency; the wire delays introduced after place and route will further lower the clock frequency.
The memory corresponding to the instruction fetch unit was optimized out by yosys because it has no write operations.
The memory corresponding to the memory access unit is actually far too small to hold microbench. To successfully run the train-scale test, the data needs to occupy 1MB of memory. This size far exceeds the number of flip-flops that can be accommodated in a real processor chip design. Setting aside the EDA tool processing time, just filling the chip with so many flip-flops would make the estimated wire delay, based on the occupied area, extraordinarily large.

Therefore, this evaluation result has little reference value; just treat it as practice for subsequent evaluations.

Since instructions are stored in memory, the IFU also needs to access memory when fetching instructions. Meanwhile, the instruction fetch process never writes data into memory, so a read-only memory is enough to support the IFU. To simulate the phenomenon of latency in the memory access process, after receiving an instruction fetch request, the memory cannot return the fetched instruction immediately, but needs to delay one cycle before returning the fetched instruction.

According to the above, for the IFU, SimpleBus involves the following signals:

output [31:0] ifu_raddr,
input  [31:0] ifu_rdata,

The timing of these signals is shown in the figure below:

           --\ /----------\ /----------\ /----------\ /----------\ /--
 ifu_raddr    X 0x80000000 X     (1)    X 0x80000004 X            X
           --/ \----------/ \----------/ \----------/ \----------/ \--
           --\ /----------\ /----------\ /----------\ /----------\ /--
 ifu_rdata    X            X 0x00000413 X     (2)    X 0x80051137 X
           --/ \----------/ \----------/ \----------/ \----------/ \--

To let the NPC implement the function of "waiting for the memory to read out the instruction", we first need to let the IFU know which stage of instruction fetching it is currently in, and adopt different strategies in different stages. This kind of "doing different things at different times" can be implemented with the state machine of digital circuits! Specifically, we can implement two states, idle and wait, for the IFU:

In the idle state, ifu_raddr needs to be set to pc, and then jump to the wait state.
In the wait state, ifu_rdata is executed as a valid instruction, and then jump to the idle state.

According to the bus protocol mentioned earlier, since communication occurs every cycle, while sending an address request to the memory, the memory also returns the data corresponding to the previous cycle. For example, in the figure above, when NPC sends the instruction fetch request with address 0x80000004, the memory simultaneously returns the data marked (2), which corresponds to the address request marked (1) in the previous cycle. If NPC executes (2) as a valid instruction, it either repeatedly executes the instruction corresponding to address 0x80000000, or executes an instruction at some other location; either way, NPC's behavior will not conform to the ISA convention. Therefore, in the idle state, NPC should not treat ifu_rdata as a valid instruction to execute; instead, it should not execute any instruction.

Previously, we assumed by default that NPC could execute one instruction every cycle, but after implementing SimpleBus, NPC has no valid instruction to execute while waiting for the instruction to return. Therefore, we need to consider "how to make NPC not execute instructions". Recalling the state machine model of computer systems, the process of a processor executing an instruction is the process of changing the processor state. In other words, as long as we find a way to keep the processor state unchanged, we can achieve the effect of "not executing instructions". The clever you should already think of the solution: the processor state is the sequential logic components; as long as their write enable signals are invalid, the values of the sequential logic components will not change. Therefore, you also need to correctly set the write enable signals of the various sequential logic components in NPC in the appropriate states.

Support SimpleBus for the IFU

According to the above, let the IFU support the SimpleBus protocol. For the instruction fetch part of the memory, you can refer to the following code:

always @(posedge clock) begin
  ifu_rdata <= pmem_read(ifu_raddr);
end

For the data access part of the LSU, no modification is needed for now; we will let it support SimpleBus next.

After implementation, try running some test programs, and confirm via waveforms that the communication between NPC and the memory is as expected. In principle, the bus protocol is transparent to upper-level programs, so programs that ran successfully before should also run successfully after implementing SimpleBus.

However, since the memory now needs 1 cycle to read out data, NPC is no longer strictly a single-cycle processor, but rather a simple multi-cycle processor:

In the 1st cycle, the IFU issues an instruction fetch request.
In the 2nd cycle, the IFU obtains the instruction and passes it to the subsequent modules for decoding and execution.

If you refactored the NPC according to the earlier suggestions, you will find that transforming the NPC into a multi-cycle processor is not difficult.

Adapt DiffTest for Multi-Cycle Processor

After modifying the NPC into a multi-cycle processor, the NPC no longer executes an instruction every cycle. To ensure the DiffTest mechanism works correctly, you need to slightly adjust the timing of checks. To do this, you may need to read some states of the RTL from the simulation environment to help you decide when to perform the DiffTest checks.

Accessing Read-Write Memory

To support write operations, new signals need to be added to the bus:

Naturally, the write address waddr and write data wdata are needed first.
Since write operations do not occur every cycle, a write enable signal wen also needs to be added.
- Although read operations also do not occur every cycle — for example, for the multi-cycle NPC above, in the wait state there is no need to send an instruction fetch request to the memory — since read operations do not change circuit states, a read enable is theoretically not necessary.
- However, in practice there is usually a read enable ren; if there are no read requests, the memory does not need to perform read operations, thereby saving energy.
A write operation may only write into some bytes of a word (for example, the sb instruction writes only 1 byte), so a write mask signal wmask also needs to be added, to specify which bytes of the data to be written.

+-----+ addr[log2(N)-1:0]  ---> +-----+
|     | wen                ---> |     |
| CPU | wdata[31:0]        ---> | MEM |
|     | wmask[3:0]         ---> |     |
|     | <---        rdata[31:0] |     |
+-----+                         +-----+

Meanwhile, the communication protocol needs to define the behavior of write operations. We use pseudocode to represent it:

if (wen) {
  // wmask_full is the result of expanding wmask bit by bit
  M[waddr] = (wdata & wmask_full) | M[waddr] & ~wmask_full;
}

Similar to the IFU, let's also have the LSU access the memory according to the SimpleBus protocol. For the LSU, since store instructions need to write to memory, SimpleBus involves the following signals:

output [31:0] lsu_addr,
output        lsu_wen,
output [31:0] lsu_wdata,
output [ 3:0] lsu_wmask,
input  [31:0] lsu_rdata,

For read operations, wdata and wmask can take any values, and the timing of the relevant signals is shown in the figure below:

           --\ /----------\ /------------------------
 lsu_addr     X 0x80400000 X
           --/ \----------/ \------------------------
           ---+            +-------------------------
 lsu_wen      |            |
              +------------+
           ------------------------------------------
 lsu_wdata
           ------------------------------------------
           ------------------------------------------
 lsu_wmask
           ------------------------------------------
           ---------------\ /----------\ /-----------
 lsu_rdata                 X 0x12345678 X
           ---------------/ \----------/ \-----------

For write operations, rdata is a don't-care signal, and the timing of the relevant signals is shown in the figure below:

           --\ /----------\ /------------------------
 lsu_addr     X 0x80400000 X
           --/ \----------/ \------------------------
              +------------+
 lsu_wen      |            |
           ---+            +-------------------------
           --\ /----------\ /------------------------
 lsu_wdata    X 0x12345678 X
           --/ \----------/ \------------------------
           --\ /----------\ /------------------------
 lsu_wmask    X     0xf    X
           --/ \----------/ \------------------------
           ------------------------------------------
 lsu_rdata
           ------------------------------------------

Support SimpleBus for the LSU

According to the above, let the LSU support the SimpleBus protocol. For the data access part of the memory, you can refer to the following code:

always @(posedge clock) begin
  lsu_rdata <= (!lsu_wen) ? pmem_read(lsu_addr) : 32'b0;
  if (lsu_wen) begin
    pmem_write(lsu_addr, lsu_wdata, lsu_wmask);
  end
end

At this point, the device access functionality in pmem_read()/pmem_write() can be retained; we will introduce how to access peripherals through the bus in subsequent chapters.

After the LSU supports SimpleBus, for a load instruction, the NPC requires 3 cycles to complete:

In the 1st cycle, the IFU issues an instruction fetch request.
In the 2nd cycle, the IFU obtains the instruction and passes it to the subsequent modules for decoding; upon discovering that it is a load instruction, it issues a memory access request through the LSU.
In the 3rd cycle, the LSU obtains the data and passes it to the WBU to write back to the register.

Therefore, you also need to modify the IFU so that the next instruction is fetched only after the load instruction completes execution.

After implementation, try running some test programs, and confirm via waveforms that the communication between NPC and the memory is as expected. Likewise, programs that ran successfully before should also run successfully after implementing SimpleBus.

More General Memory

In reality, the characteristic of a 1-cycle read latency is usually only achievable with SRAM, because SRAM can be produced with the same process used to manufacture the processor, but SRAM is very expensive. To achieve lower-cost memory, other processes with higher storage density are usually adopted to manufacture memory, such as DRAM. However, due to electrical characteristics, the read latency of these memories is usually greater than the processor's 1 cycle.

At this point the processor cannot keep sending read requests; otherwise, since the processor's request rate exceeds the memory's service rate, the memory will be continuously occupied by useless requests, seriously degrading the efficiency of the entire system. To solve this problem, the processor needs to tell the memory when it sends a valid request. To this end, we can add a new signal reqValid to the signals sent from the processor to the memory, so that the memory can use this signal to determine when there is a real request. When the processor needs to access memory, it sets addr and other signals and asserts reqValid; when the processor does not need to access memory, it deasserts reqValid. For the memory, it only needs to access the data corresponding to addr when the reqValid signal is valid; when reqValid is invalid, the memory will not perform the access operation.

On the other hand, when the memory can read out data is also unpredictable in advance. For example, DRAM periodically charges and refreshes the capacitors of memory cells; if a read request is received at this time, the data will only actually be read out after the charging refresh is complete. Therefore, the memory also needs to tell the processor when it can return valid data. Similarly, to recognize when the memory's reply arrives, we can add a new signal respValid to the signals the processor receives from the memory, so that the processor can use this signal to determine when the memory's reply is valid. Taking a read operation as an example, when the memory reads out the data, it sets rdata and asserts respValid; when the memory has not yet read out the data, it deasserts respValid. For the processor, it only considers rdata as valid data when the respValid signal is valid; when respValid is invalid, the processor will consider that the data has not yet returned and needs to continue waiting.

+-----+ reqValid           ---> +-----+
|     | addr[log2(N)-1:0]  ---> |     |
|     | wen                ---> |     |
| CPU | wdata[31:0]        ---> | MEM |
|     | wmask[3:0]         ---> |     |
|     | <---          respValid |     |
|     | <---        rdata[31:0] |     |
+-----+                         +-----+

Taking the LSU as an example, the extended SimpleBus involves the following signals:

output        lsu_reqValid,
output [31:0] lsu_addr,
output        lsu_wen,
output [31:0] lsu_wdata,
output [ 3:0] lsu_wmask,
input         lsu_respValid,
input  [31:0] lsu_rdata,

For read operations, wdata and wmask can take any values, and the timing of the relevant signals is shown in the figure below:

                  +------------+
 lsu_reqValid     |            |
               ---+            +-------------------------------------------------
               --\ /----------\ /------------------------------------------------
 lsu_addr         X 0x80400000 X
               --/ \----------/ \------------------------------------------------
               --\              /------------------------------------------------
 lsu_wen          X            X
               --/ ------------ \------------------------------------------------
               ------------------------------------------------------------------
 lsu_wdata
               ------------------------------------------------------------------
               ------------------------------------------------------------------
 lsu_wmask
               ------------------------------------------------------------------
                                                       +------------+
 lsu_respValid                                         |            |
               ----------------------------------------+            +------------
               ---------------------------------------\ /----------\ /-----------
 lsu_rdata                                             X 0x12345678 X
               ---------------------------------------/ \----------/ \-----------

For write operations, although rdata is a don't-care signal, SimpleBus still uses respValid to indicate that the write operation has been completed. The timing of the relevant signals is shown in the figure below:

                  +------------+
 lsu_reqValid     |            |
               ---+            +-------------------------------------------------
               --\ /----------\ /------------------------------------------------
 lsu_addr         X 0x80400000 X
               --/ \----------/ \------------------------------------------------
               --\ ------------ /------------------------------------------------
 lsu_wen          X            X
               --/              \------------------------------------------------
               --\ /----------\ /------------------------------------------------
 lsu_wdata        X 0x12345678 X
               --/ \----------/ \------------------------------------------------
               --\ /----------\ /------------------------------------------------
 lsu_wmask        X     0xf    X
               --/ \----------/ \------------------------------------------------
                                                       +------------+
 lsu_respValid                                         |            |
               ----------------------------------------+            +------------
               ------------------------------------------------------------------
 lsu_rdata
               ------------------------------------------------------------------

After adding the valid signals to SimpleBus, we also need to extend the state machine:

In the idle state, if memory access is currently needed, set reqValid, addr, and other signals, and jump to the wait state; if memory access is not currently needed, stay in the idle state.
In the wait state, if respValid is valid, execute the instruction continuing with rdata as the read-out data, and jump to the idle state; if respValid is invalid, stay in the wait state, thereby achieving the waiting effect.

SimpleBus Protocol with Valid Signals

According to the above, let the IFU and LSU access memory based on the valid signals. For the data access part of the memory, you can refer to the following code:

always @(posedge clock) begin
  lsu_rdata <= (lsu_reqValid && !lsu_wen) ? pmem_read(lsu_addr) : 32'b0;
  if (lsu_reqValid && lsu_wen) begin
    pmem_write(lsu_addr, lsu_wdata, lsu_wmask);
  end
  lsu_respValid <= lsu_reqValid;
end

The instruction fetch part of the memory can be modified in a similar way.

Test the SimpleBus Implementation

Add random delay functionality to the memory to test whether the bus implementation works correctly under arbitrary delays. You can add memory access delays in order from simple to complex:

Change the memory access delays to 5, 10, 20, etc. in sequence.
Add an LFSR to the memory module and use it to determine the delay of the current request.
Add LFSRs to the IFU and LSU as well, and use them to determine the delay of the corresponding valid signals.

If the NPC can still run programs correctly under random delays filled with LFSRs, it will greatly boost your confidence in your code.

Friendly Reminder

If you are learning about buses for the D-stage tape out, you can stop here.

More General Memory (2)

Sometimes, even though the processor wants to send a request, the memory may not be able to receive it immediately, possibly because the memory is busy or the request queue is full. To this end, we can add a new signal reqReady to the signals sent from the memory to the processor, so that the processor can use this signal to recognize when the memory can receive requests. When the processor needs to access memory, if reqReady is valid, it means the current request can be successfully received by the memory; if reqReady is invalid, it means the current request cannot be received by the memory, and the processor needs to wait.

On the other hand, the processor may also not be ready to receive the data returned by the memory because the previously read-out data has not yet been used up. Similarly, we can add a new signal respReady to the signals sent from the processor to the memory, so that the memory can use this signal to recognize when the processor can receive replies. When the memory needs to send a reply, if respReady is valid, it means the current reply can be successfully received by the processor; if respReady is invalid, it means the current reply cannot be received by the processor, and the memory needs to wait.

+-----+ reqValid           ---> +-----+
|     | <---           reqReady |     |
|     | addr[log2(N)-1:0]  ---> |     |
|     | wen                ---> |     |
| CPU | wdata[31:0]        ---> | MEM |
|     | wmask[3:0]         ---> |     |
|     | <---          respValid |     |
|     | respReady          ---> |     |
|     | <---        rdata[31:0] |     |
+-----+                         +-----+

Taking the LSU as an example, the extended SimpleBus involves the following signals:

output        lsu_reqValid,
input         lsu_reqReady,
output [31:0] lsu_addr,
output        lsu_wen,
output [31:0] lsu_wdata,
output [ 3:0] lsu_wmask,
input         lsu_respValid,
output        lsu_respReady,
input  [31:0] lsu_rdata,

For read operations, the timing of the relevant signals is shown in the figure below:

                  +-------------------------+
 lsu_reqValid     |                         |
               ---+                         +---------------------------------------------------
                                +------------+
 lsu_reqReady                  |            |
               ----------------+            +---------------------------------------------------
               --\ /-----------------------\ /--------------------------------------------------
 lsu_addr         X       0x80400000        X
               --/ \-----------------------/ \--------------------------------------------------
               --\                           /--------------------------------------------------
 lsu_wen          X                         X
               --/ ------------------------- \--------------------------------------------------
               ---------------------------------------------------------------------------------
 lsu_wdata
               ---------------------------------------------------------------------------------
               ---------------------------------------------------------------------------------
 lsu_wmask
               ---------------------------------------------------------------------------------
                                                         +-------------------------+
 lsu_respValid                                           |                         |
               ------------------------------------------+                         +------------
                                                                       +------------+
 lsu_respReady                                                        |            |
               -------------------------------------------------------+            +------------
               -----------------------------------------\ /-----------------------\ /-----------
 lsu_rdata                                               X       0x12345678        X
               -----------------------------------------/ \-----------------------/ \-----------

For write operations, the timing of the relevant signals is shown in the figure below:

                  +-------------------------+
 lsu_reqValid     |                         |
               ---+                         +---------------------------------------------------
                                +------------+
 lsu_reqReady                  |            |
               ----------------+            +---------------------------------------------------
               --\ /-----------------------\ /--------------------------------------------------
 lsu_addr         X       0x80400000        X
               --/ \-----------------------/ \--------------------------------------------------
               --\ ------------------------- /--------------------------------------------------
 lsu_wen          X                         X
               --/                           \--------------------------------------------------
               --\ /-----------------------\ /--------------------------------------------------
 lsu_wdata        X       0x12345678        X
               --/ \-----------------------/ ---------------------------------------------------
               --\ /-----------------------\ /--------------------------------------------------
 lsu_wmask        X           0xf           X
               --/ \-----------------------/ \--------------------------------------------------
                                                         +-------------------------+
 lsu_respValid                                           |                         |
               ------------------------------------------+                         +------------
                                                                       +------------+
 lsu_respReady                                                        |            |
               -------------------------------------------------------+            +------------
               ---------------------------------------------------------------------------------
 lsu_rdata
               ---------------------------------------------------------------------------------

As can be seen, during the completion of a read transaction, both the master and the slave need to go through two handshakes: the master first waits for reqReady to ensure the slave can receive the read request, then waits for respValid to receive the read data; while the slave first waits for reqValid to receive the read request, then waits for respReady to ensure the master can receive the read data. Of course, at the RTL implementation level, these are all state machines.

The Significance of Microarchitectural Design in Handshake Signals - Decoupling

From the perspective of microarchitectural design, the most important significance of handshake signals is to shield the processing delay of the two communicating parties. For example, when DRAM reads out data is affected by many factors, including the refresh timing, the request scheduling of the DRAM controller, whether the row buffer in the DRAM chip is hit, and even the electrical characteristics of the chip. Similarly, when the CPU sends requests and receives data is also affected by many factors, including when the program executes memory access instructions, pipeline congestion, cache status, etc. If one party in the communication has to consider these situations of the other party, such a design would be impossible to achieve: we have no way to know what state the other party is in.

Handshake signals successfully decouple the states of both parties. With handshake signals, neither party needs to care about the state of the other module; they only need to wait for the handshake. Therefore, as long as modules follow the same bus protocol, they can work smoothly once connected to the bus.

SimpleBus Protocol with Complete Handshake Signals

According to the above, let the IFU and LSU access memory based on the complete handshake signals.

After implementation, try running some test programs, and confirm via waveforms that the communication between NPC and the memory is as expected.

You can add random delays to reqReady and respReady to test the bus implementation more thoroughly.

Error Handling and Exceptions

In some cases, errors may occur when the memory processes read and write transactions, such as reading or writing an address beyond the storage range, or discovering through checksums that the read/written memory cells are damaged. In these cases, the slave should usually tell the master that an error occurred and let the master decide how to handle it. Therefore, we need to additionally transmit an error signal in the memory's reply to indicate whether the operation succeeded. If the error signal is valid, the returned read data rdata is invalid, or the write operation was unsuccessful. Of course, the error signal also needs to participate in the handshake.

+-----+ reqValid           ---> +-----+
|     | <---           reqReady |     |
|     | addr[log2(N)-1:0]  ---> |     |
|     | wen                ---> |     |
|     | wdata[31:0]        ---> |     |
| CPU | wmask[3:0]         ---> | MEM |
|     | <---          respValid |     |
|     | respReady          ---> |     |
|     | <---        rdata[31:0] |     |
|     | <---              error |     |
+-----+                         +-----+

In a processor, if errors occur in read or write operations, exceptions can be further thrown to notify the software for handling. For example, RISC-V defines 3 types of Access Fault exceptions, representing errors when fetching instructions from memory, reading data, and writing data. In this way, errors within the memory can be communicated to the processor through the bus protocol and then notified to the software through the processor's exception handling mechanism.

Bus Protocol Widely Used in the Industry - AMBA Bus Protocol

AMBA (Advanced Microcontroller Bus Architecture) is a bus protocol family widely used in the industry, which defines bus protocols for various performance scenarios. After understanding SimpleBus, you will be capable of understanding these bus protocols widely used in the industry.

APB Bus Protocol

In AMBA, APB (Advanced Peripheral Bus) is a bus protocol for low-speed peripherals. In fact, by simply transforming the SimpleBus without ready signals, we can obtain the APB bus protocol:

Rename the signals.
Add psel, used for the master to select one among multiple slaves.

reqValid  --->          penable   --->          psel      --->
addr      --->          paddr     --->          penable   --->
wen       --->          pwrite    --->          paddr     --->
wdata     --->    1     pwdata    --->    2     pwrite    --->
wmask     --->   ===>   pstrb     --->   ===>   pwdata    --->
<--- respValid          <---    pready          pstrb     --->
<---     rdata          <---    prdata          <---    pready
<---     error          <---   pslverr          <---    prdata
                                                <---   pslverr

In this way we obtain the APB4 bus protocol in the APB manualopen in new window! However, APB4's timing requirements for requests differ from SimpleBus; we won't expand on that here — just RTFM when you need to know.

AXI Bus Protocol

Because APB lacks ready signals, a slave can only receive and process one request at a time. For low-speed peripherals like the serial port, this is acceptable, because during program execution, the frequency of accessing low-speed peripherals is not high; having the low-speed peripheral process one request at a time does not bring obvious performance loss.

But on the contrary, accessing memory is a frequent operation for programs: most of the load and store instructions executed by the processor ultimately need to access memory, especially main memory. Therefore, for high-performance processors, adopting APB, which can only "receive and process one request at a time", would bring obvious performance loss, making the memory access process a bottleneck of the entire system. To support processors accessing memory more efficiently, AMBA needs to define another bus protocol different from APB — this is AXI (Advanced eXtensible Interface).

To help everyone understand AXI, we base it on the complete-handshake version of SimpleBus and make a simple transformation:

Separate the addresses of read and write requests; signals related to read operations use the r prefix, and signals related to write operations use the w prefix.
- Since the write request has its own valid signal wreqValid, the wen signal is not needed.
Rename the signals: read address uses the ar prefix, read data uses the r prefix, write request uses the w prefix, write response uses the b prefix (meaning backward).
Separate the write address and write data; write address uses the aw prefix, write data uses the w prefix.
Group the signals into 5 groups by function.

reqValid  --->           rreqValid --->           arvalid   --->
<---  reqReady           <--- rreqReady           <---   arready
addr      --->           raddr     --->           araddr    --->
wen       --->           <---rrespValid           <---    rvalid
wdata     --->    1      rrespReady--->    2      rready    --->
wmask     --->   ===>    <---     rdata   ===>    <---     rdata
<--- respValid           <---    rerror           <---     rresp
respReady --->           wreqValid --->           wvalid    --->
<---     rdata           <--- wreqReady           <---    wready
<---     error           waddr     --->           waddr     --->
                         wdata     --->           wdata     --->
                         wmask     --->           wstrb     --->
                         <---wrespValid           <---    bvalid
                         wrespReady--->           bready    --->
                         <---    werror           <---     bresp



                         arvalid   --->          araddr  ---> -+
                         <---   arready          arvalid --->  AR
                         araddr    --->          <--- arready -+
                         <---    rvalid
                  3      rready    --->    4     <--- rdata   -+
                 ===>    <---     rdata   ===>   <--- rresp    |
                         <---     rresp          <--- rvalid   R
                         awvalid   --->          rready  ---> -+
                         <---   awready
                         awaddr    --->          awaddr  ---> -+
                         wvalid    --->          awvalid --->  AW
                         <---    wready          <--- awready -+
                         wdata     --->
                         wstrb     --->          wdata   ---> -+
                         <---    bvalid          wstrb   --->  |
                         bready    --->          wvalid  --->  W
                         <---     bresp          <--- wready  -+

                                                 <--- bresp   -+
                                                 <--- bvalid   B
                                                 bready  ---> -+

In this way we obtain the AXI4-Lite bus protocol in the AXI manualopen in new window! AXI4-Lite is a simplified version of AXI. It has 5 transaction channels, namely Read Address (AR), Read Data (R), Write Address (AW), Write Data (W), and Write Response (B). The working states of these transaction channels depend only on their respective handshake signals, so they can work independently.

Compared with APB, AXI4-Lite has the following performance advantages:

Read and write transactions use separate transaction channels. In this way, the bus can transmit read and write transactions simultaneously in the same cycle, thereby enabling concurrent transmission and processing of read and write requests.
Write address and write data use separate transaction channels. For some slaves, a write operation needs to be completed in multiple stages. For example, for DRAM memory, the target memory cell needs to be activated first. For such slaves, if the write address can be received in advance, even if the write data has not yet been received, the first few stages of the write operation can still be started. By separating write address from write data, AXI allows the master to transmit the write address before the write data is ready, letting the slave start the write operation first, thereby improving the processing efficiency of write operations.
Each channel has complete handshake signals. In this way, the bus can transmit the next request to the slave before transmitting the reply to the previous request back to the master, thereby enabling concurrent transmission and processing of requests.

Of course, the current NPC is still a multi-cycle processor; as a master, it cannot send multiple memory access requests simultaneously, so it cannot take advantage of the above AXI benefits either. But for high-performance out-of-order multi-issue processors, multiple memory access requests may be sent simultaneously in the system; adopting the AXI bus protocol can greatly improve the memory access concurrency of the processor, thereby improving the processor's memory access efficiency.

In fact, the AXI4-Lite we are now implementing is only a simplified version of AXI4. The complete AXI bus specification has more signals, thereby supporting more features, such as overlap and out-of-order processing of multiple requests. If you are interested, you can also learn the relevant details through RTFM. We will gradually introduce more features we need to use in subsequent chapters.

Simultaneous Read and Write to the Same Address - RTFM

AXI separates read and write transactions, which means the same address in memory can be read and written simultaneously. In this case, which data does the memory actually read out? In fact, the specific behavior needs to be determined by RTFM: the memory device's manual specifies the behavior of this operation, and in some devices, the read result is undefined. SRAM and Block RAM in FPGAs mostly have this characteristic.

Therefore, the master side should try to avoid reading and writing the same address at the same time. If it is truly unavoidable, detection logic can be added on the master side. When detecting reading and writing the same address at the same time, the write operation can be delayed to first read out the old data; or the read operation can be delayed to first write the data and then read out the newly written data. In this way, at least the read result can be guaranteed to be well-defined.

However, in the current NPC, the IFU only issues read requests, and the LSU does not simultaneously issue read and write requests (depending on whether the currently executed instruction is a load or store), so you do not need to consider this problem in NPC for now.

In the actual use of AXI, we also need to avoid two situations related to handshake signals:

The master and slave are both waiting for the other party to first set the handshake signal to 1: the master sets valid to 1 only after waiting for the slave to set ready to 1; while the slave sets ready to 1 only after waiting for the master to set valid to 1. The result is that both parties wait indefinitely, causing a deadlockopen in new window.
The master and slave are both tentatively handshaking, but both cancel the handshake after the tentative attempt fails:
- In the 1st cycle, the master sets valid to 1, but ready is 0 at this time, so the handshake fails.
- In the 2nd cycle, the slave discovers that the master set valid to 1 in the previous cycle, so it sets ready to 1 in this cycle; but because the handshake failed in the previous cycle, the master sets valid to 0 in this cycle, so the handshake fails again this cycle.
- In the 3rd cycle, the master discovers that the slave set ready to 1 in the previous cycle, so it sets valid to 1 in this cycle; but because the handshake failed in the previous cycle, the slave sets ready to 0 in this cycle, so the handshake fails again this cycle.
- As a result, both parties still wait indefinitely, causing a livelockopen in new window.

        +---+   +---+   +---+   +---+
 clk    |   |   |   |   |   |   |   |
        +   +---+   +---+   +---+   +---+
        +-------+       +-------+
valid           |       |       |
                +-------+       +--------
                +-------+       +--------
ready           |       |       |
        +-------+       +-------+

Avoiding Deadlock and Livelock in Handshaking

To avoid the above problems, the AXI standard adds some constraints on the behavior of handshake signals. You need to RTFM to find these constraints and understand them correctly.

Please note that you must consult the official manual. If you reference materials that are not sufficiently formal, you will fall into a painful debugging black hole when integrating into the SoC.

Let NPC Support AXI4-Lite

By now, you have understood why each signal in the AXI4-Lite bus is included, and understood the design philosophy of this bus specification from the perspective of requirements. Now you can add AXI4-Lite to NPC and let NPC access memory through AXI4-Lite.

Convert the Memory Access Interfaces of IFU and LSU to AXI4-Lite

You need to correctly implement the AXI4-Lite bus protocol with handshakes on both the master and slave ends. Specifically:

Convert the memory access interfaces of IFU and LSU to AXI4-Lite.
Convert the two SimpleBus interfaces ifu and lsu of the memory into AXI4-Lite respectively.

Since the IFU only performs read operations on the memory and never writes to the memory, the handshake signals of the three channels AW, W, and B of the IFU can all be set to 0. Of course, a better practice is to use assert() on the other end of the handshake signals to ensure they remain 0.

After implementation, try running some test programs, and confirm via waveforms that the communication between NPC and the memory is as expected. If NPC can still boot RT-Thread correctly under random delays filled with LFSRs, it will greatly boost your confidence in your code.

Knowledge Gained through Reflection is True Mastery

Traditional textbooks often introduce buses only at the protocol level, without clearly introducing how to implement a bus at the RTL level. Therefore, if you only read textbooks, the bus remains a relatively abstract concept. "One Student One Chip" is a hands-on learning project, and we have already introduced how to implement a bus at the RTL level in the handouts.

However, if you find that there are still some difficulties hard to overcome during the implementation process, it's time to sound the alarm: you are very likely lacking the ability to convert requirements into code step by step. You probably need to reflect on your past learning methods, for example:

Are you overly dependent on block diagrams, leading to a lack of microarchitectural design ability, feeling at a loss without block diagrams?
Even worse, are you overly dependent on materials and books full of example code, leading to not only a lack of microarchitectural design ability but also an inability to convert designs into code? We strongly do not recommend beginners to read such materials and books before their first version of code can work!

We hope you can put "independent thinking" first in your learning, rather than being a code porter. If you find that independent thinking is very difficult from now on, it is very likely because you missed too much in your previous learning, leaving you unable to complete the next tasks on your own. If this is really the case, we suggest you restart your learning of "One Student One Chip" with the right mindset.

Maintain the Right Mindset, Gradually Grasp All Details through Iterative Development and Debugging

However, as a bus protocol widely used in the industry, AXI still has quite a few details, making it difficult for most beginners to understand AXI thoroughly upon first contact. Therefore, everyone also needs to maintain the right mindset; the idea of "writing all the bus code at once and never modifying it later" is unrealistic.

In fact, everyone gradually grasps all the details of a bus through iterative development and debugging. Even senior engineers do this when learning new knowledge; for beginners, the idea of reaching the top in one step does not conform to the objective laws of learning.

Arbiter in Bus

After the above modifications, we transformed the memory into two AXI4-Lite interfaces, connected to the IFU and LSU respectively. But in real hardware, most memories provide only one bus interface. To integrate into the subsequent SoC, we need to consider how to let the IFU and LSU access the same interface of the memory in hardware implementation.

This is actually a multi-master problem. Since the slave has only one interface, an arbiter is used to make the decision: when multiple masters access the same slave at the same time, the master that obtains the access right will be granted permission and can successfully access the slave; the requests of the other masters will be blocked at the arbiter, waiting for the master that obtained the access right to finish its access before they can obtain the next access right.

To briefly summarize, the arbiter needs to implement the following functions:

Scheduling: select a master that is currently sending a valid request.
Blocking: block the access of other masters.
Forwarding: forward the request of the master that obtained the access right to the slave, and when the slave's reply arrives, forward it back to the original master.

How exactly to select when multiple masters access simultaneously is essentially a scheduling strategy issue. In complex systems, scheduling strategies need to consider more issues: first, avoid starvation, i.e., any master can obtain the access right after a finite number of arbitrations; second, avoid deadlock, i.e., the blocking caused by the arbiter should not cause circular waiting in the entire system. However, since the current NPC is a multi-cycle processor, the masters of the IFU and LSU will not send requests simultaneously, so you don't need to consider complex scheduling strategies; choosing any simple scheduling strategy is fine.

Implement an AXI4-Lite Arbiter

Keep one AXI4-Lite interface on the memory, write an AXI4-Lite arbiter, and select one master from the IFU and LSU to communicate with the memory.

Hint: The arbiter is essentially a state machine, and the blocking and forwarding functions are fundamentally implemented by manipulating the handshake signals.

Evaluate NPC Clock Frequency and Program Performance

After implementing AXI4-Lite, the NPC can now be externally connected to a real memory. The object of evaluation will be an NPC with one AXI4-Lite interface, including the AXI4-Lite arbiter just implemented, while the AXI4-Lite-interface memory module implemented through DPI-C is not within the evaluation scope.

Following the same evaluation method, another version of yzh's NPC has a clock frequency of 297.711MHz on the nangate45 process provided by default by the yosys-sta project, so microbench can be calculated to take 1.394s to run, but the simulation took 29.861s. It can be seen that the simulation time increased, because the multi-cycle NPC has a lower IPC than the single-cycle NPC, requiring more cycles to execute the program. Although the IPC decreased, since the clock frequency increased significantly, the program actually runs faster.

Don't forget, the single-cycle NPC evaluation result above is very optimistic, even optimistic to a degree that is not feasible in practice. But the evaluation result of this multi-cycle NPC is much more realistic; at least a 1MB SRAM is achievable. However, this is still quite different from our upcoming tape-out configuration, after all, the tape-out cost of a 1MB SRAM is still very high. Next, we will integrate into the SoC, making the evaluation results closer to the tape-out scenario.

Why Implement the Bus First?

In the past, many students have been very puzzled by this question, and some even thought it was a trap dug by the handouts. The biggest reason is that they think all the code will need to be rewritten when implementing the pipeline in the future. Another voice is that if the single-cycle is written first, it will also need to be rewritten later.

The reason these students think a rewrite is needed later comes partly from traditional textbooks: traditional textbooks do explain the design principles of processors very clearly, but they almost never consider how to smoothly transition between processors of different microarchitectures. After all, this belongs to the realm of engineering practice, and traditional textbooks do not teach it as a knowledge point. So, the other reason is that these students rely too much on traditional textbooks: they just implement according to the textbook content, without thinking about how to adopt appropriate design patterns to achieve the above transition.

For beginners, this is actually not a big problem, because design patterns can only be abstracted and summarized after fully understanding various details; we should not require beginners to come up with a good design pattern at the first learning. But if you want to learn more, you should not regard the textbook content as the whole of processor design, and should not regard rewriting as a waste of time. As a beginner who has not yet thought deeply, rewriting actually contains a huge growth opportunity behind it: why are the two versions of the design so different? Can we abstract and summarize some common features from them? If we were to do this again, how could we do better?

In fact, good experiences and innovation opportunities are all summarized from problems. When you go to work in the future, you will certainly encounter more and different problems, and these problems will no longer have standard answers like textbooks. The knowledge in textbooks is limited, and relying only on textbooks, the height we can grow is also limited; but what can further help us solve unknown future problems is our thinking habits: the thinking habits formed in the process of thinking many times are much more important than the knowledge in textbooks.

Back to the problem of processor design patterns, we had already thought about and summarized relevant experiences in May 2017, and practiced them in the two Longxin Cup competitions participated in by Nanjing University in 2017 and 2018, as well as the first "One Student One Chip" program initiated by the University of Chinese Academy of Sciences in 2019, without causing too much rewriting. Many students are not aware of the situation, so they still make judgments based on their own experience.

Specifically, by using appropriate features of Chisel, changing from single-cycle to multi-cycle only requires modifying 30 lines of code, accounting for 3.75% of the total 800 lines; changing from multi-cycle to a pipeline without forwarding only requires modifying 50 lines, accounting for 5.00% of the total 1000 lines; adding forwarding to the pipeline only requires modifying 20 lines, accounting for 1.82% of the total 1100 lines. Even if you develop in Verilog, if you use correct design patterns, the proportion of code changes should be roughly the same. From the proportion point of view, it is far from the level of needing to rewrite everything.

What we want to say is, as a beginner, you need to keep a curious heart and the philosophy of "you have to practice it yourself to truly know it". When others tell you that you need to rewrite, you should not just listen to these "advice", but should think about why, and delve into and answer this question through your own practice. Because in learning, you are the protagonist.

So Why Implement the Bus First?

This is to practice the design principle of "complete first, perfect later".

If we take the single-cycle processor as the starting point, the pipeline should belong to the "perfect later" part. Therefore, implementing the bus first is actually to move the processor closer to being tape-out ready, achieving "complete first", i.e., removing any feature would either prevent the processor from running RT-Thread, or make the memory and peripherals quite different from a tape-out design.

On the basis of "complete first", we then understand the performance benefits brought by each optimization measure through quantitative evaluation methods. This is also hoping that beginners can quantitatively understand in the future the performance improvement that pipeline design brings to the system, rather than treating pipeline design as a homework of translating block diagrams into RTL code like traditional textbooks do.

Systems with Multiple Devices

So far, our system only has memory. Obviously, a real computer system not only has memory, but also other devices. Therefore, we need to consider how to let NPC access other devices.

When we previously studied devices, we introduced memory-mapped I/O, a mechanism that uses different memory addresses to indicate which device the CPU accesses. We previously implemented memory-mapped I/O in the simulation environment by using pmem_read() and pmem_write() to select the device to access based on the memory access address. But real hardware does not have the two functions pmem_read() and pmem_write() of the simulation environment.

In reality, hardware implements memory-mapped I/O through a crossbar module (sometimes written as Xbar). The Xbar is a multiplexer switch module of the bus, which can forward the request to different outputs based on the address of the bus request on the input side, thereby passing it to different downstream modules. These downstream modules may be devices, or may be another Xbar. For example, in the figure below, the Arbiter is used to select one request from the IFU and LSU to forward downstream; after the downstream Xbar receives the request, it forwards it to the downstream device based on the address in the request.

+-----+      +---------+      +------+      +-----+
| IFU | ---> |         |      |      | ---> | UART|  [0x1000_0000, 0x1000_0fff)
+-----+      |         |      |      |      +-----+
             | Arbiter | ---> | Xbar |
+-----+      |         |      |      |      +-----+
| LSU | ---> |         |      |      | ---> | SRAM|  [0x8000_0000, 0x80ff_ffff)
+-----+      +---------+      +------+      +-----+

If the requested address falls within the address space range of the serial device UART, the Xbar will forward the request to the UART; if the requested address falls within the address space range of the SRAM, it will forward the request to the SRAM. If the Xbar finds that the requested address does not belong to any downstream address space, such as 0x0400_0000, it will return an error decerr through the resp signal of AXI4-Lite, indicating an address decoding error. Here, "address decoding" means converting the requested address into the number of the downstream bus channel, and the address decoder in the Xbar can be regarded as the core module of memory-mapped I/O in hardware implementation.

The Arbiter and Xbar can also be merged into a multi-input multi-output Xbar, also called an Interconnect or bus bridge depending on the occasion. For example, the figure below is a 2-input 2-output Xbar. It can connect multiple masters and multiple slaves. First, the Arbiter records which master the current request comes from, and then decides which slave to forward it to based on the address of that request.

+-----+      +------+      +-----+
| IFU | ---> |      | ---> | UART|  [0x1000_0000, 0x1000_0fff)
+-----+      |      |      +-----+
             | Xbar |
+-----+      |      |      +-----+
| LSU | ---> |      | ---> | SRAM|  [0x8000_0000, 0x80ff_ffff)
+-----+      +------+      +-----+

Physical Memory Attributes (PMA) and Bare Metal Programming

The topology connection method above may also cause some special problems. Since the IFU can be connected to the UART through the Xbar, it means the CPU can also fetch instructions from the UART, but obviously the UART device cannot store programs and instructions.

Therefore, if a program mistakenly jumps to 0x1000_0000, the CPU will send a read request to the UART, and the UART will return a data based on the device's behavior, which may represent an encoding of the UART's internal state, or a character received by the UART. On one hand, the CPU will mistakenly execute the result returned by the UART as an instruction, causing serious errors; on the other hand, the IFU's access to the UART device may change the device's state, causing the UART to enter an unpredictable state.

To avoid causing such problems, some check mechanisms are generally added in hardware, adding several permission attributes for each address space, such as a readable flag, a writable flag, an executable flag, etc. Before the IFU issues an instruction fetch request, it first checks whether the address space to which the requested address belongs is executable; if not executable, it directly throws an exception. In RISC-V, if the address spaces of devices in the system are fixed, permission checks can be implemented through the PMA (Physical Memory Attribute) mechanism; while for modern PCI-e devices whose address spaces are dynamically allocated when the operating system is initialized, permission checks can be implemented through the PMP (Physical Memory Protection) mechanism or the virtual memory mechanism. If you are interested in these two mechanisms, you can refer to the relevant content in the RISC-V manual.

Of course, if these check mechanisms are not implemented in the CPU, you need to be especially careful when developing bare metal programs: if a bare metal program runs off the rails, the consequences can be unimaginable.

With the Xbar in place, we can now move the peripheral functions previously implemented in the simulation environment into RTL.

Implement UART Functionality with an AXI4-Lite Interface

Write a slave module with an AXI4-Lite interface, containing a device register. When a write request is sent to this device register, the lower 8 bits of the written data are treated as a character and output through $write() or printf(). For ease of testing, the address of this device register can be set to be the same as the address of the serial port in the previous simulation environment. After implementation, you also need to write an Xbar module yourself to integrate this module with UART functionality into the system.

In fact, we have not fully implemented a UART with RTL, because $write() or printf() still rely on the simulation environment to implement character output. But as a bus exercise, this is already enough; after all, implementing a UART also needs to consider many electrical details. However, we will soon integrate into the SoC, which contains a real UART controller. By testing the bus implementation through this exercise now, the future SoC integration will also go more smoothly.

Implement CLINT with an AXI4-Lite Interface

CLINT (Core Local INTerrupt controller)open in new window is a relatively general interrupt controller in RISC-V systems, a module used to maintain clock interrupts and software interrupts. However, our system does not need interrupt functionality for now, so we will first consider only the clock-related functionality.

You need to implement a CLINT module with an AXI4-Lite interface and integrate it into the system. CLINT contains a read-only device register mtime, which increases at a certain rate; the simplest implementation is to increment it by 1 each cycle. Likewise, for ease of testing, its address can be set to be the same as the address of the clock in the previous simulation environment.

However, the progression of mtime cannot directly reflect the passage of time; there is a coefficient difference between them, which needs to be read out by software and then processed. In real processor chips, this coefficient is generally equal to the clock frequency in the CLINT module, so that software can measure real time. But there is no concept of a base frequency in the simulation environment; if this coefficient equals the simulation rate, we can calculate the passage of real time from the progression of mtime in the simulation environment. Specifically, you also need to modify the relevant code of IOE so that AM_TIMER_UPTIME returns time close to real time.

Finally, you also need to consider the bit width of the mtime register. The mtime defined in the manual above is 64 bits, to avoid overflow in actual use. But the current NPC is 32-bit; if we only read out the lower 32 bits of mtime, after some time mtime will overflow, causing errors in the system's time functionality. Although it is not easy to run to the moment of mtime overflow in the simulation environment, if the NPC runs at 500MHz in the future, overflow will most likely occur. Therefore, software running on a 32-bit NPC needs to read out the lower 32 bits and the upper 32 bits of mtime in sequence, combine them into a 64-bit value, and provide it to upper-level applications.

yyz

# C4 Bus

# Bus - Communication Protocols Between Hardware Modules

# The Simplest Bus

# Asynchronous Bus

# The RTL Implementation of an Asynchronous Bus