C2 Devices and Input/Output

You have already implemented TRM in both NEMU and NPC. The next step, of course, is to make them support input and output. Before that, however, let's first get some hands-on experience with GUI application development.

Programs Supporting GUI Input and Output

You previously made the minirv processor run a program that displays an image in Logisim. Can this program also run in minirvEMU? Also, the C programs you previously developed could only perform input and output through the terminal. How are the GUI programs we commonly use developed? To build a basic understanding, let's first let you experience running NES games.

Run NES Games

Read the beginning of the PA1 handout section Before starting the pleasant PA journey -> What is NEMU?, follow the handout instructions to run NES games, and complete the checks for display, keyboard input, and sound.

You must be curious about how NES games actually run. In fact, a game itself is also an instruction sequence, but its instructions belong to an instruction set called 6502. This instruction set appeared in the 1980s and is now rarely used, and corresponding processors are not very common either. But just now, you did not run the game on a real 6502 processor. So how did the game run? What actually happened is that a 6502 instruction set emulator program, fceux, was run first. This program has very special behavior: it can simulate the execution process of 6502 instructions through software behavior, thereby simulating the execution process of the whole game!

Experiencing GUI Input and Output

For now, however, we first consider GUI-program-related content, and later return to emulators. We introduce a runtime environment called abstract machine, abbreviated AM. It provides some APIs (Application Programming Interfaces) in the form of library functions, used to implement input and output functionality commonly used in GUIs. The fceux program just mentioned implements game controls and display by calling these APIs. But fceux's behavior is still too complex for everyone, so we first use some simple example programs to experience the input and output functionality used by fceux.

Experience the Clock Functionality

Run the clock test program with the following commands:

cd am-kernels/tests/am-tests
make ARCH=native mainargs=t run

You will find that the program outputs a line in the terminal every 1 second. In addition, the program pops up a new all-black window, but this window has no functionality in the current program, so you do not need to care about it for now.

After experiencing this functionality, try reading am-kernels/tests/am-tests/src/tests/rtc.c to understand how it is implemented. The code io_read(AM_TIMER_UPTIME).us obtains the elapsed time since the program started running, in units of us.

Experience Keyboard Functionality

Run the keyboard test program with the following commands:

cd am-kernels/tests/am-tests
make ARCH=native mainargs=k run

You will find that the program pops up an all-black new window. Press keys in the new window, and you will see the program output corresponding key information in the terminal, including key name, key code, and key state.

After experiencing this functionality, try reading am-kernels/tests/am-tests/src/tests/keyboard.c to understand how it is implemented. For now, you can ignore code related to uart. The code io_read(AM_INPUT_KEYBRD) obtains a key event ev; ev.keycode indicates the key code, and ev.keydown indicates whether the key is pressed or released. Key code values can be found in abstract-machine/am/include/amdev.h. They all use the AM_KEY_ prefix, such as AM_KEY_A for the A key. In particular, AM_KEY_NONE indicates no key event.

Experience Display Functionality

Run the display test program with the following commands:

cd am-kernels/tests/am-tests
make ARCH=native mainargs=v run

You will find that the program pops up a new window and plays an animation.

After experiencing this functionality, try reading am-kernels/tests/am-tests/src/tests/video.c to understand how it is implemented. The code io_write(AM_GPU_FBDRAW, x * w, y * h, color_buf, w, h, false) means drawing a rectangular image of size w*h at screen coordinate (x * w, y * h). Image pixels are stored in row-major order in color_buf, and each pixel uses a 32-bit integer in 00RRGGBB format to describe its color.

Designing a Simple Screensaver

After understanding how to use GUI input and output in code, we will next implement a simple screensaver to learn how to develop a simple GUI program on AM.

First, try making the window that pops up display some color. We provide the following skeleton code:

#include <am.h>
#include <klib-macros.h>

void draw(uint32_t color) {
  // add code here
}

int main() {
  ioe_init(); // initialization for GUI
  while (1) {
    draw(0x000000ff);
  }
  return 0;
}

To use the functionality provided by AM, we need to write a Makefile that conforms to the AM specification, for example:

NAME = screensaver
SRCS = screensaver.c
include $(AM_HOME)/Makefile

If the filename of your C code is not screensaver.c, modify the Makefile above according to the actual filename. Then compile and run with:

make ARCH=native run

You should see the program pop up a new, all-black window. This is because you have not correctly implemented the void draw(uint32_t color) function yet. We will introduce more details about the Makefile above and AM in the D stage. For now, you only need to know how to use the functionality provided by AM to develop, compile, and run programs.

Implement Single-color Display

Implement the draw() function above. It should fill the popped-up window with the color indicated by the argument color. The window resolution is 400x300.

After implementing it, compile and run again. If your implementation is correct, you should see the popped-up window filled with blue.

The effect we need to implement for the screensaver is to make the screen gradually transition among multiple colors. The gradient algorithm needs to linearly interpolate several colors between the current color and the target color and display them in order, thereby achieving the gradient effect. Suppose the current color is c0, the gradient changes to the target color c1 after n steps, and each color component has a value range of [0, 255]. Then the color displayed at step i can be computed by linearly interpolating each of the R, G, and B components between c0 and c1.

After one gradient round ends, choose the next target color for the next round. The target color can be randomly chosen from the following colors:

0x000000, 0xff0000, 0x00ff00, 0x0000ff, 0xffff00, 0xff00ff, 0x00ffff, 0xffffff

We only need to use clock-related code to control the interval between screen color updates to implement the gradient effect.

Implement the Color Gradient Effect

Implement the color gradient effect according to the description above. You may decide how long each gradient round lasts and how many steps each round contains.

Finally, let's use keyboard functionality. Specifically, we want to implement the following key behaviors:

If the ESC key is pressed, whose key code is AM_KEY_ESCAPE, exit the program.
If any other key is pressed, speed up the current gradient round; after the key is released, restore the gradient duration.

Add Keyboard Effects

Add keyboard effects to the screensaver according to the description above.

How Does a Screensaver Protect the Screen?

In the past, people usually used CRT displays. Their working principle is to use high-speed electron beams emitted by an electron gun to bombard the phosphor layer of the screen, causing it to emit light and display images. But if the displayed image does not change, the electron beam bombards the same position of the phosphor layer for a long time, causing accelerated aging of that part of the picture tube or even breakdown of the picture tube, damaging the screen. A screensaver continuously changes the image displayed on the screen, thereby continuously changing the position and intensity of electron-beam bombardment and avoiding screen damage.

Nowadays, however, people more often use LCD displays, whose working principle differs from CRT displays. Past screensavers no longer provide protection today. The required exercise above is only a simple exercise and does not consider its protective function. In fact, if the screen image does not change, it usually means the user has not operated the computer for a long time. At that point, the best practice is to turn off the display.

After implementing the screensaver, you should have a basic understanding of GUI program development. However, we have not further explained exactly how io_read() and io_write() implement input and output. Why can calling io_read(AM_INPUT_KEYBRD) obtain key information? Why can calling io_write(AM_GPU_FBDRAW, x, y, pixels, w, h, true) draw pixels to the screen? In a real computer system, this requires cooperation among modules at multiple abstraction levels, including the runtime environment, instruction set, processor, bus, device controller, and I/O device. As your learning progresses, you will understand every detail of this process in the B stage.

Developing Your Own Mini-game

If you are interested, you can also use the APIs above to develop your own mini-game! A so-called mini-game is essentially just an infinite loop:

while (1) {
  wait_for_a_new_frame();
  process_user_keys();
  update_game_logic();
  draw_new_screen();
}

The screensaver you just developed is essentially such an infinite loop, except that its game logic is very simple and only includes handling a gradient algorithm. In addition, am-kernels provides some mini-game examples:

am-kernels/kernels/typing-game - typing game
am-kernels/kernels/snake - Snake

Develop Your Own Mini-game

Try conceiving and designing a mini-game. On one hand, this mini-game can run on the processor you design yourself in the future. On the other hand, we have long been collecting fun and interesting mini-games. If the game you design is good enough, it may be included in the game collection we maintain for other students to enjoy, and may even be demonstrated on real chips.

Some useful resources:

Displaying characters: am-kernels/kernels/typing-game/font.c provides character bitmaps. Please read the relevant code to understand how to use it.
Displaying images: you can use the command convert abc.jpg abc.txt to convert an image into a text file enumerating pixels. The file lists the color encoding of every pixel in order. You can further process this text file with a script and convert it into a C array.

If you plan to submit your designed game to us, contact a TA. To ensure your game can run in other students' environments, additionally ensure the following:

Portability: do not use floating-point numbers in the code. If decimal-related computation is needed, scale it to integers before operating, for example, to handle a 3-decimal-place number, scale it by 1000 before processing. If elementary functions such as sin need to be computed, use power-series expansion for approximation. If a 64-bit integer type is needed, use uint64_t or int64_t. If an integer type is needed to store pointers, use uintptr_t or intptr_t.
Playability: after entering the main game, if the player does nothing for a period of time, the game should fail.
Exit support: after pressing ESC, the code calls halt(0) to exit.

Implement Input/Output in NEMU

Follow the PA lecture notes to complete Phase 3 of PA2 until you see the following prompt box:

Friendly Reminder

PA2 ends here.

Input/Output in NPC

For the RISC-V architecture, input/output is implemented through MMIO. With the memory read and write functions based on DPI-C, we can now implement input/output for NPC without modifying the RTL code! We just need to perform simple address range checks in these two functions to redirect memory access requests from NPC to the correct devices. MMIO is implemented over a bus in hardware., and we will implement true MMIO later.

Regarding devices, we are not directly using NEMU's device model here. Instead, we are implementing a device model for NPC in the simulation environment that resembles the future SoC for chip fabrication. Once we implement the bus, we will then implement RTL versions of the devices based on the bus. At this point, the role of the IOE abstraction in the abstract machine (AM) becomes apparent: the device addresses and device models for NEMU and NPC are different, but after abstraction, they can both run the same NES emulator source code. Furthermore, all programs on the AM do not need to be written differently for different execution environments.

Running an NES Game in NPC

You have already completed NEMU with support for RV32IM and peripherals, and successfully run an NES game on NEMU. Similarly, we can also run an NES game on NPC with RV32E, but this first requires implementing a serial port and a clock.

We know that RISC-V processors access peripherals through MMIO. For example, in NEMU, the serial port is mapped to 0xa00003f8. Similarly, we can implement a simple serial port and a clock in NPC's simulation environment. As mentioned in the previous section, we use DPI-C to let NPC call the read and write functions pmem_read() and pmem_write() to access memory. Similar to NEMU, we can add several checks in these two functions to implement MMIO functionality. Pseudocode is shown below:

extern "C" int pmem_read(int raddr) {
  // always read 4 bytes from the address `raddr & ~0x3u` and return
  if (raddr == clock_address) { return current_time; }
  ...
}
extern "C" void pmem_write(int waddr, int wdata, char wmask) {
  // always write `wdata` into the 4 bytes aligned to the address `waddr & ~0x3u` according to the write mask `wmask`
  // each bit in `wmask` acts as a byte-level mask for one byte in `wdata`,
  // e.g. `wmask = 0x3` means only the lowest 2 bytes are written, leaving other bytes in memory unchanged
  if (waddr == uart_address) { putchar(...) }
  ...
}

Add a Serial Port and a Clock to NPC

Implement the serial port output functionality in NPC's simulation environment and run the hello program. To keep the serial port address consistent with the later SoC, you can set NPC's serial port address to 0x10000000.

Add a Clock to NPC

Implement a clock in NPC's simulation environment and run the real-time clock test from am-tests. You can implement the clock functionality based on system time. Which C library is related to system time, and how to obtain system time, is left for you to STFW.

Run the Character Version of the NES Game

Try running the character version of the NES game on NPC. Note that NPC's simulation environment does not support keyboard input at this point.

yyz

# C2 Devices and Input/Output

# Programs Supporting GUI Input and Output