C语言标准规范及其实现

和ISA手册定义指令的语义一样, C语言的语义也是通过相应手册定义的

• C语言的标准经过多年的发展, 演进出各种版本
  • 包括C90, C99, C11, C17, C23
  • 这些版本通常以标准发布的年份来命名
    • 例如, C11标准是在2011年左右发布的
• 我们以C99标准为例进行介绍

C语言标准规范及其实现(2)

C99手册的第3.12节定义了什么是实现:

particular set of software, running in a particular translation environment under
particular control options, that performs translation of programs for, and supports
execution of functions in a particular execution environment

C语言标准的实现是一系列特定软件, 用于将程序翻译到一个特定的执行环境中, 并支持该执行环境中相关功能的执行

执行环境的定义

5.1.2 Execution environments
Two execution environments are defined: freestanding and hosted. In both cases,
program startup occurs when a designated C function is called by the execution
environment... Program termination returns control to the execution environment.

执行环境有两种: 独立环境(freestanding)和宿主环境(hosted)

• 由执行环境调用一个专门的C函数, 来启动程序的执行; 程序结束时返回到执行环境
  • 回顾: 用strace观察程序启动和结束

5.1.2.1 Freestanding environment
1. In a freestanding environment (in which C program execution may take place
without any benefit of an operating system), the name and type of the function
called at program startup are implementation-defined.

执行环境的定义(2)

5.1.2.2 Hosted environments
5.1.2.2.1 Program startup
1. The function called at program startup is named main...

相对地, 在宿主环境中, C程序的执行会在操作系统的帮助下发生

• 在宿主环境下, 这个专门的C函数名称为main
• 这是大家学习C语言时接触得比较多的执行环境
  • Linux就是一个宿主环境

2. ...
— If the value of argc is greater than zero, the string pointed to by argv[0]
  represents the program name; ...
  If the value of argc is greater than one, the strings pointed to by argv[1]
  through argv[argc-1] represent the program parameters.

程序执行的语义

了解C语言标准如何定义程序执行, 有助于进一步认识程序执行的细节

C99手册的第5.1.2.3节定义了程序执行, 我们对这些定义逐条进行说明:

1 The semantic descriptions in this International Standard describe the behavior of
an abstract machine in which issues of optimization are irrelevant.

2 Accessing a volatile object, modifying an object, modifying a file, or calling a
function that does any of those operations are all side effects, which are changes
in the state of the execution environment. Evaluation of an expression in general
includes both value computations and initiation of side effects. Value computation
for an lvalue expression includes determining the identity of the designated object.

程序执行的语义(2)

3 Sequenced before is an asymmetric, transitive, pair-wise relation between
evaluations executed by a single thread, which induces a partial order among those
evaluations. Given any two evaluations A and B, if A is sequenced before B, then the
execution of A shall precede the execution of B. (Conversely, if A is sequenced
before B, then B is sequenced after A.) If A is not sequenced before or after B,
then A and B are unsequenced. Evaluations A and B are indeterminately sequenced when
A is sequenced either before or after B, but it is unspecified which. The presence
of a sequence point between the evaluation of expressions A and B implies that every
value computation and side effect associated with A is sequenced before every value
computation and side effect associated with B. (A summary of the sequence points is
given in annex C.)

• 前序于是针对在一个线程中执行的求值所定义的一个反对称和传递性的二元关系, 通过它可以得到这些求值之间的一个偏序

程序执行的语义(3)

第3条内容借助序列点的概念, 严格定义了不同求值操作之间合法的顺序关系

• 结合副作用, 从而严格定义了整个程序执行的语义

#include <stdio.h>
int f() { printf("in f()\n"); return 1; }
int g() { printf("in g()\n"); return 2; }
int h() { printf("in h()\n"); return 3; }
int main () {
  int result = f() + g() * h();
  return 0;
}

程序执行的语义(4)

4 In the abstract machine, all expressions are evaluated as specified by the
semantics. An actual implementation need not evaluate part of an expression if it
can deduce that its value is not used and that no needed side effects are produced
(including any caused by calling a function or accessing a volatile object).

• 在抽象机中, 所有表达式都按照相应的语义进行求值
• 在一个具体实现中, 如果一个表达式中的其中一部分的值没有被使用, 也没有产生副作用(包括由调用函数和访问volatile对象引起的副作用), 那么, 这部分表达式可以不进行求值

第4条内容其实指示了表达式求值过程中的优化空间

5 When the processing of the abstract machine is interrupted by receipt of a signal,
the values of objects that are neither lock-free atomic objects nor of type volatile
sig_atomic_t are unspecified, as is the state of the floating-point environment. The
value of any object modified by the handler that is neither a lock-free atomic
object nor of type volatile sig_atomic_t becomes indeterminate when the handler
exits, as does the state of the floating-point environment if it is modified by the
handler and not restored to its original state.

程序执行的语义(5)

6 The least requirements on a conforming implementation are:
— Accesses to volatile objects are evaluated strictly according to the rules of the
  abstract machine.
— At program termination, all data written into files shall be identical to the
  result that execution of the program according to the abstract semantics would
  have produced.
— The input and output dynamics of interactive devices shall take place as specified
  in 7.21.3. The intent of these requirements is that unbuffered or line-buffered
  output appear as soon as possible, to ensure that prompting messages actually
  appear prior to a program waiting for input.
This is the observable behavior of the program.

• 一个符合规范的实现至少需要满足3点要求:
  • 这就是之前提到的程序可观测行为的一致性

7 What constitutes an interactive device is implementation-defined.

8 More stringent correspondences between abstract and actual semantics may be
defined by each implementation.

回顾: C语言标准手册定义了一个状态机

• 状态集合\(S = \{<PC, V>\}\)
  • \(V = \{v_1, v_2, v_3, \dots\}\) = 程序中所有变量的取值
  • \(PC\) = 程序计数器 = 当前执行的语句位置
• 激励事件\(E = \{语句\}\)
  • 执行PC指向的语句, 改变程序的状态
• 状态转移规则\(next: S \times E \to S\)
  • 语句的语义(semantics)
• 初始状态\(S_0 = <main函数的第一条语句, V_0>\)

我们可以把这个状态机实现出来, 用它来执行C程序!

• 这里我们假设C程序运行在宿主环境

CEMU = C语言解释器 = 用程序实现C语言状态机

import sys,re
# prepend an empty line to let PC starts from 1
srcs = [''] + list(map(lambda s: s.strip(), sys.stdin.read().split('\n')))
# set PC to the next line of "int main"
state = {'PC': i + 1 for i, line in enumerate(srcs) if line.startswith('int main') }
labels = {}  # record mappings of label -> PC
[labels.setdefault(line.rstrip(':'), i) for i, line in enumerate(srcs) if re.match(r'^\w+:', line) != None]
semantics = [
  (r'^int\s+(\w+)\s*;$',          lambda s, p: exec(re.sub(p, r'\1 = 0xdeadbeef', s), {}, state)),
  (r'^int\s+(\w+)\s*=\s*(.+)?;$', lambda s, p: exec(re.sub(p, r'\1 = \2', s), {}, state)),
  (r'^\w+\s*=.+\s*;$',            lambda s, p: exec(s, {}, state)),
  (r'^printf\s*\(.+\)\s*;$',      lambda s, p: exec(s, {'printf': lambda fmt, *args: print(fmt % args, end='')}, state)),
  (r'^return\s+(.+)\s*;$',        lambda s, p: (print('Exit with %d' % eval(re.sub(p, r'\1', s), {}, state)), exit())),
  (r'^\w+:$',                     lambda s, p: 0),  # do nothing
  (r'^if\s*\((.+)\)\s*goto\s+(\w+)\s*;$',
                                  lambda s, p: exec(re.sub(p, r'if \1: PC = labels["\2"]', s), {'labels': labels}, state)),
  (r'^.*$',                       lambda s, p: print("Not implement: " + s)),
]
while True:
  print(state)
  stmt = srcs[state['PC']]    # read one line of statement
  for pattern, fn in semantics:
    if re.match(pattern, stmt) != None:   # parse it with regular expression
      fn(stmt, pattern) # execute according to the semantics
      break
  state['PC'] = state['PC'] + 1  # read PC again, since it may be changed by the if statement

解释 = 以源语言的语句为对象逐条执行

• 例如, Shell就是一个命令行解释器
• 和编译不同, 后者先翻译成目标语言, 再执行目标语言
  • 但同样通过解释方式来执行目标语言

通过CEMU观察C程序执行的过程

为了方便演示, CEMU只支持少部分较为固定的C语言语法

int main() {
  int x = 10;
  int y = 20;
  int z = x + y;
  printf("z = %d\n", z);
  return 0;
}

$ python cemu.py < a.c
{'PC': 2}
{'PC': 3, 'x': 10}
{'PC': 4, 'x': 10, 'y': 20}
{'PC': 5, 'x': 10, 'y': 20, 'z': 30}
z = 30
{'PC': 6, 'x': 10, 'y': 20, 'z': 30}
Exit with 0

int main() {
  int s = 0;
  int i = 1;
loop:
  s = s + i;
  i = i + 1;
  if (i <= 100) goto loop;
  printf("s = %d\n", s);
  return 0;
}

人生苦短, 我用python

通过各种高级语言特性轻松实现CEMU

• 开箱即用的字符串处理函数: strip(), split(), startwith()
• 方便的容器: 列表, 元组, 字典(可用字符串索引)
  • 及其迭代操作: for ... in ..., map

• 用正则表达式匹配C语句
• 用lambda函数实现C语句的语义
• 黑科技: exec()和eval()
  • 把字符串当作python代码来执行

如果用C语言来实现, 代码量至少翻10倍

CEMU的本质

状态机的4个要素同样存在

while True:
  print(state)
  stmt = srcs[state['PC']]    # read one line of statement
  for pattern, fn in semantics:
    if re.match(pattern, stmt) != None:   # parse it with regular expression
      fn(stmt, pattern) # execute according to the semantics
      break
  state['PC'] = state['PC'] + 1

#            pattern          |           fn
(r'^int\s+(\w+)\s*=\s*(.+)?;$', lambda s, p: exec(re.sub(p,r'\1 = \2',s), {}, state))

CEMU提供的运行时环境

CEMU运行在python环境中, 可以借助python的功能来向C程序提供运行时环境

• 程序运行前 - 通过sys.stdin.read()读入C程序
• 程序运行中 - 通过python的printf()实现C程序的printf()
• 程序运行后 - 通过exit()退出CEMU, 同时也退出C程序

一个案例

#include <stdio.h>
int main() {
  printf(6 - 2147483648 > 6 ? "T" : "F");
  printf(6 - 0x80000000 > 6 ? "T" : "F");
  printf("\n");
  return 0;
}

# apt-get install libc6-dev-i386
# apt-get install lib32gcc-12-dev
clang -w -std=c90 -m32 a.c && ./a.out
clang -w -std=c99 -m32 a.c && ./a.out
clang -w -std=c90 -m64 a.c && ./a.out
clang -w -std=c99 -m64 a.c && ./a.out

clang -fno-color-diagnostics -fsyntax-only -Xclang -ast-dump -w -std=c90 -m32 a.c

2147483648究竟如何被识别

根据clang输出的AST, 整理不同组合下2147483648的类型

结合AST输出和手册描述

• 64位环境下long可以表示2147483648 -> long是64位
• 32位环境下long不能表示2147483648 -> long是32位

#include <stdio.h>
int main() {
  printf("%zu\n", sizeof(long));
  return 0;
}

clang -w -m32 a.c && ./a.out
clang -w -m64 a.c && ./a.out

为什么会这样?

C99的Abstract

... Its purpose is to promote portability, reliability, maintainability, and
efficient execution of C language programs on a variety of computing systems.

3.6 byte

NOTE 2   A byte is composed of a contiguous sequence of bits, the number of which is
implementation-defined

未指定行为(Unspecified Behavior)

use of an unspecified value, or other behavior where this International Standard
provides two or more possibilities and imposes no further requirements on which is
chosen in any instance

C标准提供了多种行为可选, 具体实现需要从中选择一种

An example of unspecified behavior is the order in which the arguments to a
function are evaluated

#include <stdio.h>
void f(int x, int y) {
  printf("x = %d, y = %d\n", x, y);
}
int main() {
  int i = 1;
  f(i ++, i ++);
  return 0;
}

未指定行为(Unspecified Behavior)(2)

• 使用同一个编译器的不同的版本, 也可能会得到不同的运行结果
• 更极端地, 某编译器通过随机方式决定函数调用时的参数求值顺序, 仍然符合C语言标准!

if (rand() & 1) { 从左向右求值; }
else            { 从右向左求值; }

int i = 1;
int x = i ++;
int y = i ++;
f(x, y);

x = 1, y = 2

实现定义行为(Implementation-defined Behavior)

unspecified behavior where each implementation documents how the choice is made

一类特殊的未指定行为, 具体实现需要将选择写到文档里

• 写进文档之后就不能随便改了
  • 具体实现不仅需要遵循C语言标准, 还需要遵循自己的文档

5.2.4.2 Numerical limits

An implementation is required to document all the limits specified in this subclause

5.2.4.2.1 Sizes of integer types <limits.h>

... Their implementation-defined values shall be equal or greater in magnitude
(absolute value) to those shown, with the same sign.

例子: 整数类型的长度

C语言标准并没有明确定义类型的长度

• 不过定义了类型取值的最小范围, 具体实现可以采用更大的范围

区域特定行为(Locale-specific Behavior)

behavior that depends on local conventions of nationality, culture, and
language that each implementation documents

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <assert.h>
#define 主函数 main
#define 返回 return
char* 字符串拼接(char *串1,
  char *串2) {
  char *新串 = malloc(
    strlen(串1) +
    strlen(串2) + 1);
  assert(新串);
  strcpy(新串, 串1);
  strcat(新串, 串2);
  返回 新串;
}
int 主函数() {
  char *信息 = 字符串拼接(
    "一生一芯", "很简单");
  printf("%s\n", 信息);
  free(信息);
  返回 0;
}

未定义行为(Undefined Behavior)

behavior, upon use of a nonportable or erroneous program construct or of erroneous
data, for which this International Standard imposes no requirements

程序/数据不符合标准的行为, C语言标准对其结果不作任何约束

• 一切皆有可能: 无论发生什么, 都符合C语言标准

Possible undefined behavior ranges from ignoring the situation completely with
unpredictable results, to behaving during translation or program execution in a
documented manner characteristic of the environment (with or without the
issuance of a diagnostic message), to terminating a translation or execution
(with the issuance of a diagnostic message).

#include <stdio.h>
int main() {
  int a[10] = {0};
  printf("a[10] = %d\n", a[10]);
  return 0;
}

学会RTFM

• 想了解一切细节发生的依据, RTFM是唯一正确的选择
  • C99手册的附录J列出了C语言标准中无法精确定义的所有行为

开放讨论 - 为什么还要学习C语言?

“使用语言”和 “学习计算机”的目的不完全相同

• 使用语言的目标是提高效率
  • 开发者友好的语言特性, 更抽象也更安全的功能, 各种开箱即用的库
    • 例 - 数组越界不再是UB, 而是由运行环境马上报错
    • 这些功能需要更复杂的翻译环境和运行环境来支撑
  • 作为计算机的使用者, 我们应该掌握1~2门现代语言提升工作效率
    • python, bash, go, rust, …

开放讨论 - 公司不一定完全按标准来实现

• 从一个领域来看, 遵循标准有利于领域的稳定和繁荣
  • TCP/IP(网络), SQL(数据库), OpenGL(图形), 3GPP组织(通信), …

• 如果一家公司选择不遵循按标准中的某些要求, 那么
  • 它需要声明其产品在哪些场景不兼容标准, 让用户选择是否接受

• 如果一套标准过于糟糕, 公司不愿意遵循, 那么, 对这个领域来说
  • 最好的做法是建立另一套更好的标准
  • 而不是进入七国八制的状态(1987年中国通信市场)

• 公司选择不完全按标准来实现, 属于个体决策
• 标准的意义体现在更宏观的层面: 对领域技术的创新, 应用和演进
  • 扩展阅读 - RISC-V：全球共有开放标准，全力推进开放计算

C语言标准的具体实现和ABI

回顾C语言标准的具体实现:

• 编译器 - 负责生成程序的二进制可执行文件
  • 二进制可执行文件和ISA相关, 程序也是在相应ISA的处理器上执行
• 运行时环境 - 负责支撑程序的运行
  • 包含库函数和操作系统的部分功能, 程序在运行的前, 中, 后都需要和它们交互

实现定义行为的选择写到了哪个文档里?

C语言标准要兼容各种计算机系统, 无法精确定义很多行为的结果

但对于一个特定的计算机系统, 很多条件是确定的

• ABI作为这个计算机系统在二进制层次的约定, 可以看成是C语言标准的一种具体实现的文档
• 对于C语言标准层次中的很多实现定义行为, 其选择都会写入ABI中

RTFM: ABI中定义的基本数据类型

• System V ABI for i386 - 32位x86 Linux遵循的ABI
• ABI for RISC-V

 int8_t;  int16_t;  int32_t;  int64_t;
uint8_t; uint16_t; uint32_t; uint64_t;

char c = 0xff;
printf(c == 0xff ? "T\n" : "F\n");

ABI和计算机系统

ABI作为一种规范, 其内容包括:

• 处理器的指令集, 寄存器结构, 栈的组织, 访存类型等
• 处理器可直接访问的基本数据类型的大小, 布局, 对齐方式
• 调用约定, 用于规定函数的参数如何传递, 返回值如何获取
• 应用程序如何向操作系统发起系统调用
• 目标文件的格式, 支持的运行库等

• 程序在特定计算机系统上的运行结果, 与源代码, 编译器, 运行时环境, ISA, 硬件等都有关联
• ABI是计算机系统软件和硬件互相协助, 共同支撑程序运行的重要体现

C程序的执行

• C语言标准的实现 = 编译器 + 运行时环境 = 语言系统
  • 两种执行环境: 独立环境, 宿主环境
• 程序的执行: 副作用, 前序于, 抽象机, 程序可观测行为的一致性
  • CEMU = C语言解释器 = 用程序实现C语言状态机

• C语言标准中除了精确描述的行为, 还包含
  • Unspecified Behavior
  • Implementation-defined Behavior
  • Undefined Behavior

• 通过ABI手册了解Implementation-defined Behavior的选择
  • 同时认识C语言标准, 编译器, 操作系统, 运行库, 处理器之间的协助
  • 程序的运行结果与源代码和上述因素都有关系