Most of the time, we write C code that compiles directly to machine code for a specific CPU. But if you've ever wondered how languages like Python or Java work, you're looking at the concept of a Virtual Machine (VM). Instead of targeting a physical chip, they target a "virtual" one. They compile their source code into bytecode—a set of simplified instructions that a software program (the VM) then interprets.

To understand this, we aren't going to build a full-blown language. Instead, we'll build a small, stack-based VM. In a stack machine, you don't have general-purpose registers like EAX or RBX; you just push values onto a stack and perform operations on whatever happens to be on top.

Defining our custom instruction set

First, we need to decide what our VM can actually do. I want this VM to be able to push numbers and perform basic math. I'll use an enum for the opcodes because it keeps the code readable. If I just used raw numbers, I'd be staring at 0x01 and 0x02 and losing my mind within an hour.

typedef enum {
    OP_PUSH,
    OP_ADD,
    OP_SUB,
    OP_MUL,
    OP_HALT
} OpCode;

Our "program" will just be an array of integers. Some of these integers will be opcodes, and some will be the data (operands) those opcodes need.

Setting up the machine state

A VM is essentially just a struct that tracks its own internal state. We need a place to store the bytecode, a stack to do the work, and an instruction pointer (ip) to know where we are in the program.

#define STACK_SIZE 256

typedef struct {
    int stack[STACK_SIZE];
    int sp; // Stack Pointer
    int ip; // Instruction Pointer
    int running;
} VM;

void vm_init(VM* vm) {
    vm->sp = -1;
    vm->ip = 0;
    vm->running = 1;
}

The fetch-decode-execute loop

This is the heart of the VM. We fetch the instruction at the current ip, decode what it means using a switch statement, and then execute the logic. Let's try to calculate (10 + 5) * 2. In bytecode, that looks like: PUSH 10, PUSH 5, ADD, PUSH 2, MUL.

void vm_exec(VM* vm, int* program) {
    while (vm->running) {
        int opcode = program[vm->ip];
        
        switch (opcode) {
            case OP_PUSH:
                vm->stack[++vm->sp] = program[vm->ip]; 
                break;
            case OP_ADD: {
                int a = vm->stack[vm->sp--];
                int b = vm->stack[vm->sp--];
                vm->stack[++vm->sp] = a + b;
                break;
            }
            case OP_MUL: {
                int a = vm->stack[vm->sp--];
                int b = vm->stack[vm->sp--];
                vm->stack[++vm->sp] = a * b;
                break;
            }
            case OP_HALT:
                vm->running = 0;
                break;
        }
        vm->ip++;
    }
}

Correcting a critical pointer mistake

If you tried to run the code above, you'd hit a wall immediately. I just made a classic mistake: look at OP_PUSH. I'm reading the value from program[vm->ip], but vm->ip is currently pointing at the OP_PUSH opcode itself! I'm pushing the opcode onto the stack instead of the value that follows it.

Furthermore, I'm only incrementing vm->ip once at the bottom of the loop. For OP_PUSH, we actually need to move the pointer twice: once to get past the opcode, and once to get past the value we just pushed. Otherwise, the VM will try to execute the value 10 as if it were an opcode in the next iteration.

Here is the fix for that specific case:

case OP_PUSH:
    // Move to the next element to get the actual value
    vm->ip++; 
    vm->stack[++vm->sp] = program[vm->ip]; 
    break;

Putting it all together

Now we can define our program array and run the VM. I'll wrap it in a main function to see the result. Notice how the bytecode array is just a linear sequence of numbers.

#include <stdio.h>

int main() {
    int program[] = {
        OP_PUSH, 10,
        OP_PUSH, 5,
        OP_ADD,
        OP_PUSH, 2,
        OP_MUL,
        OP_HALT
    };

    VM vm;
    vm_init(&vm);
    vm_exec(&vm, program);

    printf("Final Result: %d\n", vm.stack[vm.sp]);
    return 0;
}

When this runs, the stack evolves like this: [10] → [10, 5] → [15] → [15, 2] → [30]. It's a simple pattern, but this is exactly how the Java Virtual Machine (JVM) handles expressions under the hood.

📋 Practical Task

Implementing the OP_PRINT Instruction

Currently, our VM only tells us the result after the entire program has halted. In a real-world scenario, you'd want to be able to inspect the state of the machine while it's running.

Your task is to modify the VM to support a new instruction: OP_PRINT.

• Add OP_PRINT to the OpCode enum.
• Implement the logic inside the vm_exec switch statement. The OP_PRINT instruction should peek at the top value of the stack and print it to the console using printf, without removing the value from the stack.
• Update the program array in main to print the intermediate result after the OP_ADD operation but before the OP_MUL operation.