life_of_sems セマ

What I learned in 2024 - Computer Science

Published on 2025-01-06

|

LEARNING, REVIEW

Table of content

  1. Introduction

2024 marked a deep dive into systems programming, distributed computing, and blockchain architectures. This technical retrospective covers implementations and insights across several fundamental computer science domains, with a particular focus on consensus systems and virtual machines.

  1. Bitcoin node

The Bitcoin protocol implements a UTXO-based model, contrasting with the account-based state model used in smart contract platforms like Ethereum. Each transaction consumes previous UTXOs as inputs and generates new UTXOs as outputs. Access control for spending UTXOs is enforced through Bitcoin Script - a purposely limited stack-based virtual machine that enables various transaction patterns including Pay-to-PubkeyHash (P2PKH), Pay-to-Script-Hash (P2SH), native Pay-to-Public-Key, and M-of-N Multi-Signature schemes.

My implementation focused on four core components:

Block Generation and Verification

  • Implemented core data structures for blocks, headers, and transactions
  • Developed a Proof-of-Work system with configurable difficulty targeting
  • Built cryptographic account management for key pair generation
  • Created comprehensive transaction validation, including UTXO set management and script execution

Transaction Processing

  • UTXO set indexing and retrieval
  • Output creation with configurable script conditions
  • Multi-threaded transaction verification pipeline
  • Script interpreter for standard transaction types

Network Layer

  • Peer-to-peer network implementation using a gossip protocol
  • Asynchronous block and transaction propagation
  • gRPC interface for client interactions

Rust Implementation Highlights

The project served as a practical exploration of Rust’s concurrency primitives:

  • Leveraged thread-scoped operations and atomic reference counting for UTXO set management
  • Implemented async workflows using Tokio for network operations
  • Integrated Tonic for gRPC service definitions and code generation
  • Balanced sync/async boundaries in the verification pipeline

Links:

  1. Distributed Systems

While building a Bitcoin node helped me understand one specific distributed system, I wanted to dive deeper into distributed systems theory more broadly. This led me to study distributed systems.

MIT’s 6.824 distributed systems course provided a comprehensive examination of distributed system architectures, from foundational papers to modern implementations. The course’s value lies in its systematic analysis of how different systems handle the fundamental challenges of distribution:

System Analysis

  • Examined early distributed systems and their evolution: Map-Reduce -> Apache Spark, Google File System
  • Analyzed consistency models in modern databases: Aurora’s storage-compute separation, DynamoDB’s eventual consistency tradeoffs, Spanner’s TrueTime implementation
  • Studied consensus mechanisms: Raft and practical Byzantine Fault Tolerance

Implementation Projects

Built three core components that exemplify key distributed systems principles:

  1. Reliable Transport Layer
  • Implemented at-least-once and exactly-once delivery semantics
  • Developed efficient message deduplication with bounded memory usage
  1. Raft Consensus Implementation
  • Leader election with term-based safety guarantees
  • Log replication with strong consistency
  • Implemented log compaction and snapshot mechanisms
  1. Distributed Key-Value Store
  • Built on Raft for consistent replication
  • Implemented linearizable read/write operations
  • Developed efficient state transfer for new nodes

Links:

  1. Operating Systems

Revisited systems programming fundamentals through TU Graz’s Operating Systems course, implementing core OS components from scratch:

I felt like I had some gaps in my understanding of operating systems that I wanted to fill. I decided to revisit the TU Graz’s Operating Systems course. The bachelor course takes you on a exhilarating ride through the inner workings of processes, threads, memory, drivers and many other concepts.

Implemented OS Components:

Thread Management

  • Implemented user-space threading library compatible with pthreads API
  • Developed virtual memory management for thread stacks
  • Built thread scheduling and context switching mechanisms
  • Implemented fork/exec

Memory Management

  • Designed demand paging system with disk swapping
  • Implemented multiple page replacement algorithms: Random, Aging, Clock
  • Built stack growth detection and handling
  • Developed memory protection with guard pages and stack canaries

  1. Ethereum Virtual Machine

This project is work in progress, but can already execute basic smart contracts.

Current work-in-progress implementing an EVM from specification. Key components implemented:

Ethereum’s blocks hold the full state of the “world”, such as: account balances, contracts and their storages (more? todo). Addresses (public keys) are either owned by users (externally owned account) or smart contracts. From a high-level, the tasks performed by Ethereum are very similar to Bitcoin’s. A contract (a program) can “live” at an address, whose functions can be called through transactions. There is also a special transaction to create a smart contract, which is triggered by defining the destination address to 0.

Core Architecture

  • Account management using k256 for cryptography
  • Block and transaction data structures
  • Bytecode interpreter with full EVM operation set

Execution Environment

  • Stack-based execution engine
  • Some EVM operations
  • Memory management with dynamic expansion
  • Storage access with storage reverts
  • Rudementary gas metering and consumption calculation

Unfinished work (To-Do in 2025)

  • Implement all EVM operations
  • Improve gas tracking and consumption calculation (EIP 1559)
  • Implement the modified merkle patricia trie
  • Explore EIPs (EIP-1559, Account Abstraction)

Links:

  1. Leetcode

This year I was on the job market, looking to shift from an automotive to a pure-software company. In most such companies, it is expected that the interview will include some technical challenge, whether it is Leetcode style problems or otherwise. In the prior years, I had fun solving the Advent of Code, and I found Leetcode’s 75 Study Plan adaquete as refresher and quite fun. It helped me identify areas that are weaker and strengthen them through deliberate practise.

Links:

  1. Additional Technical Deep Dives


I’m looking forward to learn more about the EVM, consensus algorithms, networking protocols, ZK-Proofs and many other exciting new things. That’s it for 2024! Onwards to the 2025 and all its blessings and challenges.

GitHub YouTube Twitch X