Operating Systems and App Distribution Landscape

By the end of this post, you will gain insight into the various distribution options available to developers for their apps, along with an overview of the architecture of the most popular operating systems. Additionally, you will develop an understanding of how apps ultimately run on these platforms. The essence of the material is illustrated in the diagram below. I encourage you to explore each operating system and browser pathway. If you come across any unfamiliar key term, you can refer to the explanations provided beneath the diagram.

👉 By the way, 📚🚀 I invite you to dive into my sci-fi epic, GÖD’S GATE! Alongside an action-packed storyline, I’ve woven in some cool computer science elements. Whether you’re a sci-fi fan or a computer scientist, this one’s for you. 👾👾

Download PDF

Code Execution

• Source Code (SC): Human-readable instructions written in a programming language (e.g. C++, Java, etc.) forming the original version of the software (SW) before compilation.
• Intermediate Representation (IR) / Intermediary Language: Code format between SC and Binary (e.g. Kotlin/Scala/Python/Java/Dex Bytecode, LLVM Bitcode for C/C++, Objective-C, and Swift, Microsoft’s Common Intermediate Language (CIL) for C#) that is designed to:
  (i) be independent of any machine architecture (i.e. portable),
  (ii) interpreted by a virtual machine (VM), or
  (iii) further compiled into Binary, often through a (e.g. VM’s) Just-In-Time (JIT) compiler (see below). JS engines in web browsers convert JS to an AST (see below) and then to an internal Bytecode, which is typically JIT-compiled.
• Assembly Code/Language: Assembly code is a human-readable, low-level programming language that uses mnemonic codes and symbols to represent machine-level instructions, registers, memory addresses, and other elements of a computer’s architecture. The assembly language is dependent of the CPU architecture.
• Binary (Machine Code): Executable machine code consisting of zeros and ones, compiled from SC or IR (e.g. Bytecode), that can be directly executed by a computer’s CPU.
• Transpilation: Process of converting SC in one high-level programming language into another high-level programming language (e.g. from C/C++ to JavaScript).
• Compilation: Compilation is the process through which source code written in a high-level programming language is transformed into IR or executable machine code that a computer’s CPU can understand and execute. Steps:
  1. Pre-processor (for C/C++ and objective-C): includes macros, files, compilation conditions, etc. in the SC (macros are pre-processor features that define reusable code that can be inserted into source code before compilation);
  2. Compiler:
     (i) Lexical analysis (a scanner reads and breaks down SC into tokens);
     (ii) Syntax analysis (following the language’s grammar rules, a parser creates an Abstract Syntax Tree (AST), which represents the hierarchical structure of the SC, with nodes that represent constructs like loops, conditionals, functions, etc.),
     (iii) Semantic analysis (checks the AST for errors),
     (iv) Intermediate code generation (i.e. translation into an IR),
     (v) Optimization (remove unnecessary code and optimize code, e.g. remove or include loops),
     (vi) Code generation (for each instruction in the IR, the compiler generates a sequence of instructions in processor-specific Binary (e.g. Java, C#, Objective-C and Swift using LLVM) or assembly (e.g. C/C++) code, e.g. how the CPU should store variables in its registers);
  3. Assembler: If the output of the code generation step is assembly code (from C/C++, Rust, Fortran), the assembler translates such code into Binary (so-called object code files). The assembler is typically absent in JIT compilers (se below);
  4. Linker: Assigns memory addresses to variables and functions and resolves references, i.e. if the output object code file uses a function or variable in another object code file (e.g. OS native libraries or other files from the developer), the linker combines or links them in the final executable file:
     (i) Static linking combines all the necessary libraries and object code files into a single executable at compile time. This includes code from thedeveloper as well as any third-party libraries that the program uses. Pro: self-contained / Con: larger file size.
     (ii) Dynamic linking links the native/third-party libraries at runtime rather than at compilation time, only references to the libraries are included in the executable. Pro: smaller size file, easier updates, programs can share library files in memory / Con: if the library is not present, the program will not run.
• Loading: The loader, a specialized OS module, loads an executable file from disk into memory, along with dynamically-linked components. Once loaded, the loader transfers control to the program’s logic, at which it begins execution.
• Ahead-of-Time (AOT) Compilation: AOT compilation happens before the program is run, often at build time (not at runtime). The source code or intermediate representation (IR) is fully compiled into Binary, resulting in an executable file. Programs usually start faster, but are targeted to a specific CPU architecture, i.e. the program is not portable.
• Just-In-Time (JIT) Compilation: JIT compilation happens during program execution, i.e. at runtime. The Bytecode or other IR is compiled into machine code on the fly, as the program runs. Programs usually start slower, but the IR is portable across architectures and the compiler handles the generation of the Binary for the specific platform.
• Interpretation: Process by which a programming language interpreter directly executes the instructions written in the SC without requiring them to be previously compiled into Binary, e.g. the CPython interpreter of Python SC (process: SC -> AST -> Bytecode -> interpretation by the Python Virtual Machine). That is, no additional machine code instructions are required, such instructions already exist in the interpreter’s codebase (i.e. the interpreter has a “mapping” of SC-level to machine-level instructions). In other words, the SC tells the interpreter the instruction to trigger. The interpreter reads and executes the SC line by line, translating it into operations that the computer can perform immediately. Interpretation provides portability but often runs slower due to the its overhead.
• Runtime: The environment in which a SW program executes. It consists of the SW stack that include the application code itself, libraries, variables that configure the environment, and services (e.g. memory management, I/O, etc.) that allows the SW program to run. The runtime manages the lifecycle of the SW program: start, execution, and termination.

OS Kernels

• Monolithic: A monolithic kernel is a type of OS architecture where the entire OS, including the scheduler, file system, device drivers, and memory management, is a single large program (kernel) running in a single address space. All these components have direct access to the system hardware. This design allows for high performance and efficient system calls since everything is closely integrated. E.g. Linux, FreeBSD.
• Microkernel: Runs the bare minimum amount of software required for the kernel to operate. In this design, the microkernel only includes core functionalities such as low-level address space management, thread management, and inter-process communication (IPC) management. Other services like device drivers, file systems, network stacks, and user services may run in user space or other kernel-level modules as separate processes. This separation can lead to better system stability and security, as a failure in one component does not necessarily crash the entire system.
• Hybrid: The kernel provides more services in the kernel space than a pure microkernel but still tries to maintain separation for key components to increase stability and maintainability. For example, some device drivers may run in kernel space for performance reasons, while others run in user space for increased stability.

iOS/macOS

Android

Windows

Linux

Chip Architectures

It encompasses primarily the memory and CPU design and its Instruction Set Architecture (ISA):

• ISA: Definition of the binary machine code instructions that a CPU can execute. Assembly code is a human-readable representation of that collection of commands that a particular CPU can execute (see above). The CPU is hardwired to recognize the binary pattern to execute the instruction.
• Multicore and Multithread: A chip could have multiple cores, i.e. CPUs, and each can hold the state (momentary status of a system, i.e. the values for all variables, data, and configurations) of multiple threads (the smallest unit of processing that can be scheduled by an OS).
• System on a Chip: A System on a Chip (SoC) is an integrated circuit that combines all or most components of a computer or other electronic system onto a single chip: CPU, memory, input/output ports, and often GPU, network connectivity modules, audio and video processing unites, power management circuits, neural processing units, etc. SOC aims at improving performance while being power efficient. SOC dominates the mobile device market.
• Chipset: In traditional machines, instead of SoC, the chipset is an integrated circuit of the mainboard that manages the communication between high-speed devices like the CPU, RAM, the Peripheral Component Interconnect bus, and GPU (Northbridge) and lower-speed I/O peripherals like hard drives, USB ports, and expansion slots (Southbridge).
• ARM: Chip architecture that uses RISC (Reduced Instruction Set Computing) with a smaller set of simpler and faster instructions that do less work individually but can be combined. ARM emphasizes power efficiency and thus dominates the mobile and embedded devices ecosystem, and it is making an entry into desktops and servers, which typically run x86. ARM Holdings licenses this architecture to chip manufacturers.
• X86: Chip architecture that uses CISC (Complex Instruction Set Computing) design, a large set of instructions that can perform complex tasks with a single instruction. Used where power is a critical factor, like demanding workloads in enterprise use cases and gaming. It was primarily developed by Intel and AMD.

Web Browsers

• Rendering Engine: Part of the browser engine that is responsible for parsing, translating (into the DOM and CSSOM) and interpreting HTML and CSS documents and deciding how to display them (screen visual output). It also interacts with the JS engine, etc.
• DOM (Document Object Model): Tree-Like data structure that represents the structure of the document, including elements, attributes, and relationships between them. The HTML parser breaks the HTML code into individual elements and creates corresponding DOM nodes for each element in the tree. The JS Engine (the JS code) also affects the DOM and CSSOM trees.
• CSSOM (CSS Object Model): Like the DOM, it is a tree-like data structure that represents the styles and layout of a document.
• Layout engine: The rendering engine’s core component that calculates the positioning and sizing of visual elements. It ingests the DOM and CSSDOM to create layouts.
• JS engine: Executes JS code. It parses JS into an AST, converts AST into Bytecode, and interprets it. For improved performance, frequently interpreted code is then compiled into optimized Binary by a JIT compiler. The engine has a garbage collector to free up memory no longer in use. Additionally, it provides the runtime environment with built-in objects and functions that JS can use. Further, it provides a WebAssembly (WA) runtime, and developers can add JS code for interacting with the WA module. WA is typically included to efficiently run games, real-time audio and video processing, cryptographic algorithms, emulators, machine learning, etc.
• Sandboxing: Applications in user space run in a particular memory space different to that assigned to the kernel. User space prevents apps from directly accessing the kernel and HW, which would be a security hazard. However, apps in user mode can still interact with predefined APIs, other processes, and user data. Restricting this access further is known as sandboxing, a more restricted app environment enforced by the OS, which can be finetuned with admin/user consent to allow access to features, data, network, etc. They are currently used in mobile devices running iOS and Android. In web browsers, each web app can be executed in the browser’s sandbox, and WA code can only perform instructions that the browser allows.

Virtualization

• Virtual Machine (VM): complete SW emulation of a physical computer with its own virtual CPU, memory, disk, network interface, etc. VMs are more resource intensive, but it is more isolated by the hypervisor, i.e. running apps in a VM is typically more secure than in a sandbox. There are two types of VMs:
  1. System VM: Designed to virtualize an entire computer system’s OS and HW and are managed by a hypervisor.
  2. Process VM (application-level VM, like the Android Runtime (ART) or the JVM): Runs a single process (application) as a platform-independent application that abstracts away the underlying OS and HW details.
• Hypervisor: It’s a VM monitoring SW that runs on the host machine itself (i.e. bare-metal like Xen, VMware ESXi) or on the host OS (hosted like VirtualBox, VMware Workstation), while the VM is the guest OS running on the hypervisor. A hypervisor is like an OS for OSes, coordinating their isolation, resource requirements, and execution with the underlying host OS.
• OS-Level Virtualization (Containers): Allows for multiple isolated user-space instances, which are often called containers. Containers are a lightweight, standalone, and executable SW package that includes everything needed to run a piece of SW, including the code, runtime, system tools, libraries, and settings. Allows multiple isolated applications to share the same OS kernel but run in separate user-space instances. They provide a consistent environment for applications across different environments. Containers are more efficient and faster to start than VMs because they do not include an entire OS—only the application, libraries, runtime, system tools not available in the host system, settings (user input or env vars like PATH to find commands), config files (parameters and options that dictatethe behavior of the application), and dependencies. Suited for microservices architectures, cloud computing, and other use cases where scalability, efficiency, and rapid deployment are important. An exemplary container tool is Docker, and solutions like Kubernetes help with automating, deploying, scaling, and operating containers.

Development

• Cross-Platform Native Apps: There are frameworks like Tauri, Electron, and React Native that allow a developer to wrap a web application with native code to work in any platform, however, the resulting apps may be heavy and not necessarily as performant as native apps.
• Integrated Development Environment (IDE): is a software suite that combines common tools for software development, such as a code editor, compiler, debugger, and build automation tools, into a single graphical user interface to facilitate the development process.
• Software Development Kit (SDK): collection of software tools like APIs specific to an OS/application/service, development tools (compilers, debuggers), libraries (your code calls a library to use its functions), and documentation that developers use to create applications for specific platforms or frameworks (a framework calls your code). SDKs can be integrated into IDEs, and some IDEs are designed to work with a specific SDK, e.g. Xcode and Android Studio.

References

Modern Operating Systems
https://www.goodreads.com/book/show/18762100

Windows
https://en.wikipedia.org/wiki/Architecture_of_Windows_NT
https://learn.microsoft.com/en-us/training/modules/explore-windows-architecture/3-examine-windows-client-architecture

iOS/macOS
https://en.wikipedia.org/wiki/Architecture_of_macOS
https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/Architecture/Architecture.html https://www.geeksforgeeks.org/architecture-of-ios-operating-system/

Android
https://source.android.com/docs/core/architecture
https://developer.android.com/guide/platform

Linux
https://www.javatpoint.com/architecture-of-linux

Browser Engines
https://en.wikipedia.org/wiki/Browser_engine
https://zicodeng.medium.com/explore-the-magic-behind-google-chrome-c3563dbd2739

Disclaimer
Any views expressed in this post are solely those of the author and do not represent the opinions or policies of any affiliated organizations.