Skip to main content

Containers - Quick Low Level Guide

Containers Kernel, namespace, cgroups
Kernel space and user space

Before we actually get to explain containers let's define what is a kernel.  Because you know there is no such thing in reality as a kernel it's only how we name things, and different people name things differently.

cgroups, namespaces, UFS

We are going to discuss containers, cgroups, namespace, UFS, hypervisor, user space, kernetl space and more.
 
When we say "kernel" we mean this.  We have the hardware, this is not the kernel, now above the hardware we have a few layers of software, imagine now two boxes.

User mode is all the application you run while the kernel is the lower level is all the virtual memory management scheduling, connection to hardware devices, network drivers, it's basically the abstraction on top of the hardware + the basic services which allow this.


One box is closer to the hardware and contains a few layers, the second box sits on top of the kernel box and contains libraries we already use the lower box.

The lower box is the kernel it contains all the hardware abstraction layer, all the thread shielding the interrupts and many other lower level services such as memory management, power management etc.  The kernel is like the bare bones of the hardware plus all the operating system which manages this hardware.


The layer which connects the User space to the kernel space is system calls from user space to kernel space.

The Promise

Containers are built from cgroups, namespaces, tarballs, and they are just processes.  Let's elaborate on this.

The promise is that you can package your application in a container and then just run it in production, because all would be bundled inside the container, while we all know that is not really the truth containers do help us however if you would like to understand a little under the hoods or what containers really are let's start our discovery.  And while containers do contain processes it's best practice to use them for stateless applications due to their simple nature all we want to do is to contain processes not file systems, they are after all ephemeral.

Machine

If you look at a machine running containers and list the processes with PS you would not see any containers you would actually see real processes like java processes etc.  So what is going on, where are all those containers because those would appear to be super standard boring processes with the standard process output from the ps command.

So containers run as standard processes, what is the packaging that we all talk about? when you create a container you create a tarball file, so if you wanted to package a container manually you would zip your app in a tarball.

But tarball is that all there is to containers? I could tarball a process by myself, you could but you would need to take a few more steps for it to be equal to standard container for example you would need to update cgroups which we would soon discuss controlling the container better control how much CPU allocation it gets and how much memory.

When you run a container the tarball is being unzipped and run as any process.

As you want your container to have limited access for example to disk then there is a native way to do this on Linux, it's with the standard namespaces feature, it would allow containers to see certain processes and libraries.

The cgroups would limit the memory and CPU. 

VM Vs Containers

With vms you have multiple kernels but how do these multiple vm multiple kernels talk to the actual hardware and get all the basic thread scheduling services? That is with the hypervisor, so you have a kernel for each vm and each vm talks to the kernel of the hypervisor to get the basic OS services.

You can also load multiple OS like windows and linux, but in containers you have processes, each process runs inside a container.

Container

Each container is a ting up together cgroup namespaces and UFS nifon capable file system this is a container, it looks like an OS the container

Namespace isolates and limits what you can use separate pid net mnt you have your own view of the system.
cgorups - how much you can use memory CPU

Processes from one namespace pid like if you have myapp1 namespace pid it cannot see any pid from myapp2 PID namespace

The access to resources is controlled by the cgroups.

UFS - combine multiple directories into one you get the illusion that you have a single directory.





Comments

Popular posts from this blog

Functional Programming in Scala for Working Class OOP Java Programmers - Part 1

Introduction Have you ever been to a scala conf and told yourself "I have no idea what this guy talks about?" did you look nervously around and see all people smiling saying "yeah that's obvious " only to get you even more nervous? . If so this post is for you, otherwise just skip it, you already know fp in scala ;) This post is optimistic, although I'm going to say functional programming in scala is not easy, our target is to understand it, so bare with me. Let's face the truth functional programmin in scala is difficult if is difficult if you are just another working class programmer coming mainly from java background. If you came from haskell background then hell it's easy. If you come from heavy math background then hell yes it's easy. But if you are a standard working class java backend engineer with previous OOP design background then hell yeah it's difficult. Scala and Design Patterns An interesting point of view on scala, is

Alternatives to Using UUIDs

  Alternatives to Using UUIDs UUIDs are valuable for several reasons: Global Uniqueness : UUIDs are designed to be globally unique across systems, ensuring that no two identifiers collide unintentionally. This property is crucial for distributed systems, databases, and scenarios where data needs to be uniquely identified regardless of location or time. Standardization : UUIDs adhere to well-defined formats (such as UUIDv4) and are widely supported by various programming languages and platforms. This consistency simplifies interoperability and data exchange. High Collision Resistance : The probability of generating duplicate UUIDs is extremely low due to the combination of timestamp, random bits, and other factors. This collision resistance is essential for avoiding data corruption. However, there are situations where UUIDs may not be the optimal choice: Length and Readability : UUIDs are lengthy (typically 36 characters in their canonical form) and may not be human-readable. In URLs,

Bellman Ford Graph Algorithm

The Shortest path algorithms so you go to google maps and you want to find the shortest path from one city to another.  Two algorithms can help you, they both calculate the shortest distance from a source node into all other nodes, one node can handle negative weights with cycles and another cannot, Dijkstra cannot and bellman ford can. One is Dijkstra if you run the Dijkstra algorithm on this map its input would be a single source node and its output would be the path to all other vertices.  However, there is a caveat if Elon mask comes and with some magic creates a black hole loop which makes one of the edges negative weight then the Dijkstra algorithm would fail to give you the answer. This is where bellman Ford algorithm comes into place, it's like the Dijkstra algorithm only it knows to handle well negative weight in edges. Dijkstra has an issue handling negative weights and cycles Bellman's ford algorithm target is to find the shortest path from a single node in a graph t