Large Language Models are at a pivotal point. On one hand, we still don’t understand how they work or why they work so well - could human thought be nothing more than a pattern that these models have somehow uncovered? On the other hand, there’s growing concern that LLMs are hitting a wall[1], that current approaches (i.e. throwing ever more money and computation on training) won’t fix its biggest shortcomings (e.g. hallucinations), and that the alarmingly high valuations and expectations may come crushing down soon.

But whatever the future holds, as of early 2025, LLMs have demonstrated their usefulness in various ways. One specific use case - this blog’s hobby horse - is using language models as a learning companion.

Continue: https://engineeringknowledge.blog/p/where-the-bug-is-a-feature

Have you ever tried playing chess against yourself? Unless you are a chess purist or have a split personality, it is not much fun.

The pleasure of engaging in a game like chess comes from challenging another mind (human or otherwise). From not knowing what the other is planning and the uncertainty that comes with it. When two opponents share the same brain, there are no secrets or surprises.

Another player makes the game “real.” When you have found a weakness in the opposing position and hatch a plan to exploit it, your opponent will prove your plot right or wrong. The stronger the rival, the harder your ideas will be tested. Left alone, that “clash with reality” can’t occur. You can try hard to emulate both separate sides, but as the two are coming from the same mind, all you do is evaluate yourself, which has limited value.

We are in a similar predicament every time we are trying to understand something or come up with an explanation. Because we are the creator of our thoughts as well as their recipient, we are biting our own tails whenever we try to figure out if what we think is correct or not.

To overcome the lack of outside perspective, we intend to look at a problem from all angles. The trouble is that not only is seeing the forest from the trees difficult, but that thinking itself is flawed. The mind’s default operational mode is not rationality, and it has no incentive to spot inconsistencies. Instead, it happily paints over blind spots, leaving us unaware of our imperfections.

Why is this so? Wouldn’t it be beneficial to our species to ensure that our thinking is consistent and flawless? The short answer is no. The mind is not designed to think. ¹ From an evolutionary point of view, it is more beneficial for our survival to automate thinking as much as possible. Active, hard pondering of problems consumes valuable energy that would be better spent on spotting danger or other life-preserving activities.

The brain is arguably the most wonderful thing in the universe (as far as we know, that is). It can do marvelous things like seeing, hearing, memorizing, control bodily functions, all at the same time and while giving birth to consciousness. But when it comes to learning and rational reasoning, the organ inside our skull is a broken tool, like a water bucket with holes.

Without a clear picture of what we actually think (rather than what we believe to think), efficient learning is difficult. It is essential to shine a bright light on the gaps in our reasoning and bring them out into the open. Learning is as much about gaining new insights as it is about getting rid of false beliefs.

What can we do?

Ideally, you’d have a few like-minded people at your disposal interested in your stuff who could challenge and inspire you (could this be an AI in the future? That would be tremendous).

Lacking such a sophisticated support group, the next best thing to do is to break the mind’s self-evaluation-loop by putting distance between the moment we generate an idea and the time we evaluate it.

Humanity has invented a tool for exactly that problem - pen and paper. Writing helps us capture our fuzzy thoughts and make them explicit and observable. The problem is, it can be a laborious word-wrangling exercise to express ourselves. Luckily, for our purposes we can skip most of the arduous parts of translating our intentions into sentences and simply transpose our raw thoughts onto paper. This can be done much easier and faster by making a list.

The difference between a list and “normal” writing is the intended audience. Usually, we write for others, which makes it so hard because the reader does not know anything about us. We need to give as much context and be exact as possible to avoid ambiguity. The list, on the other hand, is only meant to be read by the author. No need to explain where we are coming from, we can be as esoteric in our descriptions as we like.

There is one non-negotiable requirement, though: Everything on the list has to be written using our very own words. This is crucial to make the list fulfill its purpose.

Looking up answers and copying-pasting them into our list does not help at all - rather the opposite, it only strengthens the gaps in our understanding because it creates the illusion of knowledge. We may write down correct facts, but they are not coming from us.

This is one reason why highlighting passages in a book does very little to increase comprehension of a text. It looks like learning because the passages are important, but they are not our words, they are expressions of someone else’s thinking (underlining text passages can still be a useful activity, but the highlights are merely bookmarks, helping navigate a large amount of information).

Definitions, buzzwords, or concepts we don’t fully grasp don’t belong on the list. Unless we write down precisely this - that we don’t understand those terms (e.g., “Technology X is a distributed-self-contained management system - what the heck does that mean?”)

We list what we know (or believe to know) and what we don’t know or understand. This is a fast and fluent exercise. It does not require much thinking. Then, very soon, we will have exhausted what’s on our mind.

I love making lists. Whenever I am stuck, I start listing my thoughts. This post here started as a list (as all others have) that I then successively transformed into full sentences. The moment I got stuck, I switched back into list mode, and so forth. I’m always shifting modes. From slow and exact writing, building up a single line of argument, to the immediate and quick jotting down of multiple strands of thoughts (there are other modes, but that’s a subject for another post).

When I begin a new task, I start with a list. To clarify my intentions and beliefs and to prepare myself for the work ahead by bringing into focus the things I’m sure about and those I’m not. This way, I prime my mind to find answers for the gaps I have and to look for confirmation of what I assume to be true.

Lists are helpful for any kind of mental task. They expand our short-term memory. They force us to think sequentially and turn vagueness into explicitness. And list-making is a fun activity, even relaxing, similar to meditation and its benefits, where we step back from being mindlessly captured by thoughts and instead gain control through observing our mental states.

What do we value most in life? Health is an obvious choice, next to love and meaningful friendships. Money is certainly on the list too, but controversial, at least when pursued narrowly at the cost of everything else (though few would dispute that a good life depends on owning a sufficient amount of it).

Uncontroversial is the value of another item: knowledge. Our post-industrial society increasingly depends on the creation and transfer of know-how and technical expertise. We usually consider knowledgeable people as intelligent, which often correlates with higher social status and financial reward.

But what is knowledge? It is surprisingly hard to define. Does memorization of a lot of facts (e.g historical dates) count as knowledge? Then your internet-connected computer would be the most knowledgeable entity ever created. Facts are data points. Taken by themselves, they have limited value if we can’t explain their existence and are unable to predict future facts. In other words, we lack understanding.

Furthermore, the value of knowledge depends on its usefulness to solve a problem. In particular, to solve my problem. Ask “How much do you know about X?” and I would reply “to accomplish what?” (which is nicer than an engineer’s favorite retort of “it depends…”). I know that a combustion engine needs oil and gas to function and have a dim idea about gears. Enough knowledge for me to drive a car, understand the meaning of warning lights and abstain from shifting into reverse while the car is moving forward. If you ask me, I know all there is about cars. However, if tomorrow I wanted to start a new career as an auto mechanic, I would immediately find out that I know very little.

Figuring out the purpose of knowledge affects how we think about it and fundamentally changes how we teach and learn. By asking about the reason for knowledge we move from passive consumption of facts to an active inquiry about the why, how, how well, and so on.

How purpose and knowledge relate to each is the subject of the fascinating book called “Knowledge as Design” by D.N. Perkins.

As the title suggests, the author considers knowledge not something that is out there, given to be consumed without critical analysis (he calls this “truth mongering”) but something that has a purpose, or, in a broader sense, a design.

The first thing to note is that the word design may be misleading. If you, like me, haven’t thought much about design before you may equate it with style (fashion, brands, websites) or a blueprint to build something (design of a house). The author takes a wider look and defines design more accurately:

Design is the human endeavor of shaping objects to purposes

Almost everything is a design. A knife is an object adapted to the purpose of cutting things. Academic knowledge is also designed - Newton’s laws are a mathematical tool to explain the motion of bodies. So are processes, claims, and even historic dates as a part of a larger story (for example, the year 1492 marks a milestone in western civilization). There are only a few things that are not designs (like natural phenomena).

So how does thinking about design help us with acquiring knowledge? Knowing the purpose of a design is crucial, but purpose alone only explains the reason a design exists, it does not say anything about how it solves a problem.

The four design questions

The author gives us a tool to examine designs by formulating four design questions that make up the heart of the book.

The questions to ask about a design are the following:

1. What is its purpose?
2. What is its structure?
3. What are the model cases of it?
4. What are the arguments for and against its usefulness?

“Purpose” we have already discussed. The structure of a design deals with the parts, components, materials, relations, and so forth of the object in question. A knife has a cutting edge attached to a handle.

Model cases are examples of the object or knowledge (imagine photos of different types of knives). Model cases are in some sense mental models, but in a much narrower way (I come back to this later).

Arguments, the last of the four questions, evaluate a design. How well does it do its job? Given the purpose, is the design well-thought-out or could it be better? While the former three questions could be answered without much context, here we depend on our subjective understanding of the object at hand. I can’t say anything about the effectiveness of a design if I don’t understand what it does or how it differs from other solutions.

The book discusses at length how the four questions can be applied to practically everything. It turns out that looking at the world through design glasses is tremendously useful. Be it how to write an essay, how to come up with new insights, or how to improve learning and teaching in general.

Since knowledge plays such an important role in our life, it is worth looking at how this new way of thinking about knowledge (as design) compares to our default way (as information).

We shift from the left to the right, from the passive to the active, by connecting what we learn to each of the four design questions.

So how does all this help us with our mental models? Is one better than the other?

How do software mental models (SMM) compare with the four design questions (4DQ)?

The short answer is that 4DQs are a tool to evaluate the subject at hand that leads to a conclusion, while SMMs are a process that keeps evolving. There is overlap between the two but also significant differences. Let’s have a look at each of the four questions.

Model cases

We start with likely the most confusing part. Aren’t mental models the same as model cases? Not really. Mental models, at least the way I use them, are internal representations of reality, that can differ significantly from the actual object. Model cases are examples (of object types, procedures, formulas, etc), whereas mental models are simplifications that are (hopefully) so useful that they can explain complex processes with a few elegant analogies. Coming back to the simple example of a knife, model cases would present all the different types of knives (we could, for example, present their evolution from the stone age until today), whereas the mental model of a knife depends on what goal I have. In my case, I mostly think about knives in the context of a kitchen, so I have mental models about how the sharp side of a blade works and what knife to use for certain materials (e.g. bread versus meat). A chef has vastly more complex internal representations of knives. She might think about angles, handles, all kinds of materials, cutting techniques, and so on.

Structure

The structure of a design describes the elements and parts of the object or knowledge. Mental models include structure, too, but only those parts that are useful for achieving the goal. When learning something new, the structure of a model will be similar to a structure of a design. The more advanced the models get, the more abstract they become until they may have nothing in common anymore with the actual object.

Purpose

The purpose of a design is similar to what I consider the goal of a mental model. However, in the case of SMMs, the goal is deeply personal, whereas the purpose of a design is meant to be independent of the learner.

The goal of a mental model gives clarity about what is important and what I can leave out. The goal evolves. In a design, the purpose is fixed - the reason the design exist stems from the purpose.

In reality, the purpose of a design is still to some degree subjective, because it depends on the knowledge of the learner. A physics beginner in school will come up with a different, simpler purpose for Newton’s laws than a university graduate.

Arguments

Arguably, arguments are the most important of the four questions because they require the learner to conclude based entirely on their understanding and experience. Purpose, structure, and model cases lay the foundation that arguments are built upon.

SMMs don’t have arguments at all. If they are so important, how can this be?

The reason can be found in the fact that SMMs are a process. Arguments and purpose are two sides of a coin. The argument describes how well a design accomplishes its purpose with the outcome of an evaluation. SMMs don’t have a final step. Instead, we evaluate models during the test phase, where we figure out where the models are inaccurate. Instead of giving a judgment, we correct the model and make it better during the refinement phase (or replace it with a better model if necessary). The argument is ingrained in the process.

In summary, we could say that SMMs are the agile version of 4DQ.

The 4DQs are a very useful tool to structure thinking, learning, and writing. The book can be repetitive at times as it goes through the many possible applications of the theory, but once the main ideas are understood it is easy to skip content and focus on the parts that interest you. They are certainly useful for developing SMMs.

Can you spot the 4DQ in this post?

Writing an essay is among the use cases of the theory. Can you spot where in this post I have used the four questions?

Answer:

• The first half of this post explains the purpose and structure of the 4DQs. First I made claims about the importance of knowledge, and in particular, the value of looking at the purpose of knowledge
• Then I described the structure of the 4DQs, describing each question.
• I finished the first half by completing my description of the purpose of the 4DQ by demonstrating the value of moving from seeing knowledge as information to considering knowledge as design.
• I use model cases of 4DQs throughout the post, most of the time using the example of a knife. And, to go fully recursive, this answer is another model case of the 4DQs.
• In the second half I give arguments by comparing in detail the 4DQs with SMMs, with the conclusion that while there’s overlap between the two ideas, SMMs fundamentally differ from 4DQs by being an ongoing process instead of a final evaluation of an object or piece of knowledge.

Programming is a marvellous activity. By typing on a keyboard and feeding code to a computer, a programmer can create entirely new (virtual) worlds out of nothing. There are few constraints on what a program could be - the only limitations are memory and a CPU’s processing power. Whatever you can think of, you could, in theory, bring into existence on a computer.

But since virtual worlds are created in the mind of a developer and not based on physical reality, it can be difficult to talk about what they are or how they work. There is no immediate language to refer to. To overcome this lack of terminology, engineers often use analogies from the real world to describe what software does.

Docker is no exception. At its centre stands the “container”, the main building block of what Docker calls a “standard of shipping software”. One can’t help but see container ships towing heavy loads over oceans. Even the logo depicts the main idea:

Containers and shipping software

When we learn about new technology with strong ties to another technology, we immediately apply our existing knowledge of the substitute technology to fill gaps in our understanding. This “replacement reasoning” can be helpful or misleading, depending on how accurate the analogy turns out to be. Therefore, to avoid misconceptions, it is beneficial to evaluate existing mental models first, as they are the ground on which we build our individual models.

In the case of Docker, we want to know how helpful the idea of “container shipping” is and how far we can take the analogy.

It turns out that there are many parallels between real and virtual containers and that the properties of container shipping cover quite a lot of concepts of Docker.

Containers (those made from steel)

Shipping containers revolutionized international trade by making it possible to move freight between different modes of transport without the need to re-package goods. Due to standardization and a complex system of docking ports, the cost and time it takes to ship goods worldwide have been significantly reduced.

What makes containers so valuable is that they can be stacked neatly on top of each other, thanks to their standardized dimensions. They can also carry nearly any kind of goods and can be used interchangeably.

Docker Containers

Like shipping containers, Docker containers also increased efficiency. Most applications run on Webservers. In the past, it was not possible to execute more than one application per server. Not only was managing a fleet of invidual servers difficult, it also lead to a waste of resources, as applications can vary significantly in their requirements that change over time (servers needed to keep a capacity reserve).

Then a company called VMWare came along and invented the virtual machine. I mentioned in the intro that computers create virtual worlds. They can also create worlds inside worlds. A virtual machine is an emulation of hardware. Each emulated hardware is a new virtual computer (creating computers inside a computer), making it possible to run multiple applications side by side on the same server.

One OS per VM (Source: “Docker Deep Dive” ¹)

This makes deploying applications much more efficient. Applications run inside their own virtual machine (VM), and hardware resources can be individually distributed to meet the requirements of each application.

But VMs are big. Virtualizing the hardware requires each virtual machine to include an entire operating system.

Docker takes a different approach. Instead of virtualizing the hardware, it aims higher and abstracts away the operating system. Docker containers share the same OS resources, making them more lightweight ².

Docker containers are smaller (Source: Docker Deep Dive¹)

We talked about why Docker exists, but we still haven’t said what a container actually is.

How do Docker containers compare to physical containers?

The physical container is an enclosing made of steel that carries goods. Similarly, a Docker container provides applications with an isolated environment (the virtual equivalent of an enclosing).

Actual containers are filled with goods. A Docker container runs an application. To understand how the application ends up on a Docker container (and how that is done efficiently), we need to understand three basic concepts:

• Docker engine
• Docker image
• Containerizing an app

Docker Engine

The word “Docker” can mean two things.

First, there is Docker, the company. They develop the tools and libraries required to run Docker and maintain the Docker hub, a central repository.

Then there is Docker, the technology. There are three main components³:

• The runtime provides low-level libraries and components to run containers.
• The engine (or Daemon) provides higher-level functionality, most of all the interface for Docker commands.
• A Client is an application that talks to the Docker Daemon (e.g. the docker or docker-compose commands)

When someone learns Docker, they will spend most of their time understanding the interactions of a Docker client with the Docker daemon.

Actual containers are made out of steel. Similarly, the Docker engine + runtime are “the fabric of Docker containers”.

Docker image

The Docker glossary describes containers as a “runtime instance of an image” ⁴. An image is a collection of files that are executed when a container runs. They are, we could say, the “content” of a container - in other words, the application.

Images are constructed in layers, where each layer corresponds to a set of files of a container. More precisely, each layer is a set of file changes. When an image is built, layers are executed sequentially, possibly overriding files of a previous stage.

Image layers, overriding changes (source: “Docker Deep Dive” ¹)

Layers can be cached. That makes building images efficient because every time an image changes, we only need to update those layers that have changed.

Containerizing an application

The final piece of the Docker puzzle is the so-called process of “containerizing an application”. Or in other words, how do we turn an application into an image that we can then execute on a container?

This is done via a Docker file. In the container shipping model analogy, a Docker file is similar to a list of items that describe the content of a container. Except that it not only lists the files but also describes the order of their occurrence or modification.

Side-by-side comparison of container shipping and Docker

Finally, we get to the fun part! How far can we take the analogy of container shipping to create a mental model of Docker?

Container shipping is straightforward to visualize - starting with containers loaded with goods to the cranes lifting them onto ships and so forth until they reach their destination. For every step in the journey of a container, we can find a similar action in the Docker world. Let’s compare them side-by-side.

We can even create a little diagram that compares both worlds:

• Think of a Docker container as a vessel for applications.
• The Docker engine (and runtime) provides the enclosing for goods, similar to steel used to fabricate a physical container.
• The Dockerfile is like a description of the contents of a container.
• The image provides the content of a container, each file (change) is like a single good.
• Building and uploading an image prepares the content of a container.
• Downloading an image and creating a container is like a crane lifting a container onto a ship.
• Running the container can be compared to a ship transporting containers over the ocean.
• Stopping a container, destroying it and deleting an image is similar to a crane lifting a container off a ship, taking out the goods and scraping the empty container.