Compare commits

...

2 commits

Author SHA1 Message Date
Elia el Lazkani
68a0200f01 chore(): Adds Multi-stage docker container build post 2025-03-05 22:45:02 +00:00
Elia el Lazkani
a96c0e5b82 chore(): Jason's challenge ! 2025-02-20 22:23:41 +01:00
3 changed files with 570 additions and 0 deletions

View file

@ -3029,6 +3029,211 @@ On these small Raspberry Pis, the cluster seems to be working very well. The
/DNS/. There's a few improvements that can be done to this setup, mainly the
deployment of an /SSL/ cert. That may come in the future, time permitting. Stay
safe, until the next one !
*** DONE Multi-Stage Docker container Build :docker:linux:container:multi_stage:podman:dockerfile:
:PROPERTIES:
:EXPORT_HUGO_LASTMOD: 2025-03-05
:EXPORT_DATE: 2025-04-05
:EXPORT_FILE_NAME: multi-stage-docker-container-build
:CUSTOM_ID: multi-stage-docker-container-build
:END:
One of the hidden gems of /Docker containers/ is /multi-stage/ builds. If it
never made any sense to you, you've heard of it but have no clue what it is or
just passing along... We're going to use it in a practical example.
#+hugo: more
**** go-cmw
A while ago, I wrote a small utility in /golang/ which fetches the weather for
me and displays it in the terminal. /[[https://scm.project42.io/elia/go-cmw][go-cmw]]/ is, basically, a [[https://wttr.in/][wttr.in]] terminal
client. It simplifies the usage of the *API* and makes it easier to integrate,
for me, into other terminal tools. Who's not a big a fan of the shell huh !
Am I right !
**** Let's containerize it
Let's say we would like to write a =Dockerfile= for the project to build the
code and create a container for it.
The =Dockerfile= would probably look something like this.
#+begin_src dockerfile
# Yes, we're smart, we used a small image because it's all we need
FROM docker.io/library/golang:alpine
# Copy the directory of the code into /cmw
ADD . /cmw
# Install git as a dependency
RUN apk add git && \
# Navigate to the directory where we copied the code to
cd /cmw && \
# Get the dependencies of the project
go get -u . && \
# Build that bad boy !
go build -o cmw && \
# Move it into a path we know is in $PATH
mv cmw /usr/bin/cmw && \
# Clean up, we have security in mind
cd / && rm -rf /cmw
# Aight run it over one day !
CMD ["cmw", "-o"]
#+end_src
We've tried to use as few layers as possible to keep this image small. Let's
take a look at the image we built.
#+begin_src shell
$ podman build -t cmw .
$ podman run -e GO_CMW_LOCATION="Dublin" cmw
Weather report: Dublin
\ / Partly cloudy
_ /"".-. +12(11) °C
\_( ). ↑ 16 km/h
/(___(__) 10 km
0.0 mm
┌──────────────────────────────┬───────────────────────┤ Wed 05 Mar ├───────────────────────┬──────────────────────────────┐
│ Morning │ Noon └──────┬──────┘ Evening │ Night │
├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤
│ \ / Sunny │ \ / Sunny │ \ / Sunny │ \ / Clear │
│ .-. +9(7) °C │ .-. +12(10) °C │ .-. +8(6) °C │ .-. +6(3) °C │
│ ― ( ) ― ↑ 14-20 km/h │ ― ( ) ― ↗ 19-22 km/h │ ― ( ) ― ↑ 14-29 km/h │ ― ( ) ― ↑ 14-29 km/h │
│ `- 10 km │ `- 10 km │ `- 10 km │ `- 10 km │
/ \ 0.0 mm | 0% │ / \ 0.0 mm | 0% │ / \ 0.0 mm | 0% │ / \ 0.0 mm | 0% │
└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘
Location: Dublin, County Dublin, Leinster, Ireland [53.3497645,-6.2602731]
#+end_src
Okay, it works. So what now ? Well now, we look deeper.
Let's look at the size...
#+begin_src shell
$ podman images cmw
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/cmw latest db1730d690ed 2 minutes ago 398 MB
#+end_src
Ah! That's quite big for a small tiny client that shouldn't be bigger than a few
MBs. This is going to take forever to download on a server to run constantly (hypothetically).
How many layers does it have...
#+begin_src shell
$ podman inspect cmw | jq .[].RootFS.Layers[] | wc -l
7
#+end_src
That's quite a few layers. The more we have to download, the slower the download is.
And finally a quick vulnerability scan...
#+begin_src shell
$ trivy image localhost/cmw
...
#+end_src
Okay the /Trivy/ output is quite big but the summary is *2 Critical*, *1 High*
and *2 Medium* severity vulnerabilities all coming from the image even though
we're using the latest available.
**** Multi-stage build
We saw a few issues in the previous part of this post, let's see if we can fix
them.
The first thing I'm going to do is start thinking about my application. My
client is written in /golang/ which means that the binary should work without
any dependencies. I could build the binary on my machine and /then/ *copy* it to
the container. This path will definitely reduce our layer number but this path
is not easily packaged and reproduced on a different machine.
Besides, we said we're using /containers/ for this.
Another thing to think about are the vulnerabilities in the built image. All of
the vulnerabilities identified are related to /golang/, which makes sense. We're
using a /golang/ container image after all, even though the image is based on
the hardened /alpine/ distribution. We can do better, we can go with a container
that contains almost nothing, it should definitely be more secure.
#+begin_src dockerfile
FROM docker.io/library/golang:alpine as builder
ADD . /cmw
RUN apk add git && \
cd /cmw && \
go get -u . && \
go build -o cmw
FROM docker.io/library/alpine:latest
COPY --from=builder /cmw/cmw /cmw
CMD ["/cmw", "-o"]
#+end_src
Let me explain a bit what changed. The first change is that we named our /first
stage/ to *builder* to make it easier to reference it later. The /dependency/
installations and the code builds stay exactly the same. The cleanups were
removed as they have no purpose anymore.
The /second/ =FROM= is where the magic starts to happen. We're using, in this
/second stage/ a plain =alpine= image. This container does not have any /golang/
compiler, library or dependencies. We, /then/, =COPY= the =cmw= /binary/ from
the *builder* container and into our /alpine/ container. The rest does basically
the same.
Now, let's take a deeper look at the image.
#+begin_src shell
$ podman images cmw
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/cmw latest 978342ca6735 8 minutes ago 19.5 MB
#+end_src
The difference, in size, between the old image and this new one is *extremely
significant*, down from ~398 MB~ to /just/ ~20 MB~.
And the layers...
#+begin_src shell
$ podman inspect cmw | jq .[].RootFS.Layers[] | wc -l
2
#+end_src
Only ~2~, all the way down from ~7~.
And finally, the icing on the cake...
#+begin_src shell
$ trivy image localhost/cmw
localhost/cmw (alpine 3.21.3)
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
#+end_src
That's right, no vulnerabilities at all.
#+BEGIN_EXPORT html
<div class="admonition warning">
<p class="admonition-title">warning</p>
#+END_EXPORT
There are no vulnerabilities at the time of building this image. This does not
mean that this image will stay this way. Over time, vulnerabilities will
eventually be found. This is the reason why it is advisable to rebuild your images
frequently to keep them updated.
#+BEGIN_EXPORT html
</div>
#+END_EXPORT
**** Conclusion
~Docker~ container /multi-stage build/ is not a hard concept to grasp. As you
can see, it helps a lot in creating a small and safe container. More stages can
be built on top, some to build the frontend written in /TypeScript/ for example.
This opens up a wide range of features and opportunities for us to use to our advantage.
** K3s :@k3s:
*** DONE Building k3s on a Pi :arm:kubernetes:
:PROPERTIES:
@ -5734,6 +5939,78 @@ Next time you're working on your project, give ~direnv~ a try. It will change
the way you work for the better, I hope.
Happy Hacking !
*** DONE What 2025 blog question challenge ? :challenge:qa:questions:answers:
:PROPERTIES:
:EXPORT_HUGO_LASTMOD: 2025-02-20
:EXPORT_DATE: 2025-02-20
:EXPORT_FILE_NAME: what-2025-blog-question-challenge
:CUSTOM_ID: what-2025-blog-question-challenge
:END:
A few weeks ago, [[https://janusworx.com][Jason]] sneakily slipped me a link to a [[https://janusworx.com/work/blog-questions-challenge-2025/][blog post]] of his. He
knows that I /already/ subscribe to his blog but you cannot be as wise without
meddling with the forces of nature. Well, long story short, his meddling worked
so... here we go ?
**** Why did you make the blog in the first place?
#+hugo: more
I, honestly, cannot remember why I made it in the first place but it was a long
time ago. I do remember that my first post was about the /plugin/ I wrote for
~weechat~. After that, I sort of wrote on things I was playing around with. I
said it before, it's how I learn. Maybe the difference is that I dig deeper,
beyond the surface. It's fun, I get to document it for myself because believe it
or not, I do come back to the blog for refreshers. It also comes handy cause I
can simply send someone a link that explains a topic in more details.
**** What platform are you using to manage your blog and why did you choose it?
I use /Hugo/ to generate this blog and if you know where to look, it is /open
source/. The answer to why is a bit more complicated but to simplify it, I
prefer a static blog over CRMs. /Hugo/ is quick and I have it integrated with
/Emacs/ where I write my /posts/ in ~org~. The generation, packaging and
deployment is all automated.
**** Have you blogged on other platforms before?
This blog started its life on /WordPress/. I was never quite happy with the
performance for such a simple blog. It was then migrated to /Joomla/. That phase
did not last long because once I learned about static blog generators, the blog
was migrated to /Nikola/ quickly. My /Nikola/ setup was quite elaborate and I
wanted to simplify it, for one, and to make it faster. I checked /Hugo/ out and
liked it but migrating from ~rst~ was a pain. ~Orgmode~ helped a lot with the
migration but after a long journey, here we are running on /Hugo/.
**** How do you write your posts?
The whole blog is one ~org~ file. I open it with /Emacs/ and add a new section
under the relevant =Tag=. Once I save, /Emacs/ generates the /Markdown/ file,
somewhere. I don't have to care about its management in most cases. I can, then,
generate and run my blog locally with /Hugo/. Once I'm happy with the result, I
/commit/ and /push/. The pipeline takes care of the rest, once all the magic is
done successfully, I deploy with a manual confirmation step.
**** When do you feel most inspired to write?
I don't know if that question applies to my case. I don't even know if I do. It
takes a lot of time and effort to write these posts. Documentation is *very*
hard, ask any developer. I'm not sure inspiration is a requirement, hard work is.
**** Do you publish immediately after writing or do you let it simmer a bit as a draft?
I publish immediately. I mean the post has already took a few hours to write...
If I find a mistake in the future, I correct it and fix the timestamp of edit.
**** Your favorite post on your blog?
I don't have a favorite post. I know a few that I share frequently, though.
**** Any future plans for your blog? Maybe a redesign, changing the tag system, etc.?
I don't believe so. Nothing visible to the user, but the infrastructure is
always moving.
** Monitoring :@monitoring:
*** DONE Simple cron monitoring with HealthChecks :healthchecks:cron:
:PROPERTIES:

View file

@ -0,0 +1,212 @@
+++
title = "Multi-Stage Docker container Build"
author = ["Elia el Lazkani"]
date = 2025-04-05
lastmod = 2025-03-05
tags = ["docker", "linux", "container", "multi-stage", "podman", "dockerfile"]
categories = ["container"]
draft = false
+++
One of the hidden gems of _Docker containers_ is _multi-stage_ builds. If it
never made any sense to you, you've heard of it but have no clue what it is or
just passing along... We're going to use it in a practical example.
<!--more-->
## go-cmw {#go-cmw}
A while ago, I wrote a small utility in _golang_ which fetches the weather for
me and displays it in the terminal. _[go-cmw](https://scm.project42.io/elia/go-cmw)_ is, basically, a [wttr.in](https://wttr.in/) terminal
client. It simplifies the usage of the **API** and makes it easier to integrate,
for me, into other terminal tools. Who's not a big a fan of the shell huh !
Am I right !
## Let's containerize it {#let-s-containerize-it}
Let's say we would like to write a `Dockerfile` for the project to build the
code and create a container for it.
The `Dockerfile` would probably look something like this.
```dockerfile
# Yes, we're smart, we used a small image because it's all we need
FROM docker.io/library/golang:alpine
# Copy the directory of the code into /cmw
ADD . /cmw
# Install git as a dependency
RUN apk add git && \
# Navigate to the directory where we copied the code to
cd /cmw && \
# Get the dependencies of the project
go get -u . && \
# Build that bad boy !
go build -o cmw && \
# Move it into a path we know is in $PATH
mv cmw /usr/bin/cmw && \
# Clean up, we have security in mind
cd / && rm -rf /cmw
# Aight run it over one day !
CMD ["cmw", "-o"]
```
We've tried to use as few layers as possible to keep this image small. Let's
take a look at the image we built.
```shell
$ podman build -t cmw .
$ podman run -e GO_CMW_LOCATION="Dublin" cmw
Weather report: Dublin
\ / Partly cloudy
_ /"".-. +12(11) °C
\_( ). ↑ 16 km/h
/(___(__) 10 km
0.0 mm
┌──────────────────────────────┬───────────────────────┤ Wed 05 Mar ├───────────────────────┬──────────────────────────────┐
│ Morning │ Noon └──────┬──────┘ Evening │ Night │
├──────────────────────────────┼──────────────────────────────┼──────────────────────────────┼──────────────────────────────┤
│ \ / Sunny │ \ / Sunny │ \ / Sunny │ \ / Clear │
│ .-. +9(7) °C │ .-. +12(10) °C │ .-. +8(6) °C │ .-. +6(3) °C │
│ ― ( ) ― ↑ 14-20 km/h │ ― ( ) ― ↗ 19-22 km/h │ ― ( ) ― ↑ 14-29 km/h │ ― ( ) ― ↑ 14-29 km/h │
`- 10 km │ `- 10 km │ `- 10 km │ `- 10 km │
│ / \ 0.0 mm | 0% │ / \ 0.0 mm | 0% │ / \ 0.0 mm | 0% │ / \ 0.0 mm | 0% │
└──────────────────────────────┴──────────────────────────────┴──────────────────────────────┴──────────────────────────────┘
Location: Dublin, County Dublin, Leinster, Ireland [53.3497645,-6.2602731]
```
Okay, it works. So what now ? Well now, we look deeper.
Let's look at the size...
```shell
$ podman images cmw
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/cmw latest db1730d690ed 2 minutes ago 398 MB
```
Ah! That's quite big for a small tiny client that shouldn't be bigger than a few
MBs. This is going to take forever to download on a server to run constantly (hypothetically).
How many layers does it have...
```shell
$ podman inspect cmw | jq .[].RootFS.Layers[] | wc -l
7
```
That's quite a few layers. The more we have to download, the slower the download is.
And finally a quick vulnerability scan...
```shell
$ trivy image localhost/cmw
...
```
Okay the _Trivy_ output is quite big but the summary is **2 Critical**, **1 High**
and **2 Medium** severity vulnerabilities all coming from the image even though
we're using the latest available.
## Multi-stage build {#multi-stage-build}
We saw a few issues in the previous part of this post, let's see if we can fix
them.
The first thing I'm going to do is start thinking about my application. My
client is written in _golang_ which means that the binary should work without
any dependencies. I could build the binary on my machine and _then_ **copy** it to
the container. This path will definitely reduce our layer number but this path
is not easily packaged and reproduced on a different machine.
Besides, we said we're using _containers_ for this.
Another thing to think about are the vulnerabilities in the built image. All of
the vulnerabilities identified are related to _golang_, which makes sense. We're
using a _golang_ container image after all, even though the image is based on
the hardened _alpine_ distribution. We can do better, we can go with a container
that contains almost nothing, it should definitely be more secure.
```dockerfile
FROM docker.io/library/golang:alpine as builder
ADD . /cmw
RUN apk add git && \
cd /cmw && \
go get -u . && \
go build -o cmw
FROM docker.io/library/alpine:latest
COPY --from=builder /cmw/cmw /cmw
CMD ["/cmw", "-o"]
```
Let me explain a bit what changed. The first change is that we named our _first
stage_ to **builder** to make it easier to reference it later. The _dependency_
installations and the code builds stay exactly the same. The cleanups were
removed as they have no purpose anymore.
The _second_ `FROM` is where the magic starts to happen. We're using, in this
_second stage_ a plain `alpine` image. This container does not have any _golang_
compiler, library or dependencies. We, _then_, `COPY` the `cmw` _binary_ from
the **builder** container and into our _alpine_ container. The rest does basically
the same.
Now, let's take a deeper look at the image.
```shell
$ podman images cmw
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/cmw latest 978342ca6735 8 minutes ago 19.5 MB
```
The difference, in size, between the old image and this new one is **extremely
significant**, down from `398 MB` to _just_ `20 MB`.
And the layers...
```shell
$ podman inspect cmw | jq .[].RootFS.Layers[] | wc -l
2
```
Only `2`, all the way down from `7`.
And finally, the icing on the cake...
```shell
$ trivy image localhost/cmw
localhost/cmw (alpine 3.21.3)
Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
```
That's right, no vulnerabilities at all.
<div class="admonition warning">
<p class="admonition-title">warning</p>
There are no vulnerabilities at the time of building this image. This does not
mean that this image will stay this way. Over time, vulnerabilities will
eventually be found. This is the reason why it is advisable to rebuild your images
frequently to keep them updated.
</div>
## Conclusion {#conclusion}
`Docker` container _multi-stage build_ is not a hard concept to grasp. As you
can see, it helps a lot in creating a small and safe container. More stages can
be built on top, some to build the frontend written in _TypeScript_ for example.
This opens up a wide range of features and opportunities for us to use to our advantage.

View file

@ -0,0 +1,81 @@
+++
title = "What 2025 blog question challenge ?"
author = ["Elia el Lazkani"]
date = 2025-02-20
lastmod = 2025-02-20
tags = ["challenge", "qa", "questions", "answers"]
categories = ["misc"]
draft = false
+++
A few weeks ago, [Jason](https://janusworx.com) sneakily slipped me a link to a [blog post](https://janusworx.com/work/blog-questions-challenge-2025/) of his. He
knows that I _already_ subscribe to his blog but you cannot be as wise without
meddling with the forces of nature. Well, long story short, his meddling worked
so... here we go ?
## Why did you make the blog in the first place? {#why-did-you-make-the-blog-in-the-first-place}
<!--more-->
I, honestly, cannot remember why I made it in the first place but it was a long
time ago. I do remember that my first post was about the _plugin_ I wrote for
`weechat`. After that, I sort of wrote on things I was playing around with. I
said it before, it's how I learn. Maybe the difference is that I dig deeper,
beyond the surface. It's fun, I get to document it for myself because believe it
or not, I do come back to the blog for refreshers. It also comes handy cause I
can simply send someone a link that explains a topic in more details.
## What platform are you using to manage your blog and why did you choose it? {#what-platform-are-you-using-to-manage-your-blog-and-why-did-you-choose-it}
I use _Hugo_ to generate this blog and if you know where to look, it is _open
source_. The answer to why is a bit more complicated but to simplify it, I
prefer a static blog over CRMs. _Hugo_ is quick and I have it integrated with
_Emacs_ where I write my _posts_ in `org`. The generation, packaging and
deployment is all automated.
## Have you blogged on other platforms before? {#have-you-blogged-on-other-platforms-before}
This blog started its life on _WordPress_. I was never quite happy with the
performance for such a simple blog. It was then migrated to _Joomla_. That phase
did not last long because once I learned about static blog generators, the blog
was migrated to _Nikola_ quickly. My _Nikola_ setup was quite elaborate and I
wanted to simplify it, for one, and to make it faster. I checked _Hugo_ out and
liked it but migrating from `rst` was a pain. `Orgmode` helped a lot with the
migration but after a long journey, here we are running on _Hugo_.
## How do you write your posts? {#how-do-you-write-your-posts}
The whole blog is one `org` file. I open it with _Emacs_ and add a new section
under the relevant `Tag`. Once I save, _Emacs_ generates the _Markdown_ file,
somewhere. I don't have to care about its management in most cases. I can, then,
generate and run my blog locally with _Hugo_. Once I'm happy with the result, I
_commit_ and _push_. The pipeline takes care of the rest, once all the magic is
done successfully, I deploy with a manual confirmation step.
## When do you feel most inspired to write? {#when-do-you-feel-most-inspired-to-write}
I don't know if that question applies to my case. I don't even know if I do. It
takes a lot of time and effort to write these posts. Documentation is **very**
hard, ask any developer. I'm not sure inspiration is a requirement, hard work is.
## Do you publish immediately after writing or do you let it simmer a bit as a draft? {#do-you-publish-immediately-after-writing-or-do-you-let-it-simmer-a-bit-as-a-draft}
I publish immediately. I mean the post has already took a few hours to write...
If I find a mistake in the future, I correct it and fix the timestamp of edit.
## Your favorite post on your blog? {#your-favorite-post-on-your-blog}
I don't have a favorite post. I know a few that I share frequently, though.
## Any future plans for your blog? Maybe a redesign, changing the tag system, etc.? {#any-future-plans-for-your-blog-maybe-a-redesign-changing-the-tag-system-etc-dot}
I don't believe so. Nothing visible to the user, but the infrastructure is
always moving.