ramblings of a chaotic mind

Smaller docker containers

on 2022-08-13 by Gertjan Assies reading time 0 min, viewed 15 times, read 0 times, liked 0 times

Smaller docker containers

In the company where I work as a Site Reliability Engineer, we currently have around 60k devices in the field, we are expected to scale that to half a million by 2025. and they all need to communicate with our back-office which consists of a whole bunch of containerized microservices.
This means for everything that we do, we have to ask ourselves the age-old question:

does it scale?

and when it does, how can we make sure the costs do not scale faster than the resources we need.
One thing to help with that is to make sure our docker containers are as small as possible.
This will also help in other areas, less software running means less things can crash and a significantly smaller attack surface, so stability and security will increase too, Hoorah!

The example I’m going to expand upon works with a small go application but could be applied to almost any language that compiles to a native application.

If you want to dive into the code first, here’s the repo: https://gitlab.com/gertjana/tinyweb

Something to test with

The infamous hello world application here is done with the echo web framework just to have something I can verify working from within the container.

package main
import (  
func main() {  
  e := echo.New()  
  e.GET("/", hello)  

func hello(c echo.Context) error {  
  return c.String(http.StatusOK, "Hello, World!")  

Build a normal docker container for your app

FROM golang:1.16
RUN mkdir -p /app  
COPY main.go /app  
COPY go.* /app  
RUN CGO_ENABLED=0 GOOS=linux go build -o service main.go
EXPOSE 8000  
CMD ["/app/service"]

To set a baseline, I’m just creating a docker image from the golang:1.16 base-image, copy over the sources and compile them.

Image size: 968 Mb (968212150) almost a Gb!

A multistage docker image

FROM golang:1.16 AS builderCOPY main.go /app  
COPY go.* /app  
RUN CGO_ENABLED=0 GOOS=linux go build -o dist/service main.go
ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini-static /tini  
RUN chmod +x /tini

FROM scratch
COPY --from=builder /app/dist/service /service  
COPY --from=builder /tini /tini
ENTRYPOINT ["/tini", "--"]  
CMD ["/service"]

Here I’m doing a multistage build, still using the Golang image to build the application but then in the second stage, I copy over the compiled application to a base image called scratch.
Scratch is a special base image which you cannot pull or download, but it is basically a completely empty container to start with.

As it is empty it also does not contain an init system which normally runs on Linux systems and serves as the root process (PID 1) for all other processes, making sure the right signals are sent to the right process, zombie processes are cleared up etc.
Tini is a small init system for containers. this way the application thinks it’s running in a ‘normal’ Linux system.

Image size: 8.8 Mb (8799493)

Remove debug info

with the -s -w ldflags you can tell the compiler to not put any debug information in the binary. (for production systems it would be prudent to also create an image with the debug information in case you need to fix an urgent issue)

The Docker file is the same as above except for the following line

RUN CGO_ENABLED=0 GOOS=linux go build **-ldflags="-s -w"** -o dist/service main.go

Image size: 6.5 Mb (6559840)

Compress binary with UPX

UPX is a tool that compresses executables.

after the compile step in the build stage, the following will download and execute upx on the freshly build application.

ADD https://github.com/upx/upx/releases/download/v3.96/upx-3.96-amd64_linux.tar.xz upx-3.96-amd64_linux.tar.xz  
RUN tar -xf upx-3.96-amd64_linux.tar.xz  
RUN mv upx-3.96-amd64_linux/upx .  
RUN ./upx --brute /app/dist/service

Resulting Image size: 2.6 Mb (2567632)


to summarize the images created with their sizes

Image Debug info UPX compression Size Percentage
golang yes no 968212150 100.00%
scratch yes no 8799493 0.91%
scratch no no 6559840 0.68%
scratch no yes 2567632 0.27%

We managed to get the image size down to 0.27% of the original which is a lot! Most of the gain was in using the scratch image, which avoided having a complete Linux system installed there.
Multi-stage builds make it possible to have an image with all the tools needed to build your app and then just copy over the built application to an image that only has the minimum needed to run your application.

So I hope I’ve shown you to always try to create the minimal image you need to run your application.

I used the following to determine the image sizes:

> docker inspect -f "{{ .Size }}" gertjana/tinyweb:tiniest  


Image attribution

courtesy https://unsplash.com/@guibolduc

Opinions expressed here are my own and not the views of my employer or anyone else, (re)use is free, but quoting the source is appreciated.
This blog is licensed under a Creative Commons Attribution 4.0 International License. © 2023 by Gertjan Assies