<- previous - home - next ->
Building a Basic Container
In this section, we will build a brand new container similar to the lolcow container we’ve been using in the previous examples.
To build a singularity container, you must use the
build command. The
build command installs an OS, sets up your container’s environment and installs the apps you need. To use the
build command, we need a definition file. A Singularity definition file is a set of instructions telling Singularity what software to install in the container.
We are going to use a standard development cycle (sometimes referred to as Singularity flow) to create this container. It consists of the following steps:
- create a writable container (called a
- shell into the container with the
--writableoption and tinker with it interactively
- record changes that we like in our definition file
- rebuild the container from the definition file if we break it
- rinse and repeat until we are happy with the result
- rebuild the container from the final definition file as a read-only singularity image format (SIF) image for use in production
The Singularity source code contains several example definition files in the
/examples subdirectory. Let’s make a new directory, copy the Debian example definition file, and inspect it.
$ mkdir ~/lolcow $ cp ~/singularity/examples/debian/Singularity ~/lolcow/lolcow.def $ cd ~/lolcow $ nano lolcow.def
It should look like this:
BootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.us.debian.org/debian/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" apt-get -y --allow-unauthenticated install vim
See the Singularity docs for an explanation of each of these sections.
Developing a new container
Now let’s use this definition file as a starting point to build our
lolcow.img container. Note that the build command requires
sudo privileges. (We’ll discuss some ways around this restriction later in the class.)
$ sudo singularity build --sandbox lolcow lolcow.def
This is telling Singularity to build a container called
lolcow from the
lolcow.def definition file. The
--sandbox option in the command above tells Singularity that we want to build a special type of container (called a sandbox) for development purposes.
Singularity can build containers in several different file formats. The default is to build a SIF (singularity image format) container that uses squashfs for the file system. SIF files are compressed and immutable making them the best choice for reproducible, production-grade containers.
But if you want to shell into a container and tinker with it (like we will do here), you should build a sandbox (which is really just a directory). This is great when you are still developing your container and don’t yet know what to include in the definition file.
When your build finishes, you will have a basic Debian container saved in a local directory called
shell --writable to explore and modify containers
Now let’s enter our new container and look around.
$ singularity shell lolcow
Depending on the environment on your host system you may see your prompt change.
Let’s try installing some software. I used the programs
lolcat to produce the container that we saw in the first demo.
Singularity> sudo apt-get update bash: sudo: command not found
sudo command is not found. But even if we had installed
sudo into the
container and tried to run this command with it, or change to root using
we would still find it impossible to elevate our privileges within the
Singularity> sudo apt-get update sudo: effective uid is not 0, is /usr/bin/sudo on a file system with the 'nosuid' option set or an NFS file system without root privileges?
Once again, this is an important concept in Singularity. If you enter a container without root privileges, you are unable to obtain root privileges within the container. This insurance against privilege escalation is the reason that you will find Singularity installed in so many HPC environments.
Let’s exit the container and re-enter as root.
Singularity> exit $ sudo singularity shell --writable lolcow
Now we are the root user inside the container. Note also the addition of the
--writable option. This option allows us to modify the container. The changes will actually be saved into the container and will persist across uses.
Let’s try installing our software again.
Singularity> apt-get update Singularity> apt-get install -y fortune cowsay lolcat
Now you should see the programs successfully installed. Let’s try running the demo in this new container.
Singularity> fortune | cowsay | lolcat bash: lolcat: command not found bash: cowsay: command not found bash: fortune: command not found
It looks like the programs were not added to our
$PATH. Let’s add them and try again.
Singularity> export PATH=$PATH:/usr/games Singularity lolcow:~> fortune | cowsay | lolcat ________________________________________ / Keep emotionally active. Cater to your \ \ favorite neurosis. / ---------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
Great! Things are working properly now.
If you receive warnings from the Perl language about the
locale being incorrect, you can usually fix them with
We changed our path in this session, but those changes will disappear as soon as we exit the container just like they will when you exit any other shell. To make the changes permanent we should add them to the definition file and re-bootstrap the container. We’ll do that in a minute.
Building the final production-grade SIF file
Although it is fine to shell into your Singularity container and make changes while you are debugging, you ultimately want all of these changes to be reflected in your definition file. Otherwise if you need to reproduce it from scratch you will forget all of the changes you made. You will also want to rebuild you container into something more durable, portable, and robust than a directory.
Let’s update our definition file with the changes we made to this container.
Singularity> exit $ nano lolcow.def
Here is what our updated definition file should look like.
BootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.us.debian.org/debian/ %runscript echo "This is what happens when you run the container..." %post echo "Hello from inside the container" apt-get update apt-get -y install fortune cowsay lolcat %environment export PATH=$PATH:/usr/games
Let’s rebuild the container with the new definition file.
$ sudo singularity build lolcow.sif lolcow.def
Note that we changed the name of the container. By omitting the
--sandbox option, we are building our container in the standard Singularity file format (SIF). We are denoting the file format with the (optional)
.sif extension. A SIF file is compressed and immutable making it a good choice for a production environment.
As we saw in the previous section when we used the
inspect command to read the
runscript, Singularity stores a lot of useful metadata. For instance, if you want to see the definition file that was used to create the container you can use the
inspect command like so:
$ singularity inspect --deffile lolcow.sif BootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.us.debian.org/debian/ %runscript echo "This is what happens when you run the container..." %post apt-get update apt-get -y install fortune cowsay lolcat %environment export PATH=$PATH:/usr/games
Building from existing containers
In the preceding section we always used the following header in our definition file to build a container:
BootStrap: debootstrap OSVersion: stable MirrorURL: http://ftp.us.debian.org/debian/
This uses the program
debootstrap to build the root file system using a mirror URL. In this case, we supply a URL that is maintained by Debian. We could also use an Ubuntu URL since it is a derivative of Debian and can also be built with the
debootstrap program. If we wanted to build a CentOS container from the distribution mirror we could use the
yum package manager similarly. There are actually a ton of different ways to build containers. See this list of “bootstrap agents” in the Singularity docs.
In practice, most people do not build containers from a distribution mirror like this. Instead they tend to build containers from existing containers on the Container Library or on Docker Hub and use the
%post section to modify those containers to suit their needs.
For instance, to use an existing Debian container from the Container library as your starting point, your header would look like this:
BootStrap: library From: debian
Likewise to start from a debian container on Docker Hub, your header would contain the following:
Bootstrap: docker From: debian
You can also build a container from a base container on your local file system.
Bootstrap: localimage From: /home/student/debian.sif
Each of these methods can also be called without providing a definition file using the following shorthand. For an added bonus, none of these
build commands require root privileges.
$ singularity build debian1.sif library://debian $ singularity build debian2.sif docker://debian $ singularity build debian3.sif debian2.sif
Behind the scenes, Singularity creates a small definition file for each of these commands and then builds the corresponding container as you can see if you use the
inspect --deffile command.
$ singularity inspect --deffile debian1.sif bootstrap: library from: debian $ singularity inspect --deffile debian2.sif bootstrap: docker from: debian $ singularity inspect --deffile debian3.sif bootstrap: localimage from: debian2.sif
Note that the third command may not seem very useful because you are just copying the container called
debian2.sif to a new container called
debian3.sif. But you can also use
build in this way to convert a SIF file to a sandbox and back again:
$ singularity build --sandbox deb-sand debian3.sif $ singularity build deb-sif deb-sand/
This can be a useful trick during container development. But it can also produce a container with an uncertain build history if it is misapplied because the changes made to the sandbox will not be reflected in the containers definition file.
Security considerations and
In the preceding we’ve been executing the
build command as root via
sudo. In our examples, that is a reasonably safe thing to do, because we use disposable virtual machines for this class and we are building directly from mirrors hosted by same groups that create the OS distributions. (Though mirrors can still contain malware.)
But in general, it’s a bad idea to build a container as root. In particular you should never build a container from an untrusted base image as root on a machine you care about. This is the same as downloading random code from the internet and running it as root on your machine. (See this blog for a technical discussion.)
On operating systems with recent kernels (such as Ubuntu 18.04), you can invoke the
--fakeroot option when building containers instead. (For those interested in technical details, this feature leverages the user namespace).
$ singularity build --fakeroot container.sif container.def
Doing so allows you to pretend to be the root user inside of your container without actually granting singularity elevated privileges on host system. This is a much safer way to build and interact with your container, and it is going to become more prevalent (eventually probably even default) as more distributions ship with user namespaces enabled. For instance, this feature is enabled by default in RHEL 8.
For more about the
--fakeroot option, see the Singularity documentation.
<- previous - home - next ->