I've always played around with different Linux distributions, including those which are "DIY" like Arch Linux, or source-based like Gentoo Linux, but there has always been a project in the back of my mind that I never had the time to start: Linux from Scratch.
Since I had an extended Christmas/New Year's holiday, I thought this would be the perfect time to sit down and embark on the adventure of building my own Linux distribution from source code.
Please note that this blog post is just my experience with LFS and not a step-by-step guide to achieve a bootable system (but I will provide some tips).
Also, this blog post was an after thought since I wanted to concentrate on the build itself, so there will be a limited number of screenshots.
So what exactly is "Linux from Scratch"?
To quote the project itself: Linux From Scratch (LFS) is a project that provides you with step-by-step instructions for building your own customized Linux system entirely from source.
Basically, you are given a book which contains almost everything you need to build your own Linux distribution.
and I say almost because there are some aspects which are overlooked; more on that later.
There are multiple versions of this book, based on the targeted init system. The recommended ones are: sysv and systemd. You also have two branches: stable which is the recommended format, and the development branch which contains daily rendered snapshots. The former should be avoided for first time LFS'ers since changes can happen at any time that break the build, but since you've already embarked on this adventure, you should be prepared for things to go wrong.
Tips from my experience.
LFS itself is pretty straight forward, but you have to take some things in consideration:
- If you never did the process of installing a distribution manually, take a look at Arch Linux first.
- Since we are on the topic of Arch Linux, it is really recommended to also have the Arch Wiki prepared since some steps are overlooked, such as:
- LFS assumes you are building for a BIOS/MBR partition scheme, because the steps for building GRUB are for such a partition scheme. The LFS book says that GRUB can be built for an UEFI system, but requires several packages beyond the scope of LFS and are included in BLFS (Beyond Linux from Scratch). They also make the mistake of assuming the boot disk has been partitioned with a GUID Partition Table (GPT), which is incompatible with BIOS/MBR. This in turn renders the build unbootable. For the partitioning scheme and installation of GRUB, I recommend to also check the GRUB Arch Wiki and Partition scheme Arch Wiki pages for layout examples and different ways of installing GRUB before proceeding to make an educated decision.
- Another helpful resource is the Gentoo Handbook.
- When downloading the software packages required, if downloaded in an automated manner, make sure to log every download so that you don't have missing packages.
- One example: in version
12.2-systemd(the one I chose),expatrequired version is2.6.2, but the link provided by the book won't work as the developers of the library deemed this version vulnerable. As such, I went with2.6.4after checking everything was compatible. - If you miss some packages, you can always download the source code later and copy it to your sources, but this is not the recommended way.
- One example: in version
- The LFS book assumes that you are building the system on a physical drive:
- In my case, I wanted to boot it into a VM. As such, I created a raw disk file with
dd:
- In my case, I wanted to boot it into a VM. As such, I created a raw disk file with
# dd if=/dev/zero of=./lfs.raw bs=1G count=<size> status=progress
- Partitioned it with
cfdiskand created a loop device to mount it withlosetup. - Also, make sure the disk itself is at least 30GiB.
- If you are using multiple terminal emulators (or a terminal multiplexer like
tmux), you can add the$LFSvariable to your.shrcso you don't have to create the variable for every terminal session.- This is also to make sure you don't forget to export the
$LFSvariable, otherwise, you risk doing things on your host machine.
- This is also to make sure you don't forget to export the
- Be careful to always use
root(unless the book says to use thelfsortesterusers) and always set the correct permissions.- Otherwise, there is a big chance the files will not be recognized due to wrong file owners or wrong permissions, which can lead to not working build.
- Even if you copy and paste commands, make sure you understand what they do before running them (and also make sure they are correctly pasted).
- SBUs (or standard build units) are just an approximation of the time it takes to compile and install a package using a single core of a specific CPU. This means that even if you compile with a single core (which is not recommended, with the exception of
ninja) it could take more or less time.- It also depends of other factors such as the
gccversion on the host and the load and prioritization of the compilation process.
- It also depends of other factors such as the
- Testing takes a lot of SBUs. For simple packages, you can skip the testing suites since they take more to test than to compile and install.
- I didn't test every package, only the ones I personally deemed critical(for example math libraries like
gmp,mpc,mpfrorbinutils,gcc, for compilation, linking, assembling, etc.) to the system to make sure that they are correctly compiled. - Also, if a test suite fails, don't worry, some are meant to fail. Check with the book to make sure it is one of them.
- I didn't test every package, only the ones I personally deemed critical(for example math libraries like
- If you wonder why some packages you have to recompile, that means you aren't following the book right.
- You are compiling those packages multiple times because you are cross-compiling them. That means you are first preparing them to be ready for the
chrootenvironment, then in thechrootenvironment you are preparing them for the lfs build. In the cross-compilation section of the book, you'll see the 3 stages explained. If you skip these compilations, theconfigurescripts will most likely not work and some features of the packages will be unavailable or straight-up not working. - This is also a reminder to read every step. If you are not interested in some of the background history, you can skip it, but not the important steps.
- You are compiling those packages multiple times because you are cross-compiling them. That means you are first preparing them to be ready for the
- After creating the virtual kernel file system, the best thing to do is backup.
- Also make sure to re-enter the
chrootenvironment after backing up.
- Also make sure to re-enter the
- If you do not know what debugging symbols are, or do not understand, just let them be.
- Stripping the debugging symbols without understanding can lead to misbehaving or missing libraries. Even if you are experienced, it's best to leave them be (speaking from experience ;) ).
- Most of the
systemdconfiguration sections are after building the system and should be done when you are booted in the LFS system.- Also, read this section very carefully so you can configure your system based on the approach you took.
- For the love of God, do not set
/dev/loopXin thefstab. This is just a temporary loopback device on your host.- In my case, for my VM, it was
/dev/sdaXwhere X is the partition number. Your disk naming may vary based on disk type.
- In my case, for my VM, it was
- For the kernel compilation, I went with a tiny config.
- Also, I had to copy over from my host
libzstd.so.1.5.6because I wasn't careful enough while stripping debugging symbols (at this point, I was 7 hours in) andmakewouldn't run.
- Also, I had to copy over from my host
- GRUB will most likely complain that you have an
ext2file system (even though you don't), just use--force- If your system doesn't boot into GRUB, but you are sure you followed the correct partitioning layout and scheme for your system (in this case MBR), you can use a live cd and
chrootinto your LFS drive from that VM and use:
- If your system doesn't boot into GRUB, but you are sure you followed the correct partitioning layout and scheme for your system (in this case MBR), you can use a live cd and
grub-mkconfig > /boot/grub/grub.cfg
- This will generate a GRUB config that should just work.
In the end, if everything went well, you can just boot into your LFS build.
This is a screenshot of my first boot into my LFS build. Ignore the specs of the VM, I was just rushing to create the VM to boot it.
Conclusion
In total, the time to get to a working LFS distribution was 8 hours on my personal workstation, which has the following specs:
- Host: Lenovo ThinkStation P500
- CPU: Intel(R) Xeon(R) E5-2690 v3 (24) @ 3.50 GHz
- GPU: AMD Radeon RX 6600
- RAM: 64GB DDR4 ECC
- Kernel: Linux 6.6.69-1-lts
- gcc: 14.2.1
- make: 4.4.1
It was truly a remarkable experience, and I think in these 8 hours, I've compiled more utils than I did in my entire life.
Would I do this again? Probably, but if I want to create a customized GNU/Linux distribution for a certain use-case with specific configs for the kernel, I would just go with Gentoo since: it already has package management with portage, and you can still customize the compilation for each package, or just install a binary. portage is very powerful and, in many cases, it helps you when creating an actually daily-drive-able with built-in support for multilib and packages built by the developers targeting Gentoo.
LFS is time-consuming and results in a bare-bones system, but at the same time an educational experiment and it really shows how we take for granted all of the low-level utilities that make up a functional Linux distribution (and how perl truly is an important piece in every utility, no wonder it is called the swiss army knife for awk, sed, bash, etc.)
Join me next time, as I build BSD Unix for a PDP 11. /s