Lorenzo Bettini is an Associate Professor in Computer Science at the Dipartimento di Statistica, Informatica, Applicazioni "Giuseppe Parenti", Università di Firenze, Italy. Previously, he was a researcher in Computer Science at Dipartimento di Informatica, Università di Torino, Italy.
He has a Masters Degree summa cum laude in Computer Science (Università di Firenze) and a PhD in "Logics and Theoretical Computer Science" (Università di Siena).
His research interests cover design, theory, and the implementation of statically typed programming languages and Domain Specific Languages.
He is also the author of about 90 research papers published in international conferences and international journals.
This method consists of a two-step installation process:
use the standard EndeavourOS ISO, booting that from a PC, to install the installation image on an external device (in this example, I will use a USB stick);
then boot the PineBook Pro with the created USB stick and use Calamares to finalize the installation on the very same device you booted from.
Note that I will install EndeavourOS for Arm on an external device, NOT on the eMMC of the PineBook Pro. In this article, I’ll leave a few hints on how to do that on the internal eMMC.
First step
On a standard PC, boot the EndeavourOS ISO (in this example, I’m using the Cassini 2023-03 R2):
After adjusting the keyboard layout and connecting to the Internet, choose “EndeavourOS ARM Image Installer”.
As noted, you need first to insert a USB stick. If you plan to install it on the PineBook Pro’s internal eMMC, you must extract it and place it in a USB adapter. Then, choose “Strat ARM Installer”. That is a textual installation procedure so the installer will open a terminal in full-screen mode.
After pressing OK, you must select the ARM computer (in this case, “PineBook Pro”):
Concerning the file system, in all my experiments, BTRFS has never worked: when rebooting the USB stick (see later), the screen stays blank forever after selecting the boot media. So, the only working solution is EXT4:
Then, you have to write the device where you want to install the installer; the dialog shows all the devices, and you must write the main path of the device, NOT of a possibly existing single partition (in this case, it’s “/dev/sdb”):
Small note: unfortunately, the colors of this textual installer are not ideal 😉
Then, the procedure will prepare the device and download an archive from the Internet for the image to put on the USB stick (it’s a big image, so be patient):
Ultimately, it tells you about the temporary username and password for the installer copied on the USB. It also suggests unmounting the USB with a file manager. In the live environment, you use Thunar to unmount the USB stick. You can recognize the mounted USB stick to unmount because it should show two mounted partitions (the first one is about 128 Mb):
Umounting one of them will also unmount the other one.
Second step
It’s time to boot the PineBook Pro with the UBS stick we created with the abovementioned process. If, in the previous procedure, you created the installer on the eMMC (connected with a USB adapter), you should put the eMMC inside the PineBook Pro.
When the PineBook Pro starts, you should find a way to boot from the USB stick. If you’ve always used the Manjaro installation that comes with the PineBook Pro, you have U-Boot as the bootloader (see my previous blog post for a screenshot of U-Boot booting from the USB stick). If you’re lucky, it should give precedence to the USB stick (I’ve read that this is not always the case, depending on the version of the installed U-Boot). In this example, I have Tow-Boot as the bootloader, so when you see the message telling you to press ESCAPE (or Ctrl-C) to enter the boot menu, please do so:
And then, select the USB as the boot media (of course, if you installed the image on an SD, choose accordingly):
After some textual logs, you should get to the graphical environment for the actual installation. The Window Manager is OpenBox, so, differently from the standard EndeavourOS installer for PC, you don’t have a fully-fledged Desktop Environment (Xfce):
Now, you can choose whether to install an “Official” (e.g., KDE or GNOME) or a “Community” edition (e.g., Sway).
Remember: the installation will be performed on the same media you have just booted. In this example, it’s a USB stick. Again, if you want to install EndeavourOS on the internal eMMC, you first need to extract the eMMC, put it on a USB adapter, do the first procedure described above, put the eMMC back into the PineBook Pro, and start the installation from the eMMC.
As you can see from the screenshots above, there’s no section for partitioning the disk. The partitions have already been created during the first procedure. This installation procedure only finalizes the installation.
I’ve tried both KDE and GNOME.
Enjoy your EndeavourOS installation 🙂
If you like it on a USB stick (remember, it should be a fast USB), you might want to install it on the eMMC (see the notes in this blog post about that). I have already done that, and it works much better than the original Manjaro!
This blog post will describe my Ansible role for installing “Oh My Zsh” and several command line programs. The role also installs the starship prompt or the p10k theme. As for the other roles I’ve blogged about, this one is tested with Molecule and Docker and can be developed with Gitpod (see the linked posts above). In particular, it is tested in Arch, Ubuntu, and Fedora.
This role is for my personal installation and configuration and is not meant to be reusable.
Besides “zsh” and “git” (which are needed for installing other things, and, in general, I need it daily), this installs several command line tools, like ripgrep, procs, dust, exa, bat, zoxide. Note that, depending on the operating system, these tools must be installed differently (e.g., from the package manager or by downloading a binary distribution). In a few cases, the package names differ depending on the operating system. In such cases, the default names are defined in “vars/main.yml” and properly overridden depending on the operating system:
YAML
1
2
3
4
5
6
# vars file for oh_my_zsh
powerlinefonts: powerline-fonts
fdfind: fd-find
dustversion: v0.8.3
procsversion: v0.13.3
with_starship: true
The task also installs a few fonts (nerd fonts, with emoji, or with icon characters), which are needed because “starship” and “p10k” need a few icon characters; the same holds for other tools like exa.
“Oh My Zsh” is installed by cloning its Git repository; the same holds for some external plugins. The task also sets “zsh” as the default shell for the current user.
Finally, depending on the variable with_starship, which defaults to true, it installs the starship prompt or the p10k theme. These are handled in the corresponding included files “starship.yml” and “p10k.yml”, respectively.
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
---
# tasks file for starship
- name: Install starship (cross-shell prompt) as a package
become: true
ansible.builtin.package:
state: present
name: starship
when: ansible_os_family == 'Archlinux'
- name: Install curl (for starship installation)
become: true
ansible.builtin.package:
state: present
name: curl
when: ansible_os_family != 'Archlinux'
- name: Get starship install script
ansible.builtin.get_url:
url: https://starship.rs/install.sh
dest: /tmp/starship_install.sh
mode: '0755'
register: starship_installation_script
when: ansible_os_family != 'Archlinux'
- name: Install starship with installation script
become: true
ansible.builtin.shell:
cmd: /tmp/starship_install.sh --yes
executable: /bin/sh
when: ansible_os_family != 'Archlinux' and starship_installation_script.changed
# if the previous task hasn't changed, the shell script is already there
Note that both files copy the corresponding template for “.zshrc” (depending on starship or p10k, the contents of “.zshrc” are slightly different). For “p10k”, it also copies my theme configuration”; for “starship”, I’m OK with the default prompt. The copied “.zshrc” contains several “aliases” for the command line programs installed by this role (e.g., “ls” is aliased to “exa” commands).
Concerning Molecule, I have several scenarios. As I said, I tested this role in Arch, Ubuntu, and Fedora, so I have a scenario for each operating system. In such cases, I test the “starship” installation and verify that the tools that differ for their installations in different operating systems are installed correctly. This is the “verify.yml” (remember that this installs “starship” and NOT “p10k”, so it ensures that only the former is installed):
- name: Assert ~/.p10k.zsh is NOT present in this scenario
ansible.builtin.assert:
that: "notresult.stat.exists"
Concerning “p10k,” I have a separate scenario with a different “verify.yml” (I test this only on Arch since “starship” and “p10k” installations and configurations are the same in all three operating systems):
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
---
# This is an example playbook to execute Ansible tests.
However, this “verify.yml” could also be used for the other operating systems since it performs the same verifications concerning installed programs. It differs only for the final part.
Of course, this is tested on GitHub Actions and can be developed directly on the web IDE Gitpod.
I hope you find this post useful for inspiration on how to use Ansible to automatize your Linux installations 🙂
With TLP, the procedure is easier and more automatic.
First, you must install tlp (remember that tlp conflicts with power-profiles-daemon, so you have to disable the latter first or uninstall it). In Arch-based distros:
1
sudo pacman-Stlp tlp-rdw
Ensure that the tlp service is enabled on boot and the first time you should start it (“sudo systemctl start tlp”).
By running “sudo tlp-stat”, you should see near the end this line:
1
2
3
4
5
6
7
+++ Battery Care
Plugin: lg
Supported features: charge threshold
Driver usage:
* vendor (lg_laptop) = active (charge threshold)
Parameter value range:
* STOP_CHARGE_THRESH_BAT0: 80(on), 100(off) -- battery care limit
Edit the file “/etc/tlp.conf” and uncomment the following lines (note there’s one also for the start of charging, but that option doesn’t seem to be supported in this laptop):
1
2
START_CHARGE_THRESH_BAT0=75
STOP_CHARGE_THRESH_BAT0=80
Restart the service (“sudo systemctl restart tlp.service”), and it should be already active (run “tlp-stat” again):
That’s all. This will persist on reboot. However, this will not persist if you hibernate and return from hibernation (unless you restart the tlp service as shown above).
In this blog post, I’m showing how I customize KDE on that installation (thus, it is similar to this other one for GNOME).
I’m not using Wayland because I still don’t find it usable in Plasma. Indeed, I haven’t even installed the Wayland session: I’m using X11.
First of all, the default screen resolution of KDE is too tiny for my eyes. Thus, I set 150% (fractional) scaling:
After that, you have to log out and log in to see the setting in effect.
I also enable “Tap-to-click” in the “Touchpad” settings.
Then, I set “Ctrl+Alt+T” as a shortcut for opening the terminal (Konsole). Actually, the shortcut is already configured in the “Custom Shortcuts”, but it’s not enabled by default in this distribution. It’s just a matter of selecting the corresponding checkbox (the “Examples” checkbox must be selected first to enable the other checkbox):
Then, I install an AUR helper. I like “yay,” so I first installed the needed dependencies:
1
sudo pacman-S--needed git base-devel
And then
1
2
3
4
5
mkdir -p ~/git
cd ~/git
git clone https://aur.archlinux.org/yay.git
cd yay
makepkg -si
I install, by using “yay”, the “touchegg” program for touchpad gestures. (Plasma Wayland already provides touchpad gestures, but I prefer to use the X11 session, as I said above). For ARM, we need to install the AUR package:
1
yay-Stouchegg
You will get this warning, but proceed anyway: it compiles and works fine:
1
2
3
4
:: (1/1) Parsing SRCINFO: touchegg
-> The following packages are not compatible with your architecture:
touchegg
:: Try to build them anyway? [Y/n]
I customize touchegg touchpad gestures for KDE by creating the file “~/.config/touchegg/touchegg.conf” with the following contents:
XHTML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
<touchégg>
<settings>
<!--
Delay, in milliseconds, since the gesture starts before the animation is displayed.
Default: 150ms if this property is not set.
Example: Use the MAXIMIZE_RESTORE_WINDOW action. You will notice that no animation is
displayed if you complete the action quick enough. This property configures that time.
-->
<property name="animation_delay">150</property>
<!--
Percentage of the gesture to be completed to apply the action. Set to 0 to execute actions unconditionally.
Default: 20% if this property is not set.
Example: Use the MAXIMIZE_RESTORE_WINDOW action. You will notice that, even if the
animation is displayed, the action is not executed if you did not move your fingers far
enough. This property configures the percentage of the gesture that must be reached to
<!-- Change KP_Right to KP_Left if you prefer the other direction -->
<gesture type="SWIPE"fingers="4"direction="LEFT">
<action type="SEND_KEYS">
<repeat>false</repeat>
<modifiers>Alt_L</modifiers>
<keys>KP_Right</keys>
<on>begin</on>
</action>
</gesture>
<gesture type="PINCH"fingers="4"direction="IN">
<action type="SEND_KEYS">
<repeat>true</repeat>
<modifiers>Control_L</modifiers>
<keys>Shift_L+Tab</keys>
<decreaseKeys>Shift_L+Tab</decreaseKeys>
<on>begin</on>
</action>
</gesture>
<gesture type="PINCH"fingers="4"direction="OUT">
<action type="SEND_KEYS">
<repeat>true</repeat>
<modifiers>Control_L</modifiers>
<keys>Tab</keys>
<decreaseKeys>Shift_L+Tab</decreaseKeys>
<on>begin</on>
</action>
</gesture>
</application>
</touchégg>
In particular, note the 3-finger gestures (for the “Expose” effect, hide all windows, switch workspace, etc.).
Let’s start “touchegg” and verify that gestures work
1
touchegg
And then let’s enable it so that it automatically starts on the subsequent boots:
1
sudo systemctl enable touchegg.service
Let’s move on to ZSH, which I prefer as a shell:
1
sudo pacman-Szsh
Since I’m going to install “Oh My Zsh” and other Zsh plugins, I install these fonts (remember from the previous post that I had already installed “noto-fonts” and “noto-fonts-emoji”) and finder tool (“curl” is required for the installation of “Oh My Zsh”):
1
sudo pacman-Spowerline-fonts fzf curl
Let’s install “Oh My Zsh” by running the following command as documented on its website:
Once saved, you have to start a new terminal with zsh to see the plugins in action (remember that, until you log out and log in, the default shell is still BASH, so you might have to run “zsh” manually to switch to ZSH in the currently logged session).
Besides the syntax highlighting for commands, you have completion after “cd” (press TAB), excellent command history (with Ctrl+R), suggestions, etc.
Let’s switch to the “Starship” prompt. Let’s run the documented installation program:
1
curl-sS https://starship.rs/install.sh | sh
Now, let’s edit the ~/.zshrc file again; we comment the line starting with “ZSH_THEME,” and we add to the end of the file:
1
eval"$(starship init zsh)"
Opening another ZSH shell, we should see the fantastic Starship prompt in action, e.g.,
To quickly search for file names from the command line, I install “locate”, enable its periodic indexing and run the indexing once the first time (if you’re on a BTRFS file system, you might want to have a look at this older post of mine):
1
2
3
4
5
sudo pacman-Splocate
sudo systemctl enable plocate-updatedb.timer
sudo updatedb
Then, you should be able to look for files with the command “locate” quickly.
Gnome uses “Baloo” for file indexing and searching, e.g., from “KRunner” (Alt+Space) or the application launcher (Alt+F1 or simply Meta). I like it, and it quickly keeps the index up to date. However, by default, baloo also indexes the file contents, which uses too many resources, so I disable the content indexing feature. I blogged about configuring baloo in another post.
Speaking about KRunner, I find it extremely slow on this laptop; especially the first time you run it, it might take a few seconds to show up. I often use it to search for files or programs, and I need such a mechanism to be fast. I found that the application launcher is instead fast to start (Meta key). However, when you start typing to search, the results are all mixed: applications, files, and settings are all together, and it might not be easy to find what you need:
For this reason, I right-click on the KDE icon and select the “Alternative” “Application Menu”:
This launcher has a valuable feature to categorize the search results. For example, with the exact search string as above, I get results clearly separated:
I also use the “yakuake” drop-down terminal a lot:
1
sudo pacman-Syakuake
I run it once (it’s enabled by default by pressing “F12”), and it will start automatically upon logging in.
TL; DR: Xtext sources are now in a single Git monorepo (https://github.com/eclipse/xtext), and the build infrastructure is based entirely on Maven/Tycho (Gradle is not used anymore).
Background
A few years ago, Xtext sources were split into 6 separate GitHub repositories. I did not take part in that decision (I guess at that time, I wasn’t even a committer).
I think that at that time, the splitting was carefully thought out, aiming at making contributions and maintenance easier. In fact, while Xtext is mainly based on Eclipse, it has several core parts that are independent of Eclipse (including LSP support and plain Maven/Gradle projects support, without the Eclipse UI).
Those core parts were built with Gradle. I’m not a fan of Gradle: I’ve always found Gradle gives you too much power as a build tool, not to mention that, typically, the build breaks with new releases of Gradle. On the contrary, Maven has been around for several years and is rock solid. It’s rigid, so you have to obey its rules. But if you do, everything works.
Moreover, its files (POM) and build infrastructures always look the same in Maven projects, so you know where to look when you deal with a Maven project. For Eclipse projects, Tycho complements Maven. Tycho is not perfect, but everything works if you stick with its rules.
The rigidness of Maven/Tycho usually forces you to keep the build simple. And that’s good! 🙂
Of course, also previously, the Eclipse projects of Xtext had to be built with Maven/Tycho. Thus, the build of Xtext was split into 6 repositories with a mixture of Gradle builds and Maven/Tycho builds.
As a result, Xtext was a nightmare to build and maintain. That’s my opinion, of course, but if you read Xtext GitHub issues, PRs, and discussions, you see I wasn’t the only one to say that.
For example, the main problem was that if you changed something in one of the core projects (i.e., on its Git repository), you had to make sure that all the other projects did not break:
once the core project was built successfully, archive its artifacts (p2 repository and Maven artifacts) in Jenkins (JIRO in Eclipse)
trigger downstream jobs, making sure to use the archived artifacts corresponding to the branch with the changes
wait for all downstream jobs to finish
if anything breaks, go back from the beginning and start again!
Moreover, while it was easy to parametrize the Maven and Gradle files to use the archived Maven artifacts, it was a bit harder to use the archived p2 repositories in the target platforms of the Eclipse downstream projects. Typically, you had to temporarily modify and commit the target platform files to refer to the archived p2 repositories corresponding to the current feature branch. Before merging, you must remember to revert the changes to the target platform files.
In the end, the code in the single Git repositories was actually coupled, so the splitting in single Git repositories has always sounded wrong to me.
The release process used to be complex as well. Again, that was due to Git repository splitting and a mixture of Maven/Gradle builds.
A big change
I had always wanted to bring Xtext sources back into a single Git repository and to use Maven/Tycho only. I told the project lead Christian Dietrich about that many times, but he was always reluctant due to the time and effort it would have taken. On January 2023, I finally managed to convince him to give me a chance 🙂
I first started porting all single repositories to Maven/Tycho, removing the Gradle build.
Things were not so easy, but not even impossible. A few tests were failing in the new build infrastructure, but Christian and Sebastian Zarnekow helped me to bring everything to a working situation. We also took the chance to fix a few things in that respect.
In this stage, I also took the chance to set up the CI building also on GitHub Actions (our primary CI is, of course, Eclipse Jenkins “JIRO”).
Then, I moved to the repo merging. This part was easy. I had prepared the single Git repositories thinking about repository merging, so fixing the foreseen merge conflicts was just a matter of adjusting a few things in only a few files (mainly the parent POM).
Finally, I simplified the release infrastructure a lot. We used to have a vast Jenkinsfile for the release, with many complex operations based on Git cloning all the repositories, tagging, and tons of shell operations.
Now, the whole release is carried on during the Maven build (as it should be, in my opinion), including the uploading to the Eclipse web directories. The Jenkinsfile is very small.
The Oomph setup has been simplified as well. It takes time to have a development workspace, but that is due to dependency resolutions and the first Eclipse build. It takes place automatically.
Now, the complete Maven/Tycho build of Xtext, which consists of compilation and running all the tests (almost 40k tests!), takes about 1 hour and a half on Linux (Jenkins and GitHub Actions) and about 2 hours on macOS in GitHub Actions. It takes much less than before when you had to wait for the builds of the single Git repositories and downstream projects. The building of some single Git repositories was, of course, faster, but the overall time for building all the Git repositories was much more significant in the end.
Final thoughts
I hope this vast change improves the maintainability of the project. It should also help contributions from the community. Maybe I’m biased, but personally, I find working and contributing to Xtext much better now, as it used to be in the old days 😉
To me, the effort was very worthwhile!
In the end, it didn’t take long. Of course, I do not measure in months passed from the start of the porting (I started in the first days of January, and the first release with the new infrastructure, version 2.31.0, was at the end of May, though the first milestone with the new infrastructure was at the beginning of April). The time I spent working on that was much less. Of course, I hadn’t worked on that full-time: I did that in my spare time 🙂
In my humble opinion: If your project’s infrastructure and build are too complex, don’t change the build tools: change the latter and keep it simple. 😉 A single big Git monorepo is easier to deal with than 6 small Git repositories, especially when the single Git repositories contain code meant to belong together.
Many thanks to Christian and Sebastian for letting me work on this restructuring and promptly helping me during the process! 🙂
I’ve just started experimenting with the Wayland compositor Hyprland and wanted to use my favorite wallpaper manager, Variety. Unfortunately, Variety does not support Hyprland out of the box. However, it’s easy to make it work also on Wayland.
I’m going to use Arch Linux in this blog post.
First of all, you must install “swaybg”, a wallpaper tool for Wayland compositors, and “variety”:
1
sudo pacman-Sswaybg variety
Now, start variety and do the first-time configuration. Currently, trying to change the wallpaper will not work.
Variety creates the directory “~/.config/variety/scripts”. Edit the file “set_wallpaper” inside that directory and search for the block starting like this:
Shell
1
if[[-n$SWAYSOCK]];then
Change it like that (you could also remove the part about SWAYSOCK if you want or if you don’t plan to use “sway” at all):
This relies on the XDG_CURRENT_DESKTOP environment variable to be set accordingly, which should be like that automatically; you might want to check that:
1
2
echo$XDG_CURRENT_DESKTOP
Hyprland
Restart Variety, and now you can change the wallpaper!
This is a follow-up to the article Installing Arch Linux on a PineBook Pro (external storage); differently from the previous post, this one is based on more automatic mechanisms, so it will be easier to copy and paste commands once a few variables have been correctly and carefully set. Moreover, in this post, I’ll install KDE instead of GNOME. Finally, we’ll use BTRFS for the main partition, instead of EXT4.
This post will describe my experience installing Arch Linux on a PineBook Pro on external storage (a micro SD card in this example). Thus, the Manjaro default installation on the eMMC will not be touched. You should use a fast card, or the overall experience will be extremely slow.
First, install the official Arch Linux ARM distribution; this will not be enough to have many hardware parts working (e.g., WiFi, battery management, and sound).
Then, add the repositories with kernels and drivers for the PineBook Pro.
The first part must be performed from an existing Linux installation on the PineBook Pro. I will use the Manjaro installation that comes with the PineBook Pro. The second part will be performed on the installed Arch Linux system on an external drive (a USB stick in this example). Since after the Arch Linux installation, the WiFi is not working, for this part, you need to connect to the wired network, e.g., with an ethernet USB adapter.
I insert my SD card, which is /dev/sda. (Use “lsblk” to detect that.) By the way, typically, an SD card should be detected with a device name of the shape “/dev/mmcblkX”, but in this example, the SD card is inserted in a USB adapter, so its device name has the typical shape “/dev/sdX”.
From now on, I’m using this device. In your case, the device name might be different.
From now on, all the instructions are executed as “root” from a terminal window; thus, I first run:
1
sudo su -
I will do the following steps in a directory of the root’s home:
1
2
3
cd
mkdir pbp-install
cd pbp-install
We need to download and extract the latest release of Tow-Boot for the Pinebook Pro from https://github.com/Tow-Boot/Tow-Boot/releases. At the time of writing, the latest one is “2021.10-005”:
Now we flash Tow-Boot to /dev/sda (replace this with the device you are using).
Remember: this will wipe all the contents of the specified device. Moreover, make sure you specify the correct device, or you might overwrite the eMMC of the computer.
To make things easily reproducible and minimize the chances of specifying the wrong device name (which is extremely dangerous), I will use environment variables:
1
export DEVICE_NAME="/dev/sda"
The process creates the partition table for the device, with the first partition for Tow-Boot. This first partition must not be modified further. As you see in a minute, we skip the first partition when we further partition the disk.
The instructions must be followed strictly concerning, at least, the very first partition (the boot partition) that will be created, which must NOT touch the one created in the previous step. Then, after creating the boot partition, I’ll do things slightly differently: I will create a SWAP partition (not contemplated in the above instructions; The PineBook Pro has only 4 Gb of RAM, and it is likely to exhaust it, so it’s better to have a SWAP partition to avoid system hangs. Then, I’ll create the root partition.
These are the essential contents of my terminal where I follow the above-mentioned instructions (since I had already used this USB stick for some experiments before writing this blog post, fdisk detects an existing ext4 signature). Remember, though, that I created a SWAP partition that was not described in the above-mentioned instructions:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
Welcome to fdisk (util-linux 2.38.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
GPT PMBR size mismatch (26657 != 249737215) will be corrected by write.
The backup GPT table is not on the end of the device. This problem will be corrected by write.
Command (m for help): p
Disk /dev/sda: 119.08 GiB, 127865454592 bytes, 249737216 sectors
Disk identifier: E0CA6E57-39B2-4482-9838-21E2785CD93D
Device Start End Sectors Size Type
/dev/sda1 64 24639 24576 12M unknown
/dev/sda2 26624 550911 524288 256M Linux filesystem
/dev/sda3 550912 17328127 16777216 8G Linux swap
/dev/sda4 17328128 249735167 232407040 110.8G Linux filesystem
Filesystem/RAID signature on partition 3 will be wiped.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
Now I format the boot, the swap, and the root partitions. I will use EXT4 for the boot partition and BTRFS for the root partition.
Again, to increase reproducibility and avoid possible mistakes, I’m going to define additional environment variables, based on the one I have already created above, to refer to the 3 partitions:
1
2
3
export BOOT_PARTITION="${DEVICE_NAME}2"
export SWAP_PARTITION="${DEVICE_NAME}3"
export ROOT_PARTITION="${DEVICE_NAME}4"
It’s worthwhile to double-check that all the environment variables refer to the right partitions:
1
2
3
4
5
set|grep _PARTITION
BOOT_PARTITION=/dev/sda2
ROOT_PARTITION=/dev/sda4
SWAP_PARTITION=/dev/sda3
Remember that I’m using the environment variables set above:
1
2
3
mkfs.ext4 ${BOOT_PARTITION}
mkfs.btrfs -f ${ROOT_PARTITION}
mkswap ${SWAP_PARTITION}
Now we mount the root partition to create the BTRFS subvolumes, following a standard scheme, and we unmount it:
1
2
3
4
5
6
7
8
9
mount ${ROOT_PARTITION} /mnt
btrfs su cr /mnt/@
btrfs su cr /mnt/@home
btrfs su cr /mnt/@cache
btrfs su cr /mnt/@log
btrfs su cr /mnt/@tmp
umount /mnt
Now we have to mount all the subvolumes to the corresponding directories (the “-m” flag creates the mounting subdirectory if it does not exist); I’m enabling BTRFS compression (by default, the compression level for zstd will be 3):
1
2
3
4
5
mount -o subvol=/@,defaults,noatime,compress=zstd ${ROOT_PARTITION} /mnt
mount -o subvol=/@home,defaults,noatime,compress=zstd -m ${ROOT_PARTITION} /mnt/home
mount -o subvol=/@cache,defaults,noatime,compress=zstd -m ${ROOT_PARTITION} /mnt/var/cache
mount -o subvol=/@log,defaults,noatime,compress=zstd -m ${ROOT_PARTITION} /mnt/var/log
mount -o subvol=/@tmp,defaults,noatime,compress=zstd -m ${ROOT_PARTITION} /mnt/var/tmp
Then, we mount the boot partition on “/mnt/boot” (again, by creating that):
1
mount -m ${BOOT_PARTITION} /mnt/boot
Let’s verify that the layout of the destination filesystem is as expected:
1
2
3
4
5
6
7
8
9
10
tree /mnt/
/mnt/
├── boot
│ └── lost+found
├── home
└── var
├── cache
├── log
└── tmp
Now, we download the tarball for the rootfs of our USB stick installation. The instructions are once again taken from the link mentioned above, and they also include the verification of the contents of the downloaded archive:
Now, we must create the “/etc/fstab” on the mounted partition. To do that, we need to know the UUID of the two partitions by using “blkid”. You need to take note of the UUID from the output (which will be completely different according to the used external device):
We create the file “/etc/fstab” in “/mnt” according to the BTRFS subvolumes and to the other two partitions. This can be done by running the following command, which relies on the values of the 3 environment variables that we have just created:
Note that we must specify “rootflags=subvol=@” because the “/” of is on the subvolume “@”. Otherwise, the system can boot, but then nothing else will work.
We can now unmount the filesystems
1
2
umount /mnt/boot
umount -R /mnt
And we can reboot into the (hopefully) installed Arch Linux on the USB stick to finish a few operations. Remember that we need a wired connection for the next steps.
Upon rebooting, you should see the two entries (if you let the timeout expire, it will automatically boot the first entry):
After we get to the prompt, we can log in with “root” and password “root” (needless to say: change the password immediately).
Let’s connect a network cable (you need a USB adapter for that), and after a few seconds, we should be online. We verify that with “ifconfig”, which should show the assigned IP address for “eth0”.
Since there’s no DE yet, I suggest you keep following the official web pages (and this blog post) by connecting to the PineBook Pro via SSH so that it will be easy to copy and paste commands into the terminal window of another computer. Moreover, when logged into the PineBook Pro directly, you will see lots of logging information directly on the console (I guess this could be prevented by passing specific options to the kernel, but we’ll install a DE later, so I don’t care about that much). The SSH server is already up and running in the PineBook Pro installed system, so once we know the IP address from the output of “ifconfig”, we can connect via SSH. However, root access via SSH is disabled, so we must connect with the other predefined account “alarm” and password “alarm” (again, you might want to change this password right away):
1
ssh alarm@<ip address>
Once we’re logged in since “sudo” is not yet configured, we switch to root:
What follows are my own instructions I usually run when installing Arch.
In particular, I configure time, network time synchronization, and timezone (Italy, in my case):
1
2
3
4
5
timedatectl set-timezone Europe/Rome
timedatectl set-ntp on
hwclock --systohc
The next step is required for doing gnome-terminal work (and it’s also part of the Arch standard installation instructions):
Edit “/etc/locale.gen” and uncomment “en_US.UTF-8 UTF-8” and other needed locales.
Generate the locales by running:
1
locale-gen
Edit the “/etc/locale.conf” file, and set the LANG variable accordingly, for example, for the UTF-8 local above:
1
LANG=en_US.UTF-8
We could run a first system upgrade
1
pacman-Syu
I don’t know if that’s strictly required because we’ll add the additional repository for the PineBook Pro in a minute. However, just in case, it might be better to update the system.
Let’s reboot and verify that everything still works.
NOTE: By the way, I noted that if I want to boot from the USB stick, it’s better to use the right-hand side USB port (which is USB 2) instead of the left-hand side port (USB 3). Otherwise, the system seems to ignore the system on the USB stick and boots directly to the installed Manjaro system.
Second part
As mentioned above, I suggest connecting to the PineBook Pro via SSH. In any case, what follows must be executed as “root” (“su -“).
Let’s now add the repositories with kernels and drivers specific to PineBook Pro.
Note that this project also provides a way to install Arch Linux directly with these repositories, with a procedure similar to the one in the first part. I prefer to install official Arch Linux first and then add the additional repositories, though.
And let’s install the packages specific to the PineBook Pro (note that we’re going to install the Linux kernel patched by Manjaro for the PineBook Pro):
It’s about 680 Mb of packages to install, so please be patient.
Now, I enable the primary services (the login manager, the NetworkManager to select a network from KDE, and the profile daemon for switching between power profiles, e.g., “Balanced” and “Powersave”):
The graphical SDDM login manager should greet us this time, allowing us to get into KDE and select a WiFi connection.
NOTE: I always hear a strange noise when the login manager or the KDE DE starts. It also happens with the pre-installed Manjaro. It must be the sound card that gets activated…
IMPORTANT NOTE: Upon rebooting, the WiFi does not always work (it looks like the WiFi card is not seen at all); that also happens with Manjaro. The only solution is to shut down the computer (i.e., NOT simply rebooting it) and boot it again.
Here’s the KDE About dialog:
And of course, once installed, let’s run “neofetch”:
That’s all for now!
In a future blog post, I’ll describe my customizations to KDE (installed programs and configurations).
In a previous post, I reported the procedure for installing Arch Linux. The procedure is basically the one shown in the official Arch Wiki.
After a few manual steps, this post will show my installation script for automatically installing Arch Linux. I took inspiration from https://github.com/ChrisTitusTech/ArchTitus, but, differently from that project, my script is NOT meant to be reusable. The script is heavily tailored to my needs. I describe it in this post in case it might inspire others to follow a similar approach 🙂
I’ll describe the script by demonstrating its use for installing Arch Linux on a virtual machine (VirtualBox). However, I use the script for my computers. Also, for real computers, I perform the installation via SSH from another computer for the same reasons I have already explained.
The virtual machine preparation is the same as in my previous post, so I’ll start from the already configured machine.
I start the virtual machine with the Arch ISO mounted:
Inside the live environment, the SSH server is already up and running. However, since we’ll connect with the root account (the only one present), we must give the root account a password. By default, it’s empty, and SSH will not allow you to log in with a blank password. Choose a password. This password is temporary; if you’re in a trusted local network, you can choose an easy one.
Then, I connect to the virtual machine via SSH.
1
ssh-p2522root@127.0.0.1
From now on, I’ll insert all the commands from a local terminal connected to the virtual machine.
Initial manual steps
First, I ensure the system clock is accurate by enabling network synchronization NTP:
1
timedatectl set-ntp true
Then, I partition the disk according to my needs. My script heavily relies on this partitioning scheme consisting of four partitions:
the one for booting in UEFI mode, formatted as FAT32, 300Mb (it should be enough for UEFI, but if unsure, go on with 512Mb)
a swap partition, 20Gb (I have 16Gb, and if I want to enable hibernation, i.e., suspend to disk, that should be enough)
a partition meant to host common data that I want to share among several Linux installations on the same machine (maybe I’ll blog about that in the future), formatted as EXT4, 30Gb
the root partition, formatted as BTRFS, the rest of the disk
To do that, I’m using cfdisk, a textual partition manager, which I find easy to use. In the virtual machine, the disk is “/dev/sda”:
The partitions must be manually formatted:
1
2
3
4
mkfs.fat-F32/dev/sda1
mkswap/dev/sda2
mkfs.ext4/dev/sda3
mkfs.btrfs/dev/sda4
Sometimes, I have problems with the keyring, so I first run the following commands that ensure the keyring is up-to-date:
1
2
3
4
killall gpg-agent
rm-rf/etc/pacman.d/gnupg
pacman-key--init
pacman-key--populate archlinux
I’m going to clone the installation script from GitHub, so I need to install “git”:
1
pacman-Sy git
And now, I’m ready to use the installation script.
Running the installation script
First, I clone the installation script from GitHub:
The script has no parameter but relies on a few crucial environment variables to set appropriately. The first four variables refer to the partitions I created above. The last one is the name for the machine (in this example, it will be “arch-gnome”):
1
2
3
4
5
export EFI_PARTITION=/dev/sda1
export SWAP_PARTITION=/dev/sda2
export DATA_PARTITION=/dev/sda3
export ROOT_PARTITION=/dev/sda4
export INST_HOSTNAME=arch-gnome
The script will check that all these variables are set. However, it does not check whether the specified partitions are correct, so I always double-check the environment variables.
And now, let’s run it:
1
./install.sh
The script will do all the Arch Linux installation steps. These automatic steps correspond to the ones I showed in my previous post, where I ran them manually.
When the script finishes (it takes a few minutes), I have to perform a few additional manual operations before rebooting. I’ll detail these latter manual operations at the end of the post. In the next section, I’ll describe the script’s parts.
The installation script(s)
As I anticipated, the script actually consists of several scripts.
Note that the installation logs are saved in the “bettini” user’s home directory (the last run script will create the user). These can be inspected later.
The main script calls the other scripts.
We have the script for checking that all the needed environment variables are set (00_check.sh):
Shell
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
#!/usr/bin/env bash
set-x#echo on
echo-ne"
Check variables
"
if[[-z${EFI_PARTITION+y}]];then
echo"EFI_PARTITION is not defined"
exit1
else
echo"EFI_PARTITION ${EFI_PARTITION}"
fi
if[[-z${SWAP_PARTITION+y}]];then
echo"SWAP_PARTITION is not defined"
exit1
else
echo"SWAP_PARTITION ${SWAP_PARTITION}"
fi
if[[-z${ROOT_PARTITION+y}]];then
echo"ROOT_PARTITION is not defined"
exit1
else
echo"ROOT_PARTITION ${ROOT_PARTITION}"
fi
if[[-z${DATA_PARTITION+y}]];then
echo"DATA_PARTITION is not defined"
exit1
else
echo"DATA_PARTITION ${DATA_PARTITION}"
fi
if[[-z${INST_HOSTNAME+y}]];then
echo"INST_HOSTNAME is not defined"
exit1
else
echo"INST_HOSTNAME ${INST_HOSTNAME}"
fi
The script 01_mount-partitions.sh mounts the partitions and, for the main BTRFS partition, also creates the BTRFS subvolumes:
Note the use of the environment variable INST_HOSTNAME for creating the file /etc/hosts. I’m using en_US.UTF-8 for the language, but other local configurations are for Italy.
The script 05_bootloader.sh configures and installs GRUB. It also configures GRUB for the “mem_sleep_default” parameter (for suspend) and for hibernation; in that respect, it also configures mkinitcpio accordingly (note the “resume” hook):
Shell
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#!/usr/bin/env bash
set-x#echo on
set-eo pipefail
# This is meant to be executed with arch-chroot /mnt
As you see, most of the steps are performed by the script! 🙂
I can restart the system (in this example, the virtual machine) and enjoy the installed Arch!
That’s another reason why I love Arch Linux so much: the installation can be easily scripted!
It took me some time to finalize all the scripts, but using a virtual machine, especially with snapshots, wasn’t that hard. I encourage you to bake your installation script. It’ll be fun 🙂
By the way, before existing chroot and rebooting, I usually run my Ansible playbook for installing other programs (either KDE or GNOME) and configure the system and user according to my needs. I’ll blog about such a playbook in the future.
Then, I stopped using KVM because I’ve always found VirtualBox easier for my experiments. In particular, with VirtualBox, it is trivial to store virtual machines on an external drive (I mean, a fast external SD, of course): you specify to use a directory on the external drive, and all information about the virtual machine will be stored there. Then, you attach the drive to another computer with VirtualBox and open the virtual machine from the external drive. Easy!
Things are more complicated with KVM, QEMU, and Virtual Machine Manager. Even making QEMU access an external drive requires additional configuration steps.
In this blog post, I’ll summarize the steps to achieve that.
I’ll first show the manual export/import procedure for the machines’ metadata information. Then, I’ll show a different approach based on symlinks.
It was time to try KVM again because it’s faster than VirtualBox.
I’ll describe the installation steps for EndeavourOS and pure Arch Linux. I guess the steps for other distributions are similar.
Installation and configuration
Let’s install a few packages for KVM, QEMU, and the Virtual Machine Manager:
Now comes the crucial part for letting QEMU handle machines on external drives: we need to add our user to “/etc/libvirt/qemu.conf”. This can be done by setting the appropriate entries in the file or by simply appending the entries at the end of the file:
1
2
3
4
sudo tee-a/etc/libvirt/qemu.conf>/dev/null<<EOT
user="$(whoami)"
group="$(whoami)"
EOT
If you want to start the virtualization service and the default virtual network automatically at boot, you run:
1
2
sudo systemctl enable libvirtd.service
sudo virsh net-autostart default
Since I’m not using virtual machines daily, I prefer to start them when needed, so I don’t run the above commands. Of course, I must remember to run these commands (note, for the network is “start” instead of “autostart”) before starting the “Virtual Machine Manager”:
1
2
sudo systemctl start libvirtd.service
sudo virsh net-start default
Remember you can always use:
1
systemctl status libvirtd.service
to see the service status and possible errors shown when running this command.
OK, time to reboot now.
Let’s create a virtual machine on an external drive
I created a directory “kvm/images” on my external USB SD to store the virtual machine images.
Let’s start the “Virtual Machine Manager” program. We should see “QEMU/KVM”:
Let’s create a new virtual machine with the leftmost toolbar button.
I specify a local ISO.
I don’t create a pool for ISOs and use “Browse Local” to select an ISO in my external drive.
In this example, I will install EndeavourOS on the virtual machine. I have to select the operating system manually (start typing, and you get completions):
Time to allocate resources for the virtual machine. I’m giving the VM half my RAM and half my cores:
Now here’s the essential part of disk selection. Remember, I want to use my external drive, so I select custom storage and press “Manage”:
In the following dialog, I use the “+” button in the bottom left corner to create a new pool:
I give the pool the name “images” and specify the directory I mentioned above on my external drive:
After pressing “Finish”, I select the created pool and add a “Volume” (with the other “+” button)
I give the disk image a proper name and enough size (recall that the image will NOT allocate all the size immediately, but only on-demand):
Select the volume and press “Choose Volume”:
On the final dialog, make sure the default network is selected and that you check “Customize configuration before install” (note that I also changed the name for the virtual machine):
Let’s press “Finish,” and get to the configuration dialog. I changed the Firmware from “BIOS” to “UEFI”, pressed “Apply,” and finally, we can start the installation with “Begin Installation”.
We should not get any error from QEMU because it cannot access the external drive, thanks to the configuration shown above in the qemu.conf file!
After the GRUB menu, we should see the installer log:
And then, the EndeavourOS installer dialog:
Since I’ve already blogged about EndeavourOS installation, I’ll skip the detailed steps. I’ll install the GNOME desktop environment and let the installer use the whole disk space with the BTRFS filesystem and SWAP with hibernate (later, I might want to check whether hibernate works in the VM).
In a few minutes, the installation finishes! We get to the GRUB menu of the installed system:
And to the installed GNOME desktop:
The disk image is correctly created in the external drive:
Let’s export the virtual machine to use it from another computer. I understand that having the same software on the other host is crucial. Since I’m using EndeavourOS or Arch on my main computers, that is not a problem.
But isn’t the virtual machine already in an external drive? Why do I have to export it?
That’s the main difference with VirtualBox I mentioned at the beginning. The disk image is on an external drive, but the virtual machine information (configuration and metadata) is on a local XML file (see the listing of “/etc/libvirt/qemu” above; the XML file of the virtual machine is “eos-kvm-gnome.xml”, after the name I gave to the virtual machine when I created it).
Remember that the XML has an absolute path pointing to the disk image on the external drive:
Domain snapshot snapshot1 created from'/run/media/bettini/VMs/kvm/eos-kvm-gnome-snapshot1.xml'
❯ll/var/lib/libvirt/qemu/snapshot/eos-kvm-gnome
.rw-------8,4kroot4mar20:27謹snapshot1.xml
However, keep in mind that if you try to start a snapshot, you get this warning:
So if you don’t want to lose the current state, create another snapshot for the current state before restoring a previous one. Moreover, if the snapshot’s state is “Shutoff”, “starting” the snapshot only restores it. Then, you must start the virtual machine.
A different approach: symlinks
In the previous sections, I showed how machine information (including snapshots) and images could be put on external drives. Besides the machine images residing on external drives from the beginning, the machine metadata is still on your hard disk. In fact, you must first export them (e.g., on the external drive) and then import them on another computer.
A more radical approach consists of keeping the metadata on the external drive only and creating symlinks in each computer’s libvirt/qemu directories.
On the first computer, the XML files of machine information and snapshots have to be copied onto the external drive. IMPORTANT: don’t dump information as we did above; you need to copy the original XML files themselves. Dumping does not generate the exact XML files stored on the libvirt/qemu directories. In fact, as shown above, the dumped XML files must be imported with dedicated commands.
Now, I can start the “libvirtd.service” and the default network, and I make sure I can still access all my machines stored on the external drive, including all the machine information.
Of course, if you have never created virtual machines and want to start creating them on the external drive, it is enough to run the above commands. Then, start creating machines. Remember to select the external drive for the image location.
Then, on the other computers where I have already installed the same software for KVM, QEMU, etc., I first ensure the “libvirtd.service” is not running (in case stop it). Then, I connect my external drive and run the above commands (these will remove possible existing machines’ information, so be careful).
Of course, the above commands must be run only the first time.
Now, I can start the “libvirtd.service” and the default network, and I can access all my machines stored on the external drive, including all the machine information. Every modification (an image content or a machine configuration) will be stored on the external drive.
This approach works if you want to store ALL your machines on the external drive. You won’t have to keep the information in sync because they are stored in a single place.
If you need to keep some machines on your computers and others on different external drives, you must use the above-shown manual procedure for exporting and importing. It is then up to you to remember to re-export/re-import if you change a machine’s configuration or a snapshot.
In a previous blog post, I showed how to install Arch Linux on a PineBook Pro.
In this blog post, I’m showing how I customize Gnome on that installation.
First, Gnome 43 has “Gnome Console” as the default terminal application. I wouldn’t say I like it since it’s too basic. So I install the traditional “Gnome Terminal”:
1
sudo pacman-Sgnome-terminal
Then, I set “Ctrl+Alt+T” as a shortcut for opening the terminal:
Then, I install an AUR helper. I like “yay,” so I first installed the needed dependencies:
1
sudo pacman-S--needed git base-devel
And then
1
2
3
4
5
mkdir -p ~/git
cd ~/git
git clone https://aur.archlinux.org/yay.git
cd yay
makepkg -si
I install, by using “yay”, the helper the “Gnome Browser Connector” to install Gnome extensions from Firefox (Some extensions are already installed by default as system extensions. You can use the “Extensions” application to enable/disable extensions):
1
yay-Sgnome-browser-connector
Now I can navigate to https://extensions.gnome.org and install and enable a few extensions (you also need to install the Firefox extension add-on when asked). For example, “AppIndicator and KStatusNotifierItem Support” and “X11 Gestures”.
The last extension helps enable Touchpad gestures in the X11 session (Gnome Wayland already provides touchpad gestures, but I prefer to use the X11 session). This extension relies on “touchegg” that must be installed. For ARM, we need to install the AUR package:
1
yay-Stouchegg
You will get this warning, but proceed anyway: it compiles and works fine:
1
2
3
4
:: (1/1) Parsing SRCINFO: touchegg
-> The following packages are not compatible with your architecture:
touchegg
:: Try to build them anyway? [Y/n]
Let’s start “touchegg” and verify that gestures work
1
sudo systemctl start touchegg
And then let’s enable it so that it automatically starts on the subsequent boots:
1
sudo systemctl enable touchegg.service
Let’s move on to ZSH, which I prefer as a shell:
1
sudo pacman-Szsh
Since I’m going to install “Oh My Zsh” and other Zsh plugins, I install these fonts (remember from the previous post that I had already installed “noto-fonts” and “noto-fonts-emoji”) and finder tool (“curl” is required for the installation of “Oh My Zsh”):
1
sudo pacman-Spowerline-fonts fzf curl
Let’s install “Oh My Zsh” by running the following command as documented on its website:
Once saved, you have to start a new terminal with zsh to see the plugins in action (remember that, until you log out and log in, the default shell is still BASH, so you might have to run “zsh” manually to switch to ZSH in the currently logged session).
Besides the syntax highlighting for commands, you have completion after “cd” (press TAB), excellent command history (with Ctrl+R), suggestions, etc.
Let’s switch to the “Starship” prompt. Let’s run the documented installation program:
1
curl-sS https://starship.rs/install.sh | sh
Now, let’s edit the ~/.zshrc file again; we comment out the line starting with “ZSH_THEME,” and we add to the end of the file:
1
eval"$(starship init zsh)"
Opening another ZSH shell, we should see the fantastic Starship prompt in action, e.g.,
To quickly search for file names from the command line, I install “locate”, enable its periodic indexing and run the indexing once the first time (if you’re on a BTRFS file system, you might want to have a look at this older post of mine):
1
2
3
4
5
sudo pacman-Splocate
sudo systemctl enable plocate-updatedb.timer
sudo updatedb
Then, you should be able to look for files with the command “locate” quickly.
Gnome uses “Tracker” (in the current version, the command is “tracker3”) for file indexing and searching, e.g., from the “Activities” view. I like it, and it quickly keeps the index up to date. However, the “tracker extract” service also indexes the file contents, and that uses too many resources, so I disable that service:
1
systemctl--user mask tracker-extract-3.service
I also use the “guake” drop-down terminal a lot:
1
sudo pacman-Sguake
I run it once (it’s enabled by default by pressing “F12”), and I configure it to start automatically when Gnome starts (by running “Guake Preferences” -> “Start Guake at login”).
In this blog post, I’ll briefly discuss installing EndeavourOS on the same computer. I wrote it some months ago, so it’s not based on the new EndeavourOS Cassini version. It’s based on Endeavour OS Nova. However, the procedure and the results should be the same also with the current version of the EndeavourOS installer.
First of all, the installer detected my Ethernet card and nicely proposed using a working driver:
I choose the default.
Then, after the WiFi connection has been established, it’s time to start the installation:
First, I updated the mirrors, choosing my country (actually, it had already been detected by the installer):
I started the installer and chose the “Online” method to install KDE, not Xfce (the default DE).
I choose America English (though I’m Italian, I always prefer to have my OS in English). The location has been automatically detected again, and I’ll stick with the proposed settings:
I choose “Manual partitioning” because I want to keep Windows and my current two other Linux installations.
I mount the EFI partition to “/boot/efi” (the “boot” flag is automatically selected):
I create a new partition for the root partition on the free space, choosing BTRFS:
I also mount the existing EXT4 partition to share some common work data (including Docker images, containers, and Java-related stuff). The final layout is as follows:
When I continue, I get a warning because of the EFI partition, which is expected to be at least 300Mb; mine is smaller, but I’m sure there’s enough space, so I continue:
For the desktop, I select “Plasma KDE”.
Now we get to the package selection. Some packages are already selected by default:
I deselect from “Desktop Base” => “GPU drivers” the “xf86-video-intel” since it’s known to give a few problems (including the screenshot tool Spectacle capturing old screen contents), and I’ll rely on the default mesa. I also select the LTS kernel in additions since I prefer an LTS besides the latest kernel (in case of problems, the LTS kernel usually works best).
Moreover, I also select everything concerning printing:
After the user details, it’s time to review the partitioning, which looks reasonable.
Let’s start the installation! Remember to “Toggle log” to see what the installer is doing under the hood.
In a matter of minutes, the installation finished successfully.
Before rebooting, you might want to save the “endeavour-install.log” file generated by the installer in the home folder of the “liveuser”.
And here’s the installed system:
I set the fonts to 120 (that is, 25% bigger) so that I could read better.
The sound does not work fine. I tried to play a video on YouTube, and it worked, but now and then, I get no sound at all (even if I increase/decrease the volume, I get no sound from the DE). I guess that’s due to the “wireplumber” installed by default. On Arch News, they suggest using “pipewire-media-session” instead of “wireplumber”. So I do as suggested:
1
sudo pacman -Syu pipewire-media-session
And reboot (you have to accept the removal of “wireplumber”).
Up to now, I’ve been using Timeshift and grub-btrfs in my Linux installations because I found Timeshift easy to use and straightforward to install. I was scared by Snapper because I thought it was hard to use and complex to install. I had been fooled by many tutorials I found online, but maybe they were obsolete, or they were not using the right packages. I was wrong: using the right packages provided in Arch and AUR repositories makes it straightforward to use Snapper and grub-btrfs. You also get a program that automatically takes a snapshot when installing/updating your system.
This is more of a report than a tutorial.
I tried this procedure on EndeavourOS and Arch, and, as expected, the final result was the same. However, as shown later, Arch requires a few adjustments in the /etc/fstab file.
The BTRFS subvolume layout of EndeavourOS is ideal for snapper snapshots and for booting them with grub: the subvolume “@” for “/”, “@home” for “/home”, and separate subvolumes for “/var/log” and “/var/cache”. That’s basically the same as I use for Arch installations.
If you already have grub-btrfs because you use it with Timeshift (e.g., with the procedure described in one of my previous posts), it’s better to remove the package so that it will also remove possible enabled services and the custom configurations for timeshift:
1
sudo pacman-Rgrub-btrfs
If you were using Timeshift, you also provided a custom configuration for grub-btrfsd, which is not automatically removed during the previous command. The files must be removed explicitly:
If you don’t do that now, you will be asked to do that when installing the packages for snapper anyway.
If you installed Arch the Arch way, when generating /etc/fstab, the command has added “subvolid=…” entries in /etc/fstab, which will disturb when restoring snapper snapshots. For example, if you tried to restore a snapshot with btrfs-assistant (which we’ll install in a minute), you’ll get such a warning dialog:
Since the generated /etc/fstab contains both “subvolid=…” and “subvol=…”, I find it safe to remove the “subvolid” parts. I do that with this sed command (of course, do that at your own risk and take a backup of the file first):
1
sudo sed-i's/subvolid=.*,//'/etc/fstab
If you installed another Arch-based distro, like EndeavourOS, the /etc/fstab should already contain only “subvol=…” entries, so the above command is not required.
Install the following two packages with an AUR helper (e.g., “yay” in my case). The first one is a meta-package that will install snapper and other utilities like “snap-pac” (“Pacman hooks that use snapper to create pre/post btrfs snapshots when installing/upgrading/removing packages”) and “grub-btrfs” (the default configuration of grub-btrfs works already with snapshots created by snapper).
1
yay-Ssnapper-support btrfs-assistant
If you haven’t previously uninstalled timeshift and timeshift-autosnap, you’ll get this message, as I mentioned above:
1
2
3
4
5
:: Checking for conflicts...
:: Checking for inner conflicts...
-> Package conflicts found:
-> Installing snapper-support will remove: timeshift, timeshift-autosnap
-> Conflicting packages will have to be confirmed manually
During the installation of the above two packages, we can see a few interesting things in the log:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
...
:: Running pre-transaction hooks...
(1/1) Performing snapper pre snapshots for the following configurations...
(4/4) Performing snapper post snapshots for the following configurations...
==> root: 3
The installation creates a configuration for snapper for the root subvolume and configures the service to automatically update the grub menu for booting snapshots. It also creates the very first snapshot, “1”. Since we then install btrfs-assistant, it also creates a “pre” snapshot, “2”, and when the installation of btrfs-assistant finishes, it creates a “post” snapshot, “3”.
Let’s run btrfs-assistant:
Let’s explore its tabs:
Note the existing subvolumes and the newly created subvolume for snapshots “.snapshots”.
The next tab shows the snapshots taken during the installation command we issued to install the programs. Note the numbers of the snapshots and compare them with the installation log shown above. Moreounlike from “timeshift-autosnap”, “snap-pac” creates meaningful and comprehensible names for the snapshots.
Note that you act on a single configuration in this and in the next tab. By default, we have the one created during the installation for the root subvolume (see the “Select config” drop-down menu). If you have other configurations (e.g., for snapshots of the home subvolume), you must select the intended configuration.
With the first tab, you can create/delete snapshots. With the second tab, you can browse them or restore them:
On the last tab, you can see the enabled services and possibly perform further configuration. For the moment, I’m not touching that part: I’m OK without automatic snapshots (since I know they will be taken when installing/upgrading/removing commands) and with the automatic cleanup of old snapshots:
From btrfs-assistant, you can also select the checkbox to show existing Timeshift snapshots:
You might want to remove them once you’re sure the new setup with snapper and grub-btrfs works correctly.
Let’s do some experiments browsing the current snapshots. For example, let’s select the second one and click “Browse”:
Navigating to “/usr/bin,” we can verify that “btrfs-assistant” is not there. In fact, snapshot “2” was taken before installing btrfs-assistant.
Let’s browse snapshot “3”:
This time, “btrfs-assistant” is present in “/usr/bin”. In fact, snapshot “3” was taken after the installation of btrfs-assistant (it’s a “post” snapshot).
From the screenshots above, we can see that snapshots are also browsable from the file system: they are all inside “/.snapshots” (for the root subvolume configuration), each one with the corresponding number. You must be root to browse them.
Let’s experiment with booting snapshots from grub.
Before installing snapper and the other programs, I had previously installed “neofetch” on this machine. I’m going to remove it:
1
sudo pacman-Rneofetch
Two new snapshots have been automatically created by “snap-pac” (one before the removal and one afterward):
Let’s reboot the machine and navigate through the snapshots menus, selecting the snapshot corresponding to the state before the removal of neofetch:
Now, we’re inside that snapshot, and we can verify that neofetch is still there:
Let’s say that we want to restore this snapshot for good. Let’s run btrfs-assistant, select the snapshot we have just booted, and press “Restore”:
We get a confirmation dialog, and we can specify a name for the backup that will be taken (in this example, I’ll specify “before-restoring”):
Upon confirmation, we get a warning that urges us to reboot as soon as possible:
Let’s reboot. This time we select the default grub menu entry (not a snapshot).
We can verify once again that neofetch is still there.
From btrfs-assistant, we can see the subvolume with the backup, which we can delete once we’re sure that everything is still working:
If you are using “plocate” or “locate” (see also my older post about locate and BTRFS), you should also exclude “.snapshots” to “PRUNENAMES” (this should already contain some directories like “.git .hg .svn”:
1
sudo nano/etc/updatedb.conf
And add “.snapshots” to “PRUNENAMES”, e.g.,
1
PRUNENAMES=".git .hg .svn .snapshots"
Configuration files are in the directory “/etc/snapper/configs/“. Currently, we have only one configuration, “root” (for the root subvolume), created during the installation.
In that file, we can see the line
1
TIMELINE_CREATE="no"
corresponding to the setting in btrfs-assistant that disables the automatic timeline snapshots.
Moreover, we have the following:
1
2
3
4
5
6
7
# run daily number cleanup
NUMBER_CLEANUP="yes"
# limit for number cleanup
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="10"
NUMBER_LIMIT_IMPORTANT="5"
which again corresponds to the setting in the btrfs-assistant screenshot shown above.
Dropbox does not provide a client for the Linux Arm architecture, so you don’t have a client for a PineBook Pro.
However, you can use the open-source project Maestral:
Maestral is a lightweight Dropbox client for macOS and Linux. It provides powerful command line tools, supports gitignore patterns to exclude local files from syncing and allows syncing multiple Dropbox accounts.
The Arch AUR repository provides packages for Maestral:
1
2
3
4
5
6
7
8
9
$yay-Ss maestral
aur/maestral-qt-git1.6.3.r0.gf4f8030-1(+00.00)
Qt interfaceforthe Maestral daemon
aur/maestral-git1.6.3.r0.g7b8e7a53-1(+10.00)
Open-source Dropbox client
aur/maestral-qt1.6.3-2(+70.27)
Qt interfaceforMaestral
aur/maestral1.6.5-1(+100.32)
Open-source Dropbox client
So I’m going to install the “Qt interface for Maestral” with the “yay” AUR helper I already have on my PineBook Pro (this will install a lot of python packages, and the installation will take a few minutes):
1
yay-Smaestral-qt
Let’s run maestral GUI from the command line:
1
maestral gui
And the app appears:
Let’s click on the button to link to the Dropbox Account
And click on “here” to retrieve the authorization token. This should open the browser; alternatively, you’ll be asked to select an application to open the full URL in KDE. Unfortunately, selecting a browser in KDE does not work, and it will keep asking for the application.
Thus, in KDE, I run from the command line:
1
2
3
4
5
6
$maestral auth link
-Linking newaccount for'maestral'config
-Retrieving auth code from Dropbox
?How would you like toyou link your account?
Open Dropbox website
>Print auth URL toconsole
This allows me to print the auth URL to the console, copy it and paste it into a browser. I can then authorize Maestral on the Dropbox site.
The Dropbox website then shows me a token that I copy and paste on the Maestral window above and press “Link”. (The other run command, “maestral auth link,” can be interrupted.)
The setup proposes to select a local folder to synchronize with Dropbox. Note that by default, it proposes “Dropbox (Maestral)”, but I prefer the standard one “Dropbox”, so I modify it accordingly.
And now, it’s time to select the folders to synchronize. I start with a very minimal subset of my Dropbox folders. As noted below, the initial synchronization will take a lot of time (depending on the size of all the Dropbox contents, not the folders you select).
In the taskbar, you can see the Maestral icon that started to synchronize. The icon provides a context menu.
From the command line, you can see the status of the synchronization with
This is a brief article for installing exa in Arch Linux with an additional package for the icon fonts (in a few installations, boxes were shown instead of icons, that’s why I’m writing this blog article, hoping to save you some time).
Installing exa in Arch is just a matter of running:
1
sudo pacman-Sexa
However, you need a “Nerd” font to get the icon symbols. This is the one I install:
1
sudo pacman-Sttf-arimo-nerd
In EndeavourOS KDE, this should already be installed. I seem to understand that this is not the case for EndeavourOS GNOME. If these fonts are not installed, you can install them with the command above and make sure to reboot.
The output is excellent, and I aliased many of my previous ls commands to exa:
1
2
3
aliasl='exa -lah --color=always --group-directories-first --icons'# with headers
alias la='exa -al --color=always --group-directories-first --icons'# all files and dirs
alias ll='exa -l --color=always --group-directories-first --icons'# long format
This is the beautiful colored output you get, and note the icons for directories and known files types in Gnome (in particular, a “cup of coffee” for Java files):
The same holds for KDE:
I also have another alias for the tree output of exa:
1
alias lt='exa -aT --color=always --group-directories-first --icons'# tree listing
And this is the output:
Note the “–git-ignore” command line argument to ask exa to skip all the files that match the patterns in the current “.gitignore” file.
In Arch Linux (and Arch-based distros like EndeavourOS), it’s easy to add a network printer if you already know its address. Still, network printer discovery does not work out of the box as it happens on other distributions like Fedora or Ubuntu.
The procedure to enable network printer discovery is, of course, documented in the Arch wiki. Still, in this post, I’d like to detail the steps to achieve that just as a confirmation or as additional help documentation.
First of all, let’s install the packages for printing:
1
cups cups-filters cups-pdf cups-pk-helper libcups
I also install the following packages for drivers and HP (because I have HP printers):
We must also make sure the following packages (“avahi” and “nss-mdns”) are installed:
1
2
3
4
extra/avahi ... (Installed)
Service Discovery for Linux using mDNS/DNS-SD -- compatible with Bonjour
extra/nss-mdns ... (Installed)
glibc plugin providing host name resolution via mDNS
And that the “avahi-daemon.service” is running and enabled:
1
2
sudo systemctl start avahi-daemon
sudo systemctl enable avahi-daemon
Then, we must edit the file “/etc/nsswitch.conf” and change the line
1
hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns
into
1
hosts: mymachines mdns_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] files myhostname dns
Now, we should be able to discover local network printers.
I prefer the “system-config-printer” package for this purpose (in case you want to install it).
You can run it by searching for the application “Print Settings”. I’m showing an example in KDE:
“Unlock” by providing the password, press “Add,” and expand the “Network Printer”. If you have a firewall, like “firewalld”, you’ll be asked again for the password to change the firewall settings to enable the services for printer discovery:
Of course, you have to accept to adjust the firewall.
Then, the local network printer(s) should be discovered. In my example, my HP printer is discovered with the possible network protocols:
I chose the second one (the one with the local IP address) and HPLIP as the connection protocol (remember I had already installed the corresponding packages):
By pressing “Forward”, you wait for the drivers to be selected. You can print a “test page” and configure the printer as you see fit.
19 February 2023: exclude ansible-lint problems on tests/test.yml
27 April 2023: updated the molecule docker plugin
Last year, I got familiar with Ansible, the automation platform I now use to install and configure my Linux installations. I must thank Jeff Geerling and his excellent book “Ansible for DevOps“, which I highly recommend!
In this blog post, I’d like to document the use of Ansible and Molecule with a slightly more advanced example and how to test an Ansible role against 3 main Linux distributions, Fedora, Ubuntu, and Arch. To test the Ansible role, we will use Molecule and Docker. Finally, I’ll show how to implement a continuous integration process with GitHub Actions. The example consists of a role for installing zsh, setting it as the user’s default shell, and creating an initial “.zshrc” file. It will be a long post because it will be step-by-step.
I’m assuming Docker, Python, and Pip are already installed.
First, let’s install Ansible and Molecule (with Docker support). We’ll use pip to install these tools. This method works on all distributions since it’s independent of the ansible and molecule packages provided by the distribution (Ubuntu does not even provide a package for molecule):
What I do next is to enter the directory, remove “.travis.yml” (since we want to build on GitHub Actions), and create a Git repository (with “git init”). I’m also pushing to GitHub.
First, let’s adjust the file meta/main.yml with the information about this role and author:
YAML
1
2
3
4
5
galaxy_info:
role_name: zsh_role
author: Lorenzo Bettini
namespace: lorenzobettini
description: Install ZSH and set it as the user's shell
The role’s name should be the same as the one specified in the “init” command (I don’t know why this file has not been generated with the role_name already set). Otherwise, the other generated files for Molecule will not work.
The role’s main tasks are defined in tasks/main.yml. Currently, the generated file does not execute any task.
Manual tests
The “init” command also created a tests directory to manually and locally test the role. We are interested in automatically testing the role. However, since the role is currently empty, it is safe to try to run it against our own machine. At least, we can check that the syntax of the role is OK, and we can perform a “dry-run” without modifying anything on our machine.
The current contents of the files generated in the “tests” directory will not work out of the box.
First, the tests/test.yml playbook:
YAML
1
2
3
4
5
---
- hosts: localhost
remote_user: root
roles:
-zsh_role
Correctly refers to our role, but ansible will not be able to find the role in the default search path (because the role is the project’s path).
We can change the role reference with a relative path (the use of a relative path will require a few configurations to make linting happy, as we will see later):
YAML
1
2
3
4
5
6
---
- name: Test the role locally
hosts: localhost
remote_user: root
roles:
-../..
Then, we can try to run it, checking the syntax and doing a “dry-run”:
fatal: [localhost]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host localhost port 22: Connection refused", "unreachable": true}
PLAY RECAP **************************************************************************
The “dry-run” (–check) fails because, on my machine, there’s no SSH server, and by default, the tests/inventory file (specifying “localhost”) would imply an SSH connection:
1
localhost
To avoid SSH, we can change the file as follows:
1
localhost ansible_connection=local
Let’s try again with the “–check” argument, and now it works.
Run the complete Molecule default scenario
The “init” command created a default Molecule scenario in the file default/molecule.yml:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
---
dependency:
name: galaxy
driver:
name: docker
platforms:
- name: instance
image: quay.io/centos/centos:stream8
pre_build_image: true
provisioner:
name: ansible
verifier:
name: ansible
As we can see from this file, the Docker image used by Molecule is centos:stream8. For the moment, we’ll stick with this image.
Molecule will execute a playbook against a Docker container of this Docker image. We’re implementing a role, not a playbook. The playbook is defined in the file default/converge.yml:
YAML
1
2
3
4
5
6
7
---
- name: Converge
hosts: all
tasks:
- name: "Include lorenzobettini.zsh_role"
include_role:
name: "lorenzobettini.zsh_role"
In fact, “converge” is the action of performing the playbook against the Docker image, the “instance”. As you see, the “init” command generated this file automatically based on the role that we created.
There’s also a default/verify.yml file that is used to verify that some expected conditions are true once we run the playbook against the Docker instance. We’ll get back to this file later to write our own assertions. The contents of this generated file are as follows (the assertion is always verified):
YAML
1
2
3
4
5
6
7
8
9
10
---
# This is an example playbook to execute Ansible tests.
- name: Verify
hosts: all
gather_facts: false
tasks:
- name: Example assertion
ansible.builtin.assert:
that: true
To check that the scenario already works, we can run it end-to-end with the command “molecule test” issued from the project’s root. Remember that Molecule will download the Docker image during the first run, which takes time, depending on your Internet connection. This is the simplified output:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
> molecule test
INFO default scenario test matrix: dependency, lint, cleanup, destroy, syntax, create, prepare, converge, idempotence, side_effect, verify, cleanup, destroy
INFO Performing prerun with role_name_check=0...
...
INFO Running default > dependency
WARNING Skipping, missing the requirements file.
WARNING Skipping, missing the requirements file.
INFO Running default > lint
INFO Lint is disabled.
INFO Running default > cleanup
WARNING Skipping, cleanup playbook not configured.
INFO Running default > destroy
INFO Sanity checks: 'docker'
PLAY [Destroy] *****************************************************************
Thus running the entire scenario always implies starting from scratch, that is, from a brand new Docker container (of course, the pulled image will be reused). Note that after “converge,” the scenario checks “idempotence,” which is a desired property of Ansible roles and playbooks. After verification, the Docker instance is also destroyed. Of course, if any of these actions fail, the lifecycle stops with failure.
Setup the CI on GitHub Actions
Our role doesn’t do anything yet, but we verified that we could run the complete Molecule scenario. Before going on, let’s set up the GitHub Actions CI workflow. We’ll use the Ubuntu runner, where Docker and Python are already installed. We’ll have first to install ansible and molecule with pip, and then we run the “molecule test”.
Concerning the pip installation step, I created the file pip/requirements.txt in the project with these contents (they correspond to the pip packages we installed on our machine):
Then, I create the file .github/workflows/molecule-ci.yml with these contents:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
---
name: Molecule CI
on:
push:
paths-ignore:
-'**.md'
pull_request:
paths-ignore:
-'**.md'
jobs:
test:
name: Molecule
runs-on: ubuntu-latest
steps:
- name: Check out the codebase.
uses: actions/checkout@v2
- name: Set up Python 3.
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install test dependencies.
run: pip install -r pip/requirements.txt
- name: Run Molecule tests.
run: molecule test
env:
PY_COLORS: '1'
ANSIBLE_FORCE_COLOR: '1'
Now that our CI is in place, GitHub Actions will run the complete Molecule test scenario at each pushed commit. The environment variables at the end of the file will allow for colors in the GitHub Actions build output:
Familiarize with Molecule commands
While implementing our role, we could run single Molecule commands instead of the whole scenario (which, in any case, will be executed by the CI).
With “molecule create,” we create the Docker instance. Unless we run “molecule destroy” (which is executed by the entire scenario at the beginning), the Docker container will stay on our machine. Once the instance is created, you can enter the container with “molecule login“. This is useful to inspect the state of the container after running the playbook (with “molecule converge“) or to run a few commands before writing the tasks for our role:
The “login” command is more straightforward than running a “docker” command to enter the container (you don’t need to know its name). Remember that unless you run “molecule destroy,” you’ll find the same state if you exit the container and back in.
Once you run “molecule converge“, you can run “molecule verify” to check that the assertions hold.
To get rid of the instance, just run “molecule destroy“.
Let’s start implementing our role’s tasks
To start experimenting with Molecule for testing Ansible roles, the official Fedora Docker image is probably the easiest. In fact, such an image comes with “python” already installed (and that’s required to run Ansible playbooks). Moreover, it also contains “sudo”, another command typically used in Ansible tasks (when using “become: yes”).
Thus, let’s change the image in the file default/molecule.yml:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
---
dependency:
name: galaxy
driver:
name: docker
platforms:
- name: instance
image: fedora:36
pre_build_image: true
provisioner:
name: ansible
verifier:
name: ansible
You can commit, push, and let GitHub Actions verify that everything is still OK.
Now it’s time to edit the primary role’s file, tasks/main.yml. Let’s add the task to install ZSH. In this example, I’m using “ansible.builtin.package module – Generic OS package manager” so that we are independent of the target OS. This is useful later because we want to test our role against different Linux distributions. This Ansible module is less powerful than the specific package manager modules, but for our goals, it is sufficient. Moreover, in the Linux distributions that we’ll test, the name of the package for ZSH is always the same, “zsh”.
YAML
1
2
3
4
5
6
7
---
# tasks file for zsh_role
- name: Install ZSH
become: yes
ansible.builtin.package:
name: zsh
state: present
If we had already created the instance, we first need to run “molecule destroy” to avoid errors due to the previous Docker container.
Let’s run “molecule converge“. If you don’t have the “fedora:36” Docker image already in your cache, this command will take some time the first time. Moreover, also the task of installing the “zsh” package might take some time since the package must be downloaded from the Internet, not to mention that dnf is not the fastest package manager on earth. In fact, the Ansible package module will use the distribution package manager, that is, dnf in Fedora. Here’s the output:
1
2
3
4
5
6
7
8
9
10
11
12
PLAY [Converge] ****************************************************************
Let’s enter the container with “molecule login“. Now, zsh should be installed in the container:
Of course, you could always run the entire “molecule test,” but that takes more time, and for the moment, we don’t have anything to verify yet. The idempotency of the Anslibe package module implies Idempotency.
Change the user’s shell and verify it
Now, we want to change the user’s shell to zsh, and we will verify it. Let’s follow a Test-Driven Development approach, which I’m a big fan of. We first write the verification tasks in verify.yml, make sure that “molecule verify” fails, and then implement the task in our role to make the test succeed.
First, how to get the user’s shell? In the Docker container, the $SHELL environment variable is not necessarily set, so we directly inspect the contents of the file “/etc/passwd” and some shell commands to get the user’s current shell. To write the shell commands, we can enter the container (molecule login), assuming we have already created the instance, and perform some experiments there. Remember that when we’re inside the container, we are “root”, so in our experiments, we’ll try to get the root’s shell.
So, we have our shell piped command to get the root’s shell:
1
grep-E"^root:"/etc/passwd|awk-F:'{ print $7 }'
In verify.yml, we want to get the shell of the user executing Ansible. In our molecule tests, it will be root, but the user will be different in the general use case. Thus, we use Ansible’s fact “ansible_user_id”:
Then, we’ll compare it against the desired value, NOT “/bin/bash”, but “/bin/zsh”. Note that, by default, the generated molecule/verify.yml has “gather_facts: false”. We need to remove or set that line to true so that Ansible populates the variable with the current user. Here are the contents (we must use the module “shell” and not “command” because we need the “|”):
As usual, we commit, push, and let the CI run the whole scenario.
The verification would not be really required since we should rely on the correctness of the Ansible user module). However, I thought this could be the moment to experiment with Molecule verification.
Note that if you enter the container, the “/etc/passwd” has been modified, but you’re still on bash. That’s because the change becomes effective when you log out and log in as a user. In a Docker container, that’s not possible, as far as I know. However, since log out and login are expected in a real system, as long as the shell is modified in “/etc/passwd”, we’re fine.
Add the file .zshrc
Since we want to set up zsh for the user, we should also add to the converged system a “.zshrc” with some reasonable defaults. For example, if you enter the container and run zsh, you’ll see that you have no command history. The history should be enabled in the file “.zshrc”.
Files are searched for in the directory “files” of the project, which the “init” command created for us. I had an existing small “.zshrc” with the enabled history, command completion, and a few aliases:
Shell
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
autoload-Uz compinit
compinit
HISTFILE=~/.histfile
HISTSIZE=1000
SAVEHIST=1000
setopt autocd beep extendedglob nomatch notify
bindkey-e
aliasgrep='grep --color=auto'
aliasl.='ls -d .* --color=auto'
aliasll='ls -l --color=auto'
aliasls='ls --color=auto'
aliasl='ls -lah'
aliasla='ls -lAh'
aliasmv='mv -i'
aliasrm='rm -i'
I put such a file in files/zshrc (I prefer not to have hidden source files, so I removed the “.”). In the role, I added this task, which copies the source file into the converged system in the current user’s home directory with the name “.zshrc”:
YAML
1
2
3
4
5
- name: Copy zshrc
ansible.builtin.copy:
src: zshrc
dest: ~/.zshrc
mode: 0644
Of course, the “copy module” is idempotent and performs the action only if the source and the target files differ.
Let’s converge, enter the Docker container and run “zsh”. Now, the command history works.
Linting
Let’s enable linting to the Molecule scenario. Remember that the scenario has an initial phase for linting.
First of all, we have to install the two additional pip packages, yamllint, and ansible-lint. In our system, we run the following:
1
pip install yamllint ansible-lint
Of course, for the CI, we have to update pip/requirements.txt accordingly, adding these two packages.
Then, we have to enable the “lint:” section in default/molecule.yml:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
---
dependency:
name: galaxy
driver:
name: docker
lint: |
set-e
yamllint.
ansible-lint
platforms:
- name: instance
image: fedora:36
pre_build_image: true
provisioner:
name: ansible
verifier:
name: ansible
Before going on, let’s exclude our “tests/test.yml” file from ansible-lint: as mentioned above, using a relative path will make ansible-lint complain. However, for that simple test file we don’t care. So, let’s create a file “.ansible-lint” in the root directory with these contents:
YAML
1
2
3
# to avoid "role-name[path]: Avoid using paths when importing roles. (../..)"
exclude_paths:
-tests/
Now we can run “molecule lint“:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
> molecule lint
...
INFO Running default > lint
WARNING Listing 6 violation(s) that are fatal
meta-incorrect: Should change default metadata: license
meta/main.yml:1
meta-no-info: Role info should contain platforms
meta/main.yml:1
schema[meta]: 2.1 is not of type 'string' (warning)
meta/main.yml:1 Returned errors will not include exact line numbers, but they will mention
the schema name being used as a tag, like ``schema[playbook]``,
``schema[tasks]``.
This rule is not skippable and stops further processing of the file.
Schema bugs should be reported towards [schemas](https://github.com/ansible/schemas) project instead of ansible-lint.
If incorrect schema was picked, you might want to either:
* move the file to standard location, so its file is detected correctly.
* use ``kinds:`` option in linter config to help it pick correct file type.
fqcn[action-core]: Use FQCN for builtin module actions (include_role).
molecule/default/converge.yml:5 Use `ansible.builtin.include_role` or `ansible.legacy.include_role` instead.
no-changed-when: Commands should not change things if nothing needs doing.
molecule/default/verify.yml:8 Task/Handler: Get current user's shell
risky-shell-pipe: Shells that use pipes should set the pipefail option.
molecule/default/verify.yml:8 Task/Handler: Get current user's shell
You can skip specific rules or tags by adding them to your configuration file:
# .config/ansible-lint.yml
warn_list: # or 'skip_list' to silence them completely
- experimental # all rules tagged as experimental
- fqcn[action-core] # Use FQCN for builtin actions.
- meta-incorrect # meta/main.yml default values should be changed.
- meta-no-info # meta/main.yml should contain relevant info.
- no-changed-when # Commands should not change things if nothing needs doing.
- risky-shell-pipe # Shells that use pipes should set the pipefail option.
Commit and push. Everything should work on GitHub Actions.
Note: when we created the project with “molecule init”, the command also created a “.yamllint” configuration file in the root:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
---
# Based on ansible-lint config
extends: default
rules:
braces:
max-spaces-inside: 1
level: error
brackets:
max-spaces-inside: 1
level: error
colons:
max-spaces-after: -1
level: error
commas:
max-spaces-after: -1
level: error
comments: disable
comments-indentation: disable
document-start: disable
empty-lines:
max: 3
level: error
hyphens:
level: error
indentation: disable
key-duplicates: enable
line-length: disable
new-line-at-end-of-file: disable
new-lines:
type: unix
trailing-spaces: disable
truthy: disable
This configuration file enables and disables linting rules. This is out of the scope of this post. However, to experiment a bit, if we remove the last line “truthy: disable” and run “molecule lint,” we get new linting violations:
1
2
3
./tasks/main.yml
4:11 warning truthy value should be one of [false, true] (truthy)
10:11 warning truthy value should be one of [false, true] (truthy)
Because “become: yes” should be changed to “become: true”. I guess it’s a matter of taste whether to enable such a linting rule or not. I’ve seen many examples of Ansible files with “become: yes”. After fixing “main.yml”, there is still a warning (not an error) on the YAML file of our GitHub Actions in correspondence with the “on” line:
1
2
./.github/workflows/molecule-ci.yml
3:1 warning truthy value should be one of [false, true] (truthy)
You can find a few issues on this being considered false positive or not. A simple workaround is to add a comment in the “molecule-ci.yml” file to make yamllint skip that:
YAML
1
on: # yamllint disable-line rule:truthy
Testing with Ubuntu, the “prepare” step
Let’s say that, besides “fedora:36“, we also want to test our role against the Docker image “ubuntu:jammy“. Instead of creating a new scenario (i.e., another directory inside the directory “molecule”), let’s parameterize the molecule.yml with an environment variable, e.g., MOLECULE_DISTRO, which defaults to “fedora:36”, but that can be passed on the command line with a different value. This is the interesting part:
YAML
1
2
3
4
5
6
...
platforms:
- name: instance
image: ${MOLECULE_DISTRO:-fedora:36}
pre_build_image: true
...
Nothing changes if we run molecule commands as we did before: we still use the Fedora Docker image. If we want to try with another image, like “ubuntu:jammy”, we prefix the molecule command with that value for our environment variable.
IMPORTANT: Before trying with another Docker image, make sure you run “molecule destroy” since now we want to use a different Docker image.
Let’s try to converge with the Ubuntu Docker image…
1
MOLECULE_DISTRO=ubuntu:jammy molecule converge
What could go wrong?
1
2
3
4
5
6
7
8
9
INFO Running default > converge
PLAY [Converge] ****************************************************************
fatal: [instance]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "failed": true, "module_stderr": "/bin/sh: 1: /usr/bin/python: not found\n", "module_stdout": "", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"}
PLAY RECAP *********************************************************************
As I anticipated, while the Fedora Docker image comes with python preinstalled, the Ubuntu Docker image does not. The latter does not even have “sudo” installed, which is required for running our tasks with “becomes: yes”. The converge failed, but the Ubuntu image has been created so you can enter the Docker container and verify that these packages are not pre-installed.
One could try to add the tasks in the role to install python and sudo (not by using “package” because such Ansible modules require python already installed). However, this would not make sense: our role is meant to be executed against an actual distribution, where these two packages are already installed as base ones. Jeff Geerling provides a few Docker images meant for Ansible, where python and sudo are already installed. However, instead of using his Ubuntu image, let’s explore another Molecule step: prepare.
The prepare playbook executes actions which bring the system to a given state prior to converge. It is executed after create, and only once for the duration of the instances life. This can be used to bring instances into a particular state, prior to testing.
So, let’s modify this part in molecule.yml (this modification is not strictly required because if the directory of molecule/default contains a file “prepare.yml,” it will be automatically executed; it might still be good to know how to specify such a file, in case it’s in a different directory or it has a different name):
YAML
1
2
3
4
5
6
...
provisioner:
name: ansible
playbooks:
prepare: prepare.yml
...
Now, in molecule/prepare.yml we create the preparation playbook. This is kind of challenging because we cannot rely on Ansible facts nor on most of its modules (remember: they require python, which we want to install in this playbook). We can rely on the “ansible.builtin.raw module – Executes a low-down and dirty command”. And looking at its documentation, we can see that it fits our needs:
This is useful and should only be done in a few cases. A common case is installing python on a system without python installed by default.
So, here’s the prepare.yml playbook:
YAML
1
2
3
4
5
6
7
8
9
10
11
---
- name: Prepare
hosts: all
gather_facts: false
tasks:
- name: Install python in Ubuntu
ansible.builtin.raw: >
aptupdate&&\
aptinstall-y--no-install-recommendspython3sudo
when: "'ubuntu' is in lookup('ansible.builtin.env','MOLECULE_DISTRO')"
changed_when: false
Of course, we must run this task only when we are using Ubuntu (see the condition). We also specify “changed_when: false” to avoid linting problems (“no-changed-when # Commands should not change things if nothing needs doing.”).
Running “molecule converge” now succeeds (note the “prepare” step):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
> MOLECULE_DISTRO=ubuntu:jammy molecule converge
...
INFO Running default > create
WARNING Skipping, instances already created.
INFO Running default > prepare
INFO Sanity checks: 'docker'
PLAY [Prepare] *****************************************************************
TASK [Install python in Ubuntu] ************************************************
changed: [instance]
PLAY RECAP *********************************************************************
But it doesn’t. That’s because the “pipefail” we added to make lint happy works in bash but not in sh, which is used by default in Ubuntu (in Fedora, it was bash). It’s just a matter of adjusting that verification’s task accordingly:
YAML
1
2
3
4
5
6
7
...
- name: Get current user's shell
...
register: user_shell
args:
executable: /bin/bash
changed_when: false
And now verification succeeds in Ubuntu as well.
If you run this against Fedora (remember that you must destroy the Ubuntu instance first), the task “Install python in Ubuntu” will be skipped.
Note that if you run
Let’s try to converge with the Ubuntu Docker image…
1
MOLECULE_DISTRO=ubuntu:jammy molecule converge
and then execute
1
molecule login
Molecule will reuse the image just created: it does not recreate the Docker image even if you haven’t specified any environment variable. This means that, as mentioned above, if you want to test with another value of the environment variable (including the default case), you first have to destroy the current image. By defining several scenarios, as we will see in a minute, there’s no such limitation.
Add a GitHub Actions build matrix
Let’s modify the GitHub Actions workflow to test our role with Fedora and Ubuntu in two jobs using a build matrix. These are the relevant parts to change to use the environment variable MOLECULE_DISTRO that we introduced in the previous section:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
...
test:
name: Molecule
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
playbookdistro:
- fedora:36
- ubuntu:jammy
...
- name: Run Molecule tests.
run: molecule test
env:
PY_COLORS: '1'
ANSIBLE_FORCE_COLOR: '1'
MOLECULE_DISTRO: ${{matrix.playbookdistro}}
Now GitHub Actions will execute two jobs for each pushed commit:
Using different scenarios
We now see a different technique to test with different Linux distributions. Instead of using an environment variable to parameterize Molecule, we create another Molecule scenario. To do that, it’s enough to create another subdirectory inside the “molecule” directory. We’ll use the “Ubuntu” example to see this technique in action. (Before doing that, remember to run “molecule destroy” first).
First, let’s undo the modification we did in the file “molecule.yml”:
YAML
1
2
3
4
platforms:
- name: instance
image: fedora:36
pre_build_image: true
And let’s create another subdirectory, say “ubuntu”, inside “molecule”, where we create this “molecule.yml” file (it’s basically the same as the one inside “default” where we specify “ubuntu:jammy” and a different name for the “image”):
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
---
dependency:
name: galaxy
driver:
name: docker
lint: |
set-e
yamllint.
ansible-lint
platforms:
- name: instance-ubuntu
image: ubuntu:jammy
pre_build_image: true
provisioner:
name: ansible
verifier:
name: ansible
Let’s copy the “default/verify.yml” and “default/converge.yml” into this new directory, and let’s move the “default/prepare.yml” into this new directory, where we change the contents as follows (that is, we get rid of the “when” condition since this will be used only in this new scenario):
YAML
1
2
3
4
5
6
7
8
9
10
---
- name: Prepare
hosts: all
gather_facts: false
tasks:
- name: Install python in Ubuntu
ansible.builtin.raw: >
aptupdate&&\
aptinstall-y--no-install-recommendspython3sudo
changed_when: false
To summarize, this should be the layout of the “molecule” directory (we’ll get rid of duplicated contents in a minute):
1
2
3
4
5
6
7
8
9
10
molecule
├── default
│ ├── converge.yml
│ ├── molecule.yml
│ └── verify.yml
└── ubuntu
├── converge.yml
├── molecule.yml
├── prepare.yml
└── verify.yml
Now, running any molecule command will use the “default” scenario. If we want to execute molecule commands against the “ubuntu” scenario, we must use the argument “-s ubuntu” (where “-s” is the short form of the command line argument “–scenario-name”).
For example
1
2
3
4
5
6
7
8
# converge using Fedora
molecule converge
# converge using Ubuntu
molecule converge-subuntu
# enter the Fedora instance created with the first command
molecule login
# enter the Ubuntu instance created with the second command
molecule login-subuntu
So we can converge, verify, and experiment with the two scenarios without destroying a previously created instance.
Of course, we adapt the GitHub Actions workflow accordingly to use scenarios instead of environment variables:
Let’s move the shared files “verify.yml” and “converge.yml” to a new subdirectory, say “shared”. So the layout should be as follows:
1
2
3
4
5
6
7
8
9
molecule
├──default
│└──molecule.yml
├──shared
│├──converge.yml
│└──verify.yml
└──ubuntu
├──molecule.yml
└──prepare.yml
The last part of both “molecule.yml” files in “default” and “ubuntu” must be changed to refer to files in another directory (note that the “verifier” part has been removed since it’s specified in the “provisioner” part):
YAML
1
2
3
4
5
6
...
provisioner:
name: ansible
playbooks:
converge: ../shared/converge.yml
verify: ../shared/verify.yml
Now we reused the common files between the two scenarios. Of course, we verify that everything still works in both scenarios.
Testing with Arch, a custom Dockerfile
Let’s now test this simple role also with Arch Linux. The idea is to create another scenario, e.g., another subdirectory, say “arch”. We could follow the same technique that we used for Ubuntu because also the Arch Docker image has to be “prepared” with “python” and “sudo”. However, to try something different, let’s rely on a custom Docker image specified with a Dockerfile.
The “molecule.yml” in the “arch” directory is as follows:
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
---
dependency:
name: galaxy
driver:
name: docker
lint: |
set-e
yamllint.
ansible-lint
platforms:
- name: instance-arch
image: arch-ansible
platform: linux/amd64
dockerfile: ./Dockerfile
build_image: true
provisioner:
name: ansible
playbooks:
converge: ../shared/converge.yml
verify: ../shared/verify.yml
NOTE: the specification “platform: linux/amd64” is not required because we use a custom Dockerfile. It is required if you want to test this scenario on a Mac m1 (by the way, see my other blog post about Docker on a Mac m1): while Ubuntu and Fedora also provide Docker images for the aarch64 (arm) architecture, Arch Linux does not. So we must force the use of the Intel platform on Arm architectures (of course, on Mac m1, the Docker container will be emulated).
And in the same directory, we create the Dockerfile for our Arch Docker image:
1
2
3
4
5
6
FROM archlinux:latest
LABEL maintainer="Lorenzo Bettini"
RUN pacman -Sy --noconfirm --needed python sudo
CMD ["/bin/sh"]
For this example, the Docker image for Arch is simple because we only need “python” and “sudo” to test our role.
Now the directory layout should be as follows:
1
2
3
4
5
6
7
8
9
10
11
12
molecule
├── arch
│ ├── Dockerfile
│ └── molecule.yml
├── default
│ └── molecule.yml
├── shared
│ ├── converge.yml
│ └── verify.yml
└── ubuntu
├── molecule.yml
└── prepare.yml
Now, when Molecule creates the instance, it will use our custom Dockerfile.
We verify that the “arch” scenario also works and update the GitHub Actions workflow by adding “arch” to the scenario matrix.
I hope you find this post helpful in getting started with Ansible and Molecule.
As I wrote in this other article, I recently bought an Acer Aspire Vero, which I greatly enjoy. Of course, I immediately installed Linux on this machine. In this article, I’ll report my experience installing Ubuntu (Ubuntu 22.10 “Kinetic Kudu”) on this Acer Aspire Vero.
Although nowadays I’m mainly an Arch Linux user, when installing Linux on a brand new laptop, I typically prefer to start with Ubuntu.
Preparation
I have already downloaded the Ubuntu ISO and copied it into a USB stick with Ventoy.
I need the F12 boot menu to boot from the USB stick. This is also useful later because I typically have several Linux distributions installed on the same computer. To enter the BIOS, you must press F2 while the laptop is booting (when you see the “Acer” logo). Make sure the “F12 Boot Menu” is enabled (by default, it’s disabled):
I also disabled “Secure Boot”. If you go to the BIOS “boot” tab, you see that you cannot change the boot entries.
This will allow you to change the entry of “Secure Boot”:
Then, you can disable the Supervisor Password (you have to use the password you had previously chosen).
Then, it’s time to prepare some room in the SSD for Linux. I do that by shrinking the Windows partition from Windows itself. After installing a few programs on Windows (and performing the system updates), that’s the amount of used space:
I opened “Disk Management”, selected the primary partition, and used the context menu “Shrink Volume…”; since I’m not planning to use Windows much, 137Gb should be enough for the Windows partition after shrink:
And that’s the result:
Before installing Linux, I also disabled “fast startup” in Windows: this will allow me later to access the Windows partition from Linux (otherwise, the Windows partition would be in an inconsistent state):
OK, let’s reboot with the USB stick (I’m using Ventoy) and press F12 to get to the boot menu to choose to boot from the USB:
Ubuntu boots fine.
I decided first to try Ubuntu and see whether everything works in the live environment:
The sound works. I’d say that WiFi and Bluetooth are also working from the new GNOME 43 menu in the top-right corner (in fact, I can connect to my WiFi). Moreover, the “Balanced” profile is automatically selected, meaning power profiles also work.
Installation
The overall installation process went smoothly and fast.
I prefer to manually partition the disk because I want a swap partition (for hibernation), a standard EXT4 partition mounted on a directory that I will share with other Linux installations, and the root partition as BTRFS.
The installed system
So here we are on the installed system; as usual, I’m greeted by the initial setup dialog:
Wayland works and touchpad gestures work as well.
Grub detected my existing Windows installation so that I could boot Windows from the grub menu.
Usually, I have to increase the font size on my computers. This laptop provides 1920×1080 (16:9) on a 15,6 screen. Typically, I have to use Gnome Tweaks, but in this case, using the “Accessibility” menu and selecting “Large Text” was enough for having a readable screen (this corresponds to a font scaling factor of 1.25):
Power consumption
I selected “Power Saver” as the power profile in the Gnome menu, and I have installed “powertop”. I ran “sudo powertop –auto-tune,” and then I ran “sudo powertop” to see the power consumption without further interacting with the computer:
If I decrease the brightness a bit, it looks even better:
Since 9 hours is the declared time in the computer spec, I’d say Linux works great on this computer in this respect (even better than Windows).
See also later in this article another mechanism to improve power consumption.
Other configurations
I had to perform some additional tweaks, which I had already blogged about:
I have always liked Amarok, the (initially) default KDE media player. It’s very feature-rich, nothing compared to Elisa. Moreover, it has two crucial features that I haven’t found in any other players:
it saves statistics (play count and stars) directly into the music file
it synchronizes statistics with iPod
Unfortunately, while still maintained, you won’t find pre-built packages in mainstream distributions (e.g., Ubuntu). Thus, you must install that from sources, which is problematic. However, for Arch Linux, there’s an AUR package, which takes care of the compilation and, most of all, its dependencies.
In this blog post, I’ll summarize the steps for installing Amarok to access iPods (I still have an iPod classic).
First, you need to install the Phonon backend required by Amarok:
1
sudo pacman -S phonon-qt5-vlc
If you want to use Amarok with an iPod, you must first install
1
yay -S amarok
IMPORTANT: the iPod library (this must be present when Amarok is compiled from sources; if you forget about that, you’ll need to recompile Amarok, e.g., by specifying “–rebuild” as a command line argument to the AUR helper):
1
sudo pacman -S libgpod
Then, we’re ready to install (i.e., compile from sources) Amarok from the AUR repository (I’m using the “yay” AUR helper here, but if you use another one, use your preferred one):
1
yay -S amarok
Now be patient: it will take several minutes for the compilation to finish (about 20 minutes on a decent machine)!
If you’re on KDE, you can now enjoy Amarok.
If you’re on GNOME, there’s still something to fix. In particular, you’ll see Amarok lacks lots of icons:
You need to install Breeze icons:
1
sudo pacman -S breeze-icons
And now you can also enjoy icons:
Concerning the iPod: first, you have to mount it, and then you start Amarok so that Amarok can see the mounted iPod.
Although I’m a Linux user, I also recently bought a Mac Air M1, and I wanted to use Docker (a big part of my TDD book) to ensure that my projects based on Docker work on m1 as well.
Run “mvn docker:start” to start a MySql container with a random mapped port. The command will wait for the container to be ready (it looks for a “ready” string within 20 seconds). After the command succeeds, the container will be running in the background. Run “mvn docker:stop” to stop the started container.
To avoid errors in this shape:
1
2
[ERROR] : could not get native definition for type `POINTER`, original error message follows: java.lang.UnsatisfiedLinkError: Unable to execute or load jffi binary stub from `/var/folders/5c/90l1wfls2xl7_xvqxqvnm3jw0000gn/T/`. Set `TMPDIR` or Java property `java.io.tmpdir` to a read/write path that is not mounted "noexec".
Actually, the first time I tried this project, it did not work (not because of the above error), but after a recent update, it started to work, maybe because of this added link:
Note that not all the images are available for this architecture “aarch64”. For example, if you try to use this older version of “mysql”, you get this error:
1
2
3
docker pull mysql:5.7
5.7: Pulling from library/mysql
no matching manifest for linux/arm64/v8 in the manifest list entries
[WARNING]DOCKER>The requested image'splatform(linux/amd64)does notmatch the detected host platform(linux/arm64/v8)andno specific platform was requested
If you now enter the container, you can verify that you’re running an x86_64 image:
1
2
3
sh-4.2# uname -a
Linux cf9d5b28744a 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
sh-4.2#
However, such images will run slower because they are emulated:
Testcontainers
I also use Testcontainers to start Docker containers from the JUnit tests in my projects.
After using EndeavourOS, an Arch-based distro, for some time with much pleasure and appreciating Arch mechanisms (packages and AUR), I decided it was time to try the “real thing” and install Arch the “hard way” 🙂 Spoiler: it’s not that hard!
I thought it was hard. For sure, it’s more complicated than other distro installation procedures, but, to be honest, after using Linux for more than 20 years, I thought there was not much to be scared of 😉
I now use Arch (besides other distros) on my machines greatly. Of course, I did many experiments with virtual machines before installing Arch on bare metal. I know there are many guides and tutorials, but I’d like to summarize my steps for installing Arch (with a SWAP partition, an EXT4 partition for data to be shared among distros, and a BTRFS primary partition). In particular, in this blog post, I’ll describe my steps for installing Arch on a virtual machine, which, as I’ve just said, it’s the best way to get confident with Arch and not be scared of installing Arch on a real computer. Moreover, on many guides, I noticed a few missing points, which, instead, are essential.
Of course, the best reference is the excellent official guide, and I’ll use the official guide as a reference while following along, https://wiki.archlinux.org/title/installation_guide. Note that there are still a few parts in the guide that refer to other parts of the excellent Arch wiki, and I had a few minor problems the first time I tried the Arch installation.
Here we go!
Create and configure the virtual machine with enough disk space (dynamically allocated so you won’t waste space on your disk), let’s say 100Gb. Make sure you enable EFI in the virtual machine configuration. Of course, insert the Arch Linux ISO as a live CD in the virtual machine. I’m going to use archlinux-2022.09.03-x86_64.iso.
As described in a previous post, I’d suggest performing the installation by connecting via SSH to the virtual machine. This way, you’re using a local terminal: copy and paste will work (since you’re on a local terminal). In particular, since the Arch installer is textual, being able to copy and paste commands from a local terminal makes everything easier. Moreover, the keyboard layout will be the host system’s keyboard layout. Thus, the keyboard layout will be already configured correctly. While in the virtual machine, you’d have to configure the keyboard layout.
(On a side note, even when installing Arch on a real computer, I prefer doing that via SSH, of course, from another computer.)
Before starting the virtual machine, we must map the SSH port of the virtual machine to a local port to connect from our computer. This requires knowing the name you gave to your virtual machine. In this example, I called the virtual machine “Arch Gnome” (because I’ll then install Gnome on the Arch installation). We must run these instructions from the host computer:
Port 2522 is the one we’ll have to use later for connecting to the virtual machine via localhost. Of course, feel free to use another free port number as long as you’ll use it consistently from now on.
Start the virtual machine:
Inside the live environment, the SSH server is already up and running. However, since we’ll connect with the root account (the only one present), we must give the root account a password. By default, it’s empty, and SSH will not allow you to log in with a blank password. Choose a password. This password is temporary, and if you’re in a trusted local network, you can choose an easy one.
Now, we can connect via SSH to the virtual machine through localhost. if you have already connected via SSH to localhost, you might get an error of the shape:
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256: ...
Please contact your system administrator.
Add correct host key in <YOUR HOME>/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in <YOUR HOME>/.ssh/known_hosts:<A LINE NUMBER>
Host key for [127.0.0.1]:2522 has changed and you have requested strict checking.
Host key verification failed.
All you have to do is edit the known_hosts file by removing the offending lines and try again. You will have to remove all the lines that start with “[127.0.0.1]:2522”.
Note that we’re using port 2522 because we previously used that for creating the port mapping. Let’s connect to the virtual machine and type the password we have previously specified for the root account inside the virtual machine (Accept the fingerprint when asked.):
1
ssh-p2522root@127.0.0.1
In your local terminal, you see that you get the colors of the virtual machine (now, you’re inside the virtual machine):
Let’s set the console keyboard layout (the default layout is US; if that’s fine with you, skip the next step). This step is not strictly required for our local terminal: even if we’re inside the virtual machine, we’re using our local terminal, so we already use the correct layout. However, let’s do that anyway since we want to simulate an actual installation. Moreover, having the proper layout is good if we want to run commands directly from the VirtualBox window.
I already know the layout I want for my Italian keyboard, so I run:
1
loadkeys it
If you don’t know the exact layout, you can list the available ones with
1
ls/usr/share/kbd/keymaps/**/*.map.gz
You can verify in the VirtualBox window that the layout is applied correctly.
Since we are in a virtual machine, the machine should already be able to access the Internet if your host is correctly connected (and that’s required to install Arch Linux). However, if you want to simulate what you would do with an actual installation on bare metal, you can ping a remote host and verify that everything’s OK:
1
ping archlinux.org
Before going on, as suggested in the official guide, it’s better to make sure the system clock is accurate by enabling network synchronization NTP:
1
timedatectl set-ntp true
Partitioning the disk
How to partition the disk is your choice. In this example, I will partition the disk according to my needs. However, you need at least two partitions: one for booting in UEFI mode and one for the root filesystem.
In this example, I’ll create four partitions:
the one for booting in UEFI mode, formatted as FAT32, 300Mb (it should be enough for UEFI, but if unsure, go on with 512Mb)
a swap partition, 20Gb (I have 16Gb, and if I want to enable hibernation, i.e., suspend to disk, that should be enough)
a partition meant to host common data that I want to share among several Linux installations on the same machine (maybe I’ll blog about that in the future), formatted as EXT4, 30Gb
the root partition, formatted as BTRFS, the rest of the disk
To do that, I’m using cfdisk, a textual partition manager, which I find easy to use.
Now, it is time to get to know the device name of our disk using the command lsblk:
1
2
3
4
5
root@archiso ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 683.2M 1 loop /run/archiso/airootfs
sda 8:0 0 100G 0 disk
sr0 11:0 1 794.3M 0 rom /run/archiso/bootmnt
As you might guess, sda is the disk we are installing Arch Linux upon. In a virtual machine, that’s probably always like that. On a real machine, it might be different (for example, if you have an NVME SSD, it will be something like nvme0n1). It’s needless to say that using the correct device name is crucial, especially on a real machine, or you might end up wiping away essential data. The nice thing about a virtual machine is that you’re in a “sandbox,” so, at worst, you’ll break your virtual machine.
So I run
1
cfdisk/dev/sda
If it is a new virtual machine, you’ll be asked a partition table: choose gpt.
Start creating your partitions. Just use the menus of cfdisk; it’s easy (on the bottom, you will find some help). Once you create a partition, set the “Type” correctly. By default, the type is “Linux filesystem”. For UEFI, you have to specify the type “EFI System,” and for the swap partition, “Linux swap”.
That’s my final result:
Let’s “Write” the partition table to disk and “Quit”. We can also verify with lsblk that the result is as expected:
1
2
3
4
5
6
7
8
9
root@archiso ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 683.2M 1 loop /run/archiso/airootfs
sda 8:0 0 100G 0 disk
├─sda1 8:1 0 300M 0 part
├─sda2 8:2 0 20G 0 part
├─sda3 8:3 0 30G 0 part
└─sda4 8:4 0 49.7G 0 part
sr0 11:0 1 794.3M 0 rom /run/archiso/bootmnt
Of course, you must know which partition is meant for what. In my example, sda1 is for UEFI, sda2 for swap, sda3 for my shared data, and sda4 for root.
Format the partitions
According to my intended layout shown above, I’ll format the four partitions with the following commands:
1
2
3
4
mkfs.fat-F32/dev/sda1
mkswap/dev/sda2
mkfs.ext4/dev/sda3
mkfs.btrfs/dev/sda4
Mount the partitions
This is also delicate, so you must use the correct device names. What follows is, of course, correct according to my layout.
First of all, let’s deal with the swap partition:
1
swapon/dev/sda2
The presence of the BTRFS file system for the root partition makes things a bit more interesting (or a bit more complicated, as you prefer 😉
First, we must mount the BTRFS filesystem on /mnt. Note that we are mounting the BTRFS on /mnt only temporarily and to create the subvolumes (in a minute, we will mount the subvolumes in their final shape on /mnt, together with the other partitions):
1
mount/dev/sda4/mnt
This will allow us to create the subvolumes. Again, what follows is the BTRFS subvolume layout I prefer. You might want to choose a different one. To use Timeshift, you must have at least @ for / and @home for /home. This is how I create the subvolumes I want:
1
2
3
4
5
6
btrfs su cr/mnt/@
btrfs su cr/mnt/@home
btrfs su cr/mnt/@cache
btrfs su cr/mnt/@log
btrfs su cr/mnt/@machines
btrfs su cr/mnt/@portables
As I said, this mount was temporary, just for creating subvolumes. In fact, we now unmount /mnt:
1
umount/mnt
And we mount every single subvolume in its final “position” inside /mnt by also specifying a few additional options like the general “noatime” and the BTRFS-specific “compress” to enable the ztsd compression:
The “-m” option makes mount create the target directory if it does not exist.
Finally, we can mount the remaining partitions. The UEFI one should be mounted to “/boot/efi” inside “/mnt”. I like to mount the “common” partition in “/media/bettini/common” inside “/mnt” because that’s where I’ll use it (relying on the fact that I’ll create a user “bettini” for myself). Again, choose something else for yourself. These are the commands:
This is the final layout of /mnt, which, remember, is where our system will be installed:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
/mnt
├── boot
│ └── efi
├── home
├── media
│ └── bettini
│ └── common
│ └── lost+found
└── var
├── cache
├── lib
│ ├── machines
│ └── portables
└── log
Select the mirrors
This part, documented in the official installation guide, is usually skipped in several blog posts I found online. Instead, this step is essential.
The mirrors are specified in the file /etc/pacman.d/mirrorlist.
The guide says:
On the live system, after connecting to the internet, reflector updates the mirror list by choosing 20 most recently synchronized HTTPS mirrors and sorting them by download rate.
The higher a mirror is placed in the list, the more priority it is given when downloading a package. You may want to inspect the file to see if it is satisfactory. If it is not, edit the file accordingly, and move the geographically closest mirrors to the top of the list, although other criteria should be taken into account
You can verify that by inspecting the file /etc/pacman.d/mirrorlist. In my case, the Italian mirror is the last one, so it will be given the lowest priority. This sounds wrong to me. In particular, the documentation also points out:
This file will later be copied to the new system by pacstrap, so it is worth getting right.
Thus, I prefer to run the program reflector myself (see the reflector documentation for the single arguments; of course, I’m using “Italy” as the country because that’s where I leave; I could also specify several values separated by a comma, e.g., “Italy,Germany”):
1
2
3
4
5
6
7
8
reflector \
--country Italy \
--age 12 \
--protocol https \
--fastest 5 \
--latest 20 \
--sort rate \
--save /etc/pacman.d/mirrorlist
The following step does not seem to be required in the installation guide. However, to make sure we have an updated PGP keyring (for checking signatures of packages), at this point, I also run:
1
pacman-Sy--noconfirm archlinux-keyring
Running pacstrap
Now, it’s time to install the base packages, Linux kernel, and firmware for standard hardware using the pacstrap script. You specify the target directory, which, as you might guess, it’s /mnt, and the packages.
This is the command I run (I prefer to use the LTS kernel; if you want the latest kernel, use “linux” package instead of “linux-lts”; you can also install them both and then select one from the grub menu):
1
2
3
4
5
pacstrap /mnt base linux-lts linux-firmware \
nano vim \
intel-ucode \
btrfs-progs \
sof-firmware alsa-firmware
This command will download about 500Mb, which might take time depending on your Internet speed.
Configuring the system
Since we have already manually mounted all our partitions (on /mnt), the Arch ISO can generate for us the file fstab automatically through the command genfstab:
1
genfstab -U /mnt >> /mnt/etc/fstab
The “-U” option tells genfstab to use UUID to refer to partitions (alternatively, “-L” can be used to use labels instead).
You can have a look at the result (of course, UUID will be different in your case):
Note that although we used the option “compress=zstd” when mounting our BTRFS subvolumes, genfstab turned that into “compress=zstd:3” because “3” is the default compression value for zstd in BTRFS. If we wanted to make the compression value explicit, e.g., “1”, we should have done that when mounting the subvolumes. Of course, you can always tweak the generate fstab as you see fit.
Now, we can “enter” our installation with “chroot” or, better, with the enhanced arch-chroot, which automatically binds other things like /dev and /proc:
1
arch-chroot/mnt
The root directory is what’s inside /mnt, so / refers to what’s inside /mnt. Also, the prompt has changed to reflect this:
1
2
root@archiso ~ # arch-chroot /mnt
[root@archiso /]#
We now set the timezone of the installed system. You must use Region/City according to your location. Timezones are available in the directory /usr/share/zoneinfo/. In my case (Italy), I run:
Then, we use hwclock to set the Hardware Clock from the System Clock:
1
hwclock--systohc
Then, we edit /etc/locale.gen and uncomment the locales we need; in my case, en_US.UTF-8 UTF-8 and it_IT.UTF-8. Since I already know the two locales, instead of editing the file (where all locales are commented out), I append the two locales to the end of that file:
1
2
echo en_US.UTF-8 UTF-8 >> /etc/locale.gen
echo it_IT.UTF-8 UTF-8 >> /etc/locale.gen
And we generate the locales:
1
locale-gen
We must also create the /etc/locale.conf file, and set the LANG variable accordingly. This can be done as follows:
1
echo LANG=en_US.UTF-8 >> /etc/locale.conf
We also make permanent the initial changes to the console layout (remember, I used “it”; in your case, you need to use the code you previously specified):
1
echo KEYMAP=it >> /etc/vconsole.conf
We’ll deal with the network configuration (of the installed system) in a minute. But we can already create the files /etc/hostname and /etc/hosts. You have to choose your preferred hostname. In this example, I’m going to use “arch-vm-gnome”. So I generate the two files with the following two commands:
1
2
3
4
5
6
7
echo arch-vm-gnome >> /etc/hostname
cat >> /etc/hosts << EOF
127.0.0.1 localhost
::1 localhost
127.0.1.1 myhostname.localdomain arch-vm-gnome
EOF
The boot loader
Since we use BTRFS, we might want to tweak the file /etc/mkinitcpio.conf with these two modules:
1
MODULES=(crc32c-intel btrfs)
And regenerate the initramfs images:
1
mkinitcpio-P
Let’s now install the bootloader. I prefer GRUB. So let’s install a few packages:
1
pacman -S --noconfirm --needed grub efibootmgr
During the installation, you might want to take note of the recommendation for installing and configuring grub and the optional dependencies:
1
2
3
4
5
6
7
8
9
10
11
12
13
:: Install your bootloader and generate configuration with:
$ grub-install ...
$ grub-mkconfig -o /boot/grub/grub.cfg
Optional dependencies for grub
freetype2: For grub-mkfont usage
fuse2: For grub-mount usage
dosfstools: For grub-mkrescue FAT FS and EFI support
lzop: For grub-mkrescue LZO support
efibootmgr: For grub-install EFI support [pending]
libisoburn: Provides xorriso for generating grub rescue iso using
grub-mkrescue
os-prober: To detect other OSes when generating grub.cfg in BIOS systems
mtools: For grub-mkrescue FAT FS support
Now we install grub in the UEFI partition. Note that, unlike standard GUI Linux installations, you can specify the “–bootloader-id”, which will be the identifier of this grub installation in UEFI. This is useful if you have several bootloaders on your machine. In this example, I’m using ArchGnome:
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux-lts.img
Found fallback initrd image(s) in /boot: intel-ucode.img initramfs-linux-lts-fallback.img
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
User accounts
Don’t forget to set the following passwords, or you will not be able to log in to the installed system once you reboot later.
We can now set the root password. This is the effective password for root in the installed system. This should be chosen carefully:
1
passwd
I prefer to use “sudo”, so I first install that
1
pacman-Ssudo
This is also the moment to create your own user account; in my case, it is “bettini”.
1
2
3
useradd-mbettini
usermod-aG wheel,sys,rfkill bettini
I add the user to a few essential groups, in particular, “wheel” which makes my user a superuser account. Just relying on the group “wheel” is not enough: we must allow members of the group wheel to execute any command. This is done by uncommenting this line in the /etc/sudoers: “%wheel ALL=(ALL:ALL) ALL”. A sed command will accomplish that:
That would be enough to reboot and try our installation. However, we would have no networking (actually, since this is a virtual machine, networking should work out of the box since you don’t need to configure any WiFi network, for example) and no desktop environment. So let’s go on with some further installations and configurations:
Install Gnome
In this example, I’m going to install the GNOME desktop environment. Besides GNOME, I’m installing other necessary packages like the “NetworkManager” (for easily configuring networking in GNOME), “firewall”, “firefox”, the package for choosing a power profile (useful for laptops), and other base packages, including the kernel headers (here I’m using “linux-lts-headers” because I installed the LTS kernel; otherwise, use “linux-headers”):
Once done, we have to enable the services at boot (in particular, GDM, the login manager, and the NetworkManager):
1
2
3
4
systemctl enable gdm.service
systemctl enable NetworkManager.service
systemctl enable power-profiles-daemon.service
systemctl enable firewalld.service
Time to reboot!
Now, it’s time to leave the environment and unmount all the partitions:
1
2
exit
umount-R/mnt
And reboot into our new Arch Linux installation.
If everything goes fine, we should see the login manager, and we can enter Gnome:
So, in the end, it’s not so hard to install Arch 😉
Maybe it’s a long procedure, but… most of that can be scripted! That’s will be the subject of another blog post, so stay tuned! 🙂
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
cookielawinfo-checkbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
