AlmaLinux provides some "Live" images with various desktop environments. I installed the KDE "spin" and found out that there is a crap load of useless / unwanted packages. Here are my notes to disable them.
Packages that I don't need:
$ sudo dnf remove kmines kmahjongg kolourpaint kruler akregator kmail konversation dragon juk kamoso kaddressbook korganizer plasma-discover flatpak* sssd* java-11-openjdk-headless oddjob
You may want to keep sssd and oddjob if your workstation has membership to an Active Directory. You may also want to keep plasma-discover if you want a GUI to search for packages and have software updates notifications.
Softwares that can be a security issue on a workstation (they may listen for connections or allow unwanted remote access):
$ sudo dnf remove cockpit* krfb krdc hyperv* mariadb* open-vm-tools qemu-guest-agent spice-vdagent tcpdump openssh-server
It's a shame that AlmaLinux and RockyLinux do not provide a "minimal" KDE Live image, or a clear documentation to install a minimal set of packages. Let's hope that KDE gets more attention from Red Hat for the next major release!
If you need to install Zabbix-Agent2 on Ubuntu 16.04, you will find out that there is no available packages in Zabbix repository (unlinke Zabbix-Agent). You can try to use packages for other Linux systems, even RPMs, but you will always end up with library or ABI issues. The only way to make it work is compilation.
Install requirements:
apt install -y libpcre++-dev build-essential zlib1g-dev libssl-dev
Get Zabbix source code:
wget https://cdn.zabbix.com/zabbix/sources/stable/6.2/zabbix-6.2.4.tar.gz
tar xf zabbix-6.2.4.tar.gz
You need at least Go 1.17 (for Zabbix 6.2.4):
wget https://go.dev/dl/go1.19.3.linux-amd64.tar.gz
tar xf go1.19.3.linux-amd64.tar.gz
export PATH=$PATH:/root/go/bin
You should now be able to build Zabbix-Agent 2. I took these options from Zabbix documentation and made some ajustements from what I found in packages in Zabbix repository:
cd zabbix-6.2.4
./configure \
--enable-agent2 \
--enable-static \
--prefix=/usr \
--sysconfdir=/etc/zabbix \
--libdir=/usr/lib/zabbix \
--with-curl \
--with-openssl
Note: According Zabbix documentation, the --enable-static flag is useful if you want to create your own package and use it on other systems.
You can now build and install:
make install
You can now remove Go if you don't need it. A few steps are required to make Zabbix-Agent 2 work:
addgroup --system --quiet zabbix
adduser --quiet --system \
--disabled-login \
--ingroup zabbix \
--home /var/lib/zabbix \
--no-create-home zabbix
mkdir -p /etc/zabbix/zabbix_agent2.d/plugins.d
mkdir /run/zabbix/
chown -R zabbix:zabbix /run/zabbix
mkdir /var/log/zabbix
chown -R zabbix:zabbix /var/log/zabbix
Create /etc/logrotate.d/zabbix_agent2:
/var/log/zabbix/zabbix_agent2.log {
weekly
rotate 12
compress
delaycompress
missingok
notifempty
create 0640 zabbix zabbix
}
In the packages from Zabbix repository we have /usr/lib/tmpfiles.d/zabbix-agent2.conf:
d /run/zabbix 0755 zabbix zabbix - -
Don't forget to create a /etc/zabbix/zabbix_agent2.conf file. Here is a sample.
Finally, create a systemd unit file in /lib/systemd/system/zabbix-agent2.service:
[Unit]
Description=Zabbix Agent 2
After=syslog.target
After=network.target
[Service]
Environment="CONFFILE=/etc/zabbix/zabbix_agent2.conf"
EnvironmentFile=-/etc/default/zabbix-agent2
Type=simple
Restart=on-failure
PIDFile=/run/zabbix/zabbix_agent2.pid
KillMode=control-group
ExecStart=/usr/sbin/zabbix_agent2 -c $CONFFILE
ExecStop=/bin/sh -c '[ -n "$1" ] && kill -s TERM "$1"' -- "$MAINPID"
RestartSec=10s
User=zabbix
Group=zabbix
[Install]
WantedBy=multi-user.target
Reload, enable and start this new service:
systemctl daemon-reload
sytemctl enable --now zabbix-agent2
Check that everything works:
systemctl status zabbix-agent2
tail /var/log/zabbix/zabbix_agent2.log
Profit!
How does profiles work ?
Kdenlive use ffmpeg and MLT. So it's basically MLT syntax to pass arguments to ffmpeg. But what about CBR, VBR, CQP, CRF? What's the magic behind these acronyms ?
- CBR: Constant Bit Rate. Predictable bandwith, good for streaming. However, it's up to you to set the correct birate and it depends on the resolution of the video (720p, 1080p..) and its content (things that move fast will require more bandwith). A low value means bad quality while high means unnecessary huge files.
- VBR: Variable Bit Rate. Set a nominal and a maximal bitrate settings that the encoder will use. This is similar to CBR except that the output file should be less huge.
- CRF (Constant Rate Factor) and CQP (Constant Quantizer Parameter). I admit I do not get the difference between those two, but the idea is the same: you do not set the bitrate, but the quality level you want to achieve.
CBR and VBR are good for streaming because you need a predictable bandwith output. However when you need to locally record a video, you do not really care about the bandwith but more about the quality and the file size, in that case CRF / CQP are more suited.
Kdenlive Built-in CPU profiles:
- x264 (CRF 23)
- x265 (CRF 20)
- vb9 (CRF 23)
Kdenlive Built-in GPU profiles:
- NVENC H264 VBR (20-30Kbps)
- NVENC H265 (CBR 30Kbps)
- NVENC H264 VBR (20-30Kbps)
- NVENC H265 (CBR 30Kbps)
I added those profiles:
- NVENC H264 CQP 20: f=mp4 vcodec=h264_nvenc rc=constqp qp=20 profile=high preset=quality ab=192k ar=44100 acodec=aac bf=2
- NVENC H265 CQP 20: f=mp4 vcodec=hevc_nvenc rc=constqp qp=20 profile=high preset=quality ab=192k ar=44100 acodec=aac bf=2
Benchmark
The input file was a 45min gaming session of Star Wars: Knights of the Old Republic, in 1920x1080 60fps + encoded in H264. I did not applied any effects except cutting video/audio.
Computer specifications:
- AMD Ryzen 3700X (8c/16t @3,6GHz)
- 16GB DDR4
- Nvidia RTX3070 FE
- 500GB NVMe SSD
- Windows 10 x64
- Kdenlive 20.12.13
While VP9 is a totally free fairly good codec, it is incredibly slow (3h13 versus 0h18 for x264). H265 is slower than H264 probably for good reasons although I'm not able to see a difference in output quality. H264 CQP 20 (NVENC) is 34% faster than x264 CRF 23 (CPU). Keep in mind x264 and x265 speed highly depends on the number of CPU cores. If you can get yours hands on a 3900X (12c/24t) or a 3950X (16c/32t) then I suspect they can match or outperform NVENC.
H264 CBR 30Kbps (NVENC) is the worst for file recording, resulting in a 12,56 GB file while you can achieve 3,2 GB using H264 CQP 20 (NVENC). x265 seems to be the most efficient (only 1,92 GB) but it's probably related to the CRF 20 agressive parameter. Of course I can't just fix all parameters to 20 because the output quality will depend on the codec, yeah it's black magic.
Limitations
Before jumping to conclusion, you need to be aware of some limitations of my benchmark:
- The input file was encoded in H264, I did not tried anything else. This may explain the bad results for VP9.
- x264 and x265 performance scale with the number of CPU cores.
- I do not know if NVENC speed and quality depends on the GPU.
- NVENC CQP 20 looks fine for me, but it's purely subjective. Another value might change the benchmark.
- Kdenlive on Windows can't use NVENC to render video effects and will use the CPU, this is the reason why I did not add any.
Conclusion
I usually stick with H264 CQP 20 (NVENC) which is the fastest with a good quality output (at least to record video games) and a reasonable file size. While Nvidia recommends CQP 15, I can't tell the difference with CQP 20. Also remember that if you upload your video on Youtube, it will be reencoded in AV1 / VP9 with lower quality settings anyway.
Again, you have to understand that there is no "best encoder" for all situations. This is what I think works best for me, but it might be totally different for you.
Links
Kdenlive is a great software but I noticed a major drawback on the Windows version: render time that is really slow. For example, let's take a 45min H264 2560x1440 60fps video file and crop it to 1920x1080 ("Position and Zoom" effect on Kdenlive). Render time is about 1h15 on Kdenlive while it's only 15 minutes on Adobe Premiere Pro, the difference is insane. Why is the later so fast ? Spoiler: GPU rendering.
Let's talk about how rendering works. Kdenlive use ffmpeg and MLT. The Windows version of Kdenlive is built with an embedded minimal ffmpeg that does not support GPU hardware acceleration, which is sad. Fortunately, it is possible to download the full version of ffmpeg and install it on Kdenlive, here is how to proceed.
Download
Installation
- Run the kdenlive installer which is in fact a self-extracting archive. Extract the contents in a location, for example C:\Program Files\Kdenlive.
- Extract ffmpeg-release-full-shared.7z to a temporary location, for example C:\Users\utux\Downloads.
- Copy C:\Users\utux\Downloads\ffmpeg\bin\* to C:\Program Files\Kdenlive\bin\
- Copy C:\Users\utux\Downloads\ffmpeg\presets\* to C:\Program Files\Kdenlive\share\ffmpeg\
Configuration
Start or restart Kdenlive then click Configuration > Run Config Wizzard. Check "Nvidia hardware acceleration" and make sure it is properly detected.
The following render profiles show be available now:
- NVENC H264 CBR
- NVENC H264 VBR
- NVENC H265 CBR
- VAAPI Intel H264
- VAAPI AMD H264
Try to render a project using one of these profiles and take a look at the Performance / GPU / Video Encode section in the Task Manager:
If the GPU Encode graph is low (20% or less) that means that Kdenlive is rendering Effects on the CPU (such as "Position and zoom"). See limitations below.
Limitations
- Right now (Apr 2021) effects cannot be rendered by the GPU (at least on Windows). This may lead to frustrating situations where the GPU encoder only works at 20% and render is slow. I used to capture 1080p games in a 1440p desktop, then crop with the "Position and Zoom" effect, I changed that. Now I have set OBS Studio to record in 1080p, negating the need any transformation.
- I could not make NVENC work for proxy clips, which is bad.
- The Windows version of Kdenlive does not seems to support Movit aka "Experimental GPU processing" which add effects that can be rendered by the GPU. For example, instead of "Position and zoom" you can use "Pan and zoom (GPU)". You should try the Flatpak version (Linux).
Il y a un peu plus de 2 ans j'ai parlé de NixOS, distribution Linux atypique dans le sens où sa configuration se fait de manière déclarative à un unique emplacement.
Mon ordinateur pro et ma machine gaming sont sous Windows, j'utilise donc très souvent des machines virtuelles Linux pour pouvoir travailler. Habituellement j'utilise Virtualbox + Debian + Mate, mais j'ai décidé d'expérimenter NixOS. Alors oui je sais qu'une machine virtuelle n'est pas représentative d'un cas d'usage réel, mais ça permet de se faire une première idée.
Installation
J'ai installé NixOS avec la configuration suivante:
# Edit this configuration file to define what should be installed on
# your system. Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).
{ config, pkgs, ... }:
{
imports =
[ # Include the results of the hardware scan.
./hardware-configuration.nix
];
# Use the GRUB 2 boot loader.
boot.loader.grub.enable = true;
boot.loader.grub.version = 2;
# Define on which hard drive you want to install Grub.
boot.loader.grub.device = "/dev/sda"; # or "nodev" for efi only
# Networking
networking.hostName = "nixos"; # Define your hostname.
networking.networkmanager.enable = true;
# Select internationalisation properties.
i18n = {
consoleFont = "Lat2-Terminus16";
consoleKeyMap = "fr";
defaultLocale = "fr_FR.UTF-8";
};
# Set your time zone.
time.timeZone = "Europe/Paris";
# Repositories
nixpkgs.config.allowUnfree = true;
# List packages installed in system profile. To search, run:
# $ nix search wget
environment.systemPackages = with pkgs; [
ansible
dnsutils
firefox-bin
gimp
git
htop
keepassxc
libreoffice
networkmanagerapplet
parcellite
p7zip
sudo
syncthing
syncthing-gtk
thunderbird-bin
tree
unzip
vim
vscode
xarchiver
];
# Enable CUPS to print documents.
services.printing.enable = true;
# Enable sound.
sound.enable = true;
hardware.pulseaudio.enable = true;
# Mate
services.xserver = {
enable = true;
layout = "fr";
xkbOptions = "eurosign:e";
desktopManager = {
default = "mate";
xterm.enable = false;
mate.enable = true;
};
};
# Docker
virtualisation.docker.enable = true;
# Define a user account. Don't forget to set a password with ‘passwd’.
users.users.utux = {
isNormalUser = true;
extraGroups = [ "wheel" "networkmanager" "docker" ];
};
# Nix Garbage Collector
nix.gc = {
automatic = true;
dates = "weekly";
options = "--delete-older-than 30d";
};
# This value determines the NixOS release with which your system is to be
# compatible, in order to avoid breaking some software such as database
# servers. You should change this only after NixOS release notes say you
# should.
system.stateVersion = "19.09"; # Did you read the comment?
system.autoUpgrade.enable = true;
}
Environnement desktop
La gestion centralisée de la configuration s'applique au système, mais pas au $HOME de l'utilisateur. Les raccourcis, les thèmes de bureau, les profils se gèrent "à la main", ce qui m'amène à penser que NixOS perd un peu d'intérêt en usage desktop. Cependant, ce point pourrait bientôt changer avec Home-manager.
Lors de la rédaction de cet article, la version de Mate est la 1.22.1 donc très récente, un bon point. Dans la liste des paquets installés, notez firefox-bin et thunderbird-bin. Couplés à l'activation du repo unfree, cela permet d'installer les versions à jour et upstream. Les paquets firefox et thunderbird sont des versions ESR dépourvues du branding de Mozilla, un peu comme Iceweasel et Icedove pour Debian il y a quelques années.
La version de Keepassx fournie par Nix (0.4.4) est trop vieille pour être utilisable avec le format .kdbx, j'ai donc du installer keepassxc à la place. Vscode est disponible lui aussi grâce au repo unfree.
Python...
Globalement je retrouve donc mon environnement et mes applications favorites, mais je dois quand même parler de Python. Sur Debian, j'utilise généralement pip pour installer une version récente de Ansible et Molecule, mais sur NixOS je n'ai simplement pas compris comment ça marche :/ Il y a pourtant une documentation dédiée à Python mais elle a tendance à ne pas situer correctement les contextes ou ne pas donner assez d'exemples. Donc pas de pip pour le moment, j'ai utilisé le paquet NixOS de Ansible.
Performances
En ce qui me concerne, l'utilisation du bureau Mate dans Virtualbox sur NixOS n'est pas très fluide :/ que ce soit l'ouverture d'un onglet, le déplacement d'un fenêtre, le lancement d'un logiciel, tout accuse de saccades et de lags. Debian s'en sort mieux sur ce point, à configuration presque identique (2 vspus, 2Go de ram, vboxsvga, 32Mo vram), mais cette dernière utilise Mate 1.20.4 et pas 1.22.1.
Lorsque j'ai fait des tests sur une machine peu puissante, j'ai remarqué que nixos-rebuild arrive à mettre à genoux le système et les autres VMs, à cause des écritures sur le disque. Et à propos du disque, il faut parler de l'utilisation du stockage. Par nature, NixOS garde plusieurs versions de la configuration (overlays) et des paquets installés, pensez à des snapshots, donc on se doute que l'espace utilisé est important, très important. On peut facilement consommer 30 Go après quelques semaines, ce qui m'amène au point suivant.
Penser au nettoyage (garbage collector)
Heureusement il existe le Garbage collector qu'il faut régulièrement lancer en root:
[utux@nixos:~]$ sudo nix-collect-garbage -d
[...]
deleting '/nix/store/39s914agmm045fv7l3lz81zcvw86m3vb-xsltml_2.1.2.zip.drv'
deleting '/nix/store/5vy6k04dhrwn0951z7dnpkqyxp3r7ws0-jasper-2.0.16-bin'
deleting '/nix/store/p5c18w2cksfxw748d4f8l31rlfk8z1vg-font-adobe-100dpi-1.0.3'
deleting '/nix/store/mny4fywzpgb2yi54a0vz97f5kxlb8932-unit-systemd-modules-load.service.drv'
deleting '/nix/store/xbl8wj6293nfw3vziwqd5xmhrk5gdq6b-rand_hc-0.1.0.tar.gz.drv'
deleting '/nix/store/trash'
deleting unused links...
note: currently hard linking saves -0.00 MiB
8134 store paths deleted, 11574.19 MiB freed
Dans l'exemple ci-dessus, le système a nettoyé pas loin de 11,6 Go de paquets inutiles ! Il est possible d'automatiser ce nettoyage.
Documentation
NixOS a un manuel très détaillé et très instructif. On trouve aussi beaucoup d'informations sur des fils reddit et github. Il est tout de même fréquent de devoir aller lire le code pour savoir comment se configurer un logiciel pour NixOS, ce qui est intimidant au début mais on s'habitue.
Conclusion
Hé hé, un test ou un aperçu d'une distribution est un exercice que je n'ai pas fait depuis longtemps. Je vous encourage à tester NixOS, sur serveur ou en desktop. Cette distribution n'est pas comme toutes les autres que vous connaissez et elle vous plaira particulièrement si vous faites du devops et que vous cherchez une solution pour versionner les configurations de vos bécanes.
Je dois tout de même avouer que l'utilisation desktop est un peu laborieuse, surtout à cause des mauvaises performances dans Virtualbox (qui font que je reviens souvent à ma Debian) mais aussi à l'absence de gestion centralisée des $HOME (pour cela je testerai Home-manager.).
Utiliser NixOS met un peu de piment dans ma vie de Linuxien trop habitué à la Debian family et à la Red Hat family :)
Fil RSS des articles de cette catégorie