Linux - System Architecture

Hardware

The Linux kernel is the core framework of the operating system –> GNU/Linux. It provides a way for the rest of the system to operate with hardware, memory, networking and itself (meaning that the kernel is able to communicate with its subsystems).

The Linux kernel is monolithic:

The kernel handles all memory management and hardware device interactions
Extra functionality can be loaded and unloaded dynamically through kernel modules
Ensures that the system will not need to be rebooted into a different kernel image for added functionality

Note: Many third-pary Linux kernel modules are device drivers

Hardware Inspection

Identify connected devices in a Linux system:

# Show all devices currently connected to the PCI:
lspci
# Display more information about a specific PCI device:
lspci -s <device-address> -v
# Verify which kernel module is in use for the specified PCI device:
lspci -s <device-address> -k

# List USB devices currently connected to the machine:
lsusb
# Display more information about a specific USB device:
lsusb -v -d <device-id>
# Shows current USB device mappings as a hierarchical tree:
lsusb -t
# Verify which device is using the module:
lsusb -s <bus-number>:<device-number>

Note: Some devices may not be fully operational yet because a kernel module (also called drivers) is needed to control the device.

Note: These commands often require root privileges (shows only limited information when executed by a normal user).

Linux Kernel Utilities

# Displays information about the currently running kernel
uname
# Displays a listing of all currently loaded kernel modules:
lsmod

# Load kernel modules at runtime
modprobe <kernel-module>
# Unload kernel modules at runtime
modprobe -r <kernel-module>

Verify kernel module information:

# Displays information about a specified kernel module
modinfo <kernel-module>
# Display all available parameters and ignore the other information:
modinfo -p <kernel-module>
# Set parameters for kernel modules (See modinfo -p for possible values)
options <kernel-module> <parameter>=<value>

Note: Create customized module parameters persistently by including them in the file /etc/modprobe.conf or /etc/modprobe.d/<kernel-module>.conf.

Information- and Device Files

Pseudo-filesystems:

/proc               # Information on running processes
|__ /cpuinfo        # Information about CPU(s) found on the OS
|__ /interrupts     # Interrupts per IO device for each CPU
|__ /imports        # Registered Input/Output port regions in use
|__ /dma            # DMA (direct memory access) channels in use

/sys                # Device information and kernel data related to hardware

Note: These directories are mount points to filesystems only present in RAM space used by the kernel to store runtime configuration and information on running processes.

udev: identification and configuration of devices present on the machine (coldplug detection & hotplug detection).

Relies on /sys/sysFS
Searches for matching rule defined in /etc/udev/rules.d/

/dev                # Files are associated with a system device (storage devices)

Storage Devices

In Linux, storage devices are block devices (data is read to and from these devices in blocks of buffered data with different sizes and positions).

IDE, SSD and USB block devices will be prefixed by sd.

/dev/sda            # Master device
/dev/sdb            # Slave device

Boot the System

BIOS

BIOS MBR Structure

Pre-operating steps to boot a system:

POST (power-on self-test) to identify hardware failures at moment of power-on.
BIOS activates basic components to load the system (video input, storage media, …)
BIOS loads bootstrap from MBR (Master Boot Record - contains the first stage of the bootloader (bootstrap) and the partition table.)
Bootstrap calls second stage of the bootloader, responsible for presenting boot options and loading the kernel

Note: The bootstrap and MBR can be defined in the BIOS configuration utility.

UEFI

Does not rely on MBR (only checks settings in NVRAM –> locations of EFI applications).

EFI applications:

Bootloaders, operating system selectors, tools for system diagnostics and repair, …
Reside in EFI system partition (ESP)
Compatible filesystem (FAT12, FAT16 or FAT32)
UEFI requires a 64-bit operating system

Pre-operating steps to boot a system:

POST (power-on self-test) to identify hardware failures at moment of power-on.
UEFI activates basic components to load the system (video input, storage media, …)
UEFI reads definitions stored in NVRAM to execute EFI application stored in ESP filesystem (usually a bootloader)
If EFI application is a bootloader, it will load the kernel to start the operating system

Note: UEFI supports secure boot, to only allow execution of signed EFI applications (= authorized by hardware manufacturer).

Bootloader

GRUB (Grand Unified Bootloader): Displays a list of operating systems available to boot.

Note: GRUB can be invoked manually during boot with shift (BIOS) of esc (UEFI).

From the GRUB menu choose which kernel to load and pass kernel parameters to itue:

acpi=(on | off)             # Enables/disables ACPI support
init=<shell>                # Sets an alternative system initiator
systemd.unit=<target>       # Sets the systemd target to be activated
mem=<bytes>                 # Sets the amount of available RAM for the system
maxcpus=<cpus>              # Limits the number of processors visible to the system in symmetric multi- processor machines
quiet                       # Hides most boot messages
vga=<mode>                  # Selects a video mode
root=<partition>            # Sets root partition, distinct from the one pre-configured in the bootloader
rootflags                   # Mount options for the root filesystem
ro                          # Makes the initial mount of the root filesystem read-only
rw                          # Allows writing in the root filesystem during initial mount

Kernel parameters must be added to the file /etc/default/grub in the line GRUB_CMDLINE_LINUX to make them persistent across reboots.

# Generate a new configuration fil for the bootloader
grub-mkconfig -o /boot/grub/grub.cfg

Note: Once the OS is running, kernel parameters used for loading the current session are available for reading in the file /proc/cmdline.

System Initialization

During system initialization simple shell scripts and more complex service programs are loaded:

Scripts: short-lived tasks
Daemons: services active in the background that are responsible for instrinsic aspects of the OS

Initialization process:

Bootloader loads the kernel into RAM
Kernel, controlling CPU, start to detect and setup the fundamental aspects of the OS (basic hardware configuration and memory addressing)
Kernel opens initramfs (initial RAM filesystem - archive with temp root filesystem during boot, it allows the kernel to access to real root filesystem)
Kernel mounts all filesystems in /etc/fstab
Kernel executes system initiator (running all initialization scripts and system daemons)
Kernel removes initramfs from RAM

System initiators:

init                # Traditional system initiator
SysV standard       # Controls available daemons and resources based on runlevels (0-6)
systemd             # Modern system and services manager with compatibility layer for SysV commands and runlevels
Upstart             # Speed up boot process by parallelizing loading process of system services

Initialization Inspection

kernel ring buffer: volatile memory space where the kernel stores its (boot) messages in /var/log/.

# Display current messages in kernel ring buffer
dmesg
# Clear messages in kernel ring buffer
dmesg --clear

# Display current messages in kernel ring buffer (systemd)
journalctl -b
journalctl --boot
journalctl -k
journalctl --dmesg
# Show a list of boot numbers relative to the current boot
journalctl --list-boots
# Read messages in directories other than /var/log/journal
journalctl -D
journalctl --directory

Note: systemd based systems allow inspection of previous initialization logs.
# Display messages from current boot
journalctl -b 0
journalctl --boot=0
# Display messages from previous boot
journalctl -b -1
journalctl --boot=-1

Change Runlevels/Boot Targets and Shutdown or Reboot the System

SysVinit

/sbin/init: program responsible for managing runlevels and associated daemons/resources.

Runlevel	Description
0	System shutdown
1	Single user mode (maintenance mode)
2	Multi-user mode
3	Full multi-user mode
4	Unused (available for custom runlevel)
5	Graphical desktop mode
6	System restart

/etc/init.d/        # Contains scripts for services on the system
/etc/init.d/rc      # Script that orchestrates how the runlevel scripts run and what occurs when a runlevel changes
/etc/inittab        # Read by init to determine runlevel, contains default runlevel

# Syntax of /etc/inittab, each line in this file specifies a runlevel
<identifier>:<runlevel>:<action>:<process>

# Available actions
boot                # Process executed during system initialization (runlevel is ignored)
bootwait            # Process executed during system initialization, init waits until it finishes (runlevel is ignored)
sysinit             # Process executed after system initialization (runlevel is ignored)
wait                # Process executed for given runlevels and init waits until it finishes
restart             # Process restarted if terminated
ctrlaltdel          # Proces executed when SIGINT signal is received, triggered by `Ctrl+Alt+Del`

Example of /etc/inittab:

# Default runlevel
id:3:initdefault:

# Configuration script executed during boot
si::sysinit:/etc/init.d/rcS

# Action taken on runlevel S (single user)
~:S:wait:/sbin/sulogin

# Configuration for each execution level
l0:0:wait:/etc/init.d/rc 0
l1:1:wait:/etc/init.d/rc 1
l2:2:wait:/etc/init.d/rc 2
l3:3:wait:/etc/init.d/rc 3
l4:4:wait:/etc/init.d/rc 4
l5:5:wait:/etc/init.d/rc 5
l6:6:wait:/etc/init.d/rc 6

# Action taken upon ctrl+alt+del keystroke
ca::ctrlaltdel:/sbin/shutdown -r now

# Enable consoles for runlevels 2 and 3
1:23:respawn:/sbin/getty tty1 VC linux
2:23:respawn:/sbin/getty tty2 VC linux
3:23:respawn:/sbin/getty tty3 VC linux
4:23:respawn:/sbin/getty tty4 VC linux

# For runlevel 3, also enable serial terminals ttyS0 and ttyS1 (modem) consoles
S0:3:respawn:/sbin/getty -L 9600 ttyS0 vt320
S1:3:respawn:/sbin/mgetty -x0 -D ttyS1

Note: The system can only operate in 1 runlevel at a time and applies to the system as a whole.

# View you current runlevel
runlevel

# Change to another runlevel
telinit
# Reload init configuration
telinit -q

Note: You are also able to change the runlevels at boot: Interrupt the GRUB boot process by pressing any key during the startup, then at the GRUB selection menu, highlight a kernel to modify, press the a key to add arguments to the end of a kernel line, enter in a runlevel number.

systemd

systemd is the most widely used set of tools to manage system resources and services, which are referred to as units (name, type and configuration file) by systemd.

Types of systemd units:

service                 # Active system resources that can be initiated, interrupted and reloaded
socket                  # Filesystem- or network socket (units have corresponding service units)
deivce                  # Resolve configuration dependencies when hardware devices are identified by the kernel (udev rule must be present)
mount                   # Mount point definition in the filesystem
automount               # Mount point definition in the filesystem, but mounted automatically (units have corresponding mount units)
target                  # Grouping of other units, managed as a single unit
snapshot                # Saved state of the systemd manager

Note: A target type is often used to bring the system to a new state:

systemctl single unit commands:

# Start unit
systemctl start <unit>
# Stop unit
systemctl stop <unit>
# Restart unit
systemctl restart <unit>
# Shows the state of unit, including if it is running or not
systemctl status unit.service
# Shows active if unit is running or inactive otherwise
systemctl is-active unit.service
# Enables unit, that is, unit will load during system initialization
systemctl enable unit.service
# unit will not start with the system
systemctl disable unit.service
# Verifies if unit starts with the system
systemctl is-enabled unit.service

Note: systemctl can also control system targets.
# Manually alternate to a target
systemctl isolate <unit>.target

See Bootloader to change the default system target by adding it to the kernel parameters list or use:

# Change the default target
systemctl set-default <unit>.target
# Display the default target
systemctl get-default

Note: A default target should never point to to shutdown.target.

Unit configuration files:

/lib/systemd/system/    # Configuration files associated with every unit

systemctl unit inspection commands:

# List all available units
systemctl list-unit-files
# List available unit files with a specific type
systemctl list-unit-files --type=<type>
# List active units
systemctl list-units
# List active unit files with a specific type
systemctl list-units --type=<type>

systemctl power commands:

# Put the system in low power mode
systemctl suspend
# Copy all memory data to disk, so the current state of the system can be recovered after powering it off
systemctl hibernate
# Power off system
systemctl poweroff

Note: systemctl power event actions are logged in /etc/systemd/logind.conf or /etc/systemd/logind.conf.d/

Note: systemd power features can only be used when no other power manager is running (first check the status of acpid daemon).

Upstart

/etc/init/          # Initialization scripts used by Upstart

Upstart commands:

# List system services
initctl list

# Start service
start <service>
# Stop service
stop <service>
# Shows the state of service, including if it is running or not
status <service>

Note: Upstart does not use /etc/inittab to define runlevels, legacy commands runlevel and telinit are still usable.

Design Hard Disk Layout

Keeping Things Separated

/boot       # Bootloader-related files stored in boot partition
/home       # Personal user directories stored on separate partition
/var        # Data related to web/database server stored on separate partition
/           # Root filesystem on speedy SSD

The Boot Partition - /boot

Contains files used by the bootloader to load the operating system (safety: ensure the system will boot even in case of a root filesystem crash).

Usually located at the start of the disk
Size of around 300MB

Contents of the boot partition:

config-<version>            # Stores configuration parameters for the Linux kernel
System.map-<version>        # Look-up table matching symbol names to their corresponding position in memory (useful for debugging kernel panic)
vmlinux-<version>           # Operating system kernel
initrd.img-<version>        # Minimal root file system loaded into a RAM disk

EFI System Partition - ESP

Used by machines based on UEFI to store boot loaders and kernel images for the operating systems installed.

Formatted in FAT-based filesystem
Disk with GUID partition table (C12A7328-F81F-11D2-BA4B-00A0C93EC93B)
MBR partition ID: 0xEF
Mounted under /boot/efi

The Home Partition - /home

Each user in the system has a home directory to store personal files and preferences.

Same name as the username
Exception: /rootis the home directory of the root user

Variable Data - /var

Contains files and directories the system must be able to write to during operation.

Stability: process may misbehave and case kernel panic and filesystem corruption (the root filesystem will be unaffected in such cases).

/var/log            # System logs
/var/tmp            # Temporary files
/var/cache          # Application data
/var/www/html       # Data files for Apache Web Server
/var/lib/mysql      # Database files for the MySQL server

SWAP

The swap partition is used to swap memory pages from RAM to disk as needed.

Cannot be mounted
Alternative: use of swap files
See Standards for swap size

# Setup a swap partition
mkswap

LVM

Logical Volume Management (LVM) is storage virtualization that offers a flexible approach to managing disk space.

Physical Volume (PV): Block device on your system, disk partition or RAID array
Volume Groups (VG): Abstract underlying devices to be seen as a single logical device.
Extents: Each Logical Extent (LE) is mapped to a Physical Extent (PE).
Logical Volumes: Similar to partitions but more flexible.

# Structure of a volume group and logical volume
/dev/<vg-name>/<lv-name>

Install Boot Manager

Where is the Bootloader

MBR-partitioned disk	GPT-partitioned disk
GRUB boot code is installed on MBR	GRUB is loaded by `grubia32.efi` or `grubx64.efi` from ESP
Secondary boot loader (between MBR and first partition	Needs UEFI to boot
Max 26 partitions (3 partitions + 1 extended partition divided into max 23 logical partitions	Max 128 partitions
Max disk partition size of 2TB	Max disk partition size in the ZB (zettabyte) range

Interacting with GRUB

Shift       # Bring up the menu
← ↑ ↓ →     # Selecting menu options
Enter       # Confirming menu entry
E           # Edit an option
Esc         # Return to menu
Ctrl+x      # Boot
c           # Enter the GRUB shell
help        # Show list of available commands

GRUB Legacy

Installing GRUB Legacy from a running system:

# Install GRUB to a specific device
grub-install /dev/<disk>

# Locate the boot partition
findmnt /boot

Note: Executing grub-install on a up and running system is dangerous. It is typically installed to a new disk from a live CD/USB.

Installing GRUB Legacy from the GRUB shell:

# Invoke the GRUB shell environment
grub
# Print the help listing for GRUB, or get more info on a command
help <command>
# Search for a file in all partitions and list the device(s) the file is on
find
# Quit the GRUB shell environment
quit

# Find the device containing the /boot direcctory
find /boot/grub/stage1
# Set the boot partition
root (<disk>,<partition>)
# Boot the partition
setup (<disk>)
# Reboot the system when finished

Configuring GRUB Legacy

Boot loader related files:

/boot/grub/         # Location of bootloader related files
|__ menu.lst        # GRUB configuration file, contains menu entries
|__ i386-pc         # GRUB modules
|__ locale          # GRUB translation files
|__ fonts           # GRUB fonts

Note: Unlike GRUB2, /boot/grub/menu.lst can be edited directly.

# Basic syntax for a menu entry
title My Linux Distribution
root (hd0,0)
kernel /vmlinuz root=/dev/hda1
initrd /initrd.img

# Omit the root statement
title My Linux Distribution
kernel (hd0,0)/vmlinuz root=/dev/hda1
initrd /initrd.img

Note: Both disks and partitions are numbered from zero (Partitions in GRUB2 start from 1).

# Load modules (add line in menu entry)
module /boot/grub/i386-pc/<module>.mod

GRUB2

Installing GRUB2

# Locate the boot partition
fdisk -l /dev/sda

# Create a temporary directory under /mnt and mount the partition under it
mkdir /mnt/tmp
mount /dev/sda /mnt/tmp

# Run the install command pointing to the boot device (NOT PARTITION) and the temp directory where the boot partition is mounted
grub-install --boot-directory=/mnt/tmp /dev/sda
# If no boot partition is present, use /boot
grub-install --boot-directory=/boot /dev/sda

Configuring GRUB2

/boot/grub/         # Location of bootloader related files
|__ grub.cfg        # GRUB2 configuration file

# View the default boot entry for the grub configuration file as written in /boot/grub/grubenv
grub2-editenv list
# Create (or update) a /boot/grub2/grub.cfg file based on entries from the /etc/default/grub file
grub2-mkconfig

Note: manually changing grub.cfg is not recommended.

# Make changes to the GRUB2 config
vim /etc/default/grub
# Generate new config file after editing
update-grub

GRUB2 configuration file options:

GRUB_DEFAULT=<value>                    # Default menu entry to boot (number, name, saved)
GRUB_SAVEDEFAULT=<boolean>              # Default boot option will always be the last one selected in the boot menu
GRUB_TIMEOUT=<seconds>                  # Timeout before the default menu entry is selected
GRUB_CMDLINE_LINUX=<value>              # Lists command line options added to entries for the kernel
GRUB_CMDLINE_LINUX_DEFAULT=<value>      # Add extra parameters that will be added only to the default entry
GRUB_ENABLE_CRYPTODISK=<boolean>        # GRUB commands will look for encrypted disks to use during boot (disables automatic booting)

GRUB2 will scan for kernels and operating systems on the machine and generate corresponding menu entries on /boot/grub/grub.cfg.

/etc/grub.d                             # Directory to configure new menu entries
|__ 01_<entry-1>                        # Menu entry 1
|__ 02_<entry-2>                        # Menu entry 2
|__ 03_<entry-3>                        # Menu entry 3

# Basic syntax for a menu entry
menuentry "Default OS" {
    set root=(hd0,1)
    linux /vmlinuz root=/dev/sda1 ro quiet splash
    initrd /initrd.img
}

# Let GRUB2 search for a file with a specific UUID (ignore floppy disks)
search --set=root --fs-uuid <uuid> --no-floppy
# Let GRUB2 search for a file with a specific label
search --set=root --label=<label>

Note: find the UUID of a filesystem
ls -l /dev/disk/by-uuid/

Booting with GRUB2

Booting from GRUB2 Shell:

# Find out where the boot partition is and verify initrd is present in that partition
ls
# Set the boot partition
set root=(<disk>,<partition>)
# Load the Linux kernel 
linux /vmlinuz root=/dev/<root-filesystem-location>
# Load the initial RAM disk
initrd /initrd.img
# Boot the system
boot

Booting from the Rescue Shell:

# Find out where the boot partition is and verify initrd is present in that partition
ls
# Set the prefix
set prefix=(<disk>,<partition>)/boot/grub
# Load the modules
insmod normal
insmod linux
# Set the boot partition
set root=(<disk>,<partition>)
# Load the Linux kernel 
linux /vmlinuz root=/dev/<root-filesystem-location>
# Load the initial RAM disk
initrd /initrd.img
# Boot the system
boot

Package Management

RPM

RPM action commands:

# Install a package
rpm -i <package-name>
# Upgrade a package and install if no package was found
rpm -U <package-name>
# Only upgrade package
rpm -F <package-name>
# Remove an installed package
rpm -e <package-name>
# Ignore dependencies (try to avoid, unless 100% sure)
rpm --nodeps <package-name>
# Ignore errors (try to avoid, unless 100% sure)                 
rpm --force <package-name>
# Convert an rpm file to a cpio archive file: `rpm2cpio <package-name>.rpm | cpio -idmv`                        
rpm2cpio <package-name> 

Note: Extra parameters can be added to increase verbosity -v and installation progress tracking -h
rpm -ivh <package-name>

RPM inspection commands:

# List installed packages
rpm -qa
# Display package information
rpm -qi <package-name>
# List files in a package
rpm -ql <package-name>
# Verify installed packages
rpm -Va
# Find out which package owns a specific file
rpm -qf <file-name>

Note: Extra parameters can be added to view information for packages that are not installed yet -p
rpm -qlp <package-name>

RPM uses a database to store its .rpm files in the /var/lib/rpm directory.

# Rebuild a corrupted rpm database
rpm --rebuilddb

How to use RPM-build:

# Install required packages
yum install @"Development Tools" mock rmpdevtools rpmlint -y

# Add the mock group to user
usermod -a -G mock root

# Create rpmbuild directory tree
rpmdev-setuptree

# Get GNU hello world source files
cd /rpmbuild/SOURCES/
wget http://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz

# Build the SPEC file
cd ../SPECS/
rpmdev-newspec hello
vim hello.spec

# Build the rpm
rpmbuild -ba hello.spec

# Check the build success
ll ../SRPMS/
ll ../RPMS/x86_64/

Create a isolated environment on the server to test and verify rpm-builds with mock:

# Create an rpm test environment
mock --verbose ../SRPMS/<package>.rpm

YUM

YUM action commands:

# Install a package without prompting the user
yum -y install <package-name>                                          
# Update a package without prompting the user 
yum -y update <package-name>                                           
# Exclude a package while updating the whole system     
yum update --exclude=<package-name>
yum -x <package-name> update
# Install/update only security fixes                               
yum update --security                                              
# Upgrade the system with latest releases                      
yum upgrade -y                                                   
# Downgrade a package                    
yum downgrade <package-name>-<version>                                
# Remove a package                                      
yum remove <package-name>                                          
# Clear the information from the cache directory                   
yum clean all                                                                         

YUM inspection commands:

# List all packages
yum list all
# List installed packages
yum list installed
# Display package information
yum info <package-name>
# Check packages available for update
yum check update
# Find specific package
yum search <package-name>
# Display repository information
yum repolist all
# Display which package provides a specific file 
yum whatprovides <file-name>  

Managing software repositories:

# Add a repository to /etc/yum.repos.d/
yum-config-manager --ad-repo <repo-url>
# Enable updates to a repository
yum-config-manager --enable updates
# Disable updates to a repository
yum-config-manager --disable updates

Note: Yum stores downloaded packages and associated metadata in a cache directory (usually /var/cache/yum).

Note: Global yum configuration options: /etc/yum.conf.

DNF

DNF action commands:

# Install a package 
dnf install <package-name>                                          
# Update a package
dnf upgrade <package-name>                                                                               
# Remove a package                                      
dnf remove <package-name>                                                                                                                 

DNF inspection commands:

# List packages
dnf list [options] [<spec>...]
--all                           # All packages
--installed                     # Installed packages
--available                     # Available packages
--extras                        # extra packages (installed but unavailable in known repos)
--obsoletes                     # obselete packages (replacement in known repos)
--recent                        # recently added packages
--upgrades                      # upgradable packages
--autoremove                    # removed packages (dnf autoremove)

# Display package information
dnf info <package-name>
# List files in a package
dnf repoquery -l <package-name>
# Find specific package
dnf search <package-name>
# Display repository information
dnf repolist all
# Display which package provides a specific file 
dnf provides <file-name>

Note: <spec> specifies a transaction, range of transactions or a transaction by a package which it manipulated.

Note: dnf has a built-in help system, which shows more information for each command.
dnf help <command>

Display or change DNF actions:

# List information about given transactions in a table
dnf history [list] [<spec>...]
# List information about given transactions in a table in reverse order
dnf history [list] --reverse [<spec>...]
# Describe the given transactions
dnf history info [<spec>...]

# Repeat the specified transaction
dnf history redo <spec>

# Perform the same operations on packages as in the original transaction
dnf history replay [options] <filename>
--ignore-installed              # Don’t check installed packages in the same state as recorded in the transaction
--ignote-extras                 # Don’t check extra packages pulled into the transaction on the target system
--skip-unavailable              # Skip unavailable packages on the target system

# Undo all transactions performed after the specified transaction
dnf history rollback <spec>

# Store a specified transaction
dnf history store [options] <spec>
--output <output-file>          # Store the transaction into a file (default transaction.json)

# Perform the opposite operation to the specified transaction
dnf history undo <spec>

# Show all installonly packages
dnf history userinstalled
dnf repoquery --userinstalled

Note: <spec> specifies a transaction, range of transactions or a transaction by a package which it manipulated.

Managing software repositories:

# Add a repository
dnf config-manager --ad-repo <repo-url>
# Enable updates to a repository
dnf config-manager --set-enabled <repo-id>
# Disable updates to a repository
dnf config-manager --set-disabled <repo-id>

Managing Application Streams:

# Display details about a module
dnf module info [options] <module-name>:<version-number>
--profile                       # Print detailed information about module profiles

# Reports status of modules that are downloaded or available from a remote RHEL repo
dnf module list [options] <module-name>
--all                           # All packages
--installed                     # Installed packages
--available                     # Available packages
--obsoletes                     # obselete packages (replacement in known repos)

# Describes the repositories that provide a module
dnf module provides <spec>

# Provide system access to repo packages in a particular stream of a module
dnf module enable <spec>
# Remove system access to repo packages in a particular stream of a module
dnf module disable <spec>

# Switches the current installed module to the one defined
dnf module switch-to <module>:<stream>

# Remove installed module profiles
dnf module remove [options] <spec>
--all                           # Remove all installed module profiles

# Reset module state so it’s no longer enabled or disabled
dnf module reset <spec>

Note <pec> specifies a module name, module filepath or a particular version of a module.

NEXTLinux - Logical Volume Manager