LINUX | DEVOPS | SYSADMIN | TECHNOLOGY

Files That Run a Linux System (Every Engineer Should Know)

The essential files that define system behavior and control

Stop Memorizing Linux. Start Understanding It.

If you work with Linux long enough, you realize it’s less about memorizing commands and more about understanding how the system actually works.

Everything is File

On Linux, one of the most important troubleshooting principles is simple: everything is a file.

Services, processes, networking, devices, kernel settings, and system behavior are all exposed through files somewhere in the system. Once we understand the core files behind them, Linux stops feeling like a black box. Instead of guessing which command to run, we can inspect the files that actually own the data and control the behavior.

In this article, we’ll explore the essential Linux files every engineer should know to better understand system behavior, configuration, and kernel control.

#1. The Files That Decide How Linux Starts

Before Linux can run services, launch applications, or accept logins, it must first understand how the system should start. This behavior is controlled by configuration files that define boot parameters, mounted filesystems, kernel options, and startup processes.

/etc/fstab — Filesystem Mount Table

This file defines what gets mounted at boot, where, and with what options.

# <device>   <mount point>   <filesystem type>   <options>   <dump>   <fsck order>
UUID=abc123  /         ext4  defaults               0 1   # Root filesystem mounted at boot
UUID=def456  /data     xfs   defaults,noatime       0 2   # Data partition with no access-time updates for better performance
tmpfs        /tmp      tmpfs mode=1777,size=2G      0 0   # Temporary RAM-based filesystem for /tmp

A misconfigured fstab file can prevent Linux from booting properly because the system relies on it to mount essential filesystems during startup. If a critical entry is wrong, the boot process may fail or drop into emergency mode.

Common dangerous misconfigurations:

• Wrong UUID — system cannot find the correct partition, which may cause boot failure or emergency mode.
• Wrong mount point — filesystem is mounted in the wrong location, making important directories inaccessible or broken.
• Wrong filesystem type — mount fails because the specified type (e.g. xfs, ext4) does not match the actual filesystem.
• Invalid mount options — unsupported or incorrect options can prevent the filesystem from mounting properly.
• Missing nofail for optional disks — boot may hang or fail if a non-essential disk is disconnected or unavailable.
• Incorrect fsck order (0 1, 0 2) — improper filesystem check order can skip integrity checks or delay boot unexpectedly.

Important: Always test any changes using mount -a, which attempts to mount everything defined in fstab without requiring a reboot. This lets you catch errors early while the system is still running normally.

/etc/default/grub —The Boot Configuration

When a Linux server boots, the Grand Unified Bootloader (GRUB) loads the Linux kernel from disk into memory and then starts it, allowing the operating system to begin booting.

/etc/default/grub is the human-editable configuration file where boot options like timeout, default OS, and kernel parameters are set. These settings are then used to generate /boot/grub/grub.cfg, which GRUB actually reads during system boot.

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""

# Uncomment to disable graphical terminal (sometimes useful for troubleshooting)
#GRUB_TERMINAL=console

# Uncomment to change screen resolution in GRUB
#GRUB_GFXMODE=1024x768

# Uncomment to enable OS prober (detect other OS like Windows)
GRUB_DISABLE_OS_PROBER=false

Common dangerous /etc/default/grub misconfigurations:

• Wrong GRUB_TIMEOUT (e.g., 0) — menu disappears too fast, making it hard to select recovery or another kernel.
• Incorrect GRUB_DEFAULT — system may boot into the wrong kernel or an unavailable entry, causing boot loops or unexpected behavior.
• Bad GRUB_CMDLINE_LINUX parameters — invalid kernel options (e.g., wrong root=, broken quiet, nomodeset misuse) can prevent the system from booting properly.
• Broken quotes in kernel parameters — missing or mismatched quotes can cause GRUB to fail generating a valid config.
• Wrong GRUB_GFXMODE — unsupported resolution can result in a blank or unreadable boot screen.
• Disabling OS detection incorrectly (GRUB_DISABLE_OS_PROBER) — may hide other operating systems like Windows in dual-boot setups.
• Forgetting to run update-grub — changes in /etc/default/grub won’t apply to /boot/grub/grub.cfg, so the system boots with old settings.

Important: Always edit /etc/default/grub, not /boot/grub/grub.cfg, as the latter is automatically generated and will be overwritten. After changes, run update-grub (or grub-mkconfig -o /boot/grub/grub.cfg) to apply them.

#2. User, Identity, and Access Control

Linux is a multi-user system at its core. Every process runs as someone, every file belongs to someone, and every action is checked against permissions defined somewhere in the system.

[embed]Linux Permissions: Users, Groups, ACLs, and Sudo for Security Learn Linux users, groups, and ACLs to control access, enforce privilege boundaries, and keep our system safe from…medium.com

/etc/passwd — The User Database

Stores basic user account information. It maps every user on the system: username, UID, GID, home directory, and login shell.

Understanding this file is critical for auditing who can actually log into a machine.

root:x:0:0:root:/root:/bin/bash
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
deploy:x:1001:1001::/home/deploy:/bin/bash

root:x:0:0:root:/root:/bin/bash :

• root → superuser (full system control)
• x → password is stored in /etc/shadow (not here)
• UID 0 → special ID for root (always admin)
• GID 0 → root group
• comment → “root” (description field)
• /root → home directory
• /bin/bash → login shell (can open terminal normally)

Important: We do not edit /etc/passwd directly. Instead, we use safe tools like useradd, usermod, vipw, or passwd to manage user accounts, and these tools update the file automatically.

/etc/shadow — Hashed Passwords and Aging Policy

This file stores the actual password hashes and can only be read by root. In addition to the hash, each entry includes password aging information such as the last password change date, minimum and maximum password age, warning period before expiry, and account expiration date.

deploy:$6$rounds=5000$salt$hash...:19200:0:90:7:::

If we are troubleshooting a locked-out user or enforcing password rotation policies, this is the file we need to understand, as it controls password hashes and aging rules that determine when and how a user’s password expires or must be changed.

Important: We do not edit /etc/shadow directly. It should be managed using tools like passwd and chage, which safely update password hashes and password aging settings without risking system lockouts or file corruption.

/etc/group — User Groups and Permission Membership

Defines system groups and group memberships. Groups are used to manage shared permissions across users and services.

sudo:x:27:deploy,admin
www-data:x:33:
developers:x:1002:alice,bob

sudo:x:27:deploy,admin :

• sudo → The name of the group.
• x → Group passwords (rarely used), stored in /etc/gshadow (not here)
• 27 → Group ID
• deploy,admin → Users who are supplementary members of the group

Understanding this is important to manage user access and permissions in Linux, since groups determine which users can access specific files, run certain commands, and use system resources securely and correctly.

Important: We do not edit /etc/group directly. Instead, we use safe commands like groupadd, usermod, and gpasswd, which update the file automatically and help avoid syntax errors or permission issues.

/etc/sudoers (and /etc/sudoers.d/) — Sudo Privilege Configuration

[embed]Your sudo Is Not Safe — Here’s Where It All Goes Wrong A sysadmin’s guide to understanding sudo internals, common misconfigurations, and security hardening techniques that…medium.com

This file controls which users or groups are allowed to run commands with elevated (root) privileges using sudo, and controls what commands they can execute.

%admin  ALL=(ALL) ALL

• All users in admin group have sudo access

Modern systems use /etc/sudoers.d/ for drop-in files — cleaner and less risky than editing the main file. Auditing this file regularly is a basic security hygiene requirement.

Important: We don’t edit /etc/sudoers directly. Instead, we use sudo visudo, which safely validates the syntax before saving to prevent breaking sudo access.

/etc/ssh/sshd_config — SSH Server Configuration

[embed]SSH (Secure Shell): A Beginner’s Guide A complete guide to SSH basics, secure connections, key management, and advanced techniques for modern development.medium.com

This file defines how the SSH server runs, including login settings, authentication methods, allowed users, ports, and security rules for remote access. The defaults are rarely what we want in production.

Key directives every engineer should know:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy admin
Port 22

• Disabling password authentication and root login, and restricting which users can SSH in, is table-stakes hardening.

Important: After any change, test with sshd -t to validate syntax, and always keep a second terminal session open before restarting the daemon.

/etc/pam.d/

This directory contains configuration files for PAM (Pluggable Authentication Modules), which control how users are authenticated for services like login, sudo, SSH, and password changes.

# Example: /etc/pam.d/sshd

auth       required     pam_sepermit.so
auth       include      common-auth
account    include      common-account
password   include      common-password
session    include      common-session

• auth → checks user identity (password, keys, etc.)
• account → checks if user is allowed to log in
• password → controls password rules and changes
• session → sets up login session (limits, environment)

Note: We don’t edit PAM files unless necessary because incorrect changes can break login, SSH access, and sudo authentication, potentially locking users out of the system.

#3. Services and Process Management

Modern Linux systems run most components as background services (web servers, databases, schedulers, containers, and networking), managed by configuration that controls how they start, restart, log, depend on each other, and behave during boot.

/etc/systemd/system/ — System Service Configuration

This is where we define and manage custom system services that define how services run, including the executable, restart behavior, dependencies, environment variables, and startup behavior at boot.

[Unit]
Description=My Application
[Service]
ExecStart=/usr/local/bin/myapp
Restart=always
[Install]
WantedBy=multi-user.target

Understanding this is important to know which services start at boot, how they run, and how they are managed (start, stop, restart, and depend on other services).

/usr/lib/systemd/system/

Contains the default systemd service unit files provided by installed packages, defining how system services are originally configured to run.

Important: We don’t edit files in /usr/lib/systemd/system/ directly. Instead, we create or modify overrides in /etc/systemd/system/ so changes are safe and not overwritten by package updates.

/etc/crontab —System Cron Schedule

This file defines scheduled tasks that run automatically at specific times or intervals, and it includes the command, time schedule, and the user who runs each job.

Cron remains one of the simplest ways to automate recurring jobs such as backups, cleanup scripts, monitoring tasks, or report generation.

0 2 * * * root /usr/local/bin/backup.sh

Important: We don’t edit */etc/crontab directly in production systems. Instead, we use `crontab -e* or drop-in files in/etc/cron.d/` to safely manage scheduled tasks without breaking system-wide schedules.

#4. Networking

Linux networking is managed through configuration files, tools, and services that control IP addresses, routing, DNS, interfaces, and connectivity between systems and the internet.

[embed]Mastering Linux Networking Learn Linux networking concepts, tools, and configurations.blog.devops.dev

/etc/hosts — Local DNS Override

Before DNS resolvers are consulted, the system checks /etc/hosts. This file maps hostnames to IP addresses locally on the system, allowing name resolution without querying external DNS servers. It’s often used for testing or internal routing

127.0.0.1   localhost
127.0.1.1   myserver
192.168.1.10  web.local
192.168.1.20  db.local

• 127.0.0.1 localhost → our own machine
• 127.0.1.1 myserver → local system hostname
• 192.168.1.10 web.local → maps a name to a server IP (no DNS needed)
• 192.168.1.20 db.local → database server shortcut name

Note: We can edit /etc/hosts directly (unlike /etc/shadow or /etc/passwd), the changes take effect immediately without restarting any service, but incorrect entries can break access to websites or services.

/etc/resolv.conf — DNS resolver configuration.

This file defines which DNS servers the system uses to translate domain names (like google.com) into IP addresses for network access.

nameserver 8.8.8.8
nameserver 8.8.4.4
search localdomain

• nameserver 8.8.8.8 → primary DNS server (Google DNS)
• nameserver 8.8.4.4 → backup DNS server
• search localdomain → auto-appends domain when resolving short names

Note: On modern systems, this file is often generated automatically by tools like systemd-resolved or NetworkManager.

/etc/network/interfaces (or /etc/netplan/*.yaml) — Network Configuration

These files define how network interfaces are configured, including IP addresses, gateways, DNS, and whether settings are static or dynamic (DHCP).

• /etc/network/interfaces — Network Configuration (Debian-based)
• /etc/netplan/*.yaml — Modern Network Configuration (Ubuntu)

Note: We can edit /etc/network/interfaces or /etc/netplan/*.yaml to configure networking, but changes must be applied using tools like ifdown/ifup or netplan apply, and incorrect settings can break network connectivity.

#5. Package Management

Installing software on Linux is usually just a single command away, but behind that simplicity is a set of files controlling repositories, package sources, verification rules, and software metadata.

[embed]Installing and Managing Programs in Linux Learn how to install, update, and remove programs efficiently using Linux package managers.blog.devops.dev

/etc/apt/sources.list (and /etc/apt/sources.list.d/) — Defines software repositories

On Debian-based distributions, these files define the package repositories the system trusts and downloads software from.

deb http://archive.ubuntu.com/ubuntu jammy main restricted

• This line tells Ubuntu to use the specified repository when we run apt update or install packages with apt install, so it knows where to download software for Ubuntu 22.04 (Jammy) from the main and restricted sections.

Additional repositories, such as third-party software sources, are stored as separate files inside **/etc/apt/sources.list.d/**. Instead of placing everything in one file, Ubuntu keeps each extra software source in its own file to keep the system organized and easier to manage.

Note: We can directly edit files in /etc/apt/sources.list.d/, but it must be done carefully because incorrect repository entries can break package installation or updates, and changes should always be followed by apt update to validate them.

Different Linux systems store software sources in different locations, but all of them serve the same purpose. On RHEL-based distributions, repository definitions are typically stored in **/etc/yum.repos.d/** as .repo files.

Understanding this is important to manage software sources on Ubuntu, so the system can correctly install, update, and maintain packages from both official and third-party repositories.

/var/lib/dpkg/ (or /var/lib/rpm/) — Package Database Storage

This directory stores the system’s package database, tracking all installed software, versions, and package metadata so the package manager knows what is installed and how to manage updates or removals.

# Example content (simplified)
Package: nginx
Status: install ok installed
Version: 1.18.0
Description: high performance web server

Package: curl
Status: install ok installed
Version: 7.81.0

Important: We do not edit /var/lib/dpkg/ or /var/lib/rpm/ directly. We manage them with package management tools like apt, dpkg, dnf, and rpm.

/usr/bin/ (and /usr/local/bin/ )— User Commands (Executable Programs)

These directories contain executable programs (commands) that we can run from the shell like ls, curl,python3,vim

• /usr/bin/ usually contains distribution-managed applications
• /usr/local/bin/ is commonly used for manually installed tools and custom scripts

Important If both contain the same command name, the system usually runs the one in /usr/local/bin/ first (because it has higher priority in $PATH).

#6. System and Runtime Behavior

One of the most unique things about Linux is how much of the operating system is exposed directly through files.

Instead of hiding system state behind proprietary interfaces, Linux makes kernel settings, process information, hardware details, runtime metrics, and device behavior accessible through virtual filesystems. Once you understand where this information lives, Linux becomes far easier to inspect, debug, and control.

/etc/sysctl.conf — Kernel Runtime Parameter Configuration

[embed]/proc: The Linux Kernel Interface Every Engineer Should Know Discover Linux Internals and Kernel Tuning Techniquesmedium.com

This file is used to configure Linux kernel parameters such as networking, security, memory behavior, and system limits, which are applied at boot or manually using sysctl.

# Enable IP forwarding (used for routing/NAT)
net.ipv4.ip_forward = 1
# Improve network security (ignore ICMP broadcast requests)
net.ipv4.icmp_echo_ignore_broadcasts = 1
# Disable source routing (security hardening)
net.ipv4.conf.all.accept_source_route = 0
# Increase maximum open file handles
fs.file-max = 100000

Important: Changes must be applied using *sysctl -p*, and incorrect settings can impact system stability or break networking.

/etc/environment — System-wide Environment Variables

This file defines global environment variables (like PATH, LANG, JAVA_HOME) that are applied to all users and sessions on the system.

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"
JAVA_HOME="/usr/lib/jvm/java-11-openjdk"
LANG="en_US.UTF-8"

Understanding this is important to control system-wide environment settings, such as command paths, language, and application variables, that affect how all users and programs run on the system.

/etc/profile (and /etc/profile.d/) — System-wide Login Environment Scripts

This directory contains shell scripts that are executed when a user logs into the system, used to set environment variables and configure the shell environment for all users.

• /etc/profile → Main system-wide login shell configuration file
• /etc/profile.d/ — Extra modular scripts loaded by /etc/profile

# /etc/profile.d/custom.sh
export NODE_ENV=production
export PATH=$PATH:/opt/custom/bin

It is used for more complex setup (such as logic, exports, and conditions) than /etc/environment, because it supports shell scripts that run during user login.

~/.bashrc — User-specific Shell Configuration

[embed]Bash: An Essential Skills for Automation and Devops Mastering Bash (or at least having a very strong proficiency) is absolutely crucial for becoming truly proficient with…blog.devops.dev

This file contains personal settings for an individual user’s Bash shell, which runs every time a new interactive terminal is opened.

# User aliases
alias ll='ls -la'
alias update='sudo apt update && sudo apt upgrade'

# Environment variables
export EDITOR=nano

# Customize prompt
export PS1="\u@\h:\w$ "

~/.bashrc is our personal terminal setup file that controls how our shell behaves for our user only. Understanding this is important to customize the terminal environment, manage aliases, environment variables, and improve daily command-line workflow.

#7. Logs & Debugging

When Linux behaves unexpectedly, the system is usually already telling us what went wrong.

Linux logs record system events, errors, and application activity, helping administrators monitor system behavior, troubleshoot issues, and debug failures.

/var/log/syslog or /var/log/messages — System Log Files

These files store general system activity logs, including kernel messages, service events, and application errors, used for monitoring and troubleshooting system behavior.

May 22 10:15:01 server systemd[1]: Started OpenSSH server daemon.
May 22 10:15:05 server sshd[1023]: Accepted password for deploy from 192.168.1.50
May 22 10:16:10 server kernel: [12345.678] CPU temperature high warning
May 22 10:17:20 server systemd[1]: nginx.service: Failed with result 'exit-code'

Understanding this is important to monitor system activity, detect errors, and troubleshoot issues by reviewing what the system, services, and applications have done over time. This is usually the first place we check when diagnosing a problem.

Important: We don’t edit these files. They are automatically generated and can only be read or filtered using tools like *cat, `less*, orjournalctl`.

[embed]Journalctl: Diagnosing Linux issues the right way Stop rebooting your server to fix problemsblog.devops.dev

/var/log/auth.log — Authentication Log

This file records all authentication-related events, such as user logins, sudo usage, SSH access, and failed login attempts, and is used to monitor security and detect unauthorized access.

May 22 09:10:15 server sshd[1201]: Accepted password for deploy from 192.168.1.50 port 54321 ssh2
May 22 09:12:40 server sudo: deploy : TTY=pts/0 ; PWD=/home/deploy ; USER=root ; COMMAND=/bin/systemctl restart nginx
May 22 09:15:22 server sshd[1205]: Failed password for root from 192.168.1.60 port 53210 ssh2
May 22 09:16:10 server sshd[1205]: Invalid user admin from 192.168.1.60

Understanding this is important to track user login activity, detect failed or unauthorized access attempts, and investigate security-related issues such as sudo usage and SSH authentication problems.

[embed]SSH Logs: Detect Attacks, Intruders, and Odd Behavior Turn raw SSH logs into actionable security insightsblog.devops.dev

/var/log/dmesg — Kernel Ring Buffer Log

This file contains boot-time and hardware-related messages from the Linux kernel, including device detection, drivers, memory, and system initialization events.

[    0.000000] Linux version 6.x.x
[    0.001234] BIOS-provided physical RAM map:
[    0.123456] CPU0: Intel(R) Core(TM) i7 detected
[    1.234567] usb 1-1: new high-speed USB device detected
[    2.345678] EXT4-fs (sda1): mounted filesystem
[    3.456789] systemd[1]: Started Load Kernel Modules.

Understanding this is important to diagnose boot problems, check hardware detection issues, and review kernel-level messages related to devices, drivers, and system startup.

Important: This file is used for viewing or troubleshooting hardware and boot issues using tools like *dmesg or `journalctl -k`*.

Why These Matter

The files we’ve explored above form the control plane of a Linux system, meaning they define and control how the system boots, runs, is configured, secured, and managed across services, users, networking, packages, and runtime behavior.

• Boot issues → GRUB, /etc/fstab
• Login/permissions → /etc/passwd, /etc/shadow, /etc/group, PAM, SSH
• Network issues → /etc/hosts, /etc/resolv.conf, Netplan/interfaces
• Package issues → APT/DNF repos, /var/lib/dpkg/, /var/lib/rpm/
• Service issues → systemd unit files (/etc/systemd/system/, /usr/lib/systemd/system/)
• Cron issues → /etc/crontab, /etc/cron.d/, /var/spool/cron/
• Performance → /etc/sysctl.conf
• Environment issues → /etc/environment, /etc/profile, /etc/profile.d/, ~/.bashrc
• Logging/debugging → /var/log/*, journalctl, /var/log/dmesg

Understanding this makes us better at Linux because instead of guessing commands, we can trace the actual system files that control behavior, quickly identify the root cause of issues, and apply the correct fix with confidence.

Final Thoughts

Linux becomes easier to understand when we stop seeing it only as a set of commands and start seeing it as a system driven by files. Most of what the system does is controlled by key configuration files, system directories, and runtime interfaces that quietly define how everything works behind the scenes.

Once we know which files matter, troubleshooting becomes more predictable, configuration becomes less intimidating, and Linux stops feeling like a black box.

Thanks for reading! Hope this article helped us better understand the files that quietly power and control Linux systems behind the scenes.

Any other Linux files worth mentioning that engineers should understand?