You Know, I Learned Something Today #3

When troubleshooting or auditing services on a Linux system, especially RPM-managed ones, it’s useful to understand how a service is running, where its unit file comes from, and whether it has been modified. This short guide walks through a practical workflow using Fluent Bit as an example.

1. Check How the Service Is Running

Start by inspecting the systemd unit and its runtime status.

systemctl show -p FragmentPath fluent-bit.service

What this tells you

• FragmentPath shows the exact location of the unit file systemd is using.
• For RPM-installed services, this is often under /usr/lib/systemd/system/.

You can also inspect the unit file directly:

less /usr/lib/systemd/system/fluent-bit.service

2. Find Which RPM Owns the Service File

To determine which package installed the service:

rpm -qf /usr/lib/systemd/system/fluent-bit.service

Example output

fluent-bit-3.2.8-2.el9.x86_64

This confirms the service file is managed by the fluent-bit RPM.

3. Verify the Service File Against the RPM

To check whether the service file has been modified since installation:

rpm -V fluent-bit-3.2.8-2.el9.x86_64 | grep fluent-bit.service

Possible outputs

• No output → file is unchanged
• Output with flags like S, M, or 5 → size, mode, or checksum differs

Example:

..5....T.  /usr/lib/systemd/system/fluent-bit.service

This indicates the file content has changed.

4. Extract the Original Service File From the RPM (Without Installing)

If you want to compare against the original file shipped in the RPM, you can extract it safely.

rpm2cpio fluent-bit-3.2.8-2.el9.x86_64.rpm | cpio -idmv \
./usr/lib/systemd/system/fluent-bit.service

Now you have the original unit file exactly as shipped by the package.

5. Inspect How the Service Starts

To see the command systemd uses to start Fluent Bit:

grep ExecStart /usr/lib/systemd/system/fluent-bit.service

This is especially useful when debugging startup failures or verifying arguments.

6. Check File Ownership, Permissions, and Timestamps

To audit the service directory or unit file metadata:

stat -c '%y %U %G %n' /usr/lib/systemd/system/fluent-bit.service

Or for the whole directory:

stat -c '%y %U %G %n' /usr/lib/systemd/system/*

This helps identify unexpected permission or ownership changes.

7. Check for Timers Related to the Service

Some services are triggered by systemd timers instead of running continuously.

systemctl list-timers --all | grep fluent

If nothing appears, the service is likely started via standard service units rather than timers.

8. Search Systemd Unit Files for References

To find references to Fluent Bit across systemd configuration:

ausearch -f /usr/lib/systemd/system/fluent-bit.service

This can be helpful in audit or security contexts.

Summary

This workflow helps you:

• Identify where a service unit comes from
• Confirm which RPM owns it
• Detect local modifications
• Recover the original unit file
• Understand exactly how the service is started

These steps are invaluable when debugging production issues, validating system integrity, or documenting service behavior in DevOps environments.