Understanding Linux: Kernel Logs, Syslogs, Authentication Logs, and User Management

Alright, let’s break down Linux user management, authentication, and logging in a way that actually makes sense, especially if you’ve been on both Windows and Linux systems.

🔑 1. Unique Identifiers in Linux vs Windows

First off, let’s talk about how users are identified:

• Windows uses SIDs (Security Identifiers) — long strings like S-1-5-21-... to uniquely identify users.

• Linux, on the other hand, uses UIDs (User IDs) for users and GIDs (Group IDs) for groups.

👉 Quick Tip:

• Regular user accounts start from UID 1000 and above.

• Anything below 1000 is usually a system or service account (like daemon, syslog, etc.).

📂 2. Where Is User Info Stored in Linux?

🧾 /etc/passwd

This file holds basic user info:username, UID, GID, home directory, and shell (like /bin/bash or /usr/sbin/nologin).

cat /etc/passwd

You'll see entries like:

akash:x:1001:1001:Akash,,,:/home/akash:/bin/bash

🔐 /etc/shadow

This one is where the actual (hashed) passwords are stored — not in /etc/passwd. It’s restricted to root for a reason.

sudo cat /etc/shadow

You’ll notice hashed passwords that look something like this:

akash:$6$randomsalt$verylonghashedpassword:19428:0:99999:7:::

• That $6$ means it’s hashed using SHA-512.

• Other common password hashing algorithms in Linux: MD5, Blowfish, SHA-256, and SHA-512.

• All of these use salting and multiple hashing rounds for added security.

👥 3. Managing Users in Linux (Commands You’ll Actually Use)

Here are the most common commands:

Command	What It Does
useradd	Add a new user
userdel	Remove a user
usermod	Modify user properties
chsh	Change a user’s default shell
passwd	Set or change a user's password

All of this ties into how Linux handles access and sessions.

🛡️ 4. How Linux Handles Authentication (Thanks to PAM)

PAM stands for Pluggable Authentication Modules — and it’s the brain behind how Linux checks your credentials.

🗂️ Where are the PAM config files?

• /etc/pam.d/: Main directory for PAM config files for individual services (like sshd, login, sudo, etc.)

• /etc/security/access.conf: You can use this to allow or deny users based on IP, group, etc.

Example:

-:akash:ALL EXCEPT 192.168.1.100

This means Akash can only log in from the IP 192.168.1.100.

🧩 PAM Modules Location

These are .so files (shared object libraries) that do the heavy lifting.

• RHEL-based distros: /usr/lib64/security/

• Debian-based distros: /usr/lib/x86_64-linux-gnu/security/

Think of them like Linux’s version of Windows DLLs but for authentication logic.

🔄 How PAM Authentication Works (Step by Step)

1. User enters credentials (username + password).

2. System loads PAM config from /etc/pam.d/*.

3. Relevant modules get called from /usr/lib/....

4. Password is compared with the hash in /etc/shadow.

5. If valid, session gets started.

6. Authentication logs are written — usually in /var/log/auth.log or /var/log/secure.

7. Access is granted or denied.

   
   

Simple but powerful.

---------------------------------------------------------------------------------------

📜 5. Logging – Where to Look

🗂️ Where Does Linux Log Authentication Stuff?

Linux keeps logs under the /var/log/ directory — that’s the central place where you’ll find all sorts of system and user-related logs.

1. /var/log/auth.log (Debian/Ubuntu systems)

This is your go-to file when investigating:

• Logins via terminal, SSH, or sudo

• Session starts/stops

• Authentication failures and successes

Use tail -f /var/log/auth.log to monitor real-time logins.For older, compressed logs, use: zcat /var/log/auth.log.1.gz.

Also, fun fact: cron job logs land here too, because cron has to authenticate users before running scheduled tasks.

2. /var/log/secure (Red Hat/CentOS/Fedora systems)

Same purpose as auth.log but without cron logs.

So if you’re hunting down brute-force attempts or failed SSH logins on RHEL or CentOS, this is your place.

3. /var/log/failog

This one specifically logs failed login attempts, but here’s the twist:

• On Ubuntu/Debian, it’s there, but only if you configure pam_faillock.

• On RHEL-based systems, it’s often not enabled by default.

Use faillog -a to check all failed attempts.

4. /var/log/lastlog

Want to know when a user last logged in? Boom — this file’s got you covered.

Run lastlog -u akash to check last login time for user akash.

Neat for checking dormant accounts or for basic auditing.

5. /var/log/btmp

This file tracks failed login attempts, but it’s in binary format — so don’t try to cat it like a text file.

Use lastb or lastb -f /var/log/btmp to view it cleanly.

6. /var/log/wtmp

It logs all login/logout events, system reboots, and shutdowns.

Run last to read it.Or for a forensic dump from a dead system:

last -f /mnt/disk/var/log/wtmp

7. /run/utmp

This file is more “live.” It tracks users currently logged in.

Use who or w to view who's online right now.

🔍 How Long Are Logs Kept?

Linux usually keeps logs for 4 weeks by default, rotating them weekly. Older logs get compressed (you’ll see .gz files). So for deeper dives:

• Use zcat or zless to view archived logs

• Use strings, hexdump, or a hex editor to read binary logs like btmp, wtmp, or utmp in raw forensics scenarios

🧪 Quick Command Recap

Command	Purpose
last	Shows logins/logouts from /var/log/wtmp
lastb	Shows failed logins from /var/log/btmp
faillog -a	View all failed login attempts
who / w	Shows currently logged-in users (from utmp)
lastlog -u user	Shows last login info from lastlog

🧠 Bonus: How Syslog and Kernel Logging Works in Linux

Let’s talk Syslog — the backbone of Linux logging.

Syslog isn’t a file — it’s a standard protocol used by system processes to send log messages to a log server (or local file). It’s used across services, from SSH to cron, and it categorizes logs by:

• Facility (like auth, kernel, daemon)

• Severity (info, warning, error, critical)

Common Syslog Implementations:

• rsyslog (most common these days)

• syslog-ng

• journald (used with systemd)

🗂️ Key System Log Files

1. /var/log/syslog (Ubuntu/Debian)

This is like a catch-all system log. You’ll find:

• Kernel messages

• App-level logs

• Cron logs

• Hardware issues

It’s super useful when you’re not sure what exactly went wrong but want a timeline of everything.

Use tail -f /var/log/syslog or grep through it to find events.

2. /var/log/messages (RHEL/CentOS/Fedora)

Think of this as the Fedora-flavored syslog.

It logs similar data — services, kernel messages, application errors — but it’s the default log file for those distros.

Want to Go Pro Mode?

You can even forward logs to a central log server using rsyslog or syslog-ng. Perfect for SIEM integration or enterprise setups.

🚨 Tip: Watch Those Binary Logs

Files like wtmp, btmp, and utmp are not plain-text, so don’t expect to read them with cat. Either use the right commands (last, lastb, who, etc.) or open them in a hex editor when you’re in full forensic mode.

---------------------------------------------------------------------------------------

Let’s talk about something super important but often overlooked — kernel logs in Linux.

These logs are goldmines when it comes to diagnosing system-level problems like driver issues, boot errors, or hardware failures. But if you're like most people, kernel logging might feel a bit messy because of the variety of tools and file paths involved across distros. So, let's break it all down in plain language.

🧠 First Things First — What’s dmesg All About?

The dmesg command is your go-to tool when you want to see what the Linux kernel has been up to — especially right after booting. It shows stuff like:

• Hardware detection

• Driver loading

• Device initialization (like USBs, disks, network interfaces)

• Boot-time errors

Basically, if something is going wrong at the very core of your OS — this is where you'll see the first red flags.

🔧 How to Use It?

Just pop open a terminal and run:

dmesg

You’ll see a big wall of text. You can pipe it to less or grep for easier viewing:

dmesg | grep -i error

Now here’s the catch: this output comes from the kernel ring buffer — which is in memory. That means once the system reboots, poof — it’s gone unless you’ve saved it.

📁 But Wait — Where Is This Stuff Saved?

On some distros, it actually is saved!

🗂️ /var/log/dmesg (Only on Some Systems)

This file captures the output of dmesg from the last boot and stores it permanently. You can just run:

less /var/log/dmesg

But don’t count on it being available on all systems. For example:

• Debian/Ubuntu: You’re likely to find it.

• Fedora/RHEL/Rocky: Nope. They use journald instead (more on that in a sec).

📚 Enter /var/log/kern.log — The Persistent Hero

Now this one’s interesting.

The kern.log file contains all kernel messages — just like dmesg — but it sticks around even after rebooting.

That means you can go back and check what the kernel was doing a week ago if your system started acting weird after an update or new hardware installation.

View it like this:

less /var/log/kern.log

🕒 Bonus: Time Stamps

Unlike the raw dmesg output that shows uptime seconds (like [ 3.421546]), the messages in kern.log come with real human-readable timestamps, making it way easier to match with user events.

🧰 What About Fedora, RHEL, Rocky Linux?

Alright, now let’s talk Fedora-style distros. They’ve moved away from traditional log files and now rely heavily on systemd's journald service.

So files like /var/log/kern.log or /var/log/dmesg? You won't find them here.

Instead, everything is logged into the system journal.

✅ How to View Kernel Logs?

journalctl -k

That gives you only kernel logs from the journal. Super clean, super easy.

You can also use filters like:

journalctl -k --since "2 days ago"

---------------------------------------------------------------------------------------

💾 Persistent vs Non-Persistent Logs (This Is Crucial!)

Systemd-based distros can either store logs in memory (volatile) or on disk (persistent). Whether or not your logs survive reboot depends on how journald is set up.

• If logs are stored in /var/log/journal, they’re persistent.

• If not, and only in /run/log/journal, they’re gone after reboot.

So if you're doing forensics on a deadbox, you’d better hope /var/log/journal exists.

---------------------------------------------------------------------------------------

💀 Deadbox Log Hunting Tips

If you're working on a dead system and trying to dig into what happened before it went down:

1. Mount the disk using a live CD or forensic OS.

2. Navigate to /var/log/journal/ if it exists.

3. Use journalctl --directory=/mount/path/var/log/journal to view the logs.

   
   

If nothing's there?

You're kinda outta luck unless you've got other artifacts like /var/crash/, old syslog exports, or even swap memory to analyze.

---------------------------------------------------------------------------------------

🔚 Final Thoughts

Whether you’re on a live system, doing forensics, or trying to fix a misbehaving server, don’t forget:

Logs don’t lie — you just need to know where they’re hiding.

--------------------------------------Dean------------------------------------------