System Logging

Logs record events emitted by the kernel, services, applications, and security components. They support troubleshooting and auditing, but only when collection is working, timestamps are understood, and the relevant source is included.

Following a Log Message

A logging path has several distinct parts:

1. A source emits an event.
2. A collector accepts and enriches it.
3. Routing and retention rules choose storage or forwarding destinations.
4. A viewer queries the stored records.

On a systemd host, systemd-journald commonly collects service standard output, kernel messages, and journal-native or syslog messages. A syslog daemon such as rsyslog may also receive messages and write traditional text files or forward them. Applications can instead maintain their own files or external telemetry.

Discovering Available Logs

Do not assume every host has the same files. Inspect the active logging services and local configuration:

$ systemctl --type=service --state=running | grep -E 'journal|syslog'
$ ls -la /var/log
$ journalctl --disk-usage

/var/log/syslog is common on Debian-family systems using compatible routing, while /var/log/messages is common elsewhere. Either may be absent on a journal-only host. Application documentation and unit configuration can identify additional destinations.

Querying the Journal

Start with a bounded query instead of dumping the entire journal:

$ journalctl -b -p warning
$ journalctl -u ssh.service --since '1 hour ago'

-b selects the current boot, -p filters by priority, and -u filters by a unit. Unit names and retained boots differ by host. Use journalctl --list-boots to see available boots and journalctl -f to follow new records while reproducing an issue.

Reading Records in Context

A traditional syslog-style line can look like:

Jan 27 07:41:32 icebox anacron[4650]: Job `cron.weekly' started

It contains a timestamp, host, program and PID, then a message. Treat message text as application output, not a guaranteed structured fact. Check timezone, clock synchronization, boot ID, PID reuse, and records immediately before and after the event. Journal fields can provide stronger identifiers than the rendered text alone.

Logs may contain usernames, addresses, paths, tokens, or other sensitive data. Use least-privilege access, redact exports, and preserve originals and timestamps during an investigation.

Summary

You can now locate and query Linux logs without assuming one universal storage path.

1. Separate event sources, collectors, routing, storage, and viewers.
2. Discover the host's active logging configuration.
3. Use bounded journal queries for a unit, boot, time, or priority.
4. Correlate records in context and protect sensitive log data.