Logs and pipes
By the end of this lesson you can save a command's output and errors to files, chain small commands into a pipeline that answers a question, follow a log as it grows, and look up a service's log in the system journal.
You have already used > to make files and | to filter ps in the earlier lessons. Now you will see how they really work.
A sample log to work with
A log is a file a program writes to as it runs, one line per event, usually with a time and a level such as INFO, WARN or ERROR. Make a small one in your practice folder by pasting this whole block:
cd ~/practice
cat > app.log <<'EOF'
2026-11-09 09:00:01 INFO server started on port 8080
2026-11-09 09:00:05 INFO GET /home 200
2026-11-09 09:00:07 WARN slow response from database (1200 ms)
2026-11-09 09:00:09 ERROR database connection refused
2026-11-09 09:00:12 INFO GET /login 200
2026-11-09 09:00:15 ERROR payment service timeout
2026-11-09 09:00:18 INFO GET /home 200
2026-11-09 09:00:20 ERROR database connection refused
EOFcat > app.log <<'EOF' means "write everything up to the line EOF into app.log". It is a handy way to create a file with several lines.
Two output streams: stdout and stderr
Every command has two separate ways to print:
┌──────────┐ ── stdout (1): normal output ──▶ screen
keyboard ─────▶ │ command │
stdin (0) └──────────┘ ── stderr (2): error messages ─▶ screen- stdout, standard output, number 1: the results.
- stderr, standard error, number 2: the complaints.
- stdin, standard input, number 0: what the command reads, the keyboard unless you say otherwise.
On screen they look the same. You can see they are different when you ask ls for one file that exists and one that does not:
sam@laptop:~/practice$ ls notes.txt app.log
ls: cannot access 'notes.txt': No such file or directory
app.logThe first line came on stderr, the second on stdout.
Redirection: send output to a file
| Symbol | Example | What it does |
|---|---|---|
> | ls > out.txt | stdout to a file. Replaces what was in it |
>> | echo done >> out.txt | stdout to the end of a file. Keeps what was there |
2> | ls 2> err.txt | stderr to a file |
> file 2>&1 | ls > all.txt 2>&1 | Both to the same file |
2> /dev/null | ls nope 2> /dev/null | Throws errors away. /dev/null is a bin that swallows everything |
< | sort < names.txt | Reads stdin from a file |
sam@laptop:~/practice$ ls notes.txt app.log > out.txt
ls: cannot access 'notes.txt': No such file or directory
sam@laptop:~/practice$ cat out.txt
app.log
sam@laptop:~/practice$ ls notes.txt app.log > all.txt 2>&1
sam@laptop:~/practice$ cat all.txt
ls: cannot access 'notes.txt': No such file or directory
app.logThe error still reached the screen in the first command, because only stdout went to the file. 2>&1 means "send stream 2 wherever stream 1 is going", so it must come after > all.txt.
WARNING
> empties the file before the command even starts. sort app.log > app.log leaves you with an empty app.log. Write to a new file, or use >> when you mean to add.
Pipes: one command's output is the next one's input
The | (pipe) connects the stdout of one command to the stdin of the next:
grep ERROR app.log ──▶ cut -d' ' -f4- ──▶ sort ──▶ uniq -c ──▶ sort -rn
the error lines the message only groups counts biggest
repeats each group firstEach command does one small job. Together they answer a real question: which error happens most?
sam@laptop:~/practice$ grep ERROR app.log | cut -d' ' -f4- | sort | uniq -c | sort -rn
2 database connection refused
1 payment service timeoutBuild a pipeline one step at a time: run the first command, look at the output, add | and the next command, run it again. Here are the tools you will reach for most:
| Command | What it does | Example |
|---|---|---|
grep word | Keeps lines containing word | grep ERROR app.log |
grep -v word | Keeps lines not containing it | grep -v INFO app.log |
grep -i, -c, -E | Ignore case; count matching lines; allow a|b patterns | grep -E 'WARN|ERROR' app.log |
wc -l | Counts lines | grep ERROR app.log | wc -l |
sort | Sorts lines; -n by number, -r reversed | sort -rn |
uniq -c | Collapses repeated neighbouring lines and counts them. Sort first | sort | uniq -c |
head, tail | The first or last lines | tail -n 1 |
cut -d' ' -f4- | Keeps fields 4 onwards, split on spaces | The message part of each line |
awk '{print $3}' | Prints the third word of each line | The level column |
A few questions answered from the sample log:
sam@laptop:~/practice$ grep -c ERROR app.log
3
sam@laptop:~/practice$ awk '{print $3}' app.log | sort | uniq -c
3 ERROR
4 INFO
1 WARN
sam@laptop:~/practice$ grep -i database app.log | tail -n 1
2026-11-09 09:00:20 ERROR database connection refusedFollow a log as it grows: tail -f
Logs grow while programs run. tail -f (follow) prints the last lines and then waits, showing each new line as it is written:
tail -f app.logOpen a second terminal, go to the same folder and add a line:
echo "2026-11-09 09:01:00 ERROR disk full" >> app.logThe line appears in the first terminal straight away. To see only new errors, filter the stream: tail -f app.log | grep ERROR. Press Ctrl + C to stop following; the log itself is not touched.
tail -F (capital F) keeps following when a log is rotated, that is, renamed and replaced with a fresh file, which many systems do every day or week.
The system journal: journalctl
Services managed by systemd (from Processes and ports) write to the journal, a central log that journalctl reads:
| Command | Shows |
|---|---|
journalctl -u cron | The log of one service (-u for unit) |
journalctl -u cron -n 20 | Its last 20 lines |
journalctl -f | New lines from everything, as they arrive, like tail -f |
journalctl -p err -b | Only errors and worse, since the last boot |
journalctl --since "1 hour ago" | Everything from the last hour |
sam@laptop:~$ journalctl -u cron -n 2
Nov 09 17:30:01 laptop CRON[755534]: pam_unix(cron:session): session closed for user root
…Each line starts with the time, the machine name, and the program with its PID in square brackets. Older systems and some programs also write plain files under /var/log, which you read with less, grep and tail.
If journalctl shows only your own programs' messages, your user is not allowed to read the system journal. On Ubuntu, members of the adm group can; otherwise use sudo journalctl ….
Many logs at once: LogLens
When you have an app log, an API log and a server log, three tail -f windows get hard to follow. LogLens follows several files in one merged view, with a coloured label for each file:
loglens app.log api.log- / filters as you type, like
grepon a live stream. - e shows errors only, keeping each traceback with its error.
- p pauses, and s searches everything it has read.
- c groups similar errors and counts them, the same question as the
sort | uniq -cpipeline above.
See the LogLens beginner guide, which includes a script that writes sample logs for you to practise on. LogLens reads files only: it does not read from a pipe, so tail -f app.log | loglens does not work.
Stay safe
>overwrites without asking. Double-check the file name, and never redirect into the file you are reading.- Logs can hold secrets. Tokens, passwords, email addresses and IP addresses end up in logs. Think before you paste a log into a chat, a ticket or a public issue, and remove what does not belong there.
- Do not edit or delete system logs to make a problem go away. You lose the evidence you need. Read them, copy the part you need with
grep … > notes.txt, and leave the original alone. - Watch the disk. A program logging in a loop can fill a disk.
du -sh /var/log 2> /dev/nullshows how much space the system logs take (the2> /dev/nullhides complaints about folders only root may read).
Check yourself
Question 1 of 6
You run ls missing.txt > out.txt. The error still appears on screen and out.txt is empty. Why?
What you learned
- Commands print results on stdout (1) and errors on stderr (2).
>replaces a file,>>adds to it,2>catches errors,> file 2>&1catches both.|feeds one command's output into the next.grep,sort,uniq -c,wc -l,cutandawkare the usual building blocks.tail -ffollows a log live;journalctl -u NAMEreads a service's log.- LogLens follows several logs at once, with filters and error groups.