Why colorize terminal output when you can colorize on the reader?

Photo by Markus Spiske on Unsplash

Colorful text is a must, because we are visual creatures. We process shapes and colors way better than symbols. Confronted to a screen full of text, it is hard to make sense of it, without meaningful coloring. I even find pure text repulsive, I don’t want to look. Command like tools like ls, grep, git offer colorized output, in fact I’m only able to find them useful, when their output is colorful. Color demands attention and focuses it too.

The need for color and the problems it brings

Over the years, I colorized my software output out for usability reasons. Including color is simple, yet it adds some noise to the code. It feels harmless considering its benefits and that it can be neatly hidden by wrapping the styling code as functions that give semantic meaning to the action and its consequential colorized output.

However, colorized output on the terminal requires the control sequences. Sure color looks great on the terminal, but if you need to pipe the output, all those control sequences stay. If you open the output file with less it works, but if you don’t it adds a lot of noise, making it worse than the colorless plain text. It decreases the flexibility and usefulness of text files, as now every viewer must be capable of parsing those control sequences. As a consequence, you now need to make your software even more complex, because it needs to toggle its color output.

Colorizing output starts to look like a burden not an advantage. Software becomes more complex, you mix program logic with visualization. Software changes are expensive and require rebuilds and re-executions, it is too rigid. You want to change visualization more often than the program execution. Visualization is part of analysis, you want to look at the output under multiple colors, different highlights. Finally, at the expense of one viewing device, the terminal, you have harmed the software output usability for other displays.

A new way to work

Stop doing it! Coloring is analysis not pre-processing. I’m slowly striping away the colorized output from my software. Those magical lines of code that once made output so much readable are now disappearing. The alternative is now to focus on the reader. I always have my reader at hand, it is even a text editor, more than that it is an operating system that hides itself inside and editor. Better said: Emacs is building material!

Emacs already does syntax highlighting for my code and other text files. Why not do it for even more files? Within Emacs, I can execute a program and capture its output on a buffer, but now I can work on that buffer, search/replace/filter. Moreover, within Emacs I can tune my workflow. font-lock lets me syntax highlight the buffer. I can dynamically colorize a log file, or any file I’m looking at. Colorize timestamps, colorize some meaningful words, highlight lines, anything I want. The experience becomes much richer, and flexible.

I can take any file and think for exclusively that file how to display it. Afterward, I save those configurations for any other occasion. I can have visualization code explicitly dedicated to analyze log files from my software, I always activate it when looking at output from journalctl. It is so much better.

What does it look like?

In its simplest form, I define a new major mode, declare some regular expressions as interesting and specify their font-lock.

For example, I use a script to fetch my email, tag it using notmuch, and do some cleanup actions like removing or moving some email files. I used to launch this script from the terminal, not from a cron job. The reason was interactively unlock my password manager to connect to the email server. Because it was interactive I wanted to look at the output and I wanted to see it colorized.

Nowadays, I launch the script directly from Emacs, it opens a new buffer capturing the output and with this new mode, the buffer neatly colorizes the output. I don’t need the implement color output inside my script.

(define-derived-mode mail-sync-log-mode comint-mode "mail-sync-log"
  "Major mode for reading mail sync."
  ;; code for syntax highlighting
  (setq font-lock-defaults `(((,(rx bol "[" (1+ (or letter space)) "]") . font-lock-warning-face)
                              (,(rx bol " [mv]") . font-lock-constant-face)
                              (,(rx bol " [rm]") . font-lock-keyword-face)))))

For a more elaborate example, this is how I nowadays style log files, particularly those managed over journalctl. I have my personal major mode for the job.

(defcustom journalctl-error-keywords
  '("Failed" "failed" "Error" "error" "critical" "couldn't" "Can't" "not" "Not" "unreachable" "FATAL")
  "Keywords that mark errors in journalctl output."
  :group 'journalctl
  :type 'string)

(defvar journalctl-font-lock-keywords
  ;; note: order matters, because once colored, that part won't change.
  ;; in general, put longer words first
  `((,(regexp-opt journalctl-warn-keywords 'words) . 'journalctl-warning-face)
    (,(regexp-opt journalctl-error-keywords 'words) . 'journalctl-error-face)
    (,(regexp-opt journalctl-starting-keywords 'words) . 'journalctl-starting-face)
    (,(regexp-opt journalctl-finished-keywords 'words) . 'journalctl-finished-face)
    (,(rx bol (group (= 3 alpha) " " (= 2 digit) " " (1+ (in digit ":"))) " " ; timestamp
          (group (+ (in alphanumeric ?.))) " "                 ; host
          (group (+? not-newline)) "[" (group (+ digit)) "]:") ; service[PID]
     (1 'journalctl-timestamp-face)
     (2 'journalctl-host-face)
     (3 'journalctl-process-face)
     (4 'font-lock-comment-face))))

(define-derived-mode journalctl-mode fundamental-mode "journalctl"
  "Major mode for viewing journalctl output."
  ;; code for syntax highlighting
  (setq font-lock-defaults '((journalctl-font-lock-keywords))))

In this case the regular expressions are much more involved. Fortunately Emacs has the rx package, which allows to write them in a structure notation instead of the highly dense and unreadable string.

I match for a list of words that indicates warnings, errors, services starts and terminations. Each of those receive a particular style. I’m also able to selectively colorize individually matched groups belonging to a broader regular expression. For example the leading string prefix in a log line, which contains timestamp-host-service. This all happens independent of the software producing the logs and so I can dynamically change, what and how I want to highlight.

So I ask again: Why colorize terminal output when you can colorize on the reader?

Dr. Óscar Nájera
Dr. Óscar Nájera
Software distiller & Recovering Physicists

As scientist I studied the very small quantum world. As a hacker I distill code, because software is eating the world, and less code means less errors, less problems, more world.

Previous