Log files are one of the first things we look at when something fails. Some other times, we look at them because they are also useful to extract statistics. However, log files are not always easy to read, since too much information is displayed. This drawback can be overcome using vim syntax files, which enhance readability, making information more accessible and helping to detect problems faster.
Let’s take a look at the following example (CNR log with Intraway Extensions):
There is too much information and everything has the same level of “Importance”. So much text with this high contrast is difficult to read.
Vim syntax highlighting
Luckily, with vim we can change this very easily. There are 2 steps that are hard and they don’t have to do with vim:
-
Decide what to highlight.
-
Decide how to highlight (colors, contrast).
It’s not always easy to strike a good balance between what is highlighted and how much contrast there is. Too much highlighting or too much contrast could be even worse than no highlighting at all. As a rule of thumb, you should give a stark contrast to things that don’t appear often and are important for you. However, try not to highlight too many things.
In this example, we want the following:
-
Better visibility of dates.
-
Warnings and errors are VERY important!
-
Tags are useful (in our logs, tags are between brackets).
-
Mac addresses and IPs are important.
-
Type of packets (DISCOVERs, REQUESTs, etc.) are rare and important.
-
Selection tags.
Note: gvim has a much richer highlighting feature than vim. This syntax file was made to be used with gvim.
Look at the following syntax file:
" iwaylog.vim " Vim syntax file " Based on messages.vim - syntax file for highlighting kernel messages if exists("b:current_syntax") finish endif “ we use solarized scheme as a base. It has a very good contrast “ for log files colorscheme solarized “ We want to match lines with keywords like FATAL or ERROR syn match log_error 'c.*<(FATAL|ERROR|ERRORS|FAIL|FAILED|FAILURE).*' “ The same with WARNING, but we want a different highlighting “ for those lines. syn match log_warning 'c.*<(WARNING).*' “ Things between quotes are strings syn region log_string start=/"/ end=/"/ skip=/\./ syn match log_number '0x[0-9a-fA-F]*|[<[0-9a-f]+>]|<d[0-9a-fA-F]*' syn match log_number '{d{6,}}' “ Match the date: dd/mm/yyyy hh:mm:ss syn match log_date 'd{2}/d{2}/d{4}s*d{2}:d{2}:d{2}' “ A component is something between brackets syn match component '[[^]]*]' “ Match IP addresses syn match internet 'dd*.dd*.dd*.dd*' “ Match IPv6 address, macaddresses and some other stuff syn match internet '(x*:){5,}xx*' syn keyword dhcp_keywords DISCOVER OFFER REQUEST ACK INFORM RENEW syn keyword dhcp_keywords DHCPDISCOVER DHCPOFFER DHCPREQUEST DHCPACK DHCPINFORM DHCPRENEW syn keyword dhcp_keywords SOLICIT ADVERTISE REPLY syn keyword hard_keywords CMTS MTA CPE CM “ Now we apply color hi def link log_string String hi def link log_error ErrorMsg hi def link log_warning WarningMsg hi def log_date guifg=#bbbbbb hi def component guifg=#bbbbbb hi def internet guifg=#cccc33 hi def internet guifg=#cccc33 hi def dhcp_keywords guifg=#bbbbdd hi def hard_keywords guifg=#ddddff guibg=#444444
This file should be placed in ~/.vim/syntax/iwaylog.vim
On your .vimrc file:
au BufNewFile,BufRead *.iwaylog set filetype=iwaylog au BufNewFile,BufRead *.log set filetype=iwaylog
This tells vim to use our syntax to log or iwaylog files (for example, files ending with .log or .iwaylog).
The syntax file is quite straightforward. Let’s explain the important parts:
- keywords: These are easy, for example:
syn keyword hard_keywords CMTS MTA CPE CM -
matches: These are good for regex, for example:
syn match internet ‘dd*.dd*.dd*.dd*’ matches things like 10.0.123.5
syn match internet ‘(x*:){5,}xx*’ matches things like MAC addresses (00:11:22:33:44:55) or IPv6 (2001:470:5::1). -
links: We define how to treat keywords or matches (in our case, the groups we used for links are already defined and have the coloring defined in solarized theme). For example:
hi def link log_warning WarningMsg treats log_warning matches as WarningMsg. -
Custom coloring: Here we tell vim how to highlight keywords and matches, setting the foreground and background color. Notice that in our case we are setting the colors for the gui (gvim).
For further information, type :help syntax in vim.
Result
This is what we get after we apply the syntax highlighting with gvim.
Notice that:
- IP addresses (v4 and v6) and MAC addresses are very easy to find.
- Warning lines stand out (though it is not shown in this example, error lines stand out even more).
- Tags are a bit different from normal text, but not so much.
- Important keywords like “CM”, “CPE”, “MTA” can be found immediately.
Conclusion
With this new highlighting:
- Log files are easier to read and understand.
- We can find errors quickly.
- Warning and error lines are almost impossible to overlook.
- All the important data stands out.
If you would like to learn more about us, please click the following link to visit our website: www.intraway.com.