![[symon]](img/symon.png)
symon(8) manual:
SYMON(8) BSD System Manager's Manual SYMON(8)
NAME
symon -- system monitor
SYNOPSIS
symon [-dtuv] [-f filename]
DESCRIPTION
symon is a lightweight system monitor that measures cpu, load, filesys-
tem, interface, disk, memory, pf, pf queues, mbuf, proc and sensor sta-
tistics every 5 seconds. This information is then spooled to symux(8) for
further processing.
symon has been designed to inflict minimal performance and security
impact on the system it monitors. symux(8) has performance impact pro-
portional to the amount of streams it needs to manage. Ideally symux
should live on a different system and collect data from several symon
instances in a LAN.
By default, symon will drop privileges and chroot(2) to home of the symon
user. This behaviour is not strictly needed for the cpu, mem, mbuf, disk
debug and interface probes as these will work even when symon is started
as nobody.
The options:
-d Stop symon from becoming a daemon and show debug information on
stdout.
-f filename
Read configuration from filename instead of /etc/symon.conf.
-t Test configuration file and exit.
-u By default symon will chroot(2) into _symon user home directory.
The -u disables this behaviour.
-v Show version information.
CONFIGURATION
symon obtains configuration data from /etc/symon.conf. The configuration
file contains monitor stanzas that define what resources should be moni-
tored and to which symux(8) the information should be streamed to.
Multiple monitor statements are allowed. Whitespace, newlines and text
behind '#' are ignored. The format in BNF:
monitor-rule = "monitor" "{" resources "}" [every]
"stream" ["from" host] ["to"] host [ port ]
resources = resource [ version ] ["(" argument ")"]
[ ","|" " resources ]
resource = "cpu" | "cpuiow" | "debug" | "df" | "flukso" |
"if" | "io" | "load" | "mbuf" | "mem" | "pf" |
"pfq" | "proc" | "sensor" | "smart"
version = number
argument = number | name
every = "every" time
time = "second" | number "seconds"
host = ip4addr | ip6addr | hostname
port = [ "port" | "," ] portnumber
Note that symux(8) data files default to receiving data every 5 seconds.
Adjusting the monitoring interval will also require adjusting the associ-
ated symux(8) datafile(s).
The pf probe will return data that is collected for the loginterface set
in /etc/pf.conf(5).
The Linux io, df, and smart probes support device names via id, label,
path and uuid.
The FreeBSD io, df, and smart probes support gpt names, ufs names, ufs
ids and paths.
The OpenBSD io probe supports device uuids.
EXAMPLE
Here is an example OpenBSD symon.conf that monitors cpu, memory, pf,
interfaces xl0/de0/lo0/wi0, disks wd[0-3]/cd[0-1], debug variables debug0
to debug19 and streams that information to localhost on port 2100.
monitor { cpu(0), mem, pf, if(xl0), if(de0),
if(lo0), if(wi0), io(wd0), io(wd1),
io(wd2), io(wd3), io(cd0), io(cd1),
io(ccd0), df(sd0a), df(sd0d), df(sd0e),
debug, proc(httpd) } stream to 127.0.0.1 2100
EXAMPLE
Here is an example Linux symon.conf that monitors cpu including iowait,
memory, load, interface eth0, io and df for a set of disks every 5 sec-
onds. Smart data is to be collected every 60 seconds. Disks in the smart
and io statements are identified using ids, filesystem volumes in df
using labels.
monitor { smart(ata-Hitachi_HDS722020ALA330_JK1130ABABABAB),
smart(ata-Hitachi_HDS722020ALA330_JK1130ACACACAC),
} every 60 seconds stream to 192.168.0.2 port 2100
monitor { cpuiow(0), cpuiow(1), mem, if(eth0),
io(ata-Hitachi_HDS722020ALA330_JK1130ABABABAB),
io(ata-Hitachi_HDS722020ALA330_JK1130ACACACAC),
df(data_1),
df(data_2),
df(data_3),
df(home),
df(var),
load
} stream to 192.168.0.2 port 2100
SIGNALS
SIGHUP Causes symon to read /etc/symon.conf. symon will keep the old
configuration if errors occured during parsing of the configura-
tion file. Note that the chroot(2) may cause resources to become
unattainable, most notably the configuration file itself.
FILES
/var/run/symon.pid
Contains the program id of the symon daemon.
/etc/symon.conf
symon system wide configuration file.
BUGS
Every monitored resource mentioned /etc/symon.conf gets queried. Mention-
ing, for example, cpu(0) twice for different muxes will result in two
distinct cpu(0) measurement actions.
The proc module is too simple: memory shared between two instances of the
same process is simply counted twice.
symon does not check whether all resources mentioned in /etc/symon.conf
exist.
AUTHOR
Willem Dijkstra <wpd@xs4all.nl>. Daniel Hartmeier helped to port to big-
endian architectures. Matthew Gream helped to port symon to other BSD
platforms.
Port contributors: Marc Balmer, Tito Dal Canton, Matthew Gream, Daniel
Hartmeier, Lars Kotthoff, Constantine A. Murenin, J. Martin Petersen,
Fredrik Soderblom, Harm Schotanus and Martin van der Werff.
Valeriy Leshchinskiy maintains a windows symon client at
https://github.com/ValHazelwood/SymonClient .
SEE ALSO
symux(8)
BSD April 4, 2012 BSD
symux(8) manual:
SYMUX(8) BSD System Manager's Manual SYMUX(8)
NAME
symux -- symon stream multiplexer
SYNOPSIS
symux [-dltv] [-f filename]
DESCRIPTION
symon(8) is a lightweight system monitor that measures cpu, load,
filesystem, interface, disk, memory, pf, pf queues, mbuf, proc and sensor
statistics every 5 seconds. This information is then spooled to symux for
further processing.
symon(8) has been designed to inflict minimal performance and security
impact on the system it monitors. symux has performance impact propor-
tional to the amount of streams it needs to manage. Ideally symux should
live on a different system and collect data from several symon(8)
instances in a LAN. symux stores the incoming streams in .rrd files and
distributes the information to connected listeners. Listeners can connect
to symux on a tcp port and receive incoming symon(8) transmissions
decoded into ascii.
symux needs no specific privileges besides being able to open it's ports
and the rrd files. It should be run as nobody.
The options:
-d Stop symux from becoming a daemon and show debug information on
stdout. Use this setting to find hosts or specific statistics
that do get sent, but are ignored due to configuration.
-f filename
Read configuration from filename instead of /etc/symux.conf.
-l List rrd files found in active configuration.
-t Test configuration file and exit.
-v Show version.
CONFIGURATION
symux obtains configuration data from /etc/symux.conf. The configuration
file contains one mux stanza that defines on what host address and port
symux should listen to for incoming monitored data. There is a source
section for every host that is to be monitored. The source section
defines what data to accept and where to write that data to. In the case
that a source is of another address family than the mux stanza, i.e.
source = ipv6 with mux = ipv4, a listen port of the sources' family is
opened using the unspecified address. Whitespace, newlines and text
behind '#' are ignored. The format in BNF:
stmt = mux-stmt | source-stmt
mux-stmt = "mux" host [ port ]
host = ip4addr | ip6addr | hostname
port = [ "port" | "," ] portnumber
source-stmt = "source" host "{"
accept-stmts
[ write-stmts ]
[ datadir-stmt ] "}"
accept-stmts = accept-stmt [accept-stmts]
accept-stmt = "accept" "{" resources "}"
resources = resource [ version ] ["(" argument ")"]
[ ","|" " resources ]
resource = "cpu" | "cpuiow" | "debug" | "df" | "flukso" |
"if" | "io" | "load" | "mbuf" | "mem" | "pf" |
"pfq" | "proc" | "sensor" | "smart"
version = number
argument = number | interfacename | diskname
datadir-stmt = "datadir" dirname
write-stmts = write-stmt [write-stmts]
write-stmt = "write" resource "in" filename
Note that
port in the mux-stmt specifies the port-number for both the udp port
(incoming symon(8) traffic) and the tcp port for incoming listen-
ers.
version
is needed to distinguish between the same type of information
(i.e. io ) coming from different versions of OpenBSD. If no ver-
sion number is supplied, the latest will be assumed.
datadir
will guess filenames for all accepted streams. write statements
always take precendence over a datadir statement.
EXAMPLE
Here is an example symux.conf that listens to udp port 2100 on lo0, and
accepts cpu, memory, pf, interfaces xl0/de0/lo0/wi0, disks
wd[0-3]/cd[0-1], disk free blocks of three partition streams from a
symon(8) on localhost. symux will also listen on tcp port 2100 for
incoming listeners.
mux 127.0.0.1 2100
source 127.0.0.1 {
accept { cpu(0), mem, pf,
if(xl0), if(de0),
if(lo0), if(wi0),
io(wd0), io(wd1), io(wd2),
io(wd3), io(cd0), io(cd1),
df(sd0a), df(sd0d), df(sd0e) }
datadir "/var/www/symon/rrds/localhost"
}
LISTENERS
symux offers received symon(8) data to other programs via tcp. An example
of a listener session:
nexus:~/project/symon$ telnet 10.0.0.1 2100
Trying 10.0.0.1...
Connected to 10.0.0.1.
Escape character is '^]'.
10.0.0.1;mem::1077662160:7630848:53850112:469417984:0:25600;cpu:0:
1077662160:0.00:0.00:0.30:0.20:99.50;io:wd0:1077662160:2074:12759:
0:30736384:131780608;
10.0.0.2;mbuf::1077658247:138:74:0:0:41:0:23:0:90:360:868352:25:0:
0:0;pf::1077658247:700930123:535398451:0:352:1107229:706391:119833
9:4:0:0:2:3:29:4109383:83291:83262:980325:0:1:6:0:0;mem::107765824
7:79155200:131956736:391430144:0:536739840;cpu:0:1077658247:0.50:0
552:0;if:lo0:1077658247:147104:147104:45868177:45868177:0:0:0:0:0:
0;if:xl0:1077658247:284267:452077:150620236:273265863:372:89478:0:
0:0:0;if:de0:1077658247:1813721:1197722:729054136:568900227:101:2:
0:0:198:0;
^]
telnet> close
Connection closed.
The format is symon-version : symon-host-ip : stream-name :
stream-argument : timestamp : data
Data formats:
cpu Time spent in ( user, nice, system, interrupt, idle ). Total time
is 100, data is offered with precision 2.
cpuiow Time spent in ( user, nice, system, interrupt, idle, iowait ).
Total time is 100, data is offered with precision 2.
debug Kernel variables debug0 to debug19. ( debug0 : ... : debug19 ).
Values are 32 bit unsigned integers.
df Disk free statistics ( blocks : bfree : bavail : files : ffree :
syncwrites : asyncwrites ). Values are 64 bit unsigned integers.
load Load averages for the last 1, 5, and 15 minutes ( load1, load5,
load15 ). Data is offered with prec ision 2 and a maximum of 655.
if Alias for if2. See below.
if1 Pre OpenBSD 4.3 interface counters ( packets_in, packets_out,
bytes_in, bytes_out, multicasts_in, multicasts_out, errors_in,
errors_out, collisions, drops ). Values are 32 bit unsigned inte-
gers.
if2 Interface counters ( ipackets, opackets, ibytes, obytes, imcasts,
omcasts, ierrors, oerrors, collisions, drops ). Values are 64 bit
unsigned integers.
io Alias for io2. See below.
io1 Pre OpenBSD 3.5 io/disk counters ( total_transfers, total_seeks,
total_bytes ). Values are 64 bit unsigned integers.
io2 Io/disk counters ( rxfer, wxfer, seeks, rbytes, wbytes). Values
are 64 bit unsigned integers.
mbuf Mbuf statistics ( totmbufs : mt_data : mt_oobdata : mt_control :
mt_header : mt_ftable : mt_soname : mt_soopts : pgused : pgtotal
: totmem : totpct : m_drops : m_wait : m_drain ).
mem Alias for mem2. See below.
mem1 Pre symon 2.78 memory counters ( real_active, real_total, free,
swap_used, swap_total ). All values are in bytes rounded to page
boundaries. Values are 32 bit unsigned integers.
mem2 Memory in ( real_active, real_total, free, swap_used, swap_total
). All values are in bytes rounded to page boundaries. Values are
64 bit unsigned integers.
pf Packet filter statistics ( bytes_v4_in : bytes_v4_out :
bytes_v6_in : bytes_v6_out : packets_v4_in_pass : pack-
ets_v4_in_drop : packets_v4_out_pass : packets_v4_out_drop :
packets_v6_in_pass : packets_v6_in_drop : packets_v6_out_pass :
packets_v6_out_drop : states_entries : states_searches :
states_inserts : states_removals : counters_match : counters_bad-
offset : counters_fragment : counters_short : counters_normalize
: counters_memory ). Values are 64 bit unsigned integers.
pfq pf/altq queue statistics ( sent_bytes : sent_packets : drop_bytes
: drop_packets ). Values are 64 bit unsigned integers.
proc Process statistics ( number : uticks : sticks : iticks : cpusec :
cpupct : procsz : rsssz ).
sensor Single sensor measurement offered with 7.6 precision. Value
depends on sensor type.
smart SMART attributes ( read_error_rate: reallocated_sectors:
spin_retries: air_flow_temp: temperature: reallocations: cur-
rent_pending: uncorrectables: soft_read_error_rate:
g_sense_error_rate: temperature2: free_fall_protection ). Values
depend on drive model and may change between models.
flukso Average pwr sensor value offered with 7.6 precision. Value is a
moving average and will depend on the number of measurements seen
in a particular symon interval.
SIGNALS
SIGHUP Causes symux to read /etc/symux.conf or the file specified by the
-f flag. symux will keep the old configuration if errors occured
during parsing of the configuration file.
FILES
/var/run/symux.pid
Contains the program id of the symux daemon.
/etc/symux.conf
symux system wide configuration file.
LEGACY FORMATS
symux supports symon(8) clients that send
pre OpenBSD 3.5 disk statistics.
These streams should be identified as io1(<disk>) instead of
io(<disk>) in /etc/symux.conf. Also note that symon(8) measures
io1 or io2 depending on whether it was compiled on a host that
supports version 1 or 2. The rrd datastructures of these streams
differ and there is no easy way to change an io1 rrd into an io2
rrd.
pre symon 2.78 mem/if statistics.
These streams should be identified as if1(<interface>) and mem1()
in /etc/symux.conf. symon versions 2.78 and up will always report
if2 and mem2 statistics. The rrd files for the old and new probes
are identical and need not be changed.
symux will output what version of information it is offered by symon(8)s
on the network when started with the -d flag.
BUGS
symux writes incoming data to rrd files "in process". An rrdupdate on a
somewhat stale rrdfile -- with the last data from quite some time in the
past -- is a very expensive operation. This can cause symux to lockup
while rrdupdate is updating the rrd file. symux will be unresponsive
during this process.
AUTHOR
Willem Dijkstra <wpd@xs4all.nl>. Daniel Hartmeier helped to port to big-
endian architectures. Matthew Gream helped to port symon to other BSD
platforms.
Port contributors: Marc Balmer, Tito Dal Canton, Matthew Gream, Daniel
Hartmeier, Lars Kotthoff, Constantine A. Murenin, J. Martin Petersen,
Fredrik Soderblom, Harm Schotanus and Martin van der Werff.
SEE ALSO
symon(8)
BSD April 4, 2012 BSD
Installation instructions for symon, also accessable as the file INSTALL inside the symon-version.tar.gz:
Installation notes
==================
Privileges
==========
symux needs read and write access to its rrdfiles.
symon needs to interface with your kernel. Depending on your host system this
leads to different privilege requirements:
OpenBSD: - no privs: cpu, debug, df, if, io, mbuf, mem, proc, sensor
- rw on /dev/pf for pf
NetBSD: - no privs: cpu, debug, df, if, io, mbuf, proc
- r on /dev/sysmon for sensor
FreeBSD: - no privs: all
- non-chroot on FreeBSD 5.x for CPU ticks in proc
- rw on /dev/pf for pf and pfq
Linux: - r on /proc/net/dev: if
- r on /proc/stat: cpu, cpuiow
- r on /proc/meminfo: mem
all:
- r on chroot/etc/localtime for proper timezone logging
Real quick on OpenBSD
=====================
(cd /usr/ports/net/rrdtool && make install) &&
make &&
make install &&
vi /etc/symux.conf /etc/symon.conf &&
~symon/symux/c_smrrds.sh all &&
/usr/local/libexec/symux &&
useradd -d /var/empty -L daemon -c 'symon Account' -s /sbin/nologin _symon
/usr/local/libexec/symon
or grab the port and do
make package
pkg_add symon-mon-version.tgz on all monitored hosts
pkg_add symon-mux-version.tgz on the loghost
Install the seperate syweb package to show the data stored in the rrd files.
Less quick, but all OSes
========================
- Install rrdtool on the host that will also run your symux gatherer.
BSDs: cd /usr/ports/net/rrdtool && make install
- Check Makefile.inc for settings. Things to watch out for are:
+ PREFIX = Where does the installation tree start. Defaults to
'/usr/local'.
+ BINDIR = Where should the daemons be put, relative to $PREFIX. Defaults
to 'libexec'.
+ MANDIR = Where should the manuals be installed, relative to
$PREFIX. Defaults to 'man'.
+ SHRDIR = Where are the example configurations to be installed. Defaults
to 'share/symon'.
+ RRDDIR = $RRDDIR/include should yield rrd.h. Define SYMON_ONLY in the
environment or on the make command line to render this mute.
+ INSTALLUSER / GROUPDIR / GROUPFILE = user and groups that install should
use.
Note that:
+ you can define SYMON_ONLY if you do not want to compile symux / do not
have rrdtool installed.
+ symon/platform/os/Makefile.inc is read before Makefile.inc; define
your vars in the environment, or in Makefile.inc with != to force
overwriting the defaults.
BSDs: Run make && make install
Linux: Run pmake && pmake install || bmake && bmake install
- Create an '/etc/symon.conf' for each monitored host and one symux.conf for
the gatherer host. See the manual pages on how to specify alternative
configuration file locations, if necessary. Note that there are example
configurations for both in $PREFIX/$SHRDIR.
- Create the rrd files where the incoming symon data is to be
stored. $PREFIX/$SHRDIR/c_smrrds.sh and symux -l are your friends. Note that
syweb expects an '.../machine/*.rrd' style directory structure somewhere
under /var/www.
- Ensure that /etc/localtime is accessible by symon/symux. Failing this you
will get log messages in GMT. Note that etc is chroot/etc when symon
chroots.
- Both symon and symux will daemonize if started normally. Start them with
debugging on initially to iron out any configuration problems:
$PREFIX/$BINDIR/symux -d &
$PREFIX/$BINDIR/symon -d
- Remove -d flags and check system logs for any failures.
- Only if you need the webinterface: download and install syweb.
Getting measurements without the web
====================================
The client directory contains a perl module 'SymuxClient.pm' that can be used
to read measurements as they come in at the symux host. A sample Perl program
called 'getsymonitem.pl' shows how to use the module.
Example:
nexus$ getsymonitem.pl 127.0.0.1 2100 127.0.0.1 "cpu(0)" "idle"
93.40
Historical data can be gathered using rrdfetch(1) from symux's rrd files.
Portability
===========
This package was originally built as an OpenBSD application. It now has support
for FreeBSD, NetBSD and Linux.
Willem Dijkstra - wpd@xs4all.nl
Installation instructions for syweb, also accessable as the file INSTALL inside the syweb-version.tar.gz:
Installation ------------ Requirements to show a graph: - PHP: Language version needed is 7.4+. - Program files: Download syweb-version.tar.gz from the website and copy htdocs/ and symon/ from the archive into ~www. - Data files: Make your symon rrds accessable to php as run by your http server. Move them into ~www/symon/rrds and update symux. - Configuration: Customise ~www/htdocs/syweb/setup.inc. The supplied setup.inc is intended for chrooted httpd using php-fpm on OpenBSD. - RRDtool: Must be available in whatever web container you are using. E.g. for OpenBSD chrooted httpd, rrdtool must be available inside the chroot. - Date & time: chrooted httpd may have a different idea of timezone. Add etc/localtime to chroot if needed. Point your browser to http://127.0.0.1/syweb/configtest.php to test the configuration. This page will show if the application cannot deal with a setting in setup.inc. Point your browser to http://127.0.0.1/syweb/index.php for the main page. You can move index_noui.php to index.php if you do not need the ui. RRDTool and chrooted httpd -------------------------- RRDTool has a large number of requirements that it must be able to access in order to run. This can require copying a fair number of files inside your chrooted httpd. - Dynamically linked libraries: find out using ldd rrdtool - Dynamically linking kit: ld.so must be available inside chroot to do the dynamic linking, and must be fed a chroot adjusted ~www/var/run/ld.so.hints - Fonts data: a subset of fonts from /usr/X11R6/lib/X11/fonts/TTF/DevaVuSansMono* must be available in a ~www/fontdirectory. - Font configuration: "fontdirectory" must be specified in chroot ~www/etc/fonts/fonts.conf. And finally: - sh binary: this is needed by PHP to be able to execute rrdtool. All these steps are detailed for OpenBSD in install_rrdtool.sh.