SYMON(8)                  BSD System Manager's Manual                 SYMON(8)

NAME
     symon -- system monitor

SYNOPSIS
     symon [-dtuv] [-f filename]

DESCRIPTION
     symon is a lightweight system monitor that measures cpu, load, filesys-
     tem, interface, disk, memory, pf, pf queues, mbuf, proc and sensor sta-
     tistics every 5 seconds. This information is then spooled to symux(8) for
     further processing.

     symon has been designed to inflict minimal performance and security
     impact on the system it monitors.  symux(8) has performance impact pro-
     portional to the amount of streams it needs to manage. Ideally symux
     should live on a different system and collect data from several symon
     instances in a LAN.

     By default, symon will drop privileges and chroot(2) to home of the symon
     user.  This behaviour is not strictly needed for the cpu, mem, mbuf, disk
     debug and interface probes as these will work even when symon is started
     as nobody.

     The options:

     -d      Stop symon from becoming a daemon and show debug information on
             stdout.

     -f filename
             Read configuration from filename instead of /etc/symon.conf.

     -t      Test configuration file and exit.

     -u      By default symon will chroot(2) into _symon user home directory.
             The -u disables this behaviour.

     -v      Show version information.

CONFIGURATION
     symon obtains configuration data from /etc/symon.conf.  The configuration
     file contains monitor stanzas that define what resources should be moni-
     tored and to which symux(8) the information should be streamed to.

     Multiple monitor statements are allowed. Whitespace, newlines and text
     behind '#' are ignored. The format in BNF:

           monitor-rule = "monitor" "{" resources "}" [every]
                          "stream" ["from" host] ["to"] host [ port ]
           resources    = resource [ version ] ["(" argument ")"]
                          [ ","|" " resources ]
           resource     = "cpu" | "cpuiow" | "debug" | "df" | "flukso" |
                          "if" | "io" | "load" | "mbuf" | "mem" | "pf" |
                          "pfq" | "proc" | "sensor" | "smart"
           version      = number
           argument     = number | name
           every        = "every" time
           time         = "second" | number "seconds"
           host         = ip4addr | ip6addr | hostname
           port         = [ "port" | "," ] portnumber

     Note that symux(8) data files default to receiving data every 5 seconds.
     Adjusting the monitoring interval will also require adjusting the associ-
     ated symux(8) datafile(s).

     The pf probe will return data that is collected for the loginterface set
     in /etc/pf.conf(5).

     The Linux io, df, and smart probes support device names via id, label,
     path and uuid.

     The FreeBSD io, df, and smart probes support gpt names, ufs names, ufs
     ids and paths.

     The OpenBSD io probe supports device uuids.

EXAMPLE
     Here is an example OpenBSD symon.conf that monitors cpu, memory, pf,
     interfaces xl0/de0/lo0/wi0, disks wd[0-3]/cd[0-1], debug variables debug0
     to debug19 and streams that information to localhost on port 2100.

           monitor { cpu(0),  mem, pf, if(xl0), if(de0),
                     if(lo0), if(wi0), io(wd0), io(wd1),
                     io(wd2), io(wd3), io(cd0), io(cd1),
                     io(ccd0), df(sd0a), df(sd0d), df(sd0e),
                     debug, proc(httpd) } stream to 127.0.0.1 2100

EXAMPLE
     Here is an example Linux symon.conf that monitors cpu including iowait,
     memory, load, interface eth0, io and df for a set of disks every 5 sec-
     onds. Smart data is to be collected every 60 seconds.  Disks in the smart
     and io statements are identified using ids, filesystem volumes in df
     using labels.
           monitor { smart(ata-Hitachi_HDS722020ALA330_JK1130ABABABAB),
                     smart(ata-Hitachi_HDS722020ALA330_JK1130ACACACAC),
                   } every 60 seconds stream to 192.168.0.2 port 2100

           monitor { cpuiow(0), cpuiow(1), mem, if(eth0),
                     io(ata-Hitachi_HDS722020ALA330_JK1130ABABABAB),
                     io(ata-Hitachi_HDS722020ALA330_JK1130ACACACAC),
                     df(data_1),
                     df(data_2),
                     df(data_3),
                     df(home),
                     df(var),
                     load
                   } stream to 192.168.0.2 port 2100

SIGNALS
     SIGHUP  Causes symon to read /etc/symon.conf.  symon will keep the old
             configuration if errors occured during parsing of the configura-
             tion file. Note that the chroot(2) may cause resources to become
             unattainable, most notably the configuration file itself.

FILES
     /var/run/symon.pid
             Contains the program id of the symon daemon.

     /etc/symon.conf
             symon system wide configuration file.

BUGS
     Every monitored resource mentioned /etc/symon.conf gets queried. Mention-
     ing, for example, cpu(0) twice for different muxes will result in two
     distinct cpu(0) measurement actions.

     The proc module is too simple: memory shared between two instances of the
     same process is simply counted twice.

     symon does not check whether all resources mentioned in /etc/symon.conf
     exist.

AUTHOR
     Willem Dijkstra <wpd@xs4all.nl>. Daniel Hartmeier helped to port to big-
     endian architectures. Matthew Gream helped to port symon to other BSD
     platforms.

     Port contributors: Marc Balmer, Tito Dal Canton, Matthew Gream, Daniel
     Hartmeier, Lars Kotthoff, Constantine A. Murenin, J. Martin Petersen,
     Fredrik Soderblom, Harm Schotanus and Martin van der Werff.

SEE ALSO
     symux(8)

BSD                              April 4, 2012                             BSD
                

SYMUX(8)                  BSD System Manager's Manual                 SYMUX(8)

NAME
     symux -- symon stream multiplexer

SYNOPSIS
     symux [-dltv] [-f filename]

DESCRIPTION
     symon(8) is a lightweight system monitor that measures cpu, load,
     filesystem, interface, disk, memory, pf, pf queues, mbuf, proc and sensor
     statistics every 5 seconds. This information is then spooled to symux for
     further processing.

     symon(8) has been designed to inflict minimal performance and security
     impact on the system it monitors.  symux has performance impact propor-
     tional to the amount of streams it needs to manage. Ideally symux should
     live on a different system and collect data from several symon(8)
     instances in a LAN.  symux stores the incoming streams in .rrd files and
     distributes the information to connected listeners. Listeners can connect
     to symux on a tcp port and receive incoming symon(8) transmissions
     decoded into ascii.

     symux needs no specific privileges besides being able to open it's ports
     and the rrd files. It should be run as nobody.

     The options:

     -d      Stop symux from becoming a daemon and show debug information on
             stdout. Use this setting to find hosts or specific statistics
             that do get sent, but are ignored due to configuration.

     -f filename
             Read configuration from filename instead of /etc/symux.conf.

     -l      List rrd files found in active configuration.

     -t      Test configuration file and exit.

     -v      Show version.

CONFIGURATION
     symux obtains configuration data from /etc/symux.conf.  The configuration
     file contains one mux stanza that defines on what host address and port
     symux should listen to for incoming monitored data. There is a source
     section for every host that is to be monitored. The source section
     defines what data to accept and where to write that data to. In the case
     that a source is of another address family than the mux stanza, i.e.
     source = ipv6 with mux = ipv4, a listen port of the sources' family is
     opened using the unspecified address. Whitespace, newlines and text
     behind '#' are ignored. The format in BNF:

           stmt         = mux-stmt | source-stmt
           mux-stmt     = "mux" host [ port ]
           host         = ip4addr | ip6addr | hostname
           port         = [ "port" | "," ] portnumber
           source-stmt  = "source" host "{"
                          accept-stmts
                          [ write-stmts ]
                          [ datadir-stmt ] "}"
           accept-stmts = accept-stmt [accept-stmts]
           accept-stmt  = "accept" "{" resources "}"
           resources    = resource [ version ] ["(" argument ")"]
                          [ ","|" " resources ]
           resource     = "cpu" | "cpuiow" | "debug" | "df" | "flukso" |
                          "if" | "io" | "load" | "mbuf" | "mem" | "pf" |
                          "pfq" | "proc" | "sensor" | "smart"
           version      = number
           argument     = number | interfacename | diskname
           datadir-stmt = "datadir" dirname
           write-stmts  = write-stmt [write-stmts]
           write-stmt   = "write" resource "in" filename

     Note that

     port    in the mux-stmt specifies the port-number for both the udp port
             (incoming symon(8) traffic) and the tcp port for incoming listen-
             ers.

     version
             is needed to distinguish between the same type of information
             (i.e.  io ) coming from different versions of OpenBSD. If no ver-
             sion number is supplied, the latest will be assumed.

     datadir
             will guess filenames for all accepted streams.  write statements
             always take precendence over a datadir statement.

EXAMPLE
     Here is an example symux.conf that listens to udp port 2100 on lo0, and
     accepts cpu, memory, pf, interfaces xl0/de0/lo0/wi0, disks
     wd[0-3]/cd[0-1], disk free blocks of three partition streams from a
     symon(8) on localhost.  symux will also listen on tcp port 2100 for
     incoming listeners.

           mux 127.0.0.1 2100
           source 127.0.0.1 {
               accept { cpu(0), mem, pf,
                        if(xl0), if(de0),
                        if(lo0), if(wi0),
                        io(wd0), io(wd1), io(wd2),
                        io(wd3), io(cd0), io(cd1),
                        df(sd0a), df(sd0d), df(sd0e) }

               datadir "/var/www/symon/rrds/localhost"
           }

LISTENERS
     symux offers received symon(8) data to other programs via tcp. An example
     of a listener session:

           nexus:~/project/symon$ telnet 10.0.0.1 2100
           Trying 10.0.0.1...
           Connected to 10.0.0.1.
           Escape character is '^]'.

           10.0.0.1;mem::1077662160:7630848:53850112:469417984:0:25600;cpu:0:
           1077662160:0.00:0.00:0.30:0.20:99.50;io:wd0:1077662160:2074:12759:
           0:30736384:131780608;
           10.0.0.2;mbuf::1077658247:138:74:0:0:41:0:23:0:90:360:868352:25:0:
           0:0;pf::1077658247:700930123:535398451:0:352:1107229:706391:119833
           9:4:0:0:2:3:29:4109383:83291:83262:980325:0:1:6:0:0;mem::107765824
           7:79155200:131956736:391430144:0:536739840;cpu:0:1077658247:0.50:0
           552:0;if:lo0:1077658247:147104:147104:45868177:45868177:0:0:0:0:0:
           0;if:xl0:1077658247:284267:452077:150620236:273265863:372:89478:0:
           0:0:0;if:de0:1077658247:1813721:1197722:729054136:568900227:101:2:
           0:0:198:0;

           ^]
           telnet> close
           Connection closed.

     The format is symon-version : symon-host-ip : stream-name :
     stream-argument : timestamp : data

     Data formats:

     cpu     Time spent in ( user, nice, system, interrupt, idle ). Total time
             is 100, data is offered with precision 2.

     cpuiow  Time spent in ( user, nice, system, interrupt, idle, iowait ).
             Total time is 100, data is offered with precision 2.

     debug   Kernel variables debug0 to debug19. ( debug0 : ... : debug19 ).
             Values are 32 bit unsigned integers.

     df      Disk free statistics ( blocks : bfree : bavail : files : ffree :
             synwrites : asyncwrites). Values are 64 bit unsigned integers.

     load    Load averages for the last 1, 5, and 15 minutes. Data is offered
             with precision 2 and a maximum of 655.

     if      Alias for if2. See below.

     if1     Pre OpenBSD 4.3 interface counters ( packets_in, packets_out,
             bytes_in, bytes_out, multicasts_in, multicasts_out, errors_in,
             errors_out, collisions, drops ). Values are 32 bit unsigned inte-
             gers.

     if2     Interface counters ( packets_in, packets_out, bytes_in,
             bytes_out, multicasts_in, multicasts_out, errors_in, errors_out,
             collisions, drops ). Values are 64 bit unsigned integers.

     io      Alias for io2. See below.

     io1     Pre OpenBSD 3.5 io/disk counters ( total_transfers, total_seeks,
             total_bytes ). Values are 64 bit unsigned integers.

     io2     Io/disk counters ( total_rxfer, total_wxfer, total_seeks,
             total_rbytes, total_wbytes). Values are 64 bit unsigned integers.

     mbuf    Mbuf statistics ( totmbufs : mt_data : mt_oobdata : mt_control :
             mt_header : mt_ftable : mt_soname : mt_soopts : pgused : pgtotal
             : totmem : totpct : m_drops : m_wait : m_drain ).

     mem     Alias for mem2. See below.

     mem1    Pre symon 2.78 memory counters ( real_active, real_total, free,
             swap_used, swap_total ). All values are in bytes rounded to page
             boundaries. Values are 32 bit unsigned integers.

     mem2    Memory in ( real_active, real_total, free, swap_used, swap_total
             ). All values are in bytes rounded to page boundaries. Values are
             64 bit unsigned integers.

     pf      Packet filter statistics ( bytes_v4_in : bytes_v4_out :
             bytes_v6_in : bytes_v6_out : packets_v4_in_pass : pack-
             ets_v4_in_drop : packets_v4_out_pass : packets_v4_out_drop :
             packets_v6_in_pass : packets_v6_in_drop : packets_v6_out_pass :
             packets_v6_out_drop : states_entries : states_searches :
             states_inserts : states_removals : counters_match : counters_bad-
             offset : counters_fragment : counters_short : counters_normalize
             : counters_memory ). Values are 64 bit unsigned integers.

     pfq     pf/altq queue statistics ( sent_bytes : sent_packets : drop_bytes
             : drop_packets ). Values are 64 bit unsigned integers.

     proc    Process statistics ( number : uticks : sticks : iticks : cpusec :
             cpupct : procsz : rsssz ).

     sensor  Single sensor measurement offered with 7.6 precision. Value
             depends on sensor type.

     smart   SMART attributes ( read_error_rate: reallocated_sectors:
             spin_retries: air_flow_temp: temperature: reallocations: cur-
             rent_pending: uncorrectables: soft_read_error_rate:
             g_sense_error_rate: temperature2: free_fall_protection ). Values
             depend on drive model and may change between models.

     flukso  Average pwr sensor value offered with 7.6 precision. Value is a
             moving average and will depend on the number of measurements seen
             in a particular symon interval.

SIGNALS
     SIGHUP  Causes symux to read /etc/symux.conf or the file specified by the
             -f flag.  symux will keep the old configuration if errors occured
             during parsing of the configuration file.

FILES
     /var/run/symux.pid
             Contains the program id of the symux daemon.

     /etc/symux.conf
             symux system wide configuration file.

LEGACY FORMATS
     symux supports symon(8) clients that send

     pre OpenBSD 3.5 disk statistics.
             These streams should be identified as io1(<disk>) instead of
             io(<disk>) in /etc/symux.conf. Also note that symon(8) measures
             io1 or io2 depending on whether it was compiled on a host that
             supports version 1 or 2. The rrd datastructures of these streams
             differ and there is no easy way to change an io1 rrd into an io2
             rrd.

     pre symon 2.78 mem/if statistics.
             These streams should be identified as if1(<interface>) and mem1()
             in /etc/symux.conf. symon versions 2.78 and up will always report
             if2 and mem2 statistics. The rrd files for the old and new probes
             are identical and need not be changed.

     symux will output what version of information it is offered by symon(8)s
     on the network when started with the -d flag.

BUGS
     symux writes incoming data to rrd files "in process". An rrdupdate on a
     somewhat stale rrdfile -- with the last data from quite some time in the
     past -- is a very expensive operation. This can cause symux to lockup
     while rrdupdate is updating the rrd file.  symux will be unresponsive
     during this process.

AUTHOR
     Willem Dijkstra <wpd@xs4all.nl>. Daniel Hartmeier helped to port to big-
     endian architectures. Matthew Gream helped to port symon to other BSD
     platforms.

     Port contributors: Marc Balmer, Tito Dal Canton, Matthew Gream, Daniel
     Hartmeier, Lars Kotthoff, Constantine A. Murenin, J. Martin Petersen,
     Fredrik Soderblom, Harm Schotanus and Martin van der Werff.

SEE ALSO
     symon(8)

BSD                              April 4, 2012                             BSD
                

Installation notes
==================

Privileges
==========
symux needs read and write access to its rrdfiles.

symon needs to interface with your kernel. Depending on your host system this
leads to different privilege requirements:

OpenBSD:  - no privs: cpu, debug, df, if, io, mbuf, mem, proc, sensor
          - rw on /dev/pf for pf

NetBSD:   - no privs: cpu, debug, df, if, io, mbuf, proc
          - r on /dev/sysmon for sensor

FreeBSD:  - no privs: all
          - non-chroot on FreeBSD 5.x for CPU ticks in proc
          - rw on /dev/pf for pf and pfq

Linux:    - r on /proc/net/dev: if
          - r on /proc/stat: cpu, cpuiow
          - r on /proc/meminfo: mem

all:
          - r on chroot/etc/localtime for proper timezone logging

Real quick on OpenBSD
=====================
(cd /usr/ports/net/rrdtool && make install) &&
        make &&
        make install &&
        vi /etc/symux.conf /etc/symon.conf &&
        ~symon/symux/c_smrrds.sh all &&
        /usr/local/libexec/symux &&
        useradd -d /var/empty -L daemon -c 'symon Account' -s /sbin/nologin _symon
        /usr/local/libexec/symon

or grab the port and do
make package
pkg_add symon-mon-version.tgz on all monitored hosts
pkg_add symon-mux-version.tgz on the loghost

Install the seperate syweb package to show the data stored in the rrd files.

Less quick, but all OSes
========================
- Install rrdtool on the host that will also run your symux gatherer.
        BSDs: cd /usr/ports/net/rrdtool && make install

- Check Makefile.inc for settings. Things to watch out for are:

     + PREFIX = Where does the installation tree start. Defaults to
                '/usr/local'.

     + BINDIR = Where should the daemons be put, relative to $PREFIX. Defaults
                to 'libexec'.

     + MANDIR = Where should the manuals be installed, relative to
                $PREFIX. Defaults to 'man'.

     + SHRDIR = Where are the example configurations to be installed. Defaults
                to 'share/symon'.

     + RRDDIR = $RRDDIR/include should yield rrd.h. Define SYMON_ONLY in the
                environment or on the make command line to render this mute.

     + INSTALLUSER / GROUPDIR / GROUPFILE = user and groups that install should
                use.

  Note that:

     + you can define SYMON_ONLY if you do not want to compile symux / do not
       have rrdtool installed.

     + symon/platform/os/Makefile.inc is read before Makefile.inc; define
       your vars in the environment, or in Makefile.inc with != to force
       overwriting the defaults.

   BSDs: Run make && make install
  Linux: Run pmake && pmake install || bmake && bmake install

- Create an '/etc/symon.conf' for each monitored host and one symux.conf for
  the gatherer host. See the manual pages on how to specify alternative
  configuration file locations, if necessary. Note that there are example
  configurations for both in $PREFIX/$SHRDIR.

- Create the rrd files where the incoming symon data is to be
  stored. $PREFIX/$SHRDIR/c_smrrds.sh and symux -l are your friends. Note that
  syweb expects an '.../machine/*.rrd' style directory structure somewhere
  under /var/www.

- Ensure that /etc/localtime is accessible by symon/symux. Failing this you
  will get log messages in GMT. Note that etc is chroot/etc when symon
  chroots.

- Both symon and symux will daemonize if started normally. Start them with
  debugging on initially to iron out any configuration problems:

  $PREFIX/$BINDIR/symux -d &
  $PREFIX/$BINDIR/symon -d

- Remove -d flags and check system logs for any failures.

- Only if you need the webinterface: download and install syweb.

Getting measurements without the web
====================================

The client directory contains a perl module 'SymuxClient.pm' that can be used
to read measurements as they come in at the symux host. A sample Perl program
called 'getsymonitem.pl' shows how to use the module.

Example:

nexus$ getsymonitem.pl 127.0.0.1 2100 127.0.0.1 "cpu(0)" "idle"
93.40

Historical data can be gathered using rrdfetch(1) from symux's rrd files.

Portability
===========

This package was originally built as an OpenBSD application. It now has support
for FreeBSD, NetBSD and Linux.

Willem Dijkstra - wpd@xs4all.nl

                

Installation
------------

Download syweb-version.tar.gz from the website and copy htdocs/ and symon/ from
the archive into ~www. Install rrdtool into the chroot using
~syweb/install_rrdtool.sh.

Make your symon rrds accessable to apache. Move them into ~www/symon/rrds
and update symux.

Customise ~www/htdocs/syweb/setup.inc. The supplied setup.inc should work
fine if you are using  apache on FreeBSD and have copied all the files as
instructed. Interesting things to set in setup.inc:

$symon['cache_duration'] - set this to the number of seconds that generated
                           graphs should be kept for.

$symon['layout_dir']     - remove this to disable non-default layouts

You can also comment out specific sizes and timespans in $symon['defaults'] by
adding // in front of them. This can be used to restrict graphs to certain
sizes/timespans, as the larger ones cost hefty amounts of cpu and ram to
compute.

Point your browser to http://127.0.0.1/syweb/configtest.php to test the
configuration. This page will show if the application cannot deal with a
setting in setup.inc.

Point your browser to http://127.0.0.1/syweb/index.php for the main page.

Your graphs may be off by a couple of hours. Let apache know in what timezone
you live by adding a etc/localtime in your chrooted apache dir.

You can move index_noui.php to index.php if you do not need the ui.