Torrus Installation Instructions
  Required Software
    *   Operating System

        Torrus should easily run on any UNIX-like operation system, as long
        as the required components are provided.

        If the OS mounts /var/run directory to a temporary filesystem,
        "piddir" variable must point to a path on a persistent storage.
        Alternatively "piddir" can point to /var/run itself. The default
        path is /var/run/torrus, and temporary filesystems do not preserve
        subdirectories after a system reboot.

        Note to VmWare users: as recommended by Gord Philpott, Linux kernel
        for guest machine needs to be compiled with the following feature
        enabled: "Character devices ---> Enhanced Real Time Clock Support".
        For other guest operating systems (e.g. FreeBSD), in order to let
        the Torrus collector and monitor run properly, you need to add the
        following lines to your torrus-siteconfig.pl:

          $Torrus::Scheduler::maxSleepTime = 15;
          $Torrus::Scheduler::ignoreClockSkew = 1;

        It is recommended to use Tobi Oetiker's SDBS tool available at
        <https://github.com/oetiker/sdbs>. The tool compiles all Torrus
        prerequisites from sources, and guarantees the version
        compatibility.

    *   Perl

        Perl version 5.8.1 or higher is required. Version 5.8.0 is not
        recommended because of memory access bugs and unpredictable
        behaviour. Perl interpreter should be in PATH environment variable
        when running "./configure", otherwise use

          ./configure PERL=/path/to/perl

        Torrus also supports Perl multithreading in Device Discovery engine
        and in the Collector. Minimum requirements are as follows: Perl
        version 5.8.8 or higher, "threads" module version 1.41 or higher,
        "threads::shared" module version 1.03 or higher. Older versions had
        bugs that were leading to severe memory leaks and instability. After
        installing Perl 5.8.8, you need to upgrade the "threads" and
        "threads::shared" modules manually from CPAN. If any of the
        requirements is not met, the installer will automatically disable
        the threads support in Torrus.

    *   RRDtool

        Round Robin Database Tool is available at:
        <http://oss.oetiker.ch/rrdtool/>.

        The version that is coming with Debian and Ubuntu packages has shown
        some instability in Torrus WebUI -- most probably because of the
        bugs in graphical libraries that it uses. It is recommended to
        compile the tested versions with SDBS instead of using precompiled
        packages.

    *   libxml2

        libxml2 is the XML parser library from Gnome project
        <http://www.gnome.org>, available as a standalone package. Versions
        2.4.23 and above were tested.

    *   Berkeley DB version 4.2.52 or later

        There are two distinct packages with slightly conflicting names: The
        Berkeley DB from Sleepycat Software <http://www.sleepycat.com>, and
        "BerkeleyDB" Perl module available from CPAN. We will always refer
        the latter as *BerkeleyDB Perl module*.

        The Berkeley DB database engine is used for all data storage.
        Versions 4.2.52 and above are recommended. Older versions of
        Berkeley DB had problems with database integrity in concurrent
        usage. By default, it installs into /usr/local/BerkeleyDB.4.2/.

        Unfortunately, the BerkeleyDB Perl module cannot always find the
        include and library files in that directory. Thus either configure
        Berkeley DB with the option "--prefix=/usr/local", or you need to
        edit config.in in BerkeleyDB Perl module distribution.

    *   HTTP Server

        Torrus requires an HTTP server. It is compatible with the following
        combinations:

        *       lighttpd, ngnix or Apache with FastCGI

        *       Apache 1.3 with mod_perl 1.0 (not recommended and soon to be
                dropped)

        *       Apache 2.x with mod_perl 2.0 (not recommended and soon to be
                dropped)

        FastCGI is supported in Torrus version 1.0.9 or higher. It is the
        preferred way of using Torrus, more stable than mod_perl.

        Note: Starting from Torrus version 1.0.9, "libapreq2" is no longer
        required.

        See *Torrus Web Interface Reference* for details on HTTP server
        configuration.

    *   Perl Modules

        The Perl modules required are listed below. They are all available
        at CPAN <http://www.cpan.org>. Some of them require other modules to
        be installed. You can install all required modules with the
        following command, executed from within the distribution directory:

          perl -I `pwd`/setup_tools -MCPAN -e 'install Bundle::Torrus'

        *       XML::LibXML

                This is a Perl interface for libxml2, providing DOM
                standards compliant interface. It is recommended that you
                use version "1.54_3" or higher.

        *       BerkeleyDB perl module

                Perl interface to the database software. Version 0.19 or
                higher is required.

        *       Template-Toolkit

                This is a powerful set of tools for text template
                processing. Torrus uses it for HTML files output. See also
                the project homepage: <http://www.template-toolkit.org>

        *       Proc::Daemon

                Daemon invocation routines.

        *       Net::SNMP

                SNMP queries and traps interface. Version 4.0.3 or higher is
                required. *Note*: Torrus versions prior to 1.0.9 are
                incompatible with Net::SNMP version 6.0.0. You may need to
                downgrate the module to 5.2.0.

        *       URI::Escape

                Escape and unescape unsafe characters

        *       Apache (mod_perl version 1.0)

                Mod_perl is a Perl runtime environment integrated into
                Apache HTTP server.

        *       Apache::Session

                User session tracking helper

        *       Date::Parse and Date::Format

                Module for parsing the user input for date and time.

        *       JSON

                JSON data format support

    *   HTTP Browser requirements

        The HTTP browser must be compatible with *HTML 4.01 Strict* and
        *CSS2* specifications. The tests were made with the following
        browsers: IE 5.5, Opera 6.12B1/Solaris, Opera 7.03/Win2K, Netscape
        7.02.

  Installation Procedure
    Create the group "torrus" and a user "torrus". The user should be a
    member of this group.

    Add your Apache daemon user to the group "torrus". Other users that will
    run any of Torrus processes must be included in this group.

    For example, in Solaris (you may need to specify the GID and UID numbers
    according to your local administration policy):

      groupadd torrus
      useradd -c 'Torrus Daemon' -d /usr/local/torrus -g torrus torrus
      usermod -G www,torrus www

    Further installation process is mostly as usual (watch out "piddir"
    setting, as described above):

      ./configure
      make
      make install

    The default directory layout is described below. Should you need to
    change it, there is a number of variables and configuration options that
    you may use as "./configure" arguments. See "./configure --help" for
    details. Alternatively you can utilize the script
    "./setup_tools/configure_fhs", which is designed to provide an FHS
    compliant setup. The script is equivalent to executing

      ./configure \
       --prefix=/opt \
       --mandir=/opt/share/man \
       pkghome=/opt/torrus \
       sitedir=/etc/opt/torrus

    Do not try to change any paths by supplying them as "make" variables,
    this would most probably break the installation. The only "make"
    variable that is supported is "DESTDIR". It may be used for preparing
    the package for further distribution. For example, the following command
    would install all Torrus files as if /tmp/stage were the root of the
    filesystem:

      make DESTDIR=/tmp/stage install

    The presence of prerequisite Perl modules is checked during the
    execution of "./configure". You can disable this by giving
    *--enable-pkgonly* option.

    The installer sets up the site configuration files, but only if they
    don't already exist.

    Plugin modules should be installed separately, after the Torrus software
    is successfully installed. For each plugin, unpack the plugin
    distribution archive to some directory, and execute

      torrus install_plugin <UNPACKED_PLUGIN_DIR>

    A number of directories is set up by default under the path "/var", and
    they must be writable by all Torrus processes, including the Apache web
    server.

    You can control these directories' access rights by setting the
    following environment variables: *var_user*, *var_group*, *var_mode*,
    like follows:

      ./configure var_group=wwwrun

    Default values for operating systems other than Cygwin are:
    *var_user=torrus*, *var_group=torrus*, *var_mode=775*. Setgid bit is set
    by default for these directories.

    If available, you may place these directories on a fast media, for
    example, a robust disk array. Changing the "varprefix" variable would be
    generally enough, for example:

      ./configure varprefix=/vol1/torrus_var

    Attention for the systems that have /var/run in a temporary directory.
    By default, Torrus installer creates a directory /var/run/torrus to
    store the daemon PID files. After the server reboot, the temporary
    filesystems are cleaned up, and the directory would not exist any more.
    Use *piddir=/var/torrus/run* or similar in the ./configure arguments.

  Logging
    As of Torrus version 2.02, the collector and monitor daemons send their
    logs to a local SYSLOG server in "local0" facility by default.

    The SYSLOG facility can be altered in torrus-siteconfig.pl:

      $Torrus::Log::syslogFacility = 'local5';

    Also the following two statements would revert to the old behavior (log
    files written directly by the daemons). Please not that the daemons
    would not rotate the logs on SIGHUP:

      $Torrus::Collector::useSyslog = 0;
      $Torrus::Monitor::useSyslog = 0;

    If your system policy does not allow to use the system SYSKLOG daemon,
    you can run your own (rsyslogd is recommended) and attach it to a UNIX
    socket. In this case, the following statement in siteconfig would direct
    all logging to an alternative SYSLOG socket:

      $Torrus::Log::syslogSockOpt = ['unix', '/home/jsmith/tmp/syslog.sock'];

  Torrus directory layout
      /usr/local/share/torrus/
            Home directory for Torrus distribution files

      /usr/local/share/torrus/conf_defaults/
            torrus-config.pl and other configuration files

      /usr/local/libexec/torrus/
            Command-line executables

      @docdir@/
            POD and TXT documentation files

      /usr/local/share/examples/torrus/
            Miscelaneous example files

      /usr/local/lib/perl5/site_perl/
            Perl libraries

      /usr/local/share/torrus/plugins/
            Plugins configuration

      /usr/local/libexec/torrus/scripts/
            Scripts

      /usr/local/share/torrus/sup/
            Supplementary files, DTDs, MIBs, color schemas, web plain files

      /usr/local/share/torrus/templates/
            Renderer output templates

      /usr/local/share/torrus/xmlconfig/
            Distrubution XML files

      /usr/local/etc/torrus/
            Site configurable files

      /usr/local/etc/torrus/conf/
            Place for torrus-siteconfig.pl and other siteconfigs

      /usr/local/etc/torrus/conf/discovery/
            Devdiscover input files

      /usr/local/etc/torrus/templates/
            User-defined Renderer output templates

      /usr/local/etc/torrus/xmlconfig/
            User XML configuration files

      /usr/local/share/man
            Place for man pages. All articles have the prefix C<torrus_>

      /var/log/torrus/
            Daemon logfiles

      /var/run/torrus
            Daemon PID files

      /var/torrus/cache
            Renderer cache

      /var/torrus/db
            Berkeley DB home

      /var/torrus/session_data/lock
      /var/torrus/session_data/store
            Web interface session files

      /var/torrus/collector_rrd
            Recommended directory for RRD files generated by collectors

  Configuring Torrus
    The datasources are configured with %Torrus::Global::treeConfig hash in
    torrus-siteconfig.pl.

    In this hash, the keys give the tree names. The values for each tree
    name are pointers to hashes, with the following keys and values:
    *xmlfiles* points to an array of source XML file names; *run* points to
    a hash with the names of the daemons that would be automatically
    launched for the tree; *desription* gives a short line describing the
    tree contents.

    Two additional arrays: @Torrus::Global::xmlAlwaysIncludeFirst and
    @Torrus::Global::xmlAlwaysIncludeLast give a list of source XML files
    that are included in every tree, in the beginning or in the end of the
    XML files list. By default, this array consists of two files:
    defaults.xml and site-global.xml. The second one is resided in the
    user-configurable directory, and you can use it to override any default
    settings.

    Example:

      @Torrus::Global::xmlAlwaysIncludeFirst =
          ('defaults.xml', 'site-global.xml');

      %Torrus::Global::treeConfig = (
        'tree_A' => {
          'description' => 'The First Tree',
          'xmlfiles' => [qw(a1.xml a2.xml a3.xml)],
          'run' => { 'collector' => 1, 'monitor' => 1 } },
        'tree_B' => {
          'description' => 'The Second Tree',
          'xmlfiles' => ['b1.xml', 'b2.xml'],
          'run' => {} }
       );

    XML files are read additively within each tree, in the order as they are
    listed. The XML compiler searches for the files in several directories,
    listed in @Torrus::Global::xmlDirs. By default, this list conssists of
    two paths, one for Torrus distribution files, and the other for user
    files.

    Files generated by "devdiscover" usually contain *include* statements
    which add the vendor-specific XML files to the compilation.

    Below follows a short description of some XML files that come with
    Torrus distribution. Only site-global.xml is installed in the directory
    for user-configurable files, all others are placed in the distribution
    directory.

    *   defaults.xml

        Default parameters for the root of the datasources tree. Default
        view definitions. Note: this file is automatically overwritten by
        "make install".

    *   site-global.xml

        Parameters that you want to change or define for your site. This
        file will be compiled for every tree after defaults.xml, and this is
        the place to override the defaults. The file that is supplied with
        the Torrus distribution has some useful parameters that you may
        simply uncomment. Note: this file is never overwritten by "make
        install".

    *   snmp-defs.xml

        SNMP collector defaults. The file defines several templates used for
        collecting SNMP data. Do not change this file. You may redefine the
        needed parameters in your site-specific files and templates.

    *   vendor/<vendor>.<product>[.<subsystem>].xml

        SNMP collector definitions and templates for various hardware
        vendors and products. "devdiscover" includes some of these files
        automatically in the configuration.

    *   generic/*.xml

        SNMP collector definitions and templates for vendor-independent
        MIBs. Most files are named after corresponding RFC numbers.

    In addition, the distribution package contains several example files.

    For more details about XML configuration, see *Torrus User Guide* and
    *Torrus XML Configuration Guide*.

  Site configuration options
    In addition to *%Torrus::Global::treeConfig*, you may wish to set some
    other parameters in your site configuration file (torrus-siteconfig.pl).

    See torrus-config.pl for the complete list of varaibes that you may
    override in your site config. Among them, most interesting are:

    *   $Torrus::Renderer::companyName

        The text that you specify here will appear in the top left corner of
        all HTML pages.

    *   $Torrus::Renderer::companyURL

        The company name text will be clickable with the URL specified in
        this variable.

  Apache HTTP server configuration
    Torrus web interface is designed to run under mod_perl environment
    (<http://perl.apache.org>).

    See the *Torrus Web Interface Reference* document for detailed
    instructions on Apache configuration.

  Access Control Lists
    By default, Torrus web interface requires user authentication. You can
    disable this by changing $Torrus::CGI::authorizeUsers to zero in your
    torrus-siteconfig.pl.

    ACLs are controlled by "acledit" utility. See *Torrus Manual pages* for
    detailed description. Example:

      torrus acledit --addgroup=staff --permit=DisplayTree --for='*'
      torrus acledit --addgroup=staff --permit=GlobalSearch --for='*'
      torrus acledit --adduser=jsmith --password=mysecretpassword \
        --cn="John Smith" --addtogroup=staff
      torrus acledit --addgroup=admin \
        --permit=DisplayTree --permit=DisplayAdmInfo --for='*'

  Cron Job
    In order to clean old HTTP session data, it is recommended to run
    cleanup script in a cron job, once per day:

     #min hour mday month wday    who     command
     5    3    *    *     *       root    /usr/local/libexec/torrus/cleanup

  Startup script
    The Torrus distriubution provides a startup script which you can install
    in the init scripts directory (/etc/init.d on most Unixes or
    /usr/local/etc/rc.d/ on FreeBSD). The script "torrus" is created during
    the installation process in the subdirectory <init.d> of the
    distribution directory.

    The init script reads some parameters from
    /usr/local/share/torrus/conf_defaults/initscript.conf, and
    /usr/local/etc/torrus/conf/initscript.siteconf if the latter exists. By
    default, it makes the monitor daemons sleep for 20 minutes when they are
    launched simultaneously with collector daemons.

  Upgrade Procedure
    Follow these instructions when upgrading from previous Torrus release.

    In the previous distribution directory, look up the config.log file. It
    contains the configure options that you used in previous installation.

    Unpack the new release distribution.

    Run "./configure" with the same options you used before.

    Stop the collector and monitor processes (usually by "/etc/init.d/torrus
    stop").

    Stop Apache process.

    Run "make install".

    Start the collector and monitor processes.

    Start Apache process.

    Also it is recommended to re-compile your XML configuration. If you
    prefer not to do this, it's recommended that you clear the Web interface
    cache both in your browser and in Torrus:

      torrus clearcache

  Plugins
    As of Torrus version 2.01, plugins inherit a new release numbering
    scheme: a plugin release version is a 3-component number, where the
    first two numbers are the Torrus release version that is compatible with
    this plugin.

  Planning the disk space
    In a typical Torrus setup, there are two disk space consuming entities:
    the RRDtool data storage and the Berkeley DB database.

    RRDtool data files (or RRD's) are used to store the values that are
    gathered by the collector daemons. These are the most intensively
    updated files, so the disk speed is important here. If possible, it is
    better to spread the RRD files across several physical disks.

    Berkeley DB database is used to keep all the configuratiuon data for
    your datasource trees, and it also keeps some live status information.
    It's intensively updated during XML compilation only. During normal
    Torrus working cycle, there are only few updates, not very critical in
    time. The database is read quite intensively during collector
    initialization, but usually it's the CPU speed that causes more delay.

    For a typical Torrus installation with standard RRD creation parameters,
    the rough estimation is as follows: the Berkeley DB database occupies
    from 10 to 13 KB per collector datasource, and the RRD storage takes
    approximately 80 to 85 KB per datasource.

    For a medium-sized system, few hundred megabytes for the Berkeley DB
    database and several gigabytes for RRD storage would be sufficient.
    Larger systems require a thorough design and staging work. See *Torrus
    Scalability Guide* for more details.

  Network performance tuning
    In large installations, the SNMP collector experiences quite intensive
    input traffic bursts. Thus the UDP socket receive buffers should be
    adapted to sustain the load. By default, Torrus sets the UDP receiving
    buffer size, SO_RCVBUF, to 131071 bytes. This should fit most of
    installations. However, it's useful to check the network statistics if
    there are any UDP buffer overflows. On most systems, the commands
    "netstat -s -p udp" or "netstat -s" should show you these counters. The
    maximum buffer size is usually limited by a system kernel variable, and
    can be increased if needed. See $Torrus::Collector::SNMP::RxBuffer and
    its comments in torrus-config.pl for more details.

Author
    Copyright (c) 2002-2012 Stanislav Sinyagin <ssinyagin@k-open.com>

