Configuring Linux Monitoring in Nectus

Nectus Server 1.55 adds feature to monitor some system parameters on hosts running Linux. Enabling this feature for your infrastructure requires two steps:

Configuring of Nectus Server;
Installing and configuring of Nectus remote agents on target Linux machines.

This guide describes steps for Nectus Server configs. Installing and configuring of Linux remote agents are described here.

Linux monitoring integration

Choose the following menu item from the Nectus GUI to set global Linux monitoring parameters.

This will open the next window:

Agent mode determines whether the agent runs in passive mode (when it waits for an incoming connection from the server and then collects the required metrics) or in active mode when the agent connects to the server pro-actively and sends the collected data.
Server TCP port sets the server port number for incoming connections from the agents running in active mode. If you set Agent mode to passive then this value is not used.
Pre-shared Key determines the secret key used to encrypt and decrypt messages between Nectus Server and the monitoring agents. You can set string of any length but recommended length is from 8 to 16 characters. The exactly same key should be set in all agents’ configuration files (parameter SERVER_PRESHARED_KEY, see details here).

Click OK to save configured parameters.

Select “Linux Monitoring Settings” from the “Monitoring” menu to configure your monitoring profiles for Linux machines.

Click “Create” to create a new monitoring profile:

In the following form enter the name of your profile and choose metrics you would like to monitor:

Configure the necessary metric parameters like minimal and maximal thresholds, alerting etc.

You can create any number of profiles and assign each of them to different group of Linux machines.

Linux Servers

Select “Linux Servers” from the “Inventory” menu to configure groups and servers to be monitored.

Click “Create new Group” in the next form:

Enter “Group Name” for the new group, select “Enable monitoring” and choose the desired Monitoring Profile for this server group:

Also select the appropriate alert recipients in this form.

You can create any number of monitoring groups if you need to apply different profiles or alert recipients for different server groups.

When you have finished with the groups switch to the next tab “Servers” in the same form and click “Add Server” to add a Linux server to be monitored:

In the next window enter the Hostname (optional, it will be obtained later during the monitoring process), IP address, TCP port of the remote agent’s listening socket (default 5400) and choose the appropriate Group:

You can choose different values of TCP port for different servers according to your network configuration. But note that this value must match the parameter AGENT_PORT set in the agent’s configuration file (see details here).

After clicking “OK” you will see the server in the following list and you will be able to View, Edit and Delete the server (operations are marked with a red square).

After you have added all the desired servers and installed the remote agents the process of monitoring of your Linux hosts will start automatically.

Linux Monitoring Agent Installation

Nectus Linux Monitoring Agent is required to be installed on target machine to enable Nectus to monitor Linux system parameters such as CPU and RAM utilization, Storage and Network Utilization.

Agent collects all the required monitoring metrics and reports to the main Nectus Server every 5 minutes.

Step 1: Download Monitoring Agent Package

Download binary package appropriate to your target machine Linux flavor and version from the download page.

Step 2a: Installation for Debian-based systems (including Ubuntu)

After downloading the package run the following installation command:

$ sudo apt-get install -y <package-name>

This will install Nectus monitoring agent on your target machine. If everything is OK you’ll see an output like this:

$ sudo apt install -y ./nectus-agent_1.4-3_amd64.deb
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
Note, selecting ‘nectus-agent’ instead of ‘./nectus-agent_1.4-3_amd64.deb’
The following NEW packages will be installed:
nectus-agent
0 upgraded, 1 newly installed, 0 to remove and 81 not upgraded.
Need to get 0 B/843 kB of archives.
After this operation, 2,654 kB of additional disk space will be used.
Get:1 /home/oleg/nectus-agent_1.4-3_amd64.deb nectus-agent amd64 1.4-3 [843 kB]
Selecting previously unselected package nectus-agent.
(Reading database … 278414 files and directories currently installed.)
Preparing to unpack …/nectus-agent_1.4-3_amd64.deb …
Unpacking nectus-agent (1.4-3) …
Setting up nectus-agent (1.4-3) …
Created symlink /etc/systemd/system/multi-user.target.wants/nectus-agent.service → /lib/systemd/system/nectus-agent.service.

Step 2b: Installation for RedHat, CentOS and Amazon Linux

After downloading the package run the following installation command:

$ sudo yum install -y <package-name>.rpm

You’ll see an output like this:

$ sudo yum install -y nectus-agent-1.4-3.el9.x86_64.rpm

Last metadata expiration check: 0:00:12 ago on Wed 27 Sep 2023 10:40:52 +07.
Dependencies resolved.
==================================================================================================================
Package Architecture Version Repository Size
==================================================================================================================
Installing:
nectus-agent x86_64 1.4-3.el9 @commandline 1.0 M

Transaction Summary
==================================================================================================================
Install 1 Package
Total size: 1.0 M
Installed size: 4.3 M
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: nectus-agent-1.4-3.el9.x86_64 1/1
Installing : nectus-agent-1.4-3.el9.x86_64 1/1
Running scriptlet: nectus-agent-1.4-3.el9.x86_64 1/1
Verifying : nectus-agent-1.4-3.el9.x86_64 1/1

Installed:
nectus-agent-1.4-3.el9.x86_64

Complete!

Step 3: Configuration

After installation is complete you need to set up the agent’s basic configuration.

Nectus agent configuration file is located at: /etc/nectus/nectus-agent.conf.

Default configuration file content is listed in Appendix 1. Most of the values are set to defaults and don’t require a change except following two parameters:

SERVER_PRESHARED_KEY
AGENT_PORT

SERVER_PRESHARED_KEY is used to encrypt messages between the Agent and Nectus Server. Initially it’s set to value <YourSecretKey>.

Edit this parameter to match the key configured in Nectus GUI (Settings->General Settings ->Linux Monitoring Integration).

SERVER_PRESHARED_KEY is a global parameters that is used for all Linux Servers.

AGENT_PORT parameter determines the TCP port used by Agent to listen for incoming connections from the Nectus server. By default the TCP port is set to 5400.

AGENT_PORT is Server specific parameter that can be different on each Linux Server.

Step 4: Generate SSL Certificate (optional)

The Linux Monitoring Agent listens on pre-configured TCP port for incoming HTTPS connection and needs an SSL certificate to operate.

Default location for storing certificate files is /etc/nectus but it’s configurable (parameter SSL_CERTIFICATE_DIR in the configuration file).

You can either use any existing certificate or generate a new self-signed one.

To generate new self-signed certificate run commands:

$ cd /etc/nectus

$ openssl dhparam -out dh.pem 2048

$ openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 10000 -out certificate.pem

The directory pointed by SSL_CERTIFICATE_DIR parameter should finally contain files dh.pem, key.pem and certificate.pem.

Step 5: Open TCP port at the Firewall

If there is firewall between Nectus Server and Linux Monitoring Agent you need to permit connections on TCP port 5400 (Unless it is changed to a different port)

Make sure your Linux internal firewall also permits inbound connections on this port.

Step 6: Start Agent

Restart the agent in order for configuration changes to take effect:

$ service nectus-agent restart

Check it’s status to ensure proper operation:

$ service nectus-agent status

If case of successful start you’ll see some output like this:

$ service nectus-agent status

● nectus-agent.service – Nectus Agent

Loaded: loaded (/lib/systemd/system/nectus-agent.service; disabled; vendor preset: enabled)

Active: active (running) since Mon 2019-04-29 17:55:39 UTC; 4s ago

Process: 10247 ExecStart=/usr/bin/nectus-agent $CONFFILE (code=exited, status=0/SUCCESS)

Main PID: 10255 (nectus-agent)

Tasks: 2

Memory: 1.2M

CPU: 14ms

CGroup: /system.slice/nectus-agent.service

└─10255 /usr/bin/nectus-agent /etc/nectus/nectus-agent.conf

Apr 29 17:55:39 ubuntu-16 systemd[1]: Starting Nectus Agent…

Apr 29 17:55:39 ubuntu-16 systemd[1]: nectus-agent.service: PID file /run/nectus/nectus-agent.pid not readable

Apr 29 17:55:39 ubuntu-16 systemd[1]: Started Nectus Agent.

User account

For security reasons it’s not allowed to run Monitoring Agent as root user. If it’s trying to start as ‘root’ then the agent will drop its privileges to the user account set in configuration file

(usually /etc/nectus/nectus-agent.conf) as USER_INSTEAD_OF_ROOT.

Default USER_INSTEAD_OF_ROOT is ‘nectus’.

The agent automatically creates this user account during installation (if this user does not exist already) and provides minimal set of permissions required to run the agent code.

Troubleshooting

Nectus agent runs as a daemon process. You can use the following commands to start, stop restart and check status of the agent:

$ service nectus-agent start

$ service nectus-agent stop

$ service nectus-agent restart

$ service nectus-agent status

If there are any problems with starting the agent first check the agent’s status:

$ service nectus-agent status

Then check the log file (usually /var/log/nectus/nectus-agent.log). If it does not help then you can try to run the agent manually. Usage of the module:

nectus-agent <config-file> [-D]

config-file full path to configuration file

-D debug mode (agent starts not as a daemon but as a usual process)

Locate the executable nectus-agent in /usr/bin and try to run the agent not as a daemon:

$ /usr/bin/nectus-agent /etc/nectus/nectus-agent.conf -D

Check the output and log files at /var/log/nectus.

Upgrade

Upgrade to a new version of the agent is the same as a new installation: download the appropriate package and run either:

> sudo apt-get install -y <package-name>.deb

for Debian/Ubuntu systems or

> sudo yum install -y <package-name>.rpm

for RedHat/CentOS/Amazon Linux.

Uninstallation

To uninstall Nectus agent run the following command:

> sudo apt-get remove nectus-agent

for Debian/Ubuntu systems or

> sudo yum remove nectus-agent

for RedHat/CentOS/Amazon Linux.

Appendix 1. Default Configuration File

#########################################

#### GENERAL PARAMETERS ####

AGENT_IP 0.0.0.0 # IP address of listening socket.

# Default 0.0.0.0 (listen on all interfaces)

AGENT_PORT 5400 # TCP/IP port of the listening socket. Default 5400

SSL_CERTIFICATE_DIR /etc/nectus # Path to agent’s SSL certificate.

# Directory should contain files:

# dh.pem, certificate.pem and key.pem.

# To generate those files (self-signed certificate) run commands:

# openssl dhparam -out dh.pem 1024

# openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 10000 -out certificate.pem

# Maybe you would need root privileges for this, then run: sudo openssl etc…

SERVER_PRESHARED_KEY <YourSecretKey> # Pre-shared key to encrypt messages between

# the agent and Nectus monitoring server.

# Must be the same as set in Nectus GUI.

NUM_OF_THREADS 1 # Number of agent’s threads. Valid values are from 1 to 32.

# Default value = 1.

USER_INSTEAD_OF_ROOT nectus # Drop privileges to a specific user existing in the system.

# Only has effect if the agent is run as root.

# Default value = nectus.

#########################################

######### Logging parameters ############

LOG_DIR /var/log/nectus # Path to log files

LOG_LEVEL info

# Log verbosity level. Possible values are:

# error – only errors are reported in log

# info – default logging level

# debug – most verbose level including debug information.

LOG_FLUSH_PERIOD_SECONDS 30 # Flush log data to disk every X seconds.

# Valid values are from 1 to 900. Default value = 30.

LOG_FILE_MAX_SIZE_MBYTES 30 # Maximum size of a single log file. After reaching this size

# next log file is opened. Valid values are from 1 to 1000.

# Default value = 30.

LOG_FILE_MAX_COUNT 30 # Maximum count of log files stored. After reaching this count

# the oldest log files are deleted.

# Valid values are from 1 to 1000. Default value = 30.