Overview

InCenter v3 On-prem service is a log analytics tool that provides a set of network security and user authentication dashboards for Clavister NetWall and NetShield Firewalls, and IdAuth services.

It runs as a Kubernetes deployment under target customer’s environment.

Components

InCenter v3 On-prem consists of following micro-services:

InCenter micro-services

External services dependencies 

SMTP server

In order for the system to be able to send information by email, for instance for alerts or reports, an SMTP server can be specified. See the SMTP section in the Configuration manual for setting this up.

Install

Prerequisites

• Helm
• kubectl
• Ingress controller installed in Kubernetes cluster.

Steps

1. Create a values.yaml file with appropriate values for your environment. See the dedicated Helm values section for guidance.
2. Create a namespace:
   kubectl create namespace incenter
3. Install InCenter:
   helm install incenter --namespace incenter oci://public-registry.clavister.cloud/incenter/incenter-onprem --version 3.9.3-rc4 -f values.yaml

• version 3.9.3-rc4 is not configured to use private repo, in order to pull images below commands need to be executed
  
  

kubectl create secret docker-registry regcred -n "NAMESPACE" --docker-server=public-registry.clavister.cloud --docker-username='robot$reg_incenter+public' --docker-password="PASSWORD"
for i in analytics authentication core logdb logreceiver webui; do kubectl patch deployment $i -n "NAMESPACE" -p '{"spec":{"template":{"spec":{"imagePullSecrets":[{"name":"regcred"}]}}}}'; done

Post-install

After the installation is complete, a message is displayed with information on how to get started. The message can also be viewed again with helm status incenter.

If you encounter any problems, please see the dedicated troubleshooting section in this document.

Configure

All configuration for the On-prem release is done by setting helm values. Some values are required and must be set prior to the installation process. All values may be changed at any time using the “helm upgrade” command. Please consult the official helm documentation for up-to-date information on how this is done.

A table of supported helm values is provided below.

Helm values

Backup and Restore

MongoDB

One easy way to back-up and restore the data in the configuration database (MongoDB) is to use the mongodump and  mongorestore commands, and copying the resulting file.

Backup

Assuming the kubectl context is configured to use the Kubernetes cluster and the namespace that InCenter is installed in, these commands can be used, assuming and sh-like shell:

mongoPod=$(kubectl get pod -l app=mongodb -o name)
kubectl exec "${mongoPod}" -- mongodump --oplog --archive=/mongodb.backup.archive --gzip
kubectl cp "${mongoPod#pod/}":/mongodb.backup.archive mongodb.backup.archive

Restore

Please note that the following will completely remove all current data in the MongoDB service, and replace it with what has been stored in the archive.
Assuming that the back-up archive is in the current directory and is named mongodb.backup.archive and again, assuming the kubectl context is configured to use the Kubernetes cluster and the namespace that InCenter is installed in, these commands can be used, assuming and sh-like shell:

mongoPod=$(kubectl get pod -l app=mongodb -o name)
kubectl cp mongodb.backup.archive "${mongoPod#pod/}":/mongodb.backup.archive 
kubectl exec "${mongoPod}" -- mongorestore --oplogReplay --archive=/mongodb.backup.archive --gzip --drop

Stopping and starting back-end processes

For the safest results, the running processes using the database can be stopped before the back-up or restore is performed. To do this, the following command can be used before the ones mentioned above:

kubectl scale rs -l app_group=incenter --replicas=0

Afterwards, the processes can be started up again with:

kubectl scale deploy/core deploy/analytics deploy/authentication deploy/webui --replicas=1

OpenSearch

It is possible to back-up the data in OpenSearch by creating a snapshot and subsequently copying the files from the pod. The restore operation is similar, but in reverse order.

For both the back-up and restore operations gain access to the OpenSearch API by port-forwarding it to your local machine:

kubectl port-forward services/logdb 9200:9200

Keep the command running until done.

Backup

Register the snapshot repository:

curl --request PUT \
  --insecure \
  --url https://127.0.0.1:9200/_snapshot/repo \
  --header 'authorization: Basic ZWxhc3RpYzplbGFzdGlj' \
  --header 'content-type: application/json' \
  --data '{
    "type": "fs",
    "settings": {
      "location": "/usr/share/opensearch/snapshots"
    }
  }'

Create the snapshot:

curl --request PUT \
  --insecure \
  --url 'https://127.0.0.1:9200/_snapshot/repo/1?wait_for_completion=true' \
  --header 'authorization: Basic ZWxhc3RpYzplbGFzdGlj' \
  --header 'content-type: application/json' \
  --data '{
    "indices": "*,-security-auditlog,-.*"
  }'

Copy all snapshots to a local directory:

podName=$(kubectl get pod -l app=logdb -o jsonpath='{.items..metadata.name}')
kubectl cp ${podName}:/usr/share/opensearch/snapshots ./snapshots

Restore

Copy the local snapshots to the pod:

podName=$(kubectl get pod -l app=logdb -o jsonpath='{.items..metadata.name}')
kubectl cp ./snapshots ${podName}:/usr/share/opensearch/snapshots

Register the snapshot repository:

curl --request PUT \
  --insecure \
  --url https://127.0.0.1:9200/_snapshot/repo \
  --header 'authorization: Basic ZWxhc3RpYzplbGFzdGlj' \
  --header 'content-type: application/json' \
  --data '{
    "type": "fs",
    "settings": {
      "location": "/usr/share/opensearch/snapshots"
    }
  }'

Restore the latest snapshot:

latestSnapshot=$(curl -sS --request GET \
  --insecure \
  --url 'https://127.0.0.1:9200/_cat/snapshots/repo?s=start_time%3Adesc' \
  --header 'authorization: Basic ZWxhc3RpYzplbGFzdGlj' \
  --header 'content-type: application/json' | head -n 1 | cut -d' ' -f 1)
curl --request POST \
  --insecure \
  --url https://127.0.0.1:9200/_snapshot/repo/${latestSnapshot}/_restore \
  --header 'authorization: Basic ZWxhc3RpYzplbGFzdGlj' \
  --header 'content-type: application/json' \
  --data '{
    "rename_pattern": "(.*)",
    "rename_replacement": "$0-restored",
    "include_aliases": false
  }'

This will create indexes, which are named differently than the original indexes. Please check that the data has now been restored.

Uninstall

Steps

1. Uninstall the helm release:
   helm uninstall -n incenter incenter
2. Delete the namespace:
   kubectl delete ns incenter

Troubleshoot

General

If something is not working as expected, it is generally good to start by verifying that all micro-services are running and healthy. Here are some useful commands for that purpose:

kubectl get pods
kubectl describe deployment/<app>
kubectl logs deployment/<app>

If a solution to a problem cannot be solved using this troubleshooting guide, please do not hesitate to contact our support team.

Forgot my password

Immediately after installation a default admin account with a random password is created. See the install section for details on how to retrieve it.

If you have changed the default password and forgotten the new password, as of now there is no simple way to retrieve or reset it. If there are other admin accounts configured, one of them can be used to reset your password on the users page. If this is not possible, please contact our support team.

Reports not being sent

Responsible micro-services: analytics

Make sure that an SMTP server has been correctly configured using the smtp-prefixed helm values.

To help with troubleshooting, it is possible to send a report manually instead of waiting for its scheduled occurrence:

1. Go to the reports page in the web interface.
2. Select any report so that the side drawer is opened.
3. In the drawer, select the "more options" symbol next to the title, then select "Send now" and follow the instructions.

Alerts not being sent

Responsible micro-services: analytics

Make sure that an SMTP server has been correctly configured using the smtp-prefixed helm values.

Related articles