checkmk docker containers as hosts

In this guide, I will explain how to automatically add docker containers as hosts in checkmk

Prerequisites

1. You need to install the checkmk agent on your linux docker host(s). https://docs.checkmk.com/latest/en/agent_linux.html
2. Open web interface from checkmk server and log-in with admin account.
3. Create a new User (if not already existent) with the following name: automation
4. Asign the Role Administrator to the automation-User.

1. Following these instructions https://docs.checkmk.com/latest/en/monitoring_docker.html#install_agent_plugin to install the plugin mk_docker.py on Linux docker host(s). You only need to complete step 2.1 "Installing the agent and plug-in" in the linked instruction.
2. Next, create a host folder "Docker Container". In a later step, host objects are automatically created into this folder, each of which represents a container. In addition, set the host folder properties as follows:

3. Next, the following bash script is created on the checkmk server. This script looks in the piggyback data for container information of the Linux docker host(s) and automatically creates host objects via the REST API in the previously created host folder "Docker Container"

   nano ~/docker-container-checkmk.sh

4. Paste the following content into the nano editor. Please edit the following variables:

   • SERVER_IP: IP-Address or DNS name of your checkmk-Server.

   • SITE_NAME: Site where you want to add the host/client-system to.

     More info what a site is in checkmk: https://docs.checkmk.com/latest/en/intro_setup.html#create_site

   • PASSWORD: Password of the newly created or existent user automation

   #!/bin/bash
   
   # parameters needed to be set site-specific.
   SERVER_IP="172.25.17.3"
   SITE_NAME="cmk" # More infos: https://docs.checkmk.com/latest/de/intro_setup.html#create_site
   API_URL="http://$SERVER_IP/$SITE_NAME/check_mk/api/1.0"
   USERNAME="automation"
   PASSWORD="<PASSWORD_OF_THE_AUTOMATION_USER>"
   FOLDER="/docker_container" # (Optional) More infos: https://docs.checkmk.com/latest/de/hosts_structure.html?#folder
   # End specific parameters
   ##### below should not be changed unless you are absolutely sure in what you are doing !
   
   # Check if there are any pending changes on the checkmk server
   temp_file=$(mktemp)
   # Get Content-Length from "pending changes" object via REST-API
   curl -s -S -I -H "Authorization: Bearer $USERNAME $PASSWORD" -X GET "$API_URL/domain-types/activation_run/collections/pending_changes" > "$temp_file" 2> error.log &
   printf "\n\n\nGet Content-Length from 'pending changes' object via REST-API\n"
   # Extract the ETag value from the file content
   result=$(grep -iE '^Content-Length: ' "$temp_file" | awk '{print $2}' | tr -d '"' | tr -cd '[:alnum:]')
   # Delete the temporary file
   rm "$temp_file"
   if [ "$result" -gt 350 ]; then
    printf "$(tput setaf 1)%s$(tput sgr0)\n" "Please revert or accept pending change(s) on the checkmk server before running the script! Install aborted!"
    exit
   fi
   
   # Iterate over subfolders (recursive depth 1). Remove folder(s) and host(s) if the folder(s) are last modified time is older than 60 min
   subfolders_old=$(find /omd/sites/$SITE_NAME/tmp/check_mk/piggyback -maxdepth 1 -type d -mmin +60 -exec basename {} \; | grep -E '^[0-9a-f]{12}$')
   for subfolder_old in $subfolders_old; do
     rm -r /omd/sites/$SITE_NAME/tmp/check_mk/piggyback/$subfolder_old
   
     # Delete Host via REST-API
     printf "\n\n\nDelete host: $subfolder_old via REST_API\n\n"
     curloutput=$(curl -s -S -H "Accept: application/json" -H "Authorization: Bearer $USERNAME $PASSWORD" -X DELETE -H "Content-Type: application/json" "$API_URL/objects/host_config/$subfolder_old")
     sleep 1
   
     echo $curloutput | grep -q "Host not found"
     if [ $? -ne 0 ]; then
     # Get ETag from "pending changes" object via REST-API
     temp_file=$(mktemp)
     curl -s -S -I -H "Authorization: Bearer $USERNAME $PASSWORD" -X GET "$API_URL/domain-types/activation_run/collections/pending_changes" 1> "$temp_file"
     printf "\n\n\nGet ETag from 'pending changes' object via REST-API\n\n"
     # Extract the ETag value from the file content
     etag=$(grep -iE '^ETag: ' "$temp_file" | awk '{print $2}' | tr -d '"' | tr -cd '[:alnum:]')
     rm "$temp_file"
     sleep 1
   
     # Start activating the pending changes via REST-API
     temp_file=$(mktemp)
     BODY="{\"force_foreign_changes\": \"false\", \"redirect\": \"false\", \"sites\": [\"$SITE_NAME\"]}"
     curl -s -S -H "Accept: application/json" -H "Authorization: Bearer $USERNAME $PASSWORD" -H "If-Match: \"$etag\"" -X POST -H "Content-Type: application/json" -d "$BODY" "$API_URL/domain-types/activation_run/actions/activate-changes/invoke"  1> "$temp_file"
     printf "\n\n\nStart activating the pending changes via REST-API\n\n"
     # Extract the id value from the file content
     activation_id=$(cat $temp_file | grep -oP '(?<="id": ")[^"]+' | head -n1)
     rm "$temp_file"
     sleep 60
     fi
   
     printf "\n\n\n##################################################"
   done
   
   # Iterate over subfolders (recursive depth 1). Create host(s) after folder name(s)
   subfolders=$(find /omd/sites/$SITE_NAME/tmp/check_mk/piggyback -maxdepth 1 -type d -exec basename {} \; | grep -E '^[0-9a-f]{12}$')
   for subfolder in $subfolders; do
     # Get file name inside the folder
     file=$(find "/omd/sites/$SITE_NAME/tmp/check_mk/piggyback/$subfolder" -type f -print -quit)
   
     # Get docker host machine name
     dockerhostname=$(basename $file)
   
     # Use the ImageTag value as an alias name for the new host
     hostalias=$(grep -o '"ImageTags": \[[^]]*\]' "$file" | awk -F '[][]' '{gsub(/"/, "", $2); split($2, arr, ":"); print arr[1]}')
   
     # Create Host via REST-API
     printf "\n\n\nCreate host: $subfolder via REST_API\n\n"
     BODY="{\"folder\": \"$FOLDER\", \"host_name\": \"$subfolder\", \"attributes\": {\"alias\": \"$hostalias\", \"labels\": {\"$dockerhostname\": \"$hostalias\"}}}"
     curloutput=$(curl -s -S -H "Accept: application/json" -H "Authorization: Bearer $USERNAME $PASSWORD" -X POST -H "Content-Type: application/json" -d "$BODY" "$API_URL/domain-types/host_config/collections/all")
     sleep 1
   
     echo $curloutput | grep -q "already exists"
     if [ $? -ne 0 ]; then
        # Get ETag from "pending changes" object via REST-API
        temp_file=$(mktemp)
        curl -s -S -I -H "Authorization: Bearer $USERNAME $PASSWORD" -X GET "$API_URL/domain-types/activation_run/collections/pending_changes" 1> "$temp_file"
        printf "\n\n\nGet ETag from 'pending changes' object via REST-API\n\n"
        # Extract the ETag value from the file content
        etag=$(grep -iE '^ETag: ' "$temp_file" | awk '{print $2}' | tr -d '"' | tr -cd '[:alnum:]')
        rm "$temp_file"
        sleep 1
   
        # Start activating the pending changes via REST-API
        temp_file=$(mktemp)
        BODY="{\"force_foreign_changes\": \"false\", \"redirect\": \"false\", \"sites\": [\"$SITE_NAME\"]}"
        curl -s -S -H "Accept: application/json" -H "Authorization: Bearer $USERNAME $PASSWORD" -H "If-Match: \"$etag\"" -X POST -H "Content-Type: application/json" -d "$BODY" "$API_URL/domain-types/activation_run/actions/activate-changes/invoke"  1> "$temp_file"
        printf "\n\n\nStart activating the pending changes via REST-API\n\n"
        # Extract the id value from the file content
        activation_id=$(cat $temp_file | grep -oP '(?<="id": ")[^"]+' | head -n1)
        rm "$temp_file"
        sleep 60
      fi
   
      printf "\n\n\n##################################################"
   done

5. After creating the script please make it executable:

   chmod +x ~/docker-container-checkmk.sh

6. Now we schedule the script into the crontab so that it runs once every full hour. Type following command to edit the crontab:

   sudo contrab -e

   Now paste the following line into the editor and save the crontab (Please change the path to the script to your home directory. Get path to your home directory: pwd ~):

   0 * * * * /home/administrator/docker-container-checkmk.sh