© betadots GmbH 2024
Lennart Betz
Senior IT Automation Consultant, betadots GmbH
• Automation, HA, System Monitoring
• Puppet, Foreman/Katello, Icinga, Ansible, …
• Author dpunkt, iX, Linux Journal, Linux Magazin
• lennart.betz@betadots.de
• lbetz (GitHub, Slack)
betadots GmbH 2025
Auto Discovery with ICINGA
© betadots GmbH 2024
ICINGA
Base Knowledge
betadots GmbH 2025
CC-BY-NC-SA 4.0
ICINGA
● Started as Nagios fork in 2009
● Completely reimplemented in 2015 as ICINGA 2
○ for getting better performance and scaling
○ also based on the concept of Nagios
○ configured by files and an own DSL
○ distributed monitoring, but with only ONE central configuration
● Host-Service-Concept
○ any service belongs to ONE host
○ checked by a Plugin
● Check-Plugins
○ local plugin: executed on the host to monitor, e.g. disk, load, …
○ remote plugin: executed on a ICINGA server to connect the monitored host, e.g. http, snmp, ...
betadots GmbH 2025
ICINGA DSL Objects
Monitoring
● Host, Service, Notification, …
● Templates
● CheckCommand: Encapsulates the plugin execution
Configuration: Distributed Monitoring
● Endpoint: an ICINGA instance
● Zone: Manages multiple endpoints and connection to one parent zone
betadots GmbH 2025
© betadots GmbH 2024
OpenVox
Automated Monitoring
betadots GmbH 2025
CC-BY-NC-SA 4.0
Our Challenges for OpenVox
● Open Source Fork of last Puppet 8.10.0 release (without any EULA)
○ https://voxpupuli.org/
○ https://github.com/openvoxproject
● Install and configure Icinga on server and several agents
○ including the connection between
● Configure a base monitoring for several platforms
● Use facts to get an autodiscover of ICINGA services, e.g. disks
● Move all this information (config) to the central ICINGA config server
betadots GmbH 2025
Facts with Open Facter
os => {
architecture => "aarch64",
distro => {
codename => "bookworm",
description => "Debian GNU/Linux 12 (bookworm)",
id => "Debian",
release => {
full => "12.11",os => {
architecture => "aarch64",
distro => {
codename => "bookworm",
}
…
}
...
betadots GmbH 2025
partitions => {
/dev/mapper/vbox--vg-root => {
filesystem => "ext4",
mount => "/",
size => "62.06 GiB",
size_bytes => 66639101952,
uuid => "64dcf1be-be40-4c64-96da-6270fa8739aa"
},
/dev/mapper/vbox--vg-swap_1 => {
filesystem => "swap",
size => "980.00 MiB",
size_bytes => 1027604480,
uuid => "afbc1caa-4876-4055-a62b-214f071d156f"
},
...
}
How OpenVox/Puppet Works
betadots GmbH 2025
1. Collecting Facts establish
connection to server
2. Compile catalog (desired state)
Agent makes changes if
necessary
3. Send report
4. Store report
Hiera
● A Hierarchical Key-Value-Store
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Per-node data (yaml version)"
path: "net/%{networking.network}.yaml"
- name: "Per-OS data (yaml version)"
paths:
- "os/%{facts.os.family}.yaml"
- "os/%{facts.kernel}.yaml"
- name: "Other YAML hierarchy levels"
paths:
- "common.yaml"
betadots GmbH 2025
Example:
common.yaml
icinga::agent::ca_server: 192.168.10.100
icinga::agent::zone: main
net/172.16.200.0.yaml
icinga::agent::ca_server: 172.16.200.13
icinga::agent::zone: dmz
Solutions
● Upstream Voxpupuli modules
○ https://github.com/voxpupuli/puppet-icinga
○ https://github.com/voxpupuli/puppet-icinga2
○ https://github.com/voxpupuli/puppet-icingadb
○ https://github.com/voxpupuli/puppet-icingaweb2
● Hiera
● Facter
● PuppetDB
○ Puppet run store (exported) resources in a PGSQL (on the Puppet server)
○ Agent run on other Node collect (API query) these info
betadots GmbH 2025
© betadots GmbH 2024
Demo
Automated Monitoring
betadots GmbH 2025
CC-BY-NC-SA 4.0

OSMC 2025: Auto Discovery with Icinga by Lennart Betz.pdf

  • 1.
  • 2.
    Lennart Betz Senior ITAutomation Consultant, betadots GmbH • Automation, HA, System Monitoring • Puppet, Foreman/Katello, Icinga, Ansible, … • Author dpunkt, iX, Linux Journal, Linux Magazin • lennart.betz@betadots.de • lbetz (GitHub, Slack) betadots GmbH 2025 Auto Discovery with ICINGA
  • 3.
    © betadots GmbH2024 ICINGA Base Knowledge betadots GmbH 2025 CC-BY-NC-SA 4.0
  • 4.
    ICINGA ● Started asNagios fork in 2009 ● Completely reimplemented in 2015 as ICINGA 2 ○ for getting better performance and scaling ○ also based on the concept of Nagios ○ configured by files and an own DSL ○ distributed monitoring, but with only ONE central configuration ● Host-Service-Concept ○ any service belongs to ONE host ○ checked by a Plugin ● Check-Plugins ○ local plugin: executed on the host to monitor, e.g. disk, load, … ○ remote plugin: executed on a ICINGA server to connect the monitored host, e.g. http, snmp, ... betadots GmbH 2025
  • 5.
    ICINGA DSL Objects Monitoring ●Host, Service, Notification, … ● Templates ● CheckCommand: Encapsulates the plugin execution Configuration: Distributed Monitoring ● Endpoint: an ICINGA instance ● Zone: Manages multiple endpoints and connection to one parent zone betadots GmbH 2025
  • 6.
    © betadots GmbH2024 OpenVox Automated Monitoring betadots GmbH 2025 CC-BY-NC-SA 4.0
  • 7.
    Our Challenges forOpenVox ● Open Source Fork of last Puppet 8.10.0 release (without any EULA) ○ https://voxpupuli.org/ ○ https://github.com/openvoxproject ● Install and configure Icinga on server and several agents ○ including the connection between ● Configure a base monitoring for several platforms ● Use facts to get an autodiscover of ICINGA services, e.g. disks ● Move all this information (config) to the central ICINGA config server betadots GmbH 2025
  • 8.
    Facts with OpenFacter os => { architecture => "aarch64", distro => { codename => "bookworm", description => "Debian GNU/Linux 12 (bookworm)", id => "Debian", release => { full => "12.11",os => { architecture => "aarch64", distro => { codename => "bookworm", } … } ... betadots GmbH 2025 partitions => { /dev/mapper/vbox--vg-root => { filesystem => "ext4", mount => "/", size => "62.06 GiB", size_bytes => 66639101952, uuid => "64dcf1be-be40-4c64-96da-6270fa8739aa" }, /dev/mapper/vbox--vg-swap_1 => { filesystem => "swap", size => "980.00 MiB", size_bytes => 1027604480, uuid => "afbc1caa-4876-4055-a62b-214f071d156f" }, ... }
  • 9.
    How OpenVox/Puppet Works betadotsGmbH 2025 1. Collecting Facts establish connection to server 2. Compile catalog (desired state) Agent makes changes if necessary 3. Send report 4. Store report
  • 10.
    Hiera ● A HierarchicalKey-Value-Store version: 5 defaults: datadir: data data_hash: yaml_data hierarchy: - name: "Per-node data (yaml version)" path: "net/%{networking.network}.yaml" - name: "Per-OS data (yaml version)" paths: - "os/%{facts.os.family}.yaml" - "os/%{facts.kernel}.yaml" - name: "Other YAML hierarchy levels" paths: - "common.yaml" betadots GmbH 2025 Example: common.yaml icinga::agent::ca_server: 192.168.10.100 icinga::agent::zone: main net/172.16.200.0.yaml icinga::agent::ca_server: 172.16.200.13 icinga::agent::zone: dmz
  • 11.
    Solutions ● Upstream Voxpupulimodules ○ https://github.com/voxpupuli/puppet-icinga ○ https://github.com/voxpupuli/puppet-icinga2 ○ https://github.com/voxpupuli/puppet-icingadb ○ https://github.com/voxpupuli/puppet-icingaweb2 ● Hiera ● Facter ● PuppetDB ○ Puppet run store (exported) resources in a PGSQL (on the Puppet server) ○ Agent run on other Node collect (API query) these info betadots GmbH 2025
  • 12.
    © betadots GmbH2024 Demo Automated Monitoring betadots GmbH 2025 CC-BY-NC-SA 4.0