Tag Archives: Deployment Monitor

Fix erroneous DM Splunk Missing Forwarders alert

For some time now Splunk has been alerting me to “missing” forwarders even though all of those forwarders are working perfectly fine. It turns out to be a glitch in the Deployment Monitor app. After much digging I found this Splunk article which explains it:

https://answers.splunk.com/answers/188784/after-update-to-splunk-enterprise-62-why-does-the.html

The fix is fairly simple, thankfully. You have to edit the macros.conf of the Deployment Monitor app to add this small snippet right before the first pipe:

NOT eventType=*

The default path for this configuration file is:

/opt/splunk/etc/apps/SplunkDeploymentMonitor/default/macros.conf

The relevant stanza in my macros.conf is below:

[forwarder_metrics]
definition = index="_internal" source="*metrics.log" group=tcpin_connections NOT eventType=* | eval sourceHost=if(isnull(hostname), sourceHost,hostname) | eval connectionType=case(fwdType=="uf","universal forwarder", fwdType=="lwf", "lightweight forwarder",fwdType=="full", "heavy forwarder", connectionType=="cooked" or connectionType=="cookedSSL","Splunk forwarder", connectionType=="raw" or connectionType=="rawSSL","legacy forwarder")| eval build=if(isnull(build),"n/a",build) | eval version=if(isnull(version),"pre 4.2",version) | eval guid=if(isnull(guid),sourceHost,guid) | eval os=if(isnull(os),"n/a",os)| eval arch=if(isnull(arch),"n/a",arch) | fields connectionType sourceIp sourceHost sourcePort destPort kb tcp_eps tcp_Kprocessed tcp_KBps splunk_server build version os arch guid