HowTo: OpenNMS and Google Search Appliance

by | Jun 9, 2011 | OpenNMS, SNMP | 0 comments

First step is to enable SNMP on the gsa with a well choosen community string.

You also need the two MIB files from google, the GOOGLE-MIB and the GSA_MIB.
Copy the MIB files to your OpenNMS machine into the directory

$OPENNMS_DIR/contrib/mibparser/mibs

Next step is to convert these mibs into OpenNMS configuration. With the mibparser we can do this:

$OPENNMS_DIR/contrib/mibparser/dist/parseMib.sh \
$OPENNMS_DIR/contrib/mibparser/mibs/GOOGLE-MIB $OPENNMS_DIR/contrib/mibparser/mibs/GSA-MIB >$HOME/google.txt
<mibObj oid=".1.3.6.1.4.1.11129.1.1.1" instance="0" alias="crawlRunning" type="INTEGER" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.1" instance="0" alias="docsServed" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.2" instance="0" alias="crawlingRate" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.3" instance="0" alias="docBytes" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.4" instance="0" alias="todayDocsCrawled" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.5" instance="0" alias="docErrors" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.6" instance="0" alias="docsFound" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.7" instance="0" alias="batchCrawlRunning" type="INTEGER" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.8" instance="0" alias="batchCrawlStartTime" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.9" instance="0" alias="batchCrawlEndTime" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.1.2.10" instance="0" alias="batchCrawlEndReason" type="INTEGER" />
<mibObj oid=".1.3.6.1.4.1.11129.1.2.1" instance="0" alias="qpm" type="Integer32" />
<mibObj oid=".1.3.6.1.4.1.11129.1.3.1.1" instance="0" alias="diskHealth" type="INTEGER" />
<mibObj oid=".1.3.6.1.4.1.11129.1.3.1.2" instance="0" alias="diskErrors" type="DisplayString" />
<mibObj oid=".1.3.6.1.4.1.11129.1.3.2.1" instance="0" alias="temperatureHealth" type="INTEGER" />
<mibObj oid=".1.3.6.1.4.1.11129.1.3.2.2" instance="0" alias="temperatureErrors" type="DisplayString" />
<mibObj oid=".1.3.6.1.4.1.11129.1.3.3.1" instance="0" alias="machineHealth" type="INTEGER" />
<mibObj oid=".1.3.6.1.4.1.11129.1.3.3.2" instance="0" alias="machineErrors" type="DisplayString" />

This output still needs to be manualy converted to be useful in OpenNMS. We change the types “INTEGER” and “Integer32” into “Gauge32“. The lines with type “DisplayString” are deleted, as OpenNMS can do reports for them. The changes are going into the file $OPENNMS_DIR/etc/datacollection-config.xml. Here is the, what needs to be added to it:

This goes into the groups section:

<group name="googleGSA" ifType="ignore">
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.1" instance="0" alias="crawlRunning" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.1" instance="0" alias="docsServed" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.2" instance="0" alias="crawlingRate" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.3" instance="0" alias="docBytes" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.4" instance="0" alias="todayDocsCrawled" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.5" instance="0" alias="docErrors" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.6" instance="0" alias="docsFound" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.7" instance="0" alias="batchCrawlRunning" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.8" instance="0" alias="batchCrawlStartTime" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.9" instance="0" alias="batchCrawlEndTime" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.1.2.10" instance="0" alias="batchCrawlEndReason" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.2.1" instance="0" alias="qpm" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.3.1.1" instance="0" alias="diskHealth" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.3.2.1" instance="0" alias="temperatureHealth" type="Gauge32" />
  <mibObj oid=".1.3.6.1.4.1.11129.1.3.3.1" instance="0" alias="machineHealth" type="Gauge32" />
</group>

This goes into the systems section:

<systemDef name="googleGSA">
  <sysoidMask>.1.3.6.1.4.1.11129.1.</sysoidMask>
    <collect>
      <includeGroup>googleGSA</includeGroup>
  </collect>
</systemDef>

We also add thresholding setup into the $OPENNMS_DIR/etc/thresholds.xml:

<group name="gooleGSA" rrdRepository="/opt/opennms/share/rrd/snmp/">
  <threshold type="high" ds-name="machineHealth" ds-type="node" value="1" rearm="0" trigger="1" />
  <threshold type="high" ds-name="temperatureHealth" ds-type="node" value="1" rearm="0" trigger="1" />
  <threshold type="high" ds-name="diskHealth" ds-type="node" value="1" rearm="0" trigger="1" />
  <threshold type="low" ds-name="crawlRunning" ds-type="node" value="0" rearm="1" trigger="1" />
</group>

Before we can use this, we can check for syntax with:

 xmllint $OPENNMS_DIR/etc/datacollection-config.xml

Also check the thresholds.xml! We need to restart the OpenNMS now and add the gsa to the discovery to get data for it.

Last step is the setup of the graph definitions in the file $OPENNMS/etc/snmp-graph.properties. Here is the content:

This needs to be appended at reports string (don’t forget the “,” and the “\”)

googleGSA.crawlRunning, googleGSA.docsServed, googleGSA.crawlingRate, googleGSA.docBytes, googleGSA.todayDocsCrawled, googleGSA.docErrors, googleGSA.docsFound, googleGSA.qpm, googleGSA.diskHealth, googleGSA.temperatureHealth, googleGSA.machineHealth

This are the graph definitions:

report.googleGSA.docsServed.name=Number of document being served
report.googleGSA.docsServed.columns=docsServed
report.googleGSA.docsServed.type=nodeSnmp
report.googleGSA.docsServed.command=--title="Documents served" \
--vertical-label Documents \
DEF:docsServed={rrd1}:docsServed:AVERAGE \
AREA:docsServed#008800:"Docs" \
GPRINT:docsServed:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:docsServed:MIN:"Min \\: %10.2lf %s" \
GPRINT:docsServed:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.crawlingRate.name=The current crawling rate in pages per second.
report.googleGSA.crawlingRate.columns=crawlingRate
report.googleGSA.crawlingRate.type=nodeSnmp
report.googleGSA.crawlingRate.command=--title="Crawling rate" \
--vertical-label="per second" \
DEF:crawlingRate={rrd1}:crawlingRate:AVERAGE \
LINE2:crawlingRate#008800:"crawl rate/s" \
GPRINT:crawlingRate:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:crawlingRate:MIN:"Min \\: %10.2lf %s" \
GPRINT:crawlingRate:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.docBytes.name=The total megabytes processed so far.
report.googleGSA.docBytes.columns=docBytes
report.googleGSA.docBytes.type=nodeSnmp
report.googleGSA.docBytes.command=--title="Document size served" \
--vertical-label Bytes \
DEF:docBytes={rrd1}:docBytes:AVERAGE \
AREA:docBytes#008800:"Docs Bytes" \
GPRINT:docBytes:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:docBytes:MIN:"Min \\: %10.2lf %s" \
GPRINT:docBytes:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.todayDocsCrawled.name=The number of documents crawled today.
report.googleGSA.todayDocsCrawled.columns=todayDocsCrawled
report.googleGSA.todayDocsCrawled.type=nodeSnmp
report.googleGSA.todayDocsCrawled.command=--title="Documents crawled" \
--vertical-label="daily" \
DEF:todayDocsCrawled={rrd1}:todayDocsCrawled:AVERAGE \
AREA:todayDocsCrawled#008800:"Crawled today" \
GPRINT:todayDocsCrawled:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:todayDocsCrawled:MIN:"Min \\: %10.2lf %s" \
GPRINT:todayDocsCrawled:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.docErrors.name=Number of times an error occurred while trying to crawl a document.
report.googleGSA.docErrors.columns=docErrors
report.googleGSA.docErrors.type=nodeSnmp
report.googleGSA.docErrors.command=--title="Document Errors" \
--vertical-label="daily" \
DEF:docErrors={rrd1}:docErrors:AVERAGE \
AREA:docErrors#AA0000:"Doc Errors" \
GPRINT:docErrors:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:docErrors:MIN:"Min \\: %10.2lf %s" \
GPRINT:docErrors:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.docsFound.name=Total documents found.
report.googleGSA.docsFound.columns=docsFound
report.googleGSA.docsFound.type=nodeSnmp
report.googleGSA.docsFound.command=--title="Documents found" \
--vertical-label="Documents" \
DEF:docsFound={rrd1}:docsFound:AVERAGE \
AREA:docsFound#008800:"Docs found" \
GPRINT:docsFound:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:docsFound:MIN:"Min \\: %10.2lf %s" \
GPRINT:docsFound:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.qpm.name=Serving status in terms of queries per minute handled.
report.googleGSA.qpm.columns=qpm
report.googleGSA.qpm.type=nodeSnmp
report.googleGSA.qpm.command=--title="Queries" \
--vertical-label="per minute" \
DEF:qpm={rrd1}:qpm:AVERAGE \
LINE2:qpm#008800:"Queries/min" \
GPRINT:qpm:AVERAGE:"Avg \\: %10.2lf %s" \
GPRINT:qpm:MIN:"Min \\: %10.2lf %s" \
GPRINT:qpm:MAX:"Max \\: %10.2lf %s\\n" \
\
report.googleGSA.diskHealth.name=Disk status
report.googleGSA.diskHealth.columns=diskHealth
report.googleGSA.diskHealth.type=nodeSnmp
report.googleGSA.diskHealth.command=--title="Disk status" \
--vertical-label status \
DEF:diskHealth={rrd1}:diskHealth:AVERAGE \
AREA:diskHealth#FF0000:"0:OK 1:Warning 2:Critical" \
GPRINT:diskHealth:AVERAGE:"Avg \\: %2.2lf %s" \
GPRINT:diskHealth:MIN:"Min \\: %2.2lf %s" \
GPRINT:diskHealth:MAX:"Max \\: %2.2lf %s\\n" \
\
report.googleGSA.temperatureHealth.name=Temperature status
report.googleGSA.temperatureHealth.columns=temperatureHealth
report.googleGSA.temperatureHealth.type=nodeSnmp
report.googleGSA.temperatureHealth.command=--title="Temperature status" \
--vertical-label status \
DEF:temperatureHealth={rrd1}:temperatureHealth:AVERAGE \
AREA:temperatureHealth#FF0000:"0:OK 1:Warning 2:Critical" \
GPRINT:temperatureHealth:AVERAGE:"Avg \\: %2.2lf %s" \
GPRINT:temperatureHealth:MIN:"Min \\: %2.2lf %s" \
GPRINT:temperatureHealth:MAX:"Max \\: %2.2lf %s\\n" \
\
report.googleGSA.machineHealth.name=Machine status
report.googleGSA.machineHealth.columns=machineHealth
report.googleGSA.machineHealth.type=nodeSnmp
report.googleGSA.machineHealth.command=--title="Machine status" \
--vertical-label status \
DEF:machineHealth={rrd1}:machineHealth:AVERAGE \
AREA:machineHealth#FF0000:"0:OK 1:Warning 2:Critical" \
GPRINT:machineHealth:AVERAGE:"Avg \\: %2.2lf %s" \
GPRINT:machineHealth:MIN:"Min \\: %2.2lf %s" \
GPRINT:machineHealth:MAX:"Max \\: %2.2lf %s\\n" \

That’s it!

Submit a Comment

Your email address will not be published. Required fields are marked *

I accept that my given data and my IP address is sent to a server in the USA only for the purpose of spam prevention through the Akismet program.More information on Akismet and GDPR.

This site uses Akismet to reduce spam. Learn how your comment data is processed.