Best Practice - Publishing Metrics Follow

 

Summary

 

 

Publish metrics is an OverOps feature that enables export of application data from OverOps for use in third-party tools where it can be blended with other sources of data for advanced analytics.  OverOps supports this data export via StatsD which is an open-source implementation protocol to capture, aggregate and send metrics to modern DevOps tools such as a time-series database or metrics hub.   

Sending data from OverOps into an existing metrics hub gives organizations extended flexibility and control to conduct their own data analyses such as visualization, scorecards, correlations, and anomaly detection.  A sample use case may include correlating code quality metrics from Overops into an existing metrics hub that includes performance metrics from an APM solution, and sales metrics from a CRM repository. 

 

Sample Use Case

mceclip0.png

In such a scenario, data from OverOps could enable Operations teams to visually compare and contrast real-time application errors that are new or resurfaced, even those not logged, with real-time system performance of critical business transactions that generate revenue.  This type of correlation can augment existing alerting mechanisms set inside the metrics hub to make alerts more intelligent and reduce the noise of alerting storms.  

And developers get everything they need to quickly recreate and resolve code issues when they occur, with full context of what is happening inside the code at the origin of an error or exception, including the line of offending code with the data that was being processed by the application.

The final result is improved automations and efficiencies with less time wasted for DevOps personnel, while working within the tools they are already familiar which enables a better customer experience.

Configure Publish Metrics

When enabled, aggregate data from OverOps is sent every 60 seconds by the OverOps Remote Collector to a 3rd-party StatsD Server that is defined within the Publish Metrics settings dialog.mceclip1.png

To learn how to configure Publish Metrics, go here

 

Best Practice Guidelines

To make the most of the OverOps Publish Metrics capabilities, below are recommended practices to help drive efficiencies and avoid common pitfalls.

 

StatsD Support

Only services that support the StatsD protocol are able to read metrics from OverOps. Common services in the market that support StatsD include Telegraf, Splunk, and Datadog. Consult your third-party application documentation to verify that your service can receive telemetry data via StatsD.

 

Supported Data Formats

Third-party applications expect data that is received to be in a specific format.  Consult your third-party application documentation to verify what formats are best.  This information will be used when configuring the Publish Metrics format inside the OverOps Settings dialog.

mceclip2.png

Format Templates

Note that OverOps does provide template formats for Splunk, Influx, Graphite, and OpenTSDB.  If using any of these as a metrics hub or data repository, start with the template provided in OverOps.  Then if requirements make it necessary to add or omit specific fields, the account administrator can modify as necessary. 

If using another type of repository for the metrics hub, start with the Custom format.

mceclip3.png

Export Configurations

Export configurations define the fields to be streamed from OverOps.  For a comprehensive list of fields that can be exported from OverOps, go here.

For the most meaningful data aggregates and analysis in third-party tools,  it is critical that applications, servers, and deployments are properly named when monitoring with OverOps.  To learn best practices when naming applications, servers, and deployments, go here.

Below are the descriptions of the six stream configurations that can be defined within the Publish Metrics settings dialog. 

1 - Views

With the Views configuration, error volume metrics of every new view created is automatically sent.

mceclip4.png

2 - Events

With the Events configuration, every event publishes its own statistics, enabling the recreating and customizing of the OverOps dashboard in the tool of your choice. In addition, every event can publish a log link to the Root Cause Analysis, which can be directly accessed from the third-party graphic or alarm tool.

mceclip5.png

3 - Entry Points

With the Entry Points configuration, aggregate counts for failed calls and total invocations into the application are sent, along with average and total runtimes.

mceclip6.png

4 - Custom Metrics

The Custom Metrics configuration can be used to export custom events generated as a result of the OverOps SDK.  

mceclip7.png

5 - System

With the System configuration, over 40 system metrics are sent.  Metrics include cpu usage, memory used, garbage collection objects and time, loaded classes, threads, waiting threads, heap size, and more.

mceclip8.png

6 - Diagnostics

With the Diagnostics configuration, counts of application environments monitored are sent.

mceclip9.png

Configure 3rd-Party Applications

Once Publish Metrics integration is enabled in OverOps, you will need to configure a third-party service to receive data from OverOps.  OverOps has provided recommended approaches for various third-party services.  Always consult your third-party application documentation to verify what specific configurations are necessary.

 

UDP Network Traffic

Ensure the proper address and port are configured inside the Publish Metrics Settings dialog.  If no port is defined, 8125 is used by default.  

mceclip10.png

Data is sent from OverOps over UDP where data packets are sent to the recipient with no error checking and no verification the recipient received the packet. This ensures the environments can communicate more quickly compared to TCP protocol.  

But if network ports are not properly configured to accept data over UDP, it can appear as though OverOps is not sending data to the intended target service. Therefore, validate the recipient service is listening and can accept traffic on the UDP port defined within the Publish Metrics Settings.

 

OverOps Remote Collector

Data from OverOps is sent every 60 seconds by the OverOps Remote Collector.  In cases where data is received by a recipient service, but has stopped, validate the OverOps remote collector service is running or restart the service.

 

Comments

0 comments