The OverOps Automated Root Cause provides a powerful mechanism to get to the root of errors and exceptions in production and staging environments. The page is divided into three panes, that provide information about the event to create a complete picture.
Automated Root Cause
The Analytics pane provides important details relating to the impact of this error on your application. It is divided into two areas: the Analytics pane and chart.
The analytics pane shows you the exact type of error, when it began and how many times its and out of how many calls. The analytics chart shows you the volume of the error over the course of the selected timeframe (e.g. last hour,day, week,..). You can also filter the chart to shows the volume in specific applications or servers.
You can click the JVM or Server labels to directly to see the volume of the error on specifically on that machine or application. You can also hover over the occurrences label to see the number of times this error has occurred and out of how many calls into the method containing it.
You can use the button in the top right of the screen to open and select this error in the context of OverOps' main dashboard. You can click "Go to snapshot" on any point in the graph to jump and see the code and variable state at that moment in time.
The Analytics Pane on the left and the Event Chart on the right reflecting the volume of the error.
The Call Stack pane displays the chain of methods within the JVM leading up to the event. The first method in line is the last method on a non 3rd party code within your application. The icon in the method indicates that the variable state has been detected by the JVM micro-agent.
When an exception is caught and re-thrown once or more within the thread, Related Errors drop-down displays the error analysis (available only when such exceptions exist).
At the bottom of the stack the machine name and the JVM thread name in which this error occurred appear. 3rd party code is hidden by default and shown by switching on Show 3rd party methods at the bottom of the stack. To copy the full stack to the clipboard, click COPY STACK.
The call stack pane
By default, the Source Code View shows a decompiled Java version of the bytecode executing within the JVM at the moment of error. Hover over any highlighted variable to display its value and jump to see its full contents within the variable grid. The row in which the error occurred is highlighted, as depicted below.
The full error message appears in the title of the code pane.
When configured, OverOps uses the original source code instead of decompiling it from the JVM.
Search the source code or the variable grid for any variable name or value using Search Variables. Click here to learn more about variable search.
The source code and variable state pane
Recorded Variables displays the variable values and objects accessible from the method. Objects can be explored up to five levels deep into the code. Click the button that appears when hovering over the object, to view its contents as a JSOB. The content can be copied to the clipboard.
The Recorded Variables pane contains all local variables and parameters (including "this" in non-static methods). The first method also contains thread-local variables defined for this thread as well as SLF4J and Log4J Mapped Diagnostics Context (MDC) values. These MDC objects are often too large, for the full set of data to be available in the log, the micro-agent, however, is able to capture and record the entire object.
In some use cases, such as asynchronous message passing, these MDC objects contain a key-value map of the recorded requests, initial servlet information, and much more. They can be seen in any OverOps snapshot, and provide better visibility to the source of the bad request. This provides helpful extended visibility feature since back tracing the source of a bad request in an asynchronous environment is a known challenge.
The choice of the collected variables most relevant within an allocated timeframe is determined by the micro-agent using an adaptive machine learning algorithm. The selection process involves which and how many variable to collect, the number of items to collect, the length of string to capture, etc.
Click here to learn more about object and variable state.
Variable grid displaying the variables state JSON representation of an object
The Log View displays the last 250 log statements leading up to the event. Since the log statements are collected directly from JVM memory, any DEBUG, TRACE or INFO statements are visible regardless of whether or not they were logged to file.
Click the button to switch between code and log view.
Click here to learn more about the Log View.
Log View pane
For each error and exception detected, OverOps displays a JVM view that displays the internal JVM state at the moment of the error, including memory usage (heap and non-heap), basic system information, CPU usage and more. This enables working with the OverOps code (“classic”), log and JVM data without leaving the application. Click here to learn more about the JVM View.
The Actions toolbar provides a set of capabilities to share, mark and search through the error analysis contents:
- Send to Jira - Create a new Jira issue for an event linking it directly to the source, stack, state and statistics. Click here to learn more about Jira integration.
- Hide - Hide the event from the Dashboard Events List and Chart. Furthermore, the micro-agents will no longer capture snapshots for this event. The event will appear under the "Archive" label in the Dashboard from which it can be unhidden. Click here to learn more about hiding errors.
- Resolve - Mark an error as fixed and remove it from the Dashboard Events List and Chart. Should this error occur after a new code deployment, it will return to the Event List and Chart marked 'Resurfaced', and OverOps will notify you by email. Click here to learn more about resolving errors.
- Label - Label events to classify them according to priority ('Critical' or 'Low'), responsibility ('John' or 'QA') and version ('V1 RC2'). Click here to learn more about creating and assigning labels.
- Add/Edit Note - Attach a note to an event and share it with your teammates about which they are notified by email. Click here to learn more about sharing with teammates.
Add Timer - Track predefined methods for latency. OverOps sends an alert when the code runs longer than expected. Click here to learn more about Timers.