This incident has been resolved.
Jul 21, 15:31 PDT
A fix has been implemented and we are monitoring the results.
Jul 21, 15:12 PDT
We have checked in a fix and expect the delay to be resolved in the next 40 minutes. This is only impacting a small portion of the traffic (roughly 1%).
Jul 21, 14:56 PDT
The issue has been identified and a fix is being implemented.
Jul 21, 14:36 PDT
A degradation in the performance of the Splunk APM trace processing pipeline is causing Troubleshooting MetricSets to be delayed by more than five minutes. As a result, the APM Troubleshooting experience, service maps and Tag Spotlight do not have access to the most recent data.
The processing of metrics for Business Workflows, which also depends on this pipeline, are equally delayed. Trace data ingest is not impacted at this time; service-level and endpoint-level Monitoring MetricSets and the detectors built from them are also not impacted.
Jul 21, 14:31 PDT