Trigger Drops with Medium Load

triggers

#1

I have been going through our triggers using the ExtraHop System Health dashboard. I have already eliminated all the trigger errors that were happening. What I cannot figure out is why we have such a seemingly large amount of trigger drops (anywhere from 30K to 150K). The trigger load stays constant at around 60%, and I imagine it should only drop triggers when getting closer to 100%.

What can I look for?


#2

In 5.2, the system health page shows a trigger load breakdown by thread. Due to ordering constraints—for example, that an HTTP request trigger must execute before the corresponding HTTP response trigger—it may not be possible to evenly distribute trigger load, so 100% load is not actually achievable.

Regarding trigger drops, that is the surest indicator that the system is overloaded. Ideally, this metric should always read zero, since dropping triggers means missing out on potentially useful analysis. Generally, drops happen in bursts, so I would look not only for triggers with a high number of average cycles, but also those with a high maximum number of cycles. There are sometimes very simple changes you can make to your triggers to drastically improve performance, without giving up anything in the way of analysis or metrics.

Another thing that bears mentioning is that 5.2 includes a major upgrade of the triggers JavaScript engine. Depending on the workload, upgrading from a previous version could significantly improve trigger performance. Hope that helps!