Events Size to SIEM

Good Night,

Is there any possibility to know the exact size of the events to be sent to the system when integrating with the Detector SIEM Connector?

Thanks for your help.

Hi, @rcastillo.

Just to confirm, are you working with a SIEM that has a cost to each byte stored, and you’re trying to be sure you don’t send too much data if you enable on the SIEM Connector?

Hello @shaundavid,

That’s correct, I’m working with a SIEM, and I would like to know the specific or estimated cost (bytes) of data or events that would be sent to the SIEM.

Do you know it?

Thanks for your help.

Thanks, @rcastillo.

For the various data elements that we might send to a remote system, we have ways to count bytes locally. As a specific example, in the 8.3 firmware, we added new system health metrics that measure the volume of record data just before we send it to the record store.

Unfortunately, that’s where our visibility ends. Once the data arrives at the receiver (your SIEM), they will process it in ways that are out of our sight. The SIEM might apply compression or deduplication schemes to reduce the number of bytes they store. The SIEM might also decide to “charge” for the bytes before they compress or afterward. Each SIEM does its own thing.

However, one technique that might help you is to run a test that looks for correlations between ExtraHop and the receiving system and use that for future estimates. An example of how it goes

  1. Note the current (baseline) total capacity consumed in the SIEM, before the test is run
  2. Identify a suitably large block of representative events that will be sent from the ExtraHop to the SIEM and start sending them
  3. As you send the data from the ExtraHop to the SIEM, use an available mechanism to measure the size of the data on the ExtraHop side. For trigger-based sending, one idea is to JSON.stringify() the trigger object representation of the event data, compute its length, and then add the length to a “bytes seen” count metric.
  4. After all of the events of the test, note the total increase in bytes consumed in the SIEM since step 1.
  5. Now compare the SIEM-side byte count to the ExtraHop-side byte count.

To put specific numbers to it, if the metric you chose to measure the event in ExtraHop shows 1GB of data sent and the SIEM consumption increases by only 500MB, you can model the relationship between ExtraHop and the SIEM as being roughly 2:1. So you might expect that a future send of data that measures 80GB inside the ExtraHop will result in a 40GB increase in SIEM consumption.

I’ve applied this estimation technique to a couple of receiving systems like your SIEM. While there’s no guarantee it will work for every set of data you might send, the approach has been surprisingly good at predicting the “cost” of data in the receiving system based on ExtraHop-side metrics.

Hope that helps.