Mapping HTTP.payload information with Src/Dst IP/Port

Hi,

I’m trying to do a mapping of source, destination, IP, Port with HTTP payload where its destination port is 80 (We’re trying to look for plaintext payload in our environment).

Unfortunately, trigger API for HTTP does not support payload, hence i’m not able to commit record to extract out the information to excel format.

Custom metrics also does not seems to allow us to extract out the payload and do any mapping.

Looking through the API explorer or the trigger API does not seems to have any way for me to export/download the information in JSON format either.

So my current way of getting the information is to output and retrieve the values from the debug log which is definitely not a very efficient way.

if (event === “HTTP_RESPONSE” && HTTP.uri.includes(":80"){
let flow_info = “Flow ID: " + Flow.id + " “;
if(Flow.server.port.toString() == “8080”){
debug(flow_info + “Payload: " + HTTP.payload.toString()); +”\n”
debug(flow_info + “URI: " + HTTP.uri); +”\n”
debug(flow_info + “Source IP: " + Flow.client.ipaddr.toString()); +”\n"
debug(flow_info + “Source Port: " + Flow.client.port); + “\n”
debug(flow_info + “Destination IP: " + Flow.server.ipaddr.toString()); +”\n”
debug(flow_info + "Destination Port: " + Flow.server.port); + “\n”
}
}

Hence, is there any other ways to achieve this objective?

Hi Bryan - A few thoughts below that hopefully help get you to where you want quickly:

The trigger api does support http payload, you’re using it in your example trigger. You could extract specifics from the payload, then commit it as a custom metric detail count, and/or to a custom record.

For the custom record it would be something like, let record = Object.assign({}, HTTP.record); to make your new editable record format copy, then, you could grab the payload into a variable by doing const myPayload = HTTP.payload.toString(), add the payload to your record by doing record.payload = myPayload then commit the record like, commitRecord("http_analysis", record);

ExtraHop looks at all ports for HTTP/S, so if desired, you could ensure you are analyzing everything that’s plaintext, without worrying about ports, by doing an early exit near the top of your trigger like, if (HTTP.isEncrypted) { return; }; if you only want port 80, or to not analyze traffic you’re decrypting, something like you’re already doing will work; I’d probably use Flow port.

Finally, if you’d like all this information to be going off box somewhere vs as a custom metric or record, you can add an ODS target to your sensor(s) the trigger is running on, where the host added is the server you wish to send the JSON payload to! Create a new variable like let myJson = {}; then add to the object whatever you want to send, much like you were doing the debug output. Something like:

let myJson = {};
myJson.httpPayload = HTTP.payload.toString();
myJson.httpPort...
myJson.httpUserAgent...
   
 let headers = {
        'Content-Type': 'application/json',
        'Accept': 'text/json'
    },

        obj = {
            'path': '/',
            'headers': headers,
            'payload': JSON.stringify(myJson)
        };

    Remote.HTTP('your_ods_name_configured_on_sensor').request('POST', obj);

The above is rough code for doing an http target, you can also send to a mongodb, raw, etc…https://docs.extrahop.com/current/ods-http/

Hi,

Thank you for the help and clairification.

I’m not looking for any specific in the payload. I attempted your suggested way of extracting out the information but can’t seems to find it in my custom record based on the below.

Not sure if i could possibly be missing out something or whether is it even possible to extract such large amount of information in the payload into the custom record.

let record = Object.assign({}, HTTP.record);
const myPayload = HTTP.payload.toString();
record.payload = myPayload;

//commit record
commitRecord("junkdata_http_analysis6", record);

Absolutely! You’re almost there now – you should be able to see that field if switching to verbose view with “select all” selected, or preferably – create a custom record format (gear icon -> record formats) and define payload,

{
        "name": "payload",
        "display_name": "Payload",
        "data_type": "s"
    }

then it will be visible in the table view. https://docs.extrahop.com/current/collect-custom-records/

Does your environment have both Records and Packets? If so, you could craft a record query to find all those HTTP responses that are not encrypted, and from that result view and download all the associated packets for further analysis of the payload.

No custom trigger work would be required.

Hi Girardo,

Thank you so much for your advice. This is definitely something anyone including myself can learn and apply to better understand how to create custom triggers and record.

I can now see the payload information in the verbose view but somehow i cant export it. I tested other default record type and edited the field seen in the verbose view and was able to export based on the column selected.

Reckon, i probably need to create a custom record to export them out from the records.

1 Like