VPN Monitoring Bundle

Shout out to Paco for laying the groundwork for the VPN initiative and Pete/Kanen for the original trigger work. Really just posting on their behalf.

The crux of this VPN monitoring approach is the creation of a VPN Custom Device on which we can aggregate metrics and filter trigger executes only to the VPN traffic. There is a discussion for other deployment models (device group, L3 device discovery, etc) that might be more appropriate for other customers. The current trigger is designed to be very minimal in processing and should be able to be deployed to most environments without any concerns.

What do I get?

A network-focused view of VPN usage and performance.

Current Bundle:

VPN Network Monitoring (v2.0.0).json (157.6 KB)
Change log:

  • Fixed a bug in which byte counts were monotonically increasing only and double counting previous bytes

Previous Versions:

VPN Network Monitoring (v1.7.2).json (157.6 KB)
Change log:

  • Added a useCustomDevice flag for environments which have discovered devices for the VPN CIDR
  • Added README section to the dashboard to aid deployment
  • Added Current Active Clients tracking
    image

Caveat: the following has not been performance tested at scale
VPN Network Monitoring (v1.7.2 experimental).json (287.9 KB)
Change log (in addition to the above):

  • Experimental device classification via DNS (incomplete)
  • Integrated VPN Cloud Bundle

VPN Network Monitoring (v1.7.1).json (150.4 KB)
VPN Network Monitoring (v1.7.1 Experimental).json (280.7 KB)
Change log:

  • Added Dynamic Device group filtered on Name contains “VPN”
  • Trigger flag to support future experimental advanced analysis. Currently disabled.

VPN Network Monitoring (v1.7.0).json (153.5 KB)
Change log:

  • Modified charts to take advantage of more built-in metrics
  • Modified layout and style of several charts

VPN Network Monitoring (v1.6.2).json (158.9 KB)
VPN Network Monitoring (v1.6.1).json (157.9 KB)

1x Trigger – VPN Network Monitoring
1x Dashboard – VPN Monitoring

(Dependency on the Active Directory Rx trigger)

Internal and Internet-bound traffic breakdowns

Protocol Breakdown

Includes classification for VoIP traffic streams for several teleconferencing solutions. The trigger can be modified with additional port ranges for other solutions as required by the customer.

User-IP mappings and teleconference-specific network utlization tracking

Discussion

  • Where possible, enterprises should split tunneling to avoid excess VPN load. I like to use Proof-Negative charting (Evidence of Absence), dashboards which we want to see blank. Teleconference traffic is a good example of traffic which should be sent directly to the internet.
  • When accessing and manipulating large resources like DB and large files, consider whether a VDI solution could be used to reduce VPN load and improve performance for the end user
  • The attack surface has increased exponentially, with the need for rapid expansion of remote workers, understanding which users are currently tied to VPN IP’s will enable faster threat isolation

Questions the dashboards answer:

  • How much traffic is going over my VPN? How much of that is internet traffic vs. internal traffic?
  • How much bandwidth are the users consuming? Are they having congestion related issues?
  • How much traffic is Webex, Zoom, GoToMeeting, etc?
  • Who are the users consuming the most bandwidth and what type of traffic is that?
  • What files are users pulling down to home computers? Is that normal behavior for that user?
  • Who is the user that had malware or something malicious on their home computer?

Deployment

  1. Make sure Network Localities are properly configured as they are used for Internal vs Internet traffic.
  2. Create a custom device for all of the VPN subnets
  3. Assign the Trigger VPN Network Monitoring to the custom device. The trigger has three modifiable flags
    a. ad_bundle_active – Leave set if the AD bundle is active and you are going to use the IP-User mapping in that bundle. If set to false, the bundle will currently not perform any user mapping, though the code is in place to expand how users are determined.
    b. check_teleconference_ports – Set to false if the customer does not care about teleconference protocols or split tunneling has been validated. This will reduce some cycle load for the trigger
    c. commit_records – There is no Record Format in the bundle currently, so this is defaulted to false. As we extend the bundle functionality, we will include the appropriate Record Format and set the default value to true
  4. Modify the Active Directory trigger and set the flag const storeUserInSessionTable = true; (Line 50 in Rx v1.2)
  5. For the VPN Monitoring Dashboard, Modify Source to the custom device.
    image

Extending Functionality

Modifying Teleconferencing Port Ranges

The port ranges and platform port mappings are stored in the mapTeleconfPort function. To add additional ports, select the correct IPProto (tcp/udp) and add the port range to the list. For single value ports, use the port for both upper and lower bounds.

const vpn_teleconf_mapping = cache('vpn_teleconf_mapping', () => ({
        'tcp': [
            [5004, 5004, 'Webex'],
            [5090, 5091, 'Zoom'], 
            [8801, 8802, 'Zoom'],
        ],
        'udp': [
            [1853, 1853, 'GoToMeeting'],
            [3478, 3481, 'Zoom/Teams/Skype'],
            [5090, 5091, 'Zoom'],
            [7500, 7501, 'Webex'],
            [8200, 8200, 'GoToMeeting'],
            [8801, 8810, 'Zoom'],
            [9000, 9000, 'Webex'],
            [19302, 19309, 'Hangouts Meet'],
            // [20000, 64000],
            // [50000, 60000],
        ]
    }));

Add additional User-IP mappings

The getUser function maps an IP address to a username. Today the only mapping used is the AD Bundles Kerberos parsing, which is then stored on the Session Table. This can be expanded to use other auth methods, including replicating the AD bundle and assigning just to the VPN custom device.

function getUser(ipaddr) {
    let username;

    if (ad_bundle_active) {
        username = Session.lookup("AD_User_" + ipaddr);
        debug(`User from session table: ${username}`);
    }

    // TODO: Replicate AD Kerberos parsing 

    // TODO: Build other auth methods based on customer requirements

    return (username ? username : ipaddr);

}

I added the client VPN address pool from each on my GlobalProtect PANs as a Custom device but it still won’t show any metrics for internet traffic. Should that CIDR be added as external traffic?

@mariode You want to have the internal VPN concentrator CIDRs . When you navigate to the custom device directly, do you see metrics? You can type the name you gave the custom device into the global search in the upper right to navigate to the Custom Device.

image

A few issues I have seen, check and make sure the device is in advanced analysis

image

If you click “Edit Assignments” and click the Triggers Tab, make sure the VPN Network Monitoring trigger is assigned. If it’s not, you can assign it here.

image

Finally, check that the VPN Monitoring Dashboard has your custom device as a source

image

Hopefully that helps.

1 Like

Every time I try to modify sources it defaults to the following:

I think I know why… I have 3 VPN gateways, one for each data center for load balancing. I created a custom device for each gateway. This time I’m going to create a singe custom device with all 3 in one and see if that helps.

Another tip, you can use a Device Group as the source as well, so if you haven’t already done created a superset custom device, try throwing the three smaller custom devices into a static device group and use that as the source.

Is there a way to copy the trigger so that I have separate monitoring for each device/Data center?

How is it best to implement this bundle in environments with explicit proxy (with a view to identifying internal v external traffic)? I was thinking to either tweak the trigger (define an explicit proxy list and add logic); I guess setting the explicit proxy IP addresses as external is quicker but may have other considerations.

So based on professional services we have all of our VPN ip setup for remote discovery and not as a custom device. We did apply those groups to the trigger. but it does not pick up any metrics for these devices.
And we have found that with remote discovery of them we are getting alot of good data so do not really want to move back to custom device for the vpn ip’s, any suggestions?

Hey Mitch,

No problem. I’ll take a note to build a flag that enables both deployment models. The reasons why the current trigger does not collect any metrics for remote IP discovery today are the filter lines

for (const customDevice of client.customDevices) {
    if (customDevice.hasTrigger) {

and

for (let customDevice of server.customDevices) {
    if (customDevice.hasTrigger) {

If we replace those two lines with the following, you should start seeing metrics

if(client.device.hasTrigger) {
const customDevice = client.device;

and

if(server.device.hasTrigger) {
const customDevice = server.device;

I’ll make a note below when I get a new version reved for the remote IP model.

that is what I thought. I was trying to figure out how I can work around those. just have not figured it out yet. still very weak on triggers.

Dah I guess I need to read this better. I will give it a try now. Thanks for the quick response

That worked perfectly, thanks putting it to use now.

@mitchroberson thanks so much for the feedback. I’ve updated the bundle to include a flag useCustomDevice to support environments which have all of the VPN IPs discovered as devices. The experimental version of the bundle does not use the flag yet for the Cloud App monitoring portion, but we will try to get that configured soon.

Thanks again!

I have a customer with the following situation… I’m curious if there’s a way we can help correlate VPN Client -> External (hairpinned) Traffic…

""Problem is how the vpn traffic is handled by Palo. When a vpn user goes to the internet and not using split tunnel it hairpins back out causing that traffic to not be seen. I added a new span and now see it but the IP address is always the outside interface address. No easy way to tie back to user. “”

Thoughts?

What is the likelihood that this makes it to an ExtraHop Supported Bundle on the Bundles Gallery???

The only way to tie it back is by SSL information in the packet. if it is SSL, if it is http or some other traffic much more difficult to tie it back to the user.