One of the more common alerts I investigate here at Hurricane Labs is when geographically improbable access is detected for a user.
Geographically improbable access alerts typically fire when–as the name suggests–a user authenticates from two locations which are unlikely to be possible given the short period of time of the authentication attempts and the distance between the two locations. This alert may simply indicate a user is accessing a VPN client or some sort of web authentication, or it may lead to the discovery of malicious activity, such as account compromise.
In this blog post, I will walk you through my investigative process to give you some strategies for handling this type of alert.
Phase 1: Machinae
I begin my investigation by using one of our in-house tools, Machinae, to gather information about the two IP addresses seen in the alert. If you’re unfamiliar with Machinae, this is a tool often used by security analysts to improve their visibility by simplifying the collection and correlation of security-related data from public feeds and sites, including IP addresses, domain names, URLs, email addresses, and so on.
Another tool that I use is called geo_compare. This command will output details about the geographical location of the IP addresses, including how far apart they are and how fast the user would have to travel in order to authenticate from the two locations physically.
Additionally, I can use Machinae to investigate the IP addresses and receive output from various public sites/feeds (e.g., Telize, Fortiguard, and VirusTotal) to help determine if the IP is malicious or not.
Phase 2: Splunk
While that command is running, I’ll access the Splunk environment where the alert fired and begin searching.
Typically, the default search that we’ve setup for these alerts will provide the majority of the information I use for my investigation, but I like to expand the timeframe out to the past seven days. This way, I can see if the user has authenticated from both IP addresses within that span of time.
The screenshot below shows a search similar to what our default searches look like.
The featured data above was provided by Splunk in their Fundamentals classes.
index=network sourcetype=cisco_wsa_squid username=kpeha (src=126.96.36.199 OR src=188.8.131.52)
If, in fact, the user has authenticated from both IPs within a seven day period, and the IP addresses come back as clean when run through our Machinae tool, I will typically close the investigation as benign activity.
If I don’t find evidence that shows the user has authenticated from both IPs within the past seven days, I will expand the search out to 30, 60, and 90 day increments. If I see activity from both IP addresses in those ranges, I will again typically close the investigation as benign.
Phase 3: Pivoting
Assuming I do not find any other authentications from both IP addresses within a 90 day period, I will pivot the search away from those two specific IP addresses. I then begin searching to see all the IP addresses used by the user for authentication over the past seven days.
Data provided by Splunk in their Fundamentals classes.
index=network sourcetype=cisco_wsa_squid username=kpeha | iplocation src | stats values(src) as src by City, Region, Country
In my experience, I usually have to pivot like this when one of the IP addresses in the initial alert is registered to a telecom company such as AT&T or Verizon.
I will look for IP addresses that share a common geographical location and a common subnet scheme. Once I have found other authentication addresses, I will then run those through Machinae to get registration information as well as information about the maliciousness of the address itself.
If I find that the user is authenticating from IP addresses in similar geographical locations and/or registered to the same company as the initial alerting IPs, I will typically close the alert as benign.
Phase 4: Escalation
If throughout the entire investigation, I cannot find any evidence that the user has previously authenticated from one or both of the IP addresses in the initial alert, nor have they authenticated from similar IP addresses or geographical locations, I will escalate to the customer and advise them to investigate further.
At this point, I will typically recommend they reach out to the end user and verify the authenticity of the login events. I also recommend that they consider disabling the user account if the login attempts are not authorized or expected, and changing the end user password.
I hope this overview of my investigative route will provide some useful insight to handling geo improbable alerts. When you’re dealing with one of these situations, it can be easy to go into autopilot mode and go through the motions. Remember to take your time; if something feels off, don’t be afraid to have someone else take a look at your work and see if they come to the same conclusion.