Create Better Correlation Searches

Splunk Enterprise Security comes with 60 correlation searches. Depending on your data, some of these searches are a little noisy and create too many Notable Events. In this article I’m going to show you how to modify a correlation search to narrow the focus, and create a smaller amount of higher quality Notable Events.

A Noisy Noise

One particularly loud correlation search is Threatlist Activity Detected. This search creates Notable Events when ip address, urls, file names, and other threat items are found in your Splunk data. Depending on the threat lists you have enabled, this can be a very noisy correlation search. At a recent client of mine, this search produced over 6000 Notable Events in a 24hr period. How do you look at 6000 events in a day? You don’t.

How do you fix this? Look for ways to narrow the focus of the search.

Not All Threat Matches are Created Equal

By default, the Threat Activity Detected search creates a Notable Event, and also adds to the risk score of the source (src). This correlation search assumes that all threat list items are equally bad. In practice, some threat intel types should be a lot higher priority than others. File intel matches, for example, should be treated as somewhat high priority. Particularly if there is a match for a known file hash. If you see a known bad file hash from one of your security devices, this means the file exists somewhere in your network. This is a lot more important that one of your systems talking to a suspicious ip address on the Internet.

Narrow the Focus

First, we’ll change the Threat Activity Detected correlation search. We don’t want Notable Events for every threat match, but we do want the matches to increase the risk score for the offending systems. The different types of threat intel are grouped by the field “threat_collection”. We’ll use that field to narrow the focus of a new correlation search. Risk scores contribute to the priority of Notable Events, so we still want to associate a risk score to the activity.

• Uncheck “Create Notable Event”. Leave the rest of the search settings as default. The search will still add a risk score to the src.

Create a new search by copying the existing Threat Activity Detected search into a new correlation search.  It may help to have the existing search and your new search open side-by-side while you copy and paste/duplicate settings.

• Modify the search to look for just file intel:

  | datamodel Threat_Intelligence Threat_Activity search | `drop_dm_object_name("Threat_Activity")` | search threat_collection=file_intel | dedup threat_match_field,threat_match_value | `get_event_id` | table _raw,event_id,source,src,dest,threat*,weight | rename weight as record_weight | `per_panel_filter("ppf_threat_activity","threat_match_field,threat_match_value")` | `makesv(src)` | `makesv(dest)` | `get_threat_attribution(threat_key)` | rename source as orig_source,source_* as threat_source_*,description as threat_description | `map_notable_fields` | eval risk_score=case(isnum(record_weight), record_weight, isnum(weight), weight, 1=1, null())  | fields - *time | eval risk_object_type=case(threat_match_field="query" OR threat_match_field=="src" OR threat_match_field=="dest","system",threat_match_field=="src_user" OR threat_match_field=="user","user",1=1,"other") | eval risk_object=threat_match_value
  

•  Copy the rest of the settings from the original search and save.

You now have a more specific search for file intel, and all types threat activity will still contribute to risk scores of systems associated with the threat matches.

Conclusion

This quick modification of the original threat activity correlation search will create higher quality threat match notable events with more focus. If a search is creating too much noise, it will eventually be ignored. Look for ways to narrow the focus of the noisy correlation searches, break them into smaller pieces, and find the most important types of events.