Cyber threat hunting is a proactive security search through networks, endpoints, and datasets to hunt malicious, suspicious, or risky activities that have evaded detection by existing tools such as anti-virus or endpoint detect and respond tools.
Cyber threat hunting is a more aggressive approach than threat detection in the regard that it assumes a breach will occur or has occurred.
Prior to artificial intelligence (AI) and big data analytics, threat hunting required many man-hours to be an effective security hedge. Many security analysts had to comb through a mountain of log files to find anomalies in datasets and then investigate those anomalies one by one to make sure that what they were seeing was a legitimate security threat. These security analysts used their own knowledge and familiarity with the network to create hypotheses about potential threats, such as, but not limited to, lateral movement by a threat actor. The act of threat hunting was very specific to any given organization’s network because each analyst had to know what a safe network for their organization looked like prior to network threats.
With the advent of AI, machine learning, and big data analytics, threat hunting is done almost exclusively using super-computing technology. Once machine learning has found an anomaly, today’s security analyst can review the finding and determine if the threat is real, after which the analyst would simply have to confirm or refute what was flagged. This frees security analysts to be much more proactive in protecting an organization because far fewer analysts are required for threat hunting.
What should you look for in a threat hunting solution today?
There are many services out there today that allow an organization or an IT security company to manage threat hunting in an affordable way and on a large scale. If you use a managed IT security service, that provider can use any one of several solutions available to them. In some cases, these solutions may be channel exclusive, which means that the vendor will only sell to an actual IT partner and not to an individual business. All threat hunting solutions should have at least one dedicated Security Operations Center (SOC) that is staffed 24x7x365. This staff will be equipped to sort through false positive alerts and get to the real threats quickly. Below is a list of threat hunting tools that make up a comprehensive threat hunting solution and that a SOC might utilize:
SIEM or Security Information and Event Management is a set of tools and services offering a holistic view of an organization’s information security. A SIEM tool reads and compiles event logs from multiple data source such as firewalls or access-and-events logs from servers in an organization’s environment.
EDR, or Endpoint Detection and Response, is an integrated endpoint security solution that combines real-time continuous monitoring and collection of endpoint data with rules-based automated response and analysis capabilities. A good EDR tool is like anti-virus on steroids. But unlike anti-virus, an EDR tool does not use an antiquated signature-based definition on how to identify threats. EDR uses artificial intelligence and big data analytics to look for anomalies that a machine is not used to seeing. Once an anomaly is identified, the EDR tool will isolate that machine as not to threaten other devices on the network.
Beyond these threat hunting tools are threat hunting techniques that are used by today’s threat hunting teams, which are typically located in a centralized Security Operation Center. The individuals working at these centers search and analyze data sources, as well as baseline, stack-count, group, and cluster data. Below is a breakdown of this approach:
- Analysis – Inspecting data sources and logs (e.g., DNS and firewall), examining network, file, and user data, and reviewing security information and event management (SIEM) and intrusion detection system (IDS) alerts to identify threats.
- Baselining – Establishing the normal threat levels and then exploring possible deviations from the norm.
- Clustering – Examining large groups of related data to help isolate similar anomalous data characteristics or correlations between system and network activities. Clustering often involves the use of machine learning and AI.
- Grouping – Based on predetermined search criteria, grouping is the act of analyzing unusual or suspicious data to determine if a threat or problem exists.
- Stack Counting or Stacking – certain data values and then putting them into ‘stacks’ based on characteristics. Any outlying data is flagged for further examination.
Although this is not an exhaustive list of all the components that are built into threat hunting, it is a great start to understanding the many complex aspects that go into securing an environment using a threat hunting approach.