7 Best Practices for Dark Web Scanning

It would be great if Dark Web scans were as simple as Google searches – if you could simply plug your business’s name into a search engine, run a query and view a list of results about threats that impact your company.

Unfortunately for businesses seeking to stay a step ahead of threat actors, quite the opposite is true. It’s not just that there is no Google or search index that teams can turn to when searching for threats. It’s that if you tried to build an index of Dark Web content, you couldn’t, because the Dark Web is constantly changing. Threat actors continuously jump from one forum to another. They post content, then delete it hours later. They change their digital identities sometimes multiple times per day.

For these reasons and more, simply defining what you need to scan, let alone scanning it, is a tremendous challenge when it comes to the Dark Web.

But that doesn’t mean businesses should ignore the Dark Web. On the contrary, the Dark Web presents a wealth of information that helps organizations discover which threat actors may be targeting their IT estates and which attack techniques the actors intend to use. As Forbes notes, Dark Web scanning is critical “to gain actionable threat intelligence so that you can be forewarned about potential attacks.”

The question, then, is how to perform Dark Web scanning effectively. Although there is no simple answer, there are effective techniques that, when employed simultaneously, empower teams to identify relevant threat information lurking on the Dark Web and incorporate it into their cyberdefense strategies.

Let’s walk through those techniques and explain the role they play in Dark Web scans.

1. Automate Dark Web scanning

First and foremost, automating Dark Web scans and monitoring is paramount. The Dark Web is far too large and dynamic for manual analysis to yield effective results.

For example, consider that threat actors might share a link to a new secret Telegram channel in an existing Telegram channel, then delete the secret after, say, an hour. In this case, not only would it be difficult to find the Telegram channel manually, but you’d stand a very small chance of accessing the channel within the narrow window of time when the secret link is available. If you miss it, you could miss critical threat information that impacts your organization.

By automatically and continuously scanning the Dark Web, however, you can identify new sources as soon as they appear and begin pulling data from them into your threat intelligence database. Using this approach, even if the link or invitation to that forum or channel disappears, you can still maintain visibility into that intel source and monitor hidden activity.

2. Think and act like a data scientist

Automated Dark Web scans are likely to leave your team with reams of data. To process that data efficiently and ensure you can identify relevant information within it, you need to think like a data scientist. Leverage machine learning to structure and tag data, for instance, and store the data in a threat intelligence database or data lake against which you can run queries.

Data science techniques can also help teams work around barriers that make it difficult to harvest Dark Web data. For example, by employing data scraping strategies, they may be able to bypass captchas and bot-blocking features.

These practices are most commonly associated with the management of standard business data, not data surfaced through Dark Web scanning. But the fact is that the sprawling, unstructured nature of Dark Web data means that the only way to make sense of that data is to apply the same sophisticated data analytics and management techniques that you’d expect from a first-class business intelligence or data science team.

3. Zoom in on the data that matters

Collecting all of your threat intelligence data into a data lake is only useful if you have an efficient means of identifying relevant, actionable information within the data lake. To do this, you need to be able to home in on the data that actually matters by running custom queries against all of the data.

For example, imagine a threat actor announces in a Telegram group that he has found a major vulnerability on an IP address associated with one of your endpoints, or that he’s selling credentials with a username that includes an email address tied to your organization’s domain. To identify threats like these, you need to write queries that can parse the threat intelligence data lake for data linked to your business – like IP addresses and domain names, in these examples.

4. Analyze data sources comprehensively

Threat actors operate on a wide array of platforms, and they often jump from one platform to another when sharing threat information or discussing vulnerabilities. For that reason, it’s critical to ensure that your threat intelligence scanning extends across all corners of the Dark Web, as well as similar platforms where threat actors operate.

A complete list of all data sources relevant for threat intelligence scans is beyond the scope of this article. But for starters, you should be sure that you cover all of the following platforms:

• Telegram, whose privacy features make the platform an attractive place for threats actors seeking to discuss vulnerabilities or share tips.
• Discord, another go-to platform for threat actors seeking to share information with one another.
• Ransomware gang onion sites, which often host a list of threats and victims.
• Security feeds from websites and social media, which offer real-time insights about breaches or attempted breaches. 
• GitHub and other code repositories, which can accidentally reveal proprietary source code, host malicious code and exploits, and contain discussions related to vulnerabilities.
• Deep Web forums, which allow threat actors to discuss risks in places that the general public typically won’t access.
• App stores, where threat actors may deploy trojanized applications that masquerade as official apps released by a trusted brand or business.
• Dark Web forums and sites, where threat actors can post anonymously with less risk of being found.

5. Keep humans in the loop

Automated Dark Web scanning and data management techniques go far to help businesses harvest and act on threat intelligence data at scale. But because automated analysis will always be subject to a margin of error, it’s essential to involve humans in threat intelligence assessment, too.

Humans can increase the fidelity of your data, by reviewing the threats surfaced by your automated analytics processes to sort false positives from actual risks. This is especially important when the confidence level of automated threat alerts is not high. Humans are also critical for deciding which alerts to prioritize so that your team doesn’t end up chasing low-risk threats while more serious ones remain unaddressed.

At the threat investigation and remediation stage, too, you need humans to add context to the alerts and coordinate response – an activity that involves far more than just writing reports. When your Dark Web monitoring tools recognize a threat, it’s often necessary to coordinate with an analyst to build a cybersecurity analysis, assessments, analytics projects and deep-dive reports customized for the threat and your organization’s needs.

From there, your organization can create strategic threat intelligence and security advisory reports that provide the basis for the formulation of strategy, policy and longer-term decision-making – all of which are driven by humans. Humans are also critical for achieving a deeper understanding of a specific threat actor or group, including their place of operations, targeted countries and verticals, preferred tools and types of operations.

6. Integrate and contextualize threat intelligence data

Dark Web scanning tools are one component of a modern cybersecurity arsenal, but they’re only that – one component. Teams also need additional tools, such as:

• SIEM or SOAR platforms, where they can consume and analyze threat intelligence data and correlate it with other insights.
• Ticketing systems to help manage threat response activities.
• Extended Detection and Response (XDR) tools to help automate threat identification and remediation activities.

To leverage these solutions to maximum effect, it’s critical to connect Dark Web scanning tools to the tools where you analyze and contextualize threats. As IDC notes, “Ever-evolving threat intelligence feeds necessitate consistent cross-referencing,” and you can only achieve that when you are able to correlate and contextualize threat intelligence data alongside other sources of insight – like suspicious activity recorded in application and server logs – by analyzing the data within tools that pull all relevant information together.

7. Extend threat intelligence to suppliers

When it comes to Dark Web scans and threat intelligence, no organization is an island. On the contrary, most modern businesses rely on complex supply chains to power their digital operations, and reacting to threats often requires collaboration with suppliers to mitigate threats at the source.

This is why combining Dark Web threat intelligence with supply chain intelligence insights is vital for staying a step ahead of threat actors. When you can identify threats that impact third-party software or services your business uses, you can work efficiently with your suppliers to mitigate vulnerabilities in their products – which is not only good for them, but also keeps your business safe.

Consider a Dark Web Scanning Service

With so much data to parse, and with data sources that are constantly changing (and, in some cases, disappearing), identifying and reacting to threats that lurk on the Dark Web is no mean feat. To do it effectively, you need to deploy a variety of complex processes and tools – ranging from automated scanners, to advanced data analytics techniques, to human threat analysis and reporting and beyond.

If you can’t manage these solutions internally, a Dark Web scanning service can help. By working with a partner like Cyberint, you can obtain best-in-class Dark and Deep Web scanning services that alert you to threats targeting your business, without having to set up and manage your own complex array of Dark Web scanning and data management solutions. Learn more by requesting a demo.