Dark Insight: A Dark Web Crawler for Identification and Investigation of Drug Markets

Abstract

The dark web is growing day by day. Even in modern times, the dark web remains a platform that provides the highest degrees of anonymity and protection. It is used by criminals for communication and business purposes since it maintains their anonymity and keeps their transactions confidential. The dark web has been adapted to support the illegal sale of drugs, user information, and other illegal services, such as hacking and assassination. In addition, it is used to distribute unlawful content through the internet. In conclusion, everything that is illegal or banned by the government over the regular web is freely accessible on the dark web without hindrance. This is due to the fact that the dark web works independently of the regular web. Globally, the dark web continues to play a crucial role in the illegal drug trade. Thus, drug mafias are able to communicate and deliver drugs. Similar to the e-commerce websites we see on the surface Web, people on the dark web offer illegal substances through their own websites. To manage and restrict the global drug trade, a platform that can efficiently monitor these drug marketplaces is required. In this final year project, a web-based service is developed named "DarkInsight" that scrapes the data from the drug markets of the Dark web. This system permits law enforcement agencies to monitor these drug markets on the dark web, assisting them in evaluating the drug trade in order to prohibit it. The system will enable them to search the platforms of the dark web for pertinent data and will provide an analytical report for them. Additionally, the system will produce a PDF report for offline use. Crawled data from the dark web can also be exported for analysis and application of machine learning algorithms to help gain a deeper understanding of the dark market. Overall, this system includes features that will be useful for investigating and comprehending drug marketplaces on the dark web.