You may have heard about this mysterious place on the Internet called the ‘Deep Web’. Interest in the deep web and dark web arose following a few high profile data leaks, a growing interest in online privacy, references in mainstream media, and the widely publicized FBI investigation into online marketplace Silk Road that exposed the extent of online drug trafficking.
But what is exactly the deep web? Well it’s not actually a 'place' but rather anywhere other than the visible web that is crawled, indexed and accessed through links fetched by search engines like Google. The deep web is actually several times bigger than the indexed web, and it encompasses a lot more than the illegal or otherwise questionable activities that it is often associated with -- though there's certainly a lot of that going on, too. Let's start by clearing up some of the confusion among the different terms that tend to be thrown around while discussing the Internet's underground.
Note: This feature was originally published on 05/20/2016. We have revised it and bumped it as part of our #ThrowbackThursday initiative.
In simple terms, the Surface Web is anything that a search engine can find by crawling links and identifying content online, while the Deep Web is anything that a search engine can’t find. The Dark Web on the other hand merely represents a smaller portion of the deep web that has been intentionally hidden and is inaccessible through standard web browsers.
To exemplify this a bit further, the deep web can actually include things like academic databases, which can be accessed through a normal browser but are not accessible to search engines.
It also includes content within mainstream websites that is simply not set up to be accessed through a search engine. For example content posted on Facebook that is only available to the poster's friends, not the general public, or results for specific queries on a travel booking website -- you simply can't get to the content by clicking through links like a search engine would.
The Tor network is the most commonly known place where dark web content resides. This anonymous network consists of thousands of servers located all over the world run by volunteers and online privacy advocates. Its name is an acronym for ‘The Onion Router’ in reference to its namesake routing technique, which creates a path through randomly assigned Tor servers, or nodes, before reaching whatever website you are visiting. During this process data packets are wrapped in successive layers of packets that get "peeled off" at each node until reaching the destination.
Here is a more technical explanation, but in concise terms, the result is that no one node knows the complete path between your computer and a website, only the last place the packet was, and the next place it will be. This makes it almost impossible for anyone to trace the traffic back to you.
While finding content on the dark web can take time and plenty of patience, technically, accessing the Tor network isn't that complicated at all. You simply need to download and install the Tor browser available for Windows, macOS, and Linux. It is no different from installing any other piece of software and it's very simple to use as it's just a modified version of Firefox.
If you're serious about the security of whatever it is you're doing on the dark web, it's worth running an entirely separate environment from your main computer and internet connection. Tails OS is a bootable operating system that's already preconfigured to maximize privacy and that you can run from a USB stick or a virtual machine. Tails has an installation wizard that guides you step-by-step through the process of setting up the software so that shouldn't pose much of a challenge either. Tails received a lot of press when it was disclosed that Edward Snowden was using it to avoid NSA snooping. Its sole purpose is preserving your privacy and anonymity online by relying on the Tor network and other tools to keep your activity secret.
Layering your location by starting at a random Wi-Fi hotspot and then connecting to a VPN before loading Tor will greatly increase your odds of anonymity.
Almost any type of illegal and legally questionable products and services can be found in the seedier corners of the web. One of the most prominent examples is the Silk Road, the now defunct online marketplace where you could score anything from drugs and have them mailed to you, hire a hitman, buy passports and credit card information, weapons -- you name it. Several copycats have sprung and fallen since the Silk Road was taken down in 2013.
But even though technologies such as Tor are used to facilitate illegal activities, the technology itself isn't inherently good or bad. In fact, you might be surprised to learn that Tor was created with financial support from the US government as a tool for fostering democracy in repressive regimes, and that's exactly what it's used for a lot of the time. It's become popular with journalists and activists and even regular citizens who value privacy and freedom of expression.
Finding content inside the deep or dark web is somewhat reminiscent of the early days of the world wide web before search engines like Altavista and Google made finding content way easier. You can traverse link directories like OnionDir and The Hidden Wiki which list sites under a range of categories covering various services and interests -- blogs, hosting, forums, warez, hacking, to name a few.
There are also a few search engines like DuckDuckGo and you can find some guidance or related information on reddit, too. There will be a lot of sites that have been taken down or are offline so finding working, up-to-date links and directories can take some time.
Even though the Tor browser protects you by routing your traffic through various different IP locations, and Tails OS includes built-in encryption, nothing is 100% anonymous or secure. Tor and Tails' websites list a number of recommendations and best practices for remaining anonymous as best possible, like using HTTPS versions of websites, avoiding browser plugins or torrenting over Tor, and of course, not giving away personal details on websites you visit. We have also written our own list of security recommendations here.
Finally, using Tor shouldn't get you in trouble with the authorities on its own but merely being a Tor user can arouse suspicion with the law. The anonymous network has long been a target of government agencies like the NSA and the FBI, which have successfully de-anonymized criminals before. So to be clear, if you are up to something illegal it could land you in legal trouble.