1. 8. Hal Hodson: A visitors' guide to the dark web
2.  
3. Hal Hodson is a technology reporter at New Scientist
4.  
5. Whether for privacy or profit, much of the internet is hidden away -
6. we reveal the places that Google won't take you
7.  
8. OVER the past two decades, we've created a whole new world that's
9. visible only through a screen. Visibility and transparency have long
10. been hallmarks of the online world. We're used to being able to see
11. anything posted on the web, and we're used to the idea that anything
12. we do there may be visible to anyone else.
13.  
14. That's not true though. To understand why, consider how you find
15. things online. It's tempting to think that anything anyone has ever
16. published can be magicked to your screen with the right keywords.
17. That's partly true - if Google or Bing has indexed it, and your
18. internet service provider (ISP) lets you go there, you'll find it.
19. But those "ifs" are bigger than they may seem. It turns out that an
20. even greater proportion of the online world is dark than of the
21. physical universe: it is estimated that just 0.03 per cent of the
22. web is searchable.
23.  
24. Like dark matter and dark energy, we know the dark web is out there,
25. but we can't see it directly. Entire uncharted realms lie beyond the
26. reach of the crawlers that tirelessly catalogue the web on behalf of
27. search giants: databases, internet relay chat records, and the raw
28. data behind research papers. Much of it is unsearchable, because
29. most people aren't searching for it. And if they do search for it,
30. and don't find it, they assume it's not there to be found. "There's
31. this underlying assumption that if it's not online it doesn't
32. exist," says Ken Varnum, a systems manager at the University of
33. Michigan library in Ann Arbor.
34.  
35. But it's not just obscurities that may be overlooked. Search engines
36. and online stores exist to make money, and to do so they make
37. constant, almost imperceptible suggestions about what you should be
38. looking at. In some ways, this is benign: online shops prefer to
39. direct you to products like those you've previously shown interest
40. in. But the same algorithms can steer you away from things you might
41. like to see. And then there are things the online giants insist you
42. see. For example, last year Google banned an ad blocker program from
43. its app store. What's visible and invisible online is not
44. necessarily up to you: commercial interests can dictate what's
45. allowed on your screen.
46.  
47. Legitimate businesses aren't the only players in the invisibility
48. game any more, though. Privacy enthusiasts have developed a variety
49. of services that let their users tap into the dark web. One such
50. service, Tor, covers your online tracks by blending your internet
51. traffic into data from many servers worldwide to make you
52. functionally invisible. It has notoriously been used by criminals,
53. allowing, for example, the Silk Road black market to flourish for
54. almost three years before the FBI took it down. But it wasn't
55. invented by those looking to break laws, nor is it their exclusive
56. domain: it's widely used by activists and others who are persecuted
57. or living under surveillance.
58.  
59. That latter epithet could be applied to an increasing number of us.
60. In the past year, we've learned that government agencies and
61. marketing firms are able to keep amazingly detailed records of our
62. online activities. This might be for the purposes of national
63. security; or it might be to sell us more stuff. The problem is that
64. most of the data collected this way is visible to those who collect
65. it - but not to us.
66.  
67. Fed up, in 2012 Caleb James DeLisle started Hyperboria, an
68. alternative to the normal web. Like Tor, it binds people together in
69. a network that severs the link between them and their IP addresses.
70. Its denizens have built analogues of Twitter and Facebook there,
71. allowing them to communicate and socialise without fear of snoopers.
72.  
73. But Hyperboria is hard work, requiring far more than just a browser
74. and an internet connection. You have to access your command prompt,
75. install something called the cjdns protocol, and find someone
76. willing to vouch for you and become the bridge that lets your
77. computer connect to Hyperboria. This may explain why the user
78. headcount still languishes in the hundreds. Other anonymous networks
79. are even less popular, although they're growing in the wake of
80. Edward Snowden's revelations about NSA surveillance.
81.  
82. All of these new darkling networks rely on the existing cables and
83. servers that make up the standard internet, where ISPs are free to
84. block traffic to them. To become credible alternatives, then, they
85. will need their own wires and switches to carry their traffic. This
86. is beginning to happen. Across the US, small groups of dedicated
87. hackers are building their own wireless networks to run Hyperboria
88. on. In Portland, Oregon, you can now send an email across town
89. without ever connecting to the wider internet.
90.  
91. But can these alternatives win over the path of least resistance?
92. Perhaps. "Expediency is a very powerful thing," says Varnum.
93. Technologies are being developed to make invisibility easier. The
94. Enigmabox, designed to run on any internet connection, automatically
95. encrypts its users' email. Users don't need to understand the EUR283
96. box; they just plug it into their router.
97.  
98. Two hundred people are now on the network, says Enigmabox founder
99. Claude Hohl, based in Switzerland. While Enigmabox doesn't yet talk
100. to the other independent networks - such as the Portland meshnet or
101. Hyperboria - they're working on it. "When every alternative network
102. connects to all the others, that's when you start to have a global
103. network," he says: a true alternative to the internet.
104.  
105. Once it gets big enough, this dark internet will no doubt attract
106. attention from the very forces that have driven people into its
107. depths. But DeLisle says Hyperboria won't be as quick to succumb to
108. centralised control as today's internet, "which requires central
109. authorities controlling it".
110.  
111. Much more than our surfing habits depends on the outcome of this
112. struggle. The online world is increasingly converging with the
113. physical world. Apps annotate the space around us, revealing places
114. to eat and people to meet. We can look into the past with apps that
115. overlay history on what we see; or the future. Through our screens,
116. we can see the flow of city traffic, or the positions of stars
117. behind a cloudy sky - and those screens may soon be perched on our
118. noses: before our very eyes.
119.  
120. Technology may soon let us see almost anything we can imagine.
121. Whether we will depends on whether we fulfil the decades-old promise
122. of the internet - to make the world more visible, not less so.