Yet more reasons why data privacy is “dead on arrival”

Data has become a crucial part of our infrastructure, enabling commercial and social interactions. Rather than just tell us about the world, data acts in the world. Because data is both representation and infrastructure, sign and system.

And people are completely missing the importance of augmented reality technology: looking at screens and pecking at keyboards will surely seem quaint by 2030

 

27 November 2020 (Chania, Crete) – Over the last few weeks I have written about Google’s data slurping and that we’ll need to brace ourselves for even more data grabs given Apple’s new M1-based machines which have launched a new type of computer system. We are making a major philosophical leap in the underpinning of computers and mobile devices. And I have written about the recent U.S. elections and what they have taught us about the futility of tech regulation.

It all relates to the insane conversations I have with “experts” who think data privacy is actually possible. Data is in perpetual motion, swirling around us, only sometimes touching down long enough for us to make any sense of them. We use data, these numbers, as signs to navigate the world, relying on them to tell us where traffic is worst and what things cost and what are friends thinking and what our friends are doing and placing an Amazon order and letting Amazon “logistic” the delivery of the package, etc., etc.

And because we do this, data has become a crucial part of our infrastructure, enabling commercial and social interactions. Rather than just tell us about the world, data acts in the world. Because data is both representation and infrastructure, sign and system. As media theorist Wendy Chun puts it, data “puts in place the world it discovers”. Restrict it? Protect it? We live in a massively intermediated, platform-based environment, with endless network effects, commercial layers, and inference data points.

For those of you who followed my series last year on the Appboy and Disconnect data trackers, you know that each of us who use an Apple or Android phone (and even your laptop) will engage with 5,400+ trackers, mostly embedded in apps, most feeding to data brokers. According to the privacy firm Disconnect, you spew out an average of 1.5 gigabytes of data about yourself over the span of a month. But you need special software to track all this. This all feeds into the big General Data Protection Regulation (GDPR) problem over “transparency” and why EU citizens are struggling with “Data Privacy and Data Subject Access Requests” (DSARs) mandated under the GDPR: if you don’t know where your data is going, how can you ever hope to keep it private? (DSARs are a sham but data privacy vendors will not tell you that because they need to make money from their services. I’ll have a detailed post on “The Good, the Bad and the Ugly” of DSARs in  the coming year).

Big Tech has outgrown your office and your living room. They want to be with you literally every where you go, and constantly seduce you with entertaining and immersive experiences. The more of your attention they can monopolize, the more money they can make from selling chunks of it to advertisers and people developing software on their platforms. Four companies … Facebook, Apple, Amazon, and Microsoft with a combined market capitalization of over six trillion dollars  (as of 27 November 2020) … are mortal enemies fighting expensive digital wars of attrition in every  setting – except one.

OpenStreetMap, the free Wiki map of the world

 

Jennings Anderson, Joe Morrison, and Vicky Johnson-Dahl (cartographers and geospatial specialists) along with Hacker News have done heroic work examining OpenStreetMap. I will try and consolidate all their findings and pepper it with my own thoughts. 

OpenStreetMap (OSM) is the largest volunteered geographic information project in the world, characterized both by its map as well as the active community of the millions of mappers who produce it. What likely started as a conversation in a British pub between grad students in 2004 has spiraled out of control into an invaluable, strategic, voluntarily-maintained data asset the wealthiest companies in the world can’t afford to replicate – forming an unholy alliance of the world’s largest and wealthiest technology companies. The most valuable companies in the world are treating OSM as critical infrastructure for some of the most-used software ever written.

The numbers are incredible. Over 1.5M individuals have contributed data to it. It averages 4.5M changes per day. The stats page on the OSM Wiki is a collection of hockey sticks that look like this:

Cumulative registered users over time. Source: https://wiki.openstreetmap.org/wiki/Stats

You can think of OSM in several ways:

• A distributed community of mappers contributing information about the geography of the world to a common repository

• A free web map hosted at https://www.openstreetmap.org/

• A loosely affiliated collection of free and open source tools for mapping the world

• A real-time stream of instructions representing how to add, change, or remove cartographically projected geometries and associated metadata based on a prior state

• … a kind of Google Maps, but openly licensed

It’s hard to get people to agree on what exactly OSM is, but almost everyone agrees on one thing: it’s extraordinarily valuable and important. And for those of you who read my posts, there is nothing new here. I have been writing about the advances in geospatial technology that few have been paying attention to.

But news alert: if you’ve ever opened Snap Maps or Apple Maps or Bing Maps or even just peeked at the dash of your obnoxious neighbor’s new Tesla … you’ve used OSM.

NOTE:  as an astute commenter on Hacker News pointed out, Tesla uses Google Maps for its dashboard. They only use OpenStreetMap for the auto-summon feature, which incidentally has resulted in lots of parking lots being added to OSM by the Teslarati.

And depending on what you are using, you can check the attribution:

So, of course, one of the consequences of increased corporate involvement in OSM is a significant backlash from members within the OSM community that feel the community (and data) is being irreversibly adulterated by these profiteering intruders. None of these companies is essential to OpenStreetMap. They are contributors, but OpenStreetMap could work perfectly well without them. The mainstay of OpenStreetMap is the millions of hobbyists, individuals that contribute to OpenStreetMap.

So what’s motivating the “Big Tech Gang of Four”? Joel Spolsky wrote earlier this year about the concept of “Commoditizing Your Complement” (securing chokepoints, getting a quasi-monopoly), and Joe Morrison has written about why Facebook acquired Mapillary and then gave away all the data they had just purchased, for free. Both are relevant to this conversation.

SHORT NOTE: I’ll get to Mapillary shortly but just a few points.

This year, the big news in the geospatial world was Facebook’s acquisition of Mapillary. For those unfamiliar, Mapillary is a darling of the mapping world and one of the highest-profile geospatial startups of the last decade – launched in 2013, their mission was to create a global street-level imagery dataset to rival Google Street View.

Mapillary was the prototypical “venture-scale” business – preposterously ambitious, technically impressive, inarguably valuable for the world, and plausibly monetizable. What Mapillary accomplished in a short seven years is simply staggering. Google, with an enormous head start and untold resources at the ready to support Street View announced last year that they’ve collected over ten million miles of street imagery. Mapillary, on the other hand, crossed three million miles of mapped streets in 2018 and has more than doubled the number of images in their catalog in the years since (to over one billion!), putting them squarely in the same conversation as Google. That’s an insane accomplishment for any company, let alone a startup who, over its lifetime, raised ~$25M or about half of the annual compensation package for a typical member of Google’s C-Suite.

Why did Facebook buy Mapillary, and then give all that data away, for free? to that? Short answer: to hurt Google, but it also complements their augmented reality business. Long answer? Achieving mapping hegemony which every Big Tech firm needs/wants. For more “long answer”, see the geospatial technology paragraphs which conclude this post.

Whatever the motivations of these mega-corporations with OpenStreetMap, they’ve succeeded in carving out a niche for themselves (in effect following Spolsky’s advice) within the OSM community whether the hobbyists like it or not. I’d like to highlight a nuance often lost in this discussion –  just exactly who are these companies hiring to add data to the map? They are often already-active, enthusiastic contributors to OSM. These are people living the open data fanatic’s dream: getting paid to do a job they find so fulfilling they would otherwise do it for free in their spare time.

And, yes, for Facebook, there’s obviously a lot more to it than just sticking it to Google. Facebook, for instance, has ambitions of building new types of digital experiences that interplay with the real world (as evidenced by their focus on augmented reality and acquisition of novel user interface technology like CTRL-labs). Apple has added LiDAR to its new line of iPhones and iPods allowing customers to scan the 3D world in high fidelity among other exciting uses.

As I noted above, these firms have outgrown your office and your living room. Whether you like their motivations or not, the result is a desire to map the world in higher fidelity and at larger scale than even they can afford to accomplish independently. And that has, for better or worse, brought their interests into alignment with the grassroots OSM community.

Well, tough question. Are you a data privacy freak or antitrust freak? Well, I suppose any time the wealthiest institutions in history are quietly collaborating on something, I think it’s worth noting. There does not seem to be a precedent for such a collaboration: mega-corporations working with a global community of volunteers on a public dataset.

But if any of you have worked on enormous database projects … I am one of 3,000 working on a dataset of global data slurps, updated daily, and distributed to a selected client list … you know it requires ceaseless, grueling work to keep updated, and that means serious investment of time and money. One of the key differences between our data slurping projects and OSM is the exponential size of the OSM community of contributors. Without it, the project is “default dead,” as they say in Silicon Valley. Map data goes stale fairly quickly and therefore requires constant life support.

And yet that community is precisely what attracts corporate contributors. OSM provides two advantages over just buying privately collected data:

• Existing data is free and growing apace

• But proprietary data contributed to OSM may also be expanded upon and/or maintained at no additional expense by the community

 

During the lockdown, I wrote a very detailed brief on geospatial technologies for a small set of clients (yes, I am retired but I still do a few one-offs, and next year I’ll be doing 2-3 guest keynotes at some tech events). So I shall crib bits and pieces from that masterpiece 🙂  Oh, and a special shout-out to Andrew Wiseturey, a “map guy” at Apple, for his help. 

The term geospatial technologies is used to describe the range of modern tools contributing to the geographic mapping and analysis of the Earth and human societies. These technologies have been evolving in some form since the first maps were drawn in prehistoric times. In the 19th century, the long important schools of cartography and mapmaking were joined by aerial photography as early cameras were sent aloft on balloons and pigeons, and then on airplanes during the 20th century.

The science and art of photographic interpretation and map making was accelerated during the Second World War and during the Cold War it took on new dimensions with the advent of satellites and computers. Satellites allowed images of the Earth’s surface and human activities therein with certain limitations. Computers allowed storage and transfer of imagery together with the development of associated digital software, maps, and data sets on socioeconomic and environmental phenomena, collectively called geographic information systems (GIS).

An important aspect of a GIS is its ability to assemble the range of geospatial data into a layered set of maps which allow complex themes to be analyzed and then communicated to wider audiences. This “layering” is enabled by the fact that all such data includes information on its precise location on the surface of the Earth, hence the term “geospatial”.

In the data slurping/data protection world we see the end result of this technology all the time: Big Tech able to track peoples’ movements via mobile devices both secretly and-not-so-secretly. One of the aspects of the community data collection project I described above was logging every court case and article, world-wide, that involves deceptive and unfair practices used to obtain users’ location data, which is then (usually) exploited for lucrative advertising business. But more interesting is collecting information on a government’s ability to obtain – easily, quickly and cheaply – that precise geographical location at virtually any point in the history of the use of the device. In my U.S. searches, it has been an education on “geofence warrants”, Fourth Amendment probable cause … and bizarre “particularity requirements” I certainly never envisioned in law school.

The Mapillary acquisition

As I noted above, earlier this year Facebook acquired Swedish mapping technology company Mapillary, which collects images from tens of thousands of contributors to build immersive and up-to-date maps. Mapillary aims to solve one of the most expensive problems in mapping: keeping maps updated with “street level data” about signs, addresses and other information that can be observed from the road.

Big companies such as Apple and Google solve the problem by sending out fleets of vehicles outfitted with cameras and other sensors to gather images. But Mapillary crowdsources the images, ingesting pictures contributed from smart phones and other types of cameras, and uses “computer vision” technology to stitch them together into a three-dimensional map. Many consider that information key for self-driving car technology, although for Facebook it would also underpin its products under development like augmented reality glasses and virtual reality headsets.

It does some very cool stuff, all very much sought and admired by the geospatial community. Just two quick examples:

• OpenSfM, a popular computer vision engine for stitching together overlapping images to reconstruct places in 3D

• Vista, a free 25K image dataset labeled for semantic segmentation  –  one of the largest such open datasets in existence:

But what is really at stake here? Apple, Google, and Microsoft have invested billions into consumer mapping applications and acquired numerous companies to support those efforts. Any of those three behemoths would have felt like natural landing spots for Mapillary. But Facebook? What’s that about?

Unless you are already tapped into the geospatial industry you can be forgiven for not knowing that Facebook actually does do maps. Not only via OpenStreetMap as I detailed above, but such things as an open source, AI-assisted road mapping tool called RapID that is an impressive thing to witness in action.

Still, the mere fact Facebook has dipped its gargantuan toes into the mappy water explains why “geo” is so incredibly important to these platforms. There are three strategic reasons why Facebook made this acquisition and illustrates the importance of geospatial mapping to Big Tech:

Said Joe:

Previously, you could pay Mapillary for special commercial licensing if, for example, you wanted to use their imagery to inventory street signs and sell that data to municipal governments. Now it seems that Facebook just wants to give it all away for free. If you’re like me and have 100 ideas for how this data could form the foundation of a commercial endeavor, that is rather exciting news. It raises an obvious question, though: why would Facebook want to pay a bunch of money to take a revenue-generating company and turn it into an unabated cost-center?

Answer: to hurt Google. This is a strategy known as “commoditizing your complement” which I noted above. Essentially, if you’re getting your ass handed to you by a competitor with a software or data advantage you can’t replicate yourself, you can use permissive licensing as a weapon to erode that advantage by encouraging the world to collaborate with you. Google itself is no stranger to this strategy; their choice to acquire Android and support it as a free and open operating system for smartphones is an example of this tactic. In that instance, the complement was Apple and iOS.

Facebook links out to Google Maps in their own product.

If you’re Facebook, I would guess that sometimes it’s easier to negotiate buying a company outright than it is to negotiate a custom licensing deal that is orders of magnitude larger than any other deal that vendor will ever see in the life of their company. Because of their scale, the value of a particular data asset to Facebook may be entirely dislocated from the rest of the market. At a certain point, you just have to ask yourself, “Instead of buying that, why don’t we just buy them?”

 

Before I conclude with my wrap-up on geospatial technology, a few words on the “Data For Good” projects with which I was fortunate to work here in Europe.

In the earlier days of the coronavirus pandemic, an animated map from a company called Tectonix went viral. It showed spring breakers leaving a Florida beach to return to their homes across the U.S., as a series of tiny orange dots congregating on a beach in early March scattered across the country over the following two weeks. “It becomes clear just how massive the potential impact of just one single beach gathering can have in spreading this virus across our nation,” the video’s narrator said. “The data tells the stories we just can’t see.”

Most people were horrified but health experts saw an opportunity. How we move about in our communities – where we go and how often – greatly affects the spread of COVID-19. And few know our whereabouts better than Facebook and Google. So, in an effort to help researchers combat the pandemic, the two companies made their troves of GPS-based mobility data available. The data comes from users who opt in to location services on the companies’ platforms and is provided for public health use in an aggregated, anonymized way. Such data is vital to public health researchers’ efforts to understand trends in population movement and predict the spread of the disease, which is caused by the novel coronavirus SARS-CoV-2. Local government officials can use the data to make informed decisions on travel and social distancing interventions.

Both Facebook and Google are providing information about where people are going, but the companies differ in the way they are releasing the information. Facebook, through its program, provides mobility datasets and maps directly to researchers upon request. Facebook generates the data in file formats that support epidemiological models and case data. Laura McGorman, policy lead for Facebook’s program:

“We’re sharing the data in a way that public health researchers can use. Once a researcher signs a license agreement, they can request data through our mapping portal and get it the next day. The mobility datasets let researchers look at population movement between two points, movement patterns such as commutes, and whether people are staying close to home or visiting many parts of a town. Almost anywhere across the globe”.

Scientists have used Facebook’s data in several ways over the last few months. The program first became well known when scientists used Facebook’s mobility data to study how social distancing measures and a stay-at-home orders have affected movement near Seattle, Washington in the U.S. They found that population movement indeed declined, which led to reduced transmission of the virus. We used it in Italy to analyze how lockdown orders affect economic conditions and create an economic segregation effect.

Google launched a mobility tracking tool called “COVID-19 Community Mobility Reports”. The web-based tool is available freely to the public and provides insights on how communities have reduced or increased their visits to certain types of places. The public can go to the website and choose a region, such as a state or country. The tool then generates graphs on a downloadable PDF displaying the percentage change in visits over the last few weeks to places such as retail stories, pharmacies, parks, places of work and public transportation hubs in that region.

Example: in Athens, Greece over the summer, before the government tightened lockdowns, people had reduced their visits to grocery and pharmacy stories by 27% and to other retail locations by 45%. Visits to area parks and waterfronts increased 64%. In a blog post highlighting the resource, Google executives wrote that they believe the mobility reports could help shape business hours, inform delivery service offerings, or indicate a need to add additional buses or trains to a particular public transportation hub. The company pulls the data from Google users who have opted in to location tracking services. The information is aggregated and anonymized, and does not provide real-time data in an effort to protect privacy.