Multimer: Better Decisions Based on Human Signals

Think of where you were fourteen years ago. A lot has changed, right? Technology has changed. Information flow has changed. The way we relate to and understand each other has, in many significant ways, changed.

Fourteen years ago, my co-founder Ilias Koen and I met as students at an art and technology program in New York City. Back then, people still used portable compact disc players to listen to music on the subway. (Yes, the Discman.) Since then, we’ve helped each other build robots that draw, gloves that fly swans, and bike helmets that read brains. All of our projects, from grad school to the three companies we have co-founded, have been driven by a deep curiosity over the ways that data can unveil hidden relationships between our natural environment, our built landscapes, and ourselves as individual people. Our curiosity has been anchored in New York City, but it’s a curiosity that encompasses the world, its ecosystems, its major municipalities— especially now that the majority of the world’s population lives in cities.

In the past fourteen years, the convergence of several mature technologies and communication protocols — GPS, the Internet, cellular networks, Bluetooth, LEDs, touch screens — has empowered societies such that an individual person can generate massive amounts of new data each day — data that reflects emotions, thoughts, tasks, goals, dreams — through her computer, phone, and electronic devices. This is the kind of data that, by volume alone, often falls under the umbrella of “Big Data.”

There is another kind of big data that has been unlocked by the advances of the past fourteen years. We’ve nicknamed this data “Human Signals.” Human signals are basically the kinds of physiological signals that were once measured only in hospitals or specialized clinics: heart rates, brainwaves, muscle movement, respiration, sweat, steps — the list goes on. Our ability to collect and analyze this data has been limited. Until now.

Now, thanks to that convergence of technologies from the past fourteen years, we can measure human signal data not just from one individual, but from a whole population, and not just at one moment in time, but for days, weeks, months, and more. You may even own a device that measures some of these human signals, which have played a key role in understanding how our environments affect us. Being able to collect such large amounts of data, in such high spatial, temporal, and spectral resolution allows us to ask bigger questions, such as: when you average out the variance between individuals, days, and experiences, what does the real emotional temperature of an environment look like? What does the accurate mood map of a region look like?

These are some of the questions that drive us as we continue to develop our our location-aware biosensor MindRider and build our analytics platform Multimer. We think that their answers could have a profound impact not only on placemaking, wayfinding, and sensemaking, but on how we conceive of, report on, and plan these processes. They could have a profound impact on how media is made, and on how it is consumed. The past fourteen years — and the past fourteen decades — have shown us that changing technologies can have profound, unforeseen impacts on media — and vice versa.

We understand that profound, unforeseen impacts can yield great rewards — and great risks. While we have been working with biosensors for years, and have been formally analyzing the resulting data since 2014, we acknowledge that human signal data, like most human data, poses risks around privacy, bias, misrepresentation, and many other ethics issues. We’re honored to be supported by Matter, especially at this crucial time for media and the flow of global information. Many of the ethical risks we face in collecting and analyzing human signal data are the same kinds of risks that all new media technologies face, and only at this crucial time are we all beginning to realize this.

As a federally-funded program, we are required to adhere to the same human subjects trainings and review boards required for medical and clinical trials. To that end, we are also grateful to be supported by the National Science Foundation’s SBIR program, Urban-X/SOSV, and the E14 Fund at MIT Media Lab, where this project began way back in 2011 with my early prototype of a brain-reading bike helmet. The “drunken walk of the entrepreneur” never happens alone.

Since building that first prototype, we have been privileged to work with and measure data from many, many kinds of people, including our old friend and new data engineer Tommy Mitchell, hardware designer Yapah Berry, research coordinator Tania van Bergen, and recent partnerships manager Natalia Villegas. And while the insights from human signal data drive our research, it is the humans themselves — the diverse, rambunctious group of people that move through and work in the areas we measure — that drive our work. We believe that technology only works for the people it serves when it is made from and by the diversity of people that it represents. If nothing else, it is this belief with which we hope to make a lasting impact. This is what we fundamentally mean when we say better decisions based on human signals.