Understanding Urban Perception with Artificial Intelligence

Mapping Urban Perception

Starting in 2010, our team has been working to understand how people perceive the urban environment and the implication of such perception.

Our work began with the creation of a crowdsourcing platform called “Place Pulse,” that asked people to click on an image from a pair in response to an evaluative question, such as “which place looks safer?” or “livelier?“

This first version of Place Pulse (Place Pulse 1.0), collected more than 500,000 preferences which were used to evaluate about 4,000 images for four cities (Boston, New York, Linz, and Salzburg), and to study the spatial correlation and the inequality of the perceived quality of the urban environments (Link to Paper). A few years later, we created an updated version of the platform (Place Pulse 2.0), with images from more than 50 cities, which collected more than 1 million preferences.

From Crowdsource Data to Machine Learning Models

We used the data collected in Place Pulse 1.0 and 2.0 to train several machine learning models.

In 2014, we used the Place Pulse 1.0 dataset, together with a support vector machine model, to create a dataset scoring one million streetscapes (Link to Paper).

Streetscore allowed us to increase the resolution of urban perception maps by several orders of magnitude. With Place Pulse, we could only create evaluative maps with a neighborhood resolution (e.g. a few thousand images for a city as large as New York). This was problematic, since even in a single street segment, the quality of urban space can exhibit large degrees of variation. The introduction of machine learning methods, such as Streetscore, helped us create maps with street level resolution (hundreds of thousands of images). Soon after releasing Streetscore, we switched our support vector machine methodology for a deep learning model (Link to paper).

Example-results-from-the-Place-Pulse-20-dataset-containing-images-ranked-based-on.png

Understanding Urban Perception

Together, the crowdsourced data and the machine learning models, opened the opportunity to study several aspects of urban perception. In a collaboration with people from the Bruno Kessler Foundation (FBK) we studied the correlation, between the perception of safety, and the activity of neighborhoods (as measured by mobile phone records). We found that safer looking neighborhoods were more active, and that the correlation between the perception of safety and urban activity was larger for women and the elderly.

In another paper, published in PNAS, we used our machine learning models to study physical urban change. We found that neighborhoods densely populated by highly educated people experienced comparatively faster physical urban change. We also found evidence that physical urban change spread geographically, since neighborhoods with low values of streetscore, experienced more positive physical urban change when they bordered neighborhoods with higher streetscore values.

In less than a decade, our contributions went from showing that urban perception could be quantified, to showing the impact of urban perception for urban activity and physical urban change. Today, scholars in various academic communities continue to be inspired by these ideas and methods.

Resources

Code and Datasets

Place Pulse 1.0 dataset

Place Pulse 2.0 dataset (Data)

Place Pulse Open Source Repository (github)

Papers (in chronological order)

1.- THE COLLABORATIVE IMAGE OF THE CITY: MAPPING THE INEQUALITY OF URBAN PERCEPTION
PHILIP SALESSES, KATJA SCHECHTNER, AND CÉSAR A. HIDALGO
PLoS ONE 8(7): e68400. DOI: 10.1371/journal.pone.0068400
Published Version (PDF)

2. STREETSCORE--PREDICTING THE PERCEIVED SAFETY OF ONE MILLION STREETSCAPES
NIKHIL NAIK, JADE PHILIPOOM, RAMESH RASKAR, AND CÉSAR A. HIDALGO
CVPR Workshop on Web-scale Vision and Social Media (2014)
Published Version (PDF)

3. CITIES ARE PHYSICAL TOO: USING COMPUTER VISION TO MEASURE THE QUALITY AND IMPACT OF URBAN APPEARANCE
NIKHIL NAIK, RAMESH RASKAR, AND CÉSAR A. HIDALGO
The American Economic Review 106 (5), 128-132 (2016)
Published Version (PDF)

4. DEEP LEARNING THE CITY: QUANTIFYING PERCEPTION AT A GLOBAL SCALE
ABHIMANYU DUBEY, NIKHIL NAIK, DEVI PARAH, RAMESH RASKAR, CÉSAR A. HIDALGO
European Conference on Computer Vision, 196-212 (2016)
Published Version (PDF)

5. ARE SAFER LOOKING NEIGHBORHOODS MORE LIVELY? A MULTIMODAL INVESTIGATION INTO URBAN LIFE
MARCO DE NADAI, RADU LAURENTIU VIERIU, GLORIA ZEN, STEFAN DRAGICEVIC, NIKHIL NAIK, MICHELLE CARAVIELLE, CÉSAR A. HIDALGO, NICU SEBE, BRUNO LEPRI
Proceedings of the 2016 ACM on Multimedia Conference 1127-1135 (2016)
Published Version (PDF)

6.-COMPUTER VISION UNCOVERS PREDICTORS OF PHYSICAL URBAN CHANGE
NIKHIL NAIK, SCOTT KOMINERS, RAMESH RASKAR, EDWARD GLAESER, CESAR A. HIDALGO
Proceedings of the National Academy of Sciences (2017)
Published Version (PDF)