Fast Geographically Weighted Regression (FastGWR): a scalable algorithm to investigate spatial process heterogeneity in millions of observations

• Published in: International journal of geographical information science : IJGIS Vol. 33; no. 1; pp. 155 - 175
• Main Authors: Li, Ziqi, Fotheringham, A. Stewart, Li, Wenwen, Oshan, Taylor
• Format: Journal Article
• Language: English
• Published: Abingdon Taylor & Francis 02.01.2019; Taylor & Francis LLC
• Subjects: Calibration; Computer programs; Data management; Data points; Geographically Weighted Regression; GWR; Heterogeneity; Massive data points; Message passing; parallel computing; Parallel processing; Parameter estimation; Software; Software packages; Source code; spatial analysis; Spatial heterogeneity; statistical software
• ISSN: 1365-8816; 1362-3087; 1365-8824
• DOI: 10.1080/13658816.2018.1521523

Geographically Weighted Regression (GWR) is a widely used tool for exploring spatial heterogeneity of processes over geographic space. GWR computes location-specific parameter estimates, which makes its calibration process computationally intensive. The maximum number of data points that can be handled by current open-source GWR software is approximately 15,000 observations on a standard desktop. In the era of big data, this places a severe limitation on the use of GWR. To overcome this limitation, we propose a highly scalable, open-source FastGWR implementation based on Python and the Message Passing Interface (MPI) that scales to the order of millions of observations. FastGWR optimizes memory usage along with parallelization to boost performance significantly. To illustrate the performance of FastGWR, a hedonic house price model is calibrated on approximately 1.3 million single-family residential properties from a Zillow dataset for the city of Los Angeles, which is the first effort to apply GWR to a dataset of this size. The results show that FastGWR scales linearly as the number of cores within the High-Performance Computing (HPC) environment increases. It also outperforms currently available open-sourced GWR software packages with drastic speed reductions - up to thousands of times faster - on a standard desktop.