A novel systems solution for accurate colorimetric measurement through smartphone-based augmented reality

• Published in: PloS one Vol. 18; no. 6; p. e0287099
• Main Authors: Zhang, Guixiang, Song, Shuang, Panescu, Jenny, Shapiro, Nicholas, Dannemiller, Karen C., Qin, Rongjun
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 15.06.2023; Public Library of Science (PLoS)
• Subjects: Algorithms; Analysis; Augmented Reality; Biology and Life Sciences; Calibration; Cameras; Color; Color matching; Colorimetry; Computer and Information Sciences; Data processing; Digital imaging; Engineering and Technology; Evaluation; Experiments; Food safety; Image processing; Lighting; Measurement; Medical imaging; Medicine and Health Sciences; Physical Sciences; Reading; Reflectance; Regression models; Research and Analysis Methods; Sensors; Smart phones; Smartphones; Technology application; United States
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0287099

Quantifying the colors of objects is useful in a wide range of applications, including medical diagnosis, agricultural monitoring, and food safety. Accurate colorimetric measurement of objects is a laborious process normally performed through a color matching test in the laboratory. A promising alternative is to use digital images for colorimetric measurement, due to their portability and ease of use. However, image-based measurements suffer from errors caused by the non-linear image formation process and unpredictable environmental lighting. Solutions to this problem often perform relative color correction among multiple images through discrete color reference boards, which may yield biased results due to the lack of continuous observation. In this paper, we propose a smartphone-based solution, that couples a designated color reference board with a novel color correction algorithm, to achieve accurate and absolute color measurements. Our color reference board contains multiple color stripes with continuous color sampling at the sides. A novel correction algorithm is proposed to utilize a first-order spatial varying regression model to perform the color correction, which leverages both the absolute color magnitude and scale to maximize the correction accuracy. The proposed algorithm is implemented as a “human-in-the-loop” smartphone application, where users are guided by an augmented reality scheme with a marker tracking module to take images at an angle that minimizes the impact of non-Lambertian reflectance. Our experimental results show that our colorimetric measurement is device independent and can reduce up to 90% color variance for images collected under different lighting conditions. In the application of reading pH values from test papers, we show that our system performs 200% better than human reading. The designed color reference board, the correction algorithm, and our augmented reality guiding approach form an integrated system as a novel solution to measure color with increased accuracy. This technique has the flexibility to improve color reading performance in systems beyond existing applications, evidenced by both qualitative and quantitative experiments on example applications such as pH-test reading.