Towards a Unified Architecture Powering Scalable Learning Models with IoT Data Streams, Blockchain, and Open Data

• Published in: Information (Basel) Vol. 14; no. 6; p. 345
• Main Authors: Debauche, Olivier, Nkamla Penka, Jean Bertin, Hani, Moad, Guttadauria, Adriano, Ait Abdelouahid, Rachida, Gasmi, Kaouther, Ben Hardouz, Ouafae, Lebeau, Frédéric, Bindelle, Jérôme, Soyeurt, Hélène, Gengler, Nicolas, Manneback, Pierre, Benjelloun, Mohammed, Bertozzi, Carlo
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.06.2023
• Subjects: Agricultural production; Algorithms; Analysis; Application programming interface; Artificial intelligence; Big Data; Blockchain; cloud computing; Computer architecture; Cryptography; Cybersecurity; Data analysis; Data mining; Data processing; Data transmission; Datasets; Deep learning; edge computing; fog computing; Fraud prevention; Internet of Things; Knowledge discovery; Machine learning; Open data; Sensors; Time series; Training; Belgium
• ISSN: 2078-2489; 2078-2489
• DOI: 10.3390/info14060345

The huge amount of data produced by the Internet of Things need to be validated and curated to be prepared for the selection of relevant data in order to prototype models, train them, and serve the model. On the other side, blockchains and open data are also important data sources that need to be integrated into the proposed integrative models. It is difficult to find a sufficiently versatile and agnostic architecture based on the main machine learning frameworks that facilitate model development and allow continuous training to continuously improve them from the data streams. The paper describes the conceptualization, implementation, and testing of a new architecture that proposes a use case agnostic processing chain. The proposed architecture is mainly built around the Apache Submarine, an unified Machine Learning platform that facilitates the training and deployment of algorithms. Here, Internet of Things data are collected and formatted at the edge level. They are then processed and validated at the fog level. On the other hand, open data and blockchain data via Blockchain Access Layer are directly processed at the cloud level. Finally, the data are preprocessed to feed scalable machine learning algorithms.