Cost Effective Generic Machine Learning Operation: A Case Study

In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures to bring the pipeline at Cloudera, Horton works platform or cloud premises like AWS, GCP or Azure. This paper gives a very detailed understa...

Full description

Saved in:
Bibliographic Details
Published in2023 International Conference on Data Science and Network Security (ICDSNS) pp. 1 - 6
Main Authors Jain, Samridhi, Kumar, Puneet
Format Conference Proceeding
LanguageEnglish
Published IEEE 28.07.2023
Subjects
Online AccessGet full text
DOI10.1109/ICDSNS58469.2023.10245408

Cover

Abstract In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures to bring the pipeline at Cloudera, Horton works platform or cloud premises like AWS, GCP or Azure. This paper gives a very detailed understanding of operationalization of a typical ML pipelines to adhere all the elements and artifacts without even using any Docker, Kubernetes or even any API generating platforms like Flask or FastAPI. Using the combination of a simple Python/R along with SQL and Shell scripts we can manage the entire workflow at on premises with a very low-cost approach. From some angle this mechanism would not be comparable with the architectures like market ready MLOps platforms like Azure Devops, MLflow, Kubeflow, Apache Airflow, Databricks with Data factory or Sagemaker Studio workflow but from conceptual point of view, suffice almost 90% of the requirements with efficient manner. We have also done a latest review related to MLOps in recent past and also listed out the several research gaps that can be solved in future research.
AbstractList In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures to bring the pipeline at Cloudera, Horton works platform or cloud premises like AWS, GCP or Azure. This paper gives a very detailed understanding of operationalization of a typical ML pipelines to adhere all the elements and artifacts without even using any Docker, Kubernetes or even any API generating platforms like Flask or FastAPI. Using the combination of a simple Python/R along with SQL and Shell scripts we can manage the entire workflow at on premises with a very low-cost approach. From some angle this mechanism would not be comparable with the architectures like market ready MLOps platforms like Azure Devops, MLflow, Kubeflow, Apache Airflow, Databricks with Data factory or Sagemaker Studio workflow but from conceptual point of view, suffice almost 90% of the requirements with efficient manner. We have also done a latest review related to MLOps in recent past and also listed out the several research gaps that can be solved in future research.
Author Kumar, Puneet
Jain, Samridhi
Author_xml – sequence: 1
  givenname: Samridhi
  surname: Jain
  fullname: Jain, Samridhi
  email: Samridhijain100@gmail.com
  organization: Chandigarh University,Department of Computer science & Engineering,Mohali,India
– sequence: 2
  givenname: Puneet
  surname: Kumar
  fullname: Kumar, Puneet
  email: Professor.pkumar@gmail.com
  organization: Chandigarh University,Department of Computer science & Engineering,Mohali,India
BookMark eNo1z71OwzAUQGEjwQClb8BgHiDB1z_UlwVVoS2V0nYIzJVzew2WwKmSgNS3ZwCms33SuRLnucssxC2oEkDh3bp6araN8_YeS620KUFp66zyZ2KKM_TGKaPAobsUj1U3jHIRI9OYvlmuOHOfSG4CvafMsubQ55Tf5O7IfRhTlx_kXFZhYNmMX4fTtbiI4WPg6V8n4nW5eKmei3q3WlfzukgAOBakKCodnSOnLRpPboZ4cNxG7bVHg7rVPlhNlqhtDUNrDVDwBBAg6Ggm4ubXTcy8P_bpM_Sn_f-X-QEGQUdD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ICDSNS58469.2023.10245408
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Xplore
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798350301595
EndPage 6
ExternalDocumentID 10245408
Genre orig-research
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i119t-c0cf02f55c524938c5799d5ebf28289392b28a42c4ccbb3e1b431ca8c11a1a2f3
IEDL.DBID RIE
IngestDate Wed Sep 27 05:40:29 EDT 2023
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i119t-c0cf02f55c524938c5799d5ebf28289392b28a42c4ccbb3e1b431ca8c11a1a2f3
PageCount 6
ParticipantIDs ieee_primary_10245408
PublicationCentury 2000
PublicationDate 2023-July-28
PublicationDateYYYYMMDD 2023-07-28
PublicationDate_xml – month: 07
  year: 2023
  text: 2023-July-28
  day: 28
PublicationDecade 2020
PublicationTitle 2023 International Conference on Data Science and Network Security (ICDSNS)
PublicationTitleAbbrev ICDSNS
PublicationYear 2023
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.8588687
Snippet In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures...
SourceID ieee
SourceType Publisher
StartPage 1
SubjectTerms cloud platform
Costs
Data science
Industries
Low Budget Architecture
Machine learning
ML Engineering
ML pipeline
MLOps
Network security
Organizations
Pipelines
Title Cost Effective Generic Machine Learning Operation: A Case Study
URI https://ieeexplore.ieee.org/document/10245408
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NSwMxEA22B_GkYsVvInjddZNNdhMvIqulCq1CLfRWktlEitCWuj3orzfJbhUFwVsIgSQMmZdJ5r1B6CK1AFJTEgmufQkzxiORahWxjFmhXDyRCc9G7g-y3og9jPm4IasHLowxJiSfmdg3w19-OYeVfypzJ5x6wTjRQq08lzVZaxOdN7qZl_fF7XAw9IjqGSg0jdfjf1ROCcDR3UaD9ZR1vshrvKp0DB-_1Bj_vaYd1Pnm6OGnL_TZRRtmtoeui_lbhWtFYufGcBCVngLuh5RJgxs11Rf8uDC16a_wDS4ckmGfUPjeQaPu3XPRi5oSCdGUEFlFkIBNqOUcuIujUgE8l7LkRtsQSrnLj6ZCMQoMQOvUEO0uDKAEEKKIojbdR-3ZfGYOEAZWCs1FmUAumJaJZtYmzv0wWVqrsvwQdfzuJ4taBWOy3vjRH_3HaMsbwb-DUnGC2tVyZU4dgFf6LBjuE1wHmrY
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PS8MwFA46QT2pOPG3Eby2tmnSpl5EqmPTrQrbYLeRvCYyhG1od9C_3iTtFAXBWwiEJDzyvrzkfd9D6CLSAKkkoceZtCXMKPN4JIVHY6q5MPFEzC0buZfH7SG9H7FRTVZ3XBillEs-U75tur_8YgYL-1RmTjixgnF8Fa0xE1YkFV1rHZ3XypmXney2n_ctploOCon85YgftVMcdLS2UL6ctMoYefEXpfTh45ce479XtY2a3yw9_PSFPztoRU130XU2eytxpUlsHBl2stITwD2XNKlwraf6jB_nqjL-Fb7BmcEybFMK35to2LobZG2vLpLgTcIwLT0IQAdEMwbMRFIRB5akacGU1C6YMtcfSbigBCiAlJEKpbkygOAQhiIUREd7qDGdTdU-wkALLhkvAkg4lWkgqdaBcUA0LbQWcXKAmnb343mlgzFebvzwj_4ztNEe9Lrjbid_OEKb1iD2VZTwY9QoXxfqxMB5KU-dET8Br6ieBw
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2023+International+Conference+on+Data+Science+and+Network+Security+%28ICDSNS%29&rft.atitle=Cost+Effective+Generic+Machine+Learning+Operation%3A+A+Case+Study&rft.au=Jain%2C+Samridhi&rft.au=Kumar%2C+Puneet&rft.date=2023-07-28&rft.pub=IEEE&rft.spage=1&rft.epage=6&rft_id=info:doi/10.1109%2FICDSNS58469.2023.10245408&rft.externalDocID=10245408