Cost Effective Generic Machine Learning Operation: A Case Study
In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures to bring the pipeline at Cloudera, Horton works platform or cloud premises like AWS, GCP or Azure. This paper gives a very detailed understa...
Saved in:
Published in | 2023 International Conference on Data Science and Network Security (ICDSNS) pp. 1 - 6 |
---|---|
Main Authors | , |
Format | Conference Proceeding |
Language | English |
Published |
IEEE
28.07.2023
|
Subjects | |
Online Access | Get full text |
DOI | 10.1109/ICDSNS58469.2023.10245408 |
Cover
Abstract | In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures to bring the pipeline at Cloudera, Horton works platform or cloud premises like AWS, GCP or Azure. This paper gives a very detailed understanding of operationalization of a typical ML pipelines to adhere all the elements and artifacts without even using any Docker, Kubernetes or even any API generating platforms like Flask or FastAPI. Using the combination of a simple Python/R along with SQL and Shell scripts we can manage the entire workflow at on premises with a very low-cost approach. From some angle this mechanism would not be comparable with the architectures like market ready MLOps platforms like Azure Devops, MLflow, Kubeflow, Apache Airflow, Databricks with Data factory or Sagemaker Studio workflow but from conceptual point of view, suffice almost 90% of the requirements with efficient manner. We have also done a latest review related to MLOps in recent past and also listed out the several research gaps that can be solved in future research. |
---|---|
AbstractList | In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures to bring the pipeline at Cloudera, Horton works platform or cloud premises like AWS, GCP or Azure. This paper gives a very detailed understanding of operationalization of a typical ML pipelines to adhere all the elements and artifacts without even using any Docker, Kubernetes or even any API generating platforms like Flask or FastAPI. Using the combination of a simple Python/R along with SQL and Shell scripts we can manage the entire workflow at on premises with a very low-cost approach. From some angle this mechanism would not be comparable with the architectures like market ready MLOps platforms like Azure Devops, MLflow, Kubeflow, Apache Airflow, Databricks with Data factory or Sagemaker Studio workflow but from conceptual point of view, suffice almost 90% of the requirements with efficient manner. We have also done a latest review related to MLOps in recent past and also listed out the several research gaps that can be solved in future research. |
Author | Kumar, Puneet Jain, Samridhi |
Author_xml | – sequence: 1 givenname: Samridhi surname: Jain fullname: Jain, Samridhi email: Samridhijain100@gmail.com organization: Chandigarh University,Department of Computer science & Engineering,Mohali,India – sequence: 2 givenname: Puneet surname: Kumar fullname: Kumar, Puneet email: Professor.pkumar@gmail.com organization: Chandigarh University,Department of Computer science & Engineering,Mohali,India |
BookMark | eNo1z71OwzAUQGEjwQClb8BgHiDB1z_UlwVVoS2V0nYIzJVzew2WwKmSgNS3ZwCms33SuRLnucssxC2oEkDh3bp6araN8_YeS620KUFp66zyZ2KKM_TGKaPAobsUj1U3jHIRI9OYvlmuOHOfSG4CvafMsubQ55Tf5O7IfRhTlx_kXFZhYNmMX4fTtbiI4WPg6V8n4nW5eKmei3q3WlfzukgAOBakKCodnSOnLRpPboZ4cNxG7bVHg7rVPlhNlqhtDUNrDVDwBBAg6Ggm4ubXTcy8P_bpM_Sn_f-X-QEGQUdD |
ContentType | Conference Proceeding |
DBID | 6IE 6IL CBEJK RIE RIL |
DOI | 10.1109/ICDSNS58469.2023.10245408 |
DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Xplore IEEE Proceedings Order Plans (POP All) 1998-Present |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
EISBN | 9798350301595 |
EndPage | 6 |
ExternalDocumentID | 10245408 |
Genre | orig-research |
GroupedDBID | 6IE 6IL CBEJK RIE RIL |
ID | FETCH-LOGICAL-i119t-c0cf02f55c524938c5799d5ebf28289392b28a42c4ccbb3e1b431ca8c11a1a2f3 |
IEDL.DBID | RIE |
IngestDate | Wed Sep 27 05:40:29 EDT 2023 |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-i119t-c0cf02f55c524938c5799d5ebf28289392b28a42c4ccbb3e1b431ca8c11a1a2f3 |
PageCount | 6 |
ParticipantIDs | ieee_primary_10245408 |
PublicationCentury | 2000 |
PublicationDate | 2023-July-28 |
PublicationDateYYYYMMDD | 2023-07-28 |
PublicationDate_xml | – month: 07 year: 2023 text: 2023-July-28 day: 28 |
PublicationDecade | 2020 |
PublicationTitle | 2023 International Conference on Data Science and Network Security (ICDSNS) |
PublicationTitleAbbrev | ICDSNS |
PublicationYear | 2023 |
Publisher | IEEE |
Publisher_xml | – name: IEEE |
Score | 1.8588687 |
Snippet | In this research, we have proposed a mechanism to implement a typical Mops pipeline for small scale organization who cannot afford the operational expenditures... |
SourceID | ieee |
SourceType | Publisher |
StartPage | 1 |
SubjectTerms | cloud platform Costs Data science Industries Low Budget Architecture Machine learning ML Engineering ML pipeline MLOps Network security Organizations Pipelines |
Title | Cost Effective Generic Machine Learning Operation: A Case Study |
URI | https://ieeexplore.ieee.org/document/10245408 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NSwMxEA22B_GkYsVvInjddZNNdhMvIqulCq1CLfRWktlEitCWuj3orzfJbhUFwVsIgSQMmZdJ5r1B6CK1AFJTEgmufQkzxiORahWxjFmhXDyRCc9G7g-y3og9jPm4IasHLowxJiSfmdg3w19-OYeVfypzJ5x6wTjRQq08lzVZaxOdN7qZl_fF7XAw9IjqGSg0jdfjf1ROCcDR3UaD9ZR1vshrvKp0DB-_1Bj_vaYd1Pnm6OGnL_TZRRtmtoeui_lbhWtFYufGcBCVngLuh5RJgxs11Rf8uDC16a_wDS4ckmGfUPjeQaPu3XPRi5oSCdGUEFlFkIBNqOUcuIujUgE8l7LkRtsQSrnLj6ZCMQoMQOvUEO0uDKAEEKKIojbdR-3ZfGYOEAZWCs1FmUAumJaJZtYmzv0wWVqrsvwQdfzuJ4taBWOy3vjRH_3HaMsbwb-DUnGC2tVyZU4dgFf6LBjuE1wHmrY |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3PS8MwFA46QT2pOPG3Eby2tmnSpl5EqmPTrQrbYLeRvCYyhG1od9C_3iTtFAXBWwiEJDzyvrzkfd9D6CLSAKkkoceZtCXMKPN4JIVHY6q5MPFEzC0buZfH7SG9H7FRTVZ3XBillEs-U75tur_8YgYL-1RmTjixgnF8Fa0xE1YkFV1rHZ3XypmXney2n_ctploOCon85YgftVMcdLS2UL6ctMoYefEXpfTh45ce479XtY2a3yw9_PSFPztoRU130XU2eytxpUlsHBl2stITwD2XNKlwraf6jB_nqjL-Fb7BmcEybFMK35to2LobZG2vLpLgTcIwLT0IQAdEMwbMRFIRB5akacGU1C6YMtcfSbigBCiAlJEKpbkygOAQhiIUREd7qDGdTdU-wkALLhkvAkg4lWkgqdaBcUA0LbQWcXKAmnb343mlgzFebvzwj_4ztNEe9Lrjbid_OEKb1iD2VZTwY9QoXxfqxMB5KU-dET8Br6ieBw |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2023+International+Conference+on+Data+Science+and+Network+Security+%28ICDSNS%29&rft.atitle=Cost+Effective+Generic+Machine+Learning+Operation%3A+A+Case+Study&rft.au=Jain%2C+Samridhi&rft.au=Kumar%2C+Puneet&rft.date=2023-07-28&rft.pub=IEEE&rft.spage=1&rft.epage=6&rft_id=info:doi/10.1109%2FICDSNS58469.2023.10245408&rft.externalDocID=10245408 |