ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

An Exploratory Study on Machine Learning Model Stores

•
•
Z
Zhen Ming (Jack) Jiang

The document discusses machine learning (ML) model stores and some unique software engineering challenges for building ML applications. It identifies several key information elements that ML model stores typically provide, including the ML algorithm, framework, training set, license, size, usage statistics, and user feedback. The document also examines differences between how ML engineers and end users package and integrate ML models, and considerations for representing model quality, pricing schemes, and capacity planning for ML applications hosted in these stores.

1 of 42
Downloaded 13 times
An Exploratory Study on the Machine Learning Model Stores Zhen Ming (Jack) Jiang
Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning
Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning .. Bug Prediction Test Case Prioritization Automated Bug Triage .
Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning .. Bug Prediction Test Case Prioritization Automated Bug Triage . Test ML Applications
Machine Learning (ML) & Software Engineering (SE) Software Engineering Machine Learning .. Bug Prediction Test Case Prioritization Automated Bug Triage . Test ML Applications Anything else???
An Exploratory Study on Machine Learning Model Stores
An Exploratory Study on Machine Learning Model Stores
An Exploratory Study on Machine Learning Model Stores
An Exploratory Study on Machine Learning Model Stores
ML Model Stores
ML Model Stores January 2018 June 2018 November 2018 November 2018 April 2019 (beta)
ML Model Stores January 2018 June 2018 November 2018 November 2018 April 2019 (beta) What are the unique SE practices/challenges for building ML applications?
Structure Contents
Structure
What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018 November 2018 April 2019 (beta)
What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018 April 2019 (beta)
What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018
What kind of information elements do ML model stores provide? January 2018 June 2018 November 2018 October 2008 July 2008
Identifying Information Elements
Identifying Information Elements Owner
Identifying Information Elements ML Algorithm Owner
Identifying Information Elements Framework ML Algorithm Owner
Identifying Information Elements Framework Training set ML Algorithm Owner
Identifying Information Elements LicenseFramework Training set ML Algorithm Size Usage Statistics User feedback User Manual Owner
Identifying Information Elements LicenseFramework Training set ML Algorithm Size Usage Statistics User feedback User Manual Owner
Identifying Information Elements LicenseFramework Training set ML Algorithm Size Usage Statistics User feedback User Manual Owner
vs.ML Engineers End Users
An Exploratory Study on Machine Learning Model Stores
To package an ML application for release - Source code -Trained ML model(s) -Training set info Integrating such a model … - API calls, re-train on new data?
To package an ML application for release - Source code -Trained ML model(s) -Training set info Integrating such a model … - API calls, re-train on new data?
An Exploratory Study on Machine Learning Model Stores
Quality representation - Statistics vs. QoS measures (reliability) - Requirements for different ML applications
An Exploratory Study on Machine Learning Model Stores
[Pricing] Subscription-based - Pricing scheme vs. actual usage context Capacity planning for ML applications
Structure Contents
How unique are the ML models provided by each model store? Very few cross-store ML applications - Vendor lock-in - Need for cross-platform frameworks
Same ML Models Different Usage Context
Same ML Models Different Usage Context
Minor Releases Different Product Pages
Minor Releases Different Product Pages
Minor Releases Different Product Pages A new version for an ML application is: - same ML model re-trained on another dataset? or - same dataset with different ML implementations?
An Exploratory Study on Machine Learning Model Stores

More Related Content

An Exploratory Study on Machine Learning Model Stores