Ray Summit 2020 has ended
View More Details for Ray Summit & Registration Information.
Please note: All Sessions are in Pacific Daylight Time (PDT), UTC-7

Thursday, October 1 • 11:10am - 11:40am
Introducing Ray Serve: Scalable and Programmable ML Serving Framework - Simon Mo, Anyscale

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.

After data scientists train a machine learning (ML) model, the model needs to be served for interactive scoring or batch predictions. The go-to solution is often to wrap the model inside a Flask microservice. But when is that not enough? In this talk, I will discuss the short-comings of the Flask-only solution and then discuss the more common alternative, the “tensor prediction service” approach used by TFServing, SageMaker, and others. I will then introduce an easy-to-use, scalable ML serving system “Ray Serve” that overcomes the deficiencies of the two approaches. I will highlight the architectural innovations in Ray Serve.

avatar for Simon Mo

Simon Mo

Software Engineer, Anyscale
Simon Mo is a software engineer at Anyscale. Before Anyscale, he was a student at UC Berkeley participating in research at the RISELab. He focuses on studying and building systems for machine learning, in particular, how to make ML model serving systems more efficient, ergonomic... Read More →

Thursday October 1, 2020 11:10am - 11:40am PDT
Virtual 1
  Ray and Its Libraries
  • Slides Included Yes