Loading…
Ray Summit 2020 has ended
View More Details for Ray Summit & Registration Information.
Please note: All Sessions are in Pacific Daylight Time (PDT), UTC-7

Back To Schedule
Thursday, October 1 • 11:45am - 12:15pm
Adopting Ray @ LinkedIn - Jonathan Hung & Nitin Pasumarthy, LinkedIn

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.


In this talk, we will present our effort to run Ray on YARN, and the integration of Ray on LinkedIn’s open-sourced offline infrastructure: Azkaban (workflow scheduling service) and TonY (Tensorflow on Yarn). We will provide a demo of running a Ray job end-to-end, discuss the architectural decisions and talk about our cooperation with the Ray team on this effort.
In the later half of the talk, we will share how we used Ray Tune on Kubernetes in a real world use case. Tune helped us identify promising model configurations, using state of the art Bayesian optimization algorithms like TPE & PBT, with minimal supervision. The data pipeline is also optimized using Tune to extract the maximum throughput and we were able to train 2x faster by reducing the GPU idle times.

Speakers
JH

Jonathan Hung

Senior Software Engineer, LinkedIn
Jonathan Hung is a senior software engineer on the Hadoop development team at LinkedIn.



Thursday October 1, 2020 11:45am - 12:15pm PDT
Virtual 2
  Ray in the Enterprise
  • Slides Included Yes