Ray Summit 2020 has ended
View More Details for Ray Summit & Registration Information.
Please note: All Sessions are in Pacific Daylight Time (PDT), UTC-7

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Research Meets Industry [clear filter]
Wednesday, September 30

2:00pm PDT

Hyperparameter Tuning and Visualization in Deep Learning - Lukas Biewald, Weights & Biases
Deep learning models are incredibly powerful but often tricky to adapt to new use cases. Whether you’re finetuning a pretrained net on new data, trying to build an intuition for a complex model, or throwing a variety of architectures at a unique problem, hyperparameter exploration can help. We will share high-level approaches and useful visualizations for hyperparameter search, grounded in concrete examples from semantic segmentation, image classification, language understanding, and other domains. Though we will focus on Weights & Biases Sweeps as a comprehensive tool for this task, these practices are framework-agnostic, and we hope they can accelerate your progress regardless of your dev setup.

avatar for Lukas Biewald

Lukas Biewald

Founder, CEO, Weights & Biases

Wednesday September 30, 2020 2:00pm - 2:30pm PDT
Virtual 3

2:35pm PDT

EfficientBERT: Trading off Model Size and Performance - Meghana Ravikumar, SigOpt
With the publication of BERT, transfer learning was suddenly accessible for NLP, unlocking a plethora of model zoos and boosting performances for domain specific problems.  Although BERT has accelerated many modeling efforts, its size is limiting. In this talk, we will explore how to reduce the size of BERT while retaining its capacity in the context of English Question Answering tasks. We’ll show how scalable hyperparameter optimization can help you tackle difficult modeling problems, draw insights, and make informed decisions.

Our approach encompasses fine-tuning, distillation, architecture search, and hyperparameter optimization at scale. First, we fine-tune BERT on SQUAD 2.0 (our teacher model) and use distillation to compress fine-tuned BERT to a smaller model (our student model). Then, combining SigOpt and Ray, we use multimetric Bayesian optimization at scale to find the optimal architecture for the student model. Finally, we explore the trade-offs of our hyperparameter decisions to draw insights for our student model’s architecture.

avatar for Meghana Ravikumar

Meghana Ravikumar

Machine Learning Engineer, SigOpt
Meghana has worked with machine learning in academia and in industry, and is happiest working on natural language processing. Prior to SigOpt, she worked in biotech, employing NLP to mine and classify biomedical literature. When she’s not reading papers, developing models/tools... Read More →

Wednesday September 30, 2020 2:35pm - 3:05pm PDT
Virtual 3
Thursday, October 1

9:00am PDT

State Management in Distributed Online Learning with Ray - Edmon Begoli, Oak Ridge National Laboratory
Distributed online learning systems are a machine learning systems that learn in a real-time, over a continuously arriving data and in a distributed manner. The challenges of distributed online learning are many, and all are non trivial. In this talk, we'll focus on the challenge of consistent and scalable maintenance of the state of learning in a fully distributed manner, with a special emphasis on how we used Ray actors to facilitate propagation of learning parameters, conflict resolution, and transactional consistency of updates. We will discuss our technical approaches and challenges in the context of our work on real-time suicide prevention predictive algorithms, fraud detection, and infectious disease surveillance.

avatar for Edmon Begoli

Edmon Begoli

Director - Scalable Protected Data Facilities (SPDF), National Center for Computational Sciences (NCCS), Oak Ridge National Laboratory
Edmon Begoli, PhD is a director of ORNL's organization for research on protected data (SPDF), where he is responsible for research and design of large-scale systems for resilient and reliable computing on protected data. He also serves as the Principal Investigator (PI) for the joint... Read More →

Thursday October 1, 2020 9:00am - 9:30am PDT
Virtual 3

9:35am PDT

Cloudstate—Towards Stateful Serverless - Jonas Bonér & James Roper, Lightbend
The Serverless experience is revolutionary and will grow to dominate the future of Cloud. Function-as-a-Service (FaaS) however—with its ephemeral, stateless, and short-lived functions—is only the first step. FaaS is great for processing-intensive, parallelizable workloads, moving data from A to B providing enrichment and transformation along the way. But it is quite limited and constrained in what use-cases it addresses well, which makes it very hard/inefficient to implement general-purpose application development and distributed systems protocols.

What’s needed is a next-generation Serverless platform and programming model for general-purpose application development in the new world of real-time data and event-driven systems. What is missing is ways to manage distributed state in a scalable and available fashion, support for long-lived virtual stateful services, ways to physically co-locate data and processing, and options for choosing the right data consistency model for the job.

This talk will discuss the challenges, requirements, and introduce you to our proposed solution: Cloudstate—an Open Source project building the next generation Stateful Serverless and leveraging state models such as Event Sourcing, CQRS, and CRDTs, running on Akka, gRPC, Kubernetes, and GraalVM, in a polyglot fashion with support for Go, JavaScript, Java, Swift, Scala, Python, Kotlin, and more.

avatar for James Roper

James Roper

Cloud Architect, Lightbend
James is a long time open source contributor and Reactive systems expert. He is the creator of Cloudstate, the framework that brings distributed state management to the serverless world. He also created the Lagom Reactive microservices framework and is a core contributor to Play... Read More →
avatar for Jonas Bonér

Jonas Bonér

CTO, Lightbend
Jonas Bonér is founder and CTO of Lightbend, creator of the Akka project, initiator and co-author of the Reactive Manifesto, Chair of the Reactive Foundation, and a Java Champion. Learn more at: http://jonasboner.com... Read More →

Thursday October 1, 2020 9:35am - 10:05am PDT
Virtual 4
  • Timezone
  • Filter By Date Ray Summit 2020 Sep 29 -Oct 1, 2020
  • Filter By Venue Virtual
  • Filter By Type
  • Anyscale Academy
  • BoF
  • Break
  • Case Studies
  • Keynote Sessions
  • Natural Language Processing
  • Ray and Its Libraries
  • Ray in the Enterprise
  • Reinforcement Learning
  • Research Meets Industry
  • Sponsored Office Hours
  • Slides Included