Ray Summit 2020 has ended
View More Details for Ray Summit & Registration Information.
Please note: All Sessions are in Pacific Daylight Time (PDT), UTC-7

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Reinforcement Learning [clear filter]
Thursday, October 1

9:00am PDT

AlphaDow: Reinforcement Learning for Industrial Production Scheduling - Adam Kelloway, Dow Chemical
Adam has deployed reinforcement learning trained agents that optimize a production planning problem at Dow, Inc and deliver measurable business impact. Inspired by the successes of AlphaGO, this ambitious project is named AlphaDow. The production scheduling problem at Dow is non-trivial. Dow’s continuous manufacturing plants transition through a product sequence which must minimize the costs of transitions and maximize product availability for customers all whilst minimizing inventory on hand. Human schedulers use experience and heuristics to select the best sequence today. The trained RL agents augment this decision-making process. Adam has used Ray extensively to distribute his training among Azure based compute clusters. This significantly decreased the time needed to train an RL agent capable of producing optimal production schedules. Adam will present several of the challenges faced when training and deploying RL agents for industrial production scheduling and how Ray has helped him to overcome those challenges by facilitating the scale required to solve these complex problems and achieve real-world business impact.

avatar for Adam Kelloway

Adam Kelloway

Innovation Manager, Dow Chemical
Adam Kelloway works as an Innovation Manager for Dow's Digital Fulfillment Center. (DFC) The DFC is leading the development and deployment of AI/ML technologies within Dow's integrated supply chain. Adam has previously worked on optimized production scheduling using Mixed Integer... Read More →

Thursday October 1, 2020 9:00am - 9:30am PDT
Virtual 2
  Reinforcement Learning
  • Slides Included Yes

9:35am PDT

Project Bonsai - How Microsoft is Using Ray and RLlib Inside a Machine Teaching Service for Autonomous Systems - Edilmo Palencia, Microsoft
Machine Learning and, in special, Reinforcement Learning could be hard. But we believe that is possible to make it accessible to subject matter experts in any industry, regardless of knowledge computer science or Machine Learning.

At Microsoft Project Bonsai we are working in Machine Teaching, a new complementary approach to Machine Learning that could be used by those without AI expertise, to build AI agents that powers intelligent control systems through a unique combination of advance machine learning techniques, simulation, and deployment capabilities.

In this presentation we are going to talk about how we use RLlib and Ray to achieve our vision. And we will perform a quick demo of the platform using a special simulator created specially for learning about Bonsai and Machine Teaching.

avatar for Edilmo Palencia

Edilmo Palencia

Principal AI Engineer, Microsoft

Thursday October 1, 2020 9:35am - 10:05am PDT
Virtual 2
  Reinforcement Learning
  • Slides Included Yes

10:10am PDT

Utilizing Reinforcement Learning Techniques for Supply Chain with Amazon SageMaker - Anna Luo, AWS
Reinforcement learning jobs are computationally expensive requiring multiple CPU and GPU instances. Researchers and practitioners need to manually set up the instances and manage utilization, especially in production. This process can be time consuming and adds additional operational overhead. Amazon SageMaker RL allows customers to focus on their RL research without the need to manage servers. It provides pre-built containers that supports RLlib with both TensorFlow and PyTorch framework. We will present how you can bring in your RL problem and use Amazon SageMaker RL to perform the training. You can easily manage and reproduce your experiments with either single or multiple instances. For production use, we will showcase how you can deploy the trained model with a single click and monitor the endpoint status.

avatar for Anna Luo

Anna Luo

Applied Scientist, Amazon Web Services
Anna Luo is an Applied Scientist in the AWS. She works on utilizing reinforcement learning techniques for different domains including supply chain and recommender system, in which Ray is extensively used. She received her Ph.D. in Statistics from University of California, Santa Barbara... Read More →

Thursday October 1, 2020 10:10am - 10:40am PDT
Virtual 2

11:10am PDT

Connecting Reinforcement Learning to Simulation Software - Max Pumperla, Pathmind
Simulation software like AnyLogic is frequently used across many industries to model complex workflows of real-world applications. You can use these models to solve actual problems, and many of the questions that customers have can be framed as reinforcement learning tasks. However, by design, most simulation software providers do not have an integration with tools like Ray RLlib to apply reinforcement learning to their models directly.

We show how Pathmind leverages the best of both worlds by connecting existing simulation models to modern reinforcement learning tools. In particular, we discuss the special challenges we face and how we use Ray to address them by walking you through a concrete example.

avatar for Max Pumperla

Max Pumperla

Deep Learning Engineer, Pathmind
Max is a deep learning engineer and prolific open source contributor. He’s maintainer of Hyperopt, DL4J core developer, Keras contributor and author of several Python libraries. He’s the author of “Deep Learning and the Game of Go” and Coursera instructor for “Applied AI... Read More →

Thursday October 1, 2020 11:10am - 11:40am PDT
Virtual 2
  Reinforcement Learning
  • Slides Included Yes

12:20pm PDT

Advanced Ray RLlib: New Features and New Possibilities - Sven Mika, Anyscale
Deep Reinforcement Learning (DRL) has become the technology of choice when trying to autonomously learn optimal policies for agents interacting with each other in arbitrarily complex environments. RLlib - Ray's open source RL library - addresses the problems of finding well performing algorithms for a given environment, configuring and customizing them, as well as running these algos in large-scale, distributed workloads to yield good decision making policies. This talk focuses on features and algorithms recently added to RLlib and how you can use these to solve novel environment- and problem types. Topics include in particular: Model-based RL, a new exploration API for RLlib, PyTorch support, curiosity and intrinsic motivation, and contextual bandits.

avatar for Sven Mika

Sven Mika

Machine Learning Engineer, Anyscale
Sven is a machine learning engineer at Anyscale Inc. and lead developer of Ray RLlib, one of the most popular reinforcement learning open-source libraries. He is responsible for implementing recently published, promising algorithms, designing user-friendly APIs, asserting high levels... Read More →

Thursday October 1, 2020 12:20pm - 12:50pm PDT
Virtual 1

12:20pm PDT

Information Gain Regulation in Reinforcement Learning with the Digital Twins' Level of Realism In RlLib - Géza Szabó, Ericsson Research & József Pető, Budapest University of Technology and Economics
Digital Twin (DT) is widely used in various industrial sectors to optimize the operations and maintenance of physical assets, system and manufacturing processes. Our goal is to introduce an architecture in which the radio access control happens automatically to minimize the utilized radio resources while still maximizing the production KPIs of the robot cell. To achieve this, we apply Reinforcement Learning (RL) in a simulated environment to explore the environment fast, while the DT ensures that the learned policy can be applied on the real world environment as well. We show that the application of Ultra Reliable Low Latency Communication (URLLC) connection can be reduced to approx. 30% of the total radio time while achieving real-world accurate robot control. We discuss the implementation of the concept within RLlib.

avatar for Géza Szabó

Géza Szabó

Senior Researcher, Ericsson Research
Géza Szabó joined Ericsson Research as an undergraduate student in 2005. He wrote his MSc thesis on Ericsson topic in the Budapest University of Technology and Economics, about comparing various application traffic classification methods. He pursued further this topic in his PhD... Read More →
avatar for József Pető

József Pető

PhD Student, Budapest University of Technology and Economics
József Pető received his MSc. degree in Computer Engineering from the Budapest University of Technology and Economics in 2018.He is currently working toward his Ph.D. degree. His current areas of interest and research include cloud robotics, digital twin, robot simulation, machine... Read More →

Thursday October 1, 2020 12:20pm - 12:50pm PDT
Virtual 2

5:25pm PDT

Distributed Reinforcement Learning for Robotic Assembly - Rodger Luo, Autodesk Research
Robotic systems for automated assembly have been widely used in manufacturing, where the environment can be carefully and precisely controlled, but they are still in infancy in architectural construction. A main reason is that current robotic systems are not adaptive to the diversity of the real world, especially in unstructured settings. RL-based robotic systems are a promising direction given their adaptability to uncertainties.

This talk will introduce a group of projects from Autodesk Research to improve RL-based robotic systems for assembly tasks. We focus on robotic assembly tasks that involve contact forces, because such tasks are widespread in industrial applications and yet challenging for robots to do. All the algorithm developments are built on top of Ray’s framework.

avatar for Rodger Luo

Rodger Luo

Principal AI Research Scientist, Autodesk Research
Rodger (Jieliang) Luo is a Principal AI Research Scientist at the Autodesk AI Lab in San Francisco. He received his Ph.D. from UCSB in 2020, where he explored the intersection of machine learning, robotics, and creativity. His current research focus is understanding how to learn complex... Read More →

Thursday October 1, 2020 5:25pm - 5:55pm PDT
Virtual 3
  Reinforcement Learning
  • Slides Included Yes
  • Timezone
  • Filter By Date Ray Summit 2020 Sep 29 -Oct 1, 2020
  • Filter By Venue Virtual
  • Filter By Type
  • Anyscale Academy
  • BoF
  • Break
  • Case Studies
  • Keynote Sessions
  • Natural Language Processing
  • Ray and Its Libraries
  • Ray in the Enterprise
  • Reinforcement Learning
  • Research Meets Industry
  • Sponsored Office Hours
  • Slides Included