If you are building web services in Python that need to scale, this talk is for you! We show how the Python web serving ecosystem integrates with Ray and how Ray makes it easy to scale up web services and to manage their state. We address practical challenges such as how to use Ray tasks and actors effectively, how to write asynchronous code, and how to do type checking and testing. We also cover deployment challenges like monitoring and tracing in a distributed web service. The talk draws on lessons we learned from building Anyscale. We will dive into Anyscale’s architecture and the technology choices we made and how they evolved.
