[Music] hi [Music] in the next few minutes I will show you how to implement the Next Generation and open machine learning operations with practicus AI successful AI systems have several operational requirements and different teams that build and consume the AI models will have different domain expertise tools and analytics requirements it is also very common to utilize multiple data lakes with different data sources posted on multiple clouds and on-prem and potentially in different geographies in order to provide a unified user experience and Federated governance modern data mesh architectures are becoming popular practice AI helps you with all these requirements by only using open source Cloud native technology also preventing vendor lock-in let's start with deploying AI models and different personas can choose different methods behind the scenes everything leads to an open well-documented and repeatable infrastructure as code we will take a look at the user experience first and then discuss the technical implementation of how this all works I will load some data from my data Lake and then build a model to predict customer churn using automl when done I will deploy my model as an API after choosing a model location the model will be available for others to consume right away another user can click the predict button choose a model and version and make predictions instantly in addition to deploying and consuming models using the app you can also use other methods for example after the automl is completed you can export its code to Jupiter make changes or start from scratch and deploy your custom model as a new version now let's take a look at how this all works practicas AI uses model prefixes names and versions as logical elements that you can control independent of each other these are then attached to physical kubernetes deployments offering you a dynamic service mesh for traditional devops when a new version is required both versions can be Canary deployed until the old version is retired this approach does not always work for ML Ops it is very common to have multiple users creating model versions with different capacity and software package requirements you might want to use these versions simultaneously for a b testing with different traffic weight percentage requirements and for a prolonged period of time when you have hundreds of models and model versions a day traditional devops techniques will not be sufficient let's dive a little deeper into the mechanics and for this demo we will be using the web admin UI I will click on model hosting and then add a new physical deployment you can choose any amount of CPU and RAM and then optionally activate Auto scaling too technical users can prefer to use their custom containers or simply ask to install components each time a deployment starts we can then Grant access to groups or individual users you will see that a new kubernetes deployment will be ready in a few seconds now let's create a model prefix which is essentially a group of models a prefix will also Define the URL for hand-coded models you can Define production settings such as connection strings and also secrets we can then Define who can deploy consume and manage models the last logical groups are models and versions you can Define any number of model versions the physical deployment that each model version runs optional stage and optional traffic weight up to 100 active model versions after every change the open API documentation is instantly refreshed any developer internal or external to your organization can simply view the open API documentation check the code samples search for models made available to them and then make predictions with any programming language batching and compression support is automatically built in so you can easily make hundreds of millions of predictions let's talk about a more advanced scenario where practicus AI is deployed in multiple locations all using different public and private clouds and accessing different data sources business users can easily explore the separate systems their data sources and models using a single interface and developers can consume the models using fine-grained access control tokens you can also enable global apis and developers can use a single URL to automatically use the closest cloud if there is a failure the traffic will automatically route to the next Cloud vendor and then automatically route back to the original cloud when it is back online offering you the highest availability for Mission critical AI models since our mlops is built only using open source the entire system will continue operating even if you completely uninstall practice AI you can even add new models or model versions manually using practicus AI design as a blueprint in addition to building and hosting New models using open source you can also modernize your legacy or proprietary systems by simply wrapping them with our modern ml apps as a result you would get many of the practicus AI ml Ops benefits without making changes to your existing code thank you for watching and please do not hesitate to contact us if you have any questions