AI-Driven Personalized Recommendations in SAP

Hey, good morning and good afternoon, everyone that's joining now live. My name is Menina Chow. I am part of the SAP Community Team and your host today for this SAP Community Call. We'll be talking about elevating user experiences with AI-powered personalized recommendation services. And with me is Benjamin Tan, who is a full-stack machine learning developer, and Stephen Fu, who is product owner. both working in SAP artificial intelligence. Now before we start real quick, you already saw it, some of you already used it, we have a live chat. You are very welcome to use it, chat with us, and of course already address your questions, raise your questions, and then as always in the community calls we will have time at the end to address them. We also have an additional Q&A support that's also working with the chat. So maybe you noticed the one or other expert that's joining. But yeah, have fun and enjoy. The floor is yours, Benjamin and Stephen. Thank you. So I will start first. So my name is Stephen. So I joined SAP eight years ago. So I have been starting working with SAP AI for the past five years. So working on this topic about three years and more. So yeah, we're happy to be here to... share our findings and share our new offerings, personalized recommendations with you. So let's spend a couple moments on the legal disclaimer and yeah, let's maybe we can start. So before we can jump start to talk about personalized recommendations, so we like to briefly touch on the background and the offering platform. So Intelligent Enterprise uses innovative technologies such as artificial intelligence, machine learning and robotic automation to turn insights into action in across business in real time. Artificial intelligence is no longer the future for business is here and now so this fact has been validated by many statistics and many proved by the reports which we have been received daily so this shows us how urgently that's many experts on business are in need for artificial intelligence and the impact it could make. So SAP offers a wealth of business services that are based on SAP business technology platform. So this service combines our best practice from more than 30 industries with rapid time to value and flexibility to meet your business needs. So by combining SAP AI business services, with innovative AI technology from SAP, you can kickstart your AI projects. So this is a high-level overview of SAP BDP. So put our service in perspective, we are located within the AI business services suite, which is part of the BDP, alongside with other services such as service ticket intelligence and document information extraction. So we have many internal stakeholders which have successfully integrated our service such as SAP Commerce Cloud, Intelligent Selling Services, and our service is not only for internal stakeholders but also publicly available since May, while CPA and Pay-as-you-go with free tier options. So now let's talk about recommendations. So our objective is to provide recommendations or user from a long list of potential items in a given context. So there are many challenges along the way in these steps. So for the users, you have costar users, which have never seen before. You have users who have shifting habits. Even if you have seen them before, they might not behave in the same way anymore. So for the items, you have new product launch, costar items with no history. You have to adapt latest trend, seasonality. you have different marketing and promotion strategies, and you have to use recent instructions to get users intent in order to generate relevant and personalized reports. So you have to solve all these problems to deliver a good recommendation in a very short time, and then you have to keep it up to date most of the time, which is a very complicated problem. So we have solved the problem to you while this UNIC introduced the personalized recommendation service. So we have worked with internal and external stakeholders to understand the problem, to apply the state-of-the-art machine learning approaches, master the ability to extract the insight from the customer data, to build the machine learning models to deliver highly personalized recommendations. So we're able to address all the problems which I mentioned in the previous slides, the costar users, shifting habits, costar items. is able to adopt, adapt latest trends, promotion strategies, use the given context in real time. So as a result, the user engagement is enhanced, the item discovery is improved, and our machine learning model is not a black box. The model owner can still influence the recommendation output in many ways, such as filtering, boosting which we will show later to retrain business control to create relevance and to meet KPIs. So with a model trained from the training data which I shared in the previous slides that's customer training data, the service can be used in many diverse scenarios across business in all means such as next click, next share, affinity recommendation and smart search. So a next click scenario predicts the next item that the user is likely to click or view or purchase based on their historical clickstream. So for instance in an e-commerce setting after customer has viewed several products the model understands what the product the user is trying to find and recommend them other products within the same functionality. In contrast the next scenario predicts the item the user is most likely to purchase based on the user's item which is recently purchased. So for instance, given user has recently bought a laptop and the model predicts the item and the user is likely to purchase next such as a mouse, keyboard, microphone, headset or other electronic accessories are related to the item which the user just bought. And another scenario is affinity recommendation, which can be used in multimedia platform and recommend user content based on their predicted attributes. So for instance, if the user has been viewing football related contents, we are able to detect the user's interest and recommend similar or relevant items based on the content. Last but not least, smart search is also extended from the model which we built from the customer data. So we use natural language processing query to suggest an appropriate request with the help of the user's past behavior. This means that the search is more than just a simple query string match. So for instance, if the user recently viewed stalker products and the recommendation which they will receive if they search for the keyword ball as shown below the screen, so we will show more stalker balls instead of other balls. in the context. So recommendation topic is the domain agnostic. So it only depends on the user item interaction such as product purchase, cost learning, or job token. So in this case, it can be applied in multiple domains which can be seen like in retail which is adopted by SAP Commerce Cloud, human resources for learning such as SS. SAP SuccessFactors Learning, Career Path Explorer, and also SAP Limons Learning. So this is all the embedded solutions which we have delivered together with other line of business within SAP. And it can also be used for content delivery such as SAP Community or any live streaming platforms which can be also adapted. So it's also available for public to find the next readable. domains which can be applied. Yeah, so stakeholders and customers always ask, what is the difference between your service and your competitors? So the key features of this service can be seen here. So I will quickly run through the key features and later our AI developer Benjamin Tan will show you in detail with a demo UI. So firstly, we can provide sophisticated personalized recommendations. quick and relevant alternative recommendations, and user session scopes, affinity prediction, and then we can also explain to perform smart search. So for all these recommendations and search results, we can explain our results. the ability to identify the confidence score for each of the predictions and the detailed contribution of each of the attributes from the relevant items or the user metadata. Next, for customizable strategy, so for new customers who have not enough clickstream, we can train models based on the item catalog only to kick start with our customers. So for the new items, new users, or coastal platforms, we can handle these scenarios very efficiently. We can also upload feature importance to help our customer to continuously improve training data quality and training efficiency because we know the garbage in, garbage out. So the more training data, the better of the training data, the better model performs. So we support customer attribute boosting, affinity boosting for different promotion or marketing strategies. We support batch and inference modes, real-time inference modes, and then for real-time inference we can update the model for the data changes in between two trainings so that customer doesn't have to train very frequently so that the model can be still up to date almost at time. And the whole solution is easy to adopt and fully manage service with scalable architecture in SAP Cloud Platform. So there's a single set of API which manage trigger training, monitor training status, model serving status, making inference calls, and further recommendations. So here is the overview of this service from AI Business Services, personalized recommendation. So if there's any burning questions, feel free to ask. We'll be very happy to answer before we show you a demo. Should we have a break for some questions? I think we can move forward unless there's anyone who already has some questions in the chat. If not, they're probably excited to see the demo and then we can go to Q&A. Yeah sure, I better stop sharing now. Okay, let me share my screen. Okay, can everyone see my screen? Yes. Okay, great. Hi, thank you, Stephen, for the introduction. And hi, everyone. I'm Benjamin, and I'm an AI developer in the same team. I've been in SAP for two years. And for today's demo, I had a hand in integrating our service into the demo UI. So I can attest that our service is quite easy to integrate. But yeah. So thank you very much for attending today's session. And today I'll be giving two short demos on our service. And through these demos, we hope to showcase some of the interesting capabilities of personalized recommendations, as well as how the recommendations can actually be used to elevate the user experience. So for the first demo, as you can see over here, this demo is an instance developed by one of our internal stakeholders, Intelligent Selling Services, with our recommendations engine embedded into. this site to provide recommendations to our customers. So we will be emulating a customer's perspective of browsing an e-commerce site specialized in camera sales, as you can see here, but also highlighting the various personalized recommendation features that are used to improve the user experience as we go along. Yeah, okay. So I'm a new user that's visiting this e-commerce site for the first time, and I'm familiar with the site and its products. So... The first thing I see here is actually two carousels showing trending products. So before we begin, I'd like to point out that as I'm a new user, I do not have any context for recommendations. So where context refers to actually like my clickstream history, like Steven has mentioned previously. For example, click item clicks, item views, add to cart events and so on. So this is what we refer to as a co-start scenario. And in this co-start scenario, personalized recommendation service will actually leverage on the aggregated training data, contextual training data, to actually recommend the most popular or trending items to the new customers you can see over here. So this is a great way to get our customers started. So now I'm interested in buying a camera, for example. So this camera looks nice. This DSLR. So let me click on it. Yeah. OK, great. So as you can see on this page, we can only see the camera item details. But we can actually see this carousel over here. So this carousel is actually powered by personalized recommendations, similar item recommendations, where we try to predict, sorry, we try to recommend items that are most similar to the items that are currently being viewed. So this is very useful in scenarios that we like to retain the customer's attention during specific events. So for example, if the current item is out of stock, for example, we will recommend items similar to this item in hopes of retaining his purchasing intent and thereby increasing conversion rates. So let me click. So I'm looking at the cameras now and I see this camera over here includes a lens in the package. So it's interesting to me. Let me add this in. Yeah, sorry, let me add this in. Okay, let me view this item, sorry. Okay, so I'm satisfied with this item, so let me add this to the cart. Yeah, okay, great. Okay, so as we look below, we can actually see that the site also tracks my item views. So you can see your recently viewed products in this carousel here. So this shows that we can actually use this contextual data to provide richer recommendations subsequently. Yeah, okay. So let's go back to our homepage. Okay, great. Okay, so now that I'm back in the homepage, I actually noticed that there's a new carousel that we've never seen before, this one over here. And yeah, so this carousel is actually powered by personalized recommendations, next item recommendations, where we try to take in the customer's clickstream history and contacts to recommend items that the customer is likely to view or purchase next. Yeah, so this is only possible because we have been collecting contextual information through our journey on this site. Yeah, okay. So it just so happens that after looking at cameras and buying a camera, I want to buy some accessories, right? So let me add this tripod. Let me build this tripod. Yeah, okay. So I'm satisfied with this item. So let me add this item to the cart. Yeah, okay, great. So we can also see that similar item recommendations also recommend items that are similar to this. So extra tripods over here. Okay so lastly, I know that I'll need a memory card for my camera. So let me go to flash memory, this category over here, to actually view memory cards. Okay so there's so many memory cards to choose from. As a novice, I do not know what to choose right? Yeah so however, at the bottom we actually see a carousel that actually... shows some memory card recommendations. So this carousel is also powered by Next Item recommendations with an added twist to it. So we apply filtering on top of these recommendations layer to actually filter by flash memory only, so that we can see what are the memory cards that other customers will like after purchasing a certain camera. So I see this 32 gig memory card over here, and I'm satisfied with it. So let me add this item to my cart as well. Okay, so with this, we can actually see how recommendations can be used to elevate the user experience browsing through an e-commerce scenario. However, while this demo UI can actually show how we can apply recommendations, it does not showcase all of our features. So in addition to this customer persona that I was trying out in the demo earlier, we have actually another important persona called a merchandiser. So this merchandiser is actually responsible for managing and executing marketing and business and, sorry, marketing and sales strategies. And so in this next UI, I'll actually show you how the merchandiser is able to sort of manage or execute marketing strategies. So you can see this over here. This is also provided by Intelligent Selling Services, our internal stakeholder. And, okay, let's click on. this over here and we can click on influencers okay so over here we're able to see how uh the merchandiser or administrator can actually tweak some of the parameters to actually fine-tune the the results the search results so let's say i change something over here you can actually see the the preview being updated in real time so this is something merchandisers use to preview uh the selection the recommendations or the the item uh selection before going live. Yeah. However, we are unable to use this to showcase all of our features. So actually, we have prepared another UI for that. Sorry, I think, yeah. OK. Over here. OK. Yeah. So if we've had another UI to actually showcase our features in much more detail. Yeah. So for this demo, we'll be using a movie catalog data set. So it's quite different from our previous demo. But it just is also used to showcase how our personalized recommendation service can indeed be used across multiple LOBs, multiple scenarios. So as you can see in our demo app, we have similar components such as the clickstream history or context. and personalized recommendations, which is our next item recommendations, and our alternative or similar item recommendations over here. Okay, so let's do a quick recall. We can recall that next item recommendations will recommend items that the user is most likely to view next. So for example, if I have item A and B in my clickstream history, these next item recommendations will try to recommend me item C, which is the next item in the sequence. Whereas for alternative recommendations, it tries to predict or recommend the items that are most similar to an item being viewed. So for example, if I'm looking at item A, so similar item recommendations, you try to recommend A prime, A double prime, and so on. Yeah. OK, great. So I have prepared this demo with some items in my clickstream history or context, and we can see the recommendations over here. Yeah, the movie recommendations. Okay, so the first feature that I'll be showing off today will be actually our boosting feature. So boosting is something similar to what we saw in the Merchandiser app earlier. So it allows us to boost certain items based on their features, based on their tags or categories and so on. And it's actually very useful for merchandisers or administrators to actually execute their marketing campaigns by promoting certain items based on attributes. For example, in Christmas time, you want to promote Christmas related items. So you can actually promote them using this feature over here. Okay, so let's imagine that this week is Star Wars week, for example. So May the 4th. May the 4th. So let me try to boost Star Wars. So before that, we can actually take a look at our clickstream. While we have some Star Wars items in the clickstream, you can see that most of the items here are more related to Lord of the Rings, which is our currently viewed item. Okay, so let me start by boosting by Star Wars. Okay, as you can see here, once we have added a boosting score for Star Wars, the results have actually changed. So as we tweak the scores a bit more, we can actually see Star Wars being pushed to the front. And that's intuitively how boosting is supposed to work, right? So let's say I'm a merchandiser and it's Christmas week. I want to boost Christmas related items like decorations, chocolate sweets and things like that. I can actually boost using this tag. And this is how we preview the items for our recommendations. Yeah. Okay. So, yeah, that's all for our boosting. So it's very useful for, let's say, not only merchandisers, but sorry, not only e-commerce, but for other LOBs as well, learning recommendations and things like that. Okay. So the next. feature that we'll be showing today will actually be our explainability feature. So as Steven mentioned earlier, explainability aims to actually justify or explain the recommendations provided through our recommendation engine. So there are three components to our ML explainability feature. So you can see over here. Okay, yeah, so the three components are the overall score, the confidence score, as well as our item at attribute contributions and lastly our context attention. Yeah so these three scores together will actually provide justification to all our our recommendations and we can also compare how this accessibility shows differently based on different items. So for example in our first item over here the top rank item rank one you can see that a lot of the rings the two towers is shown next right and Under sequence attention, we can see that a lot of the rings, Fellowship of the Ring actually contributes 90% to the overall score. Whereas maybe let's take an item in our recommendations that's not related, maybe Men in Black. So this is rank number eight in our recommendations, right? And as you can see, a lot of the rings only contributes 55%, whereas the other two sci-fi movies actually contribute 45%. So you can see how explainability gives us more clarity into why we recommend the items. Okay, great. Okay, so that's for explainability. Yeah, so merchandisers can actually use this feature to understand and to actually fine-tune their parameters as they are preparing or reviewing the items for recommendation. And also for users, users can actually benefit from this feature as well in certain scenarios. For example, in learning recommendations, knowing the context behind recommended learning materials will actually provide... actually provide the user with a better understanding of his learning journey. So yeah, that's for ML accountability for you. Okay, so the last feature that we will show today is actually the Co-Start feature. Yeah, so Co-Start, as I mentioned before, is a common problem with recommended services where we do not have enough information of this item or user, but we want to use them in recommendations anyway. So for example, I have a last-minute item addition to the catalog and I have no time to retrain the model. As Stephen has mentioned earlier, we have no time to train the model. We actually still use this Co-Start feature to actually supplement. to ensure that there's no gap in our recommendations. So let's see this lot of the rings, Fellowship of the Ring, we have Adventure, Fantasy, right? And for text, maybe let's pinpoint some. High Fantasy, Wizards, Trilogy. Okay, so as an example, let me remove this item from the clickstream. Okay, and let me try to add it as a co-start item. Okay, co-start item, yeah. So let me add this item back as a co-start item. OK, something like this. OK, so the categories are adventure and fantasy. And for the text, I remember wizards, trilogy, I think high fantasy maybe, and the.. OK. OK, these four texts. So with this, you're trying to emulate adding an actual movie item. And just to reiterate, this item is not part of our training data. Yeah. And so as you can see, by adding this co-start item, we can actually see that the recommendations are actually quite accurate, although it's not identical to the one we see earlier. And this is because our recommendations solution, our process recommendations solution is able to extract key components of the coastal items metadata and approximate the embeddings to actually produce these recommendations, even though it's not part of the training data at all. So we can also actually view the explainability. for these items as well. So this feature allows merchandisers the flexibility of adding new items without worrying about the visibility factor of these newly posted items. We're also streamlining the model lifecycle management so you don't have to keep retraining every time we add a new item into the catalog. And in addition to co-start items, we also handle co-start users as well. So this is a much more frequent occurrence where let's say I'm a new user and I register with the website. I do not have any history or any context and definitely the model does not have enough time to train in between the time I register right but we are still able to use this costar user and item feature to actually onboard the user or item into the recommendations engine yeah so with this I guess we have come to the end of our demo so during this demo we have not only shown how hyper personalized recommendations can actually improve the overall customer UX and to drive sales of course, user experience. But we have also demonstrated many of our features that allow merchandisers to realize the sales and marketing strategies, such as the COSAT scenarios we have shown, filtering and boosting, and ML explainability. However, of course, that's not all we have in store. As Stephen has mentioned earlier as well, we do have affinity recommendations, smart search, and even real-time metadata updating that we have not shown yet. But due to time constraints, we would like to... and our demo here. So please feel free to contact anyone from our team in the contact list. Should you have any questions or would like to find out more about our service? Yeah, so with that, thank you very much for your kind attention. I'll pass it back to Maynita. Thank you very much. Thank you, Benjamin, for the demo. You showed two use cases and I would be interested at first, like what other use cases exist for personalized recommendation that customers are already using maybe? Yeah, maybe I can take that. So we started from the project with Commerce Cloud. So we kick off the collaboration with Commerce Cloud. The team which we work with is intelligent selling services, which we bring product recommendations for the Commerce Cloud customers. And then additionally, we work with SuccessFactors to bring learning recommendations for I think more than 100 customers for now. And also we have also worked with SenseFactors for a new offering, which is called Career Path Recommender. So like I'm a developer, which the next step can be. So based on that, the drop taken history, we train the model and then it was serving also more than 100 customers for SenseFactors. Similarly, we work with SAP DeepMos, which is another learning platform to bring learning content recommendation for e-sealers. Yeah, very, very interesting and good to know. Good to know the various use cases for personalized recommendation service. Question to the audience. Are you already using personalized recommendation service? Let us know, share your experience. And of course, now that we have Steve, Stephen and Benjamin with us, raise your question on what you saw and what you heard. Let me continue. Yeah, how about like... and I'm sure there are probably a few developers that are watching that are interested in this service. Can developers already test and like try out the service on their own? Yes, so this service is publicly available since this May and then is offered with a free tier option so anyone is eligible to sign up account in SAP BTP they can this service for free and then there's a free tier and then there is a limitation that we support up to two trainings per month and then up to 1000 inferences per month as well. So it's a fully managed service which we only need training data from the customers and then after the training is triggered, after the training data is provided and then we take care of all the workflows including the... the data validation and training and then that's a model deployment and evaluation and then the model is deployed on AI core which is also running on the BDP platform so actually after this call maybe we can drop a link that we have actually posted a blog post on step by step how to do each of the step how to sign up the service how to get a token with the token how do I do trigger training how do I monitor the training status is very straightforward. So we have created multiple steps of step-by-step blog posts. And then we also have a video of work through. So that's, I believe a customer can use that very easily, can follow the steps. And then if anyone have any questions, they can feel free to ask a question in the blog posts. And then we have the whole team watching, follow the channel so we can. follow up visa customers directly. Awesome. Yes. We've posted the link to the blog series on getting started with personalized recommendation, which you posted. And yeah, I think there are already great feedback and voices coming back to you. What was so, yeah, the highlight of this blog series, you would say, from getting started, the setup? model training and serving and so on? I would say the highlight is that it's easiness to use. And then customer finds that they do not need to do any extra steps. So they do not need to remember any like new URLs or credentials. They do not need to worry about the data privacy and all that because everything we have is taken care of in the solution. So the data which provided to... us, we will do the encryption and then no one can read the data in any way because the data there's an end-to-end encryption that is being put in place and then even our operation of developer team there's no way to decrypt it yet so that it's very secure and after the training is done and we destroy the data and because we have destroyed the whole training container as well so nobody is so it's like no there's no worry about the GDPR concern or personal data concern in any way and then yeah so we hear the feedback is that the most difficult part is the data preparation so because not many people they have the enough data for example that especially the collision data is not very easily available but we have this a kickstart of this support of costat and freezing start so even without a question which we majorly learn from we can also build a model and at least we can have a model so that the customer can serve recommendation and then from there they can collect more data for training and then that's a uh ethically we can improve the whole recommendation experience because we believe our delete perfection uh oh sorry uh into uh Continuous improvement is better than delayed perfection. So we want to help our customer to have recommendations as early as possible. And that is why probably having the feedback and really comments on this coming from those that are using it, very crucial for this continuous improvement. Thank you for your response. Yeah, so you would say for anyone interested in learning. getting started you definitely refer to this blog series where you really outline the different steps and there's also tutorials i understand right yes so so yeah definitely recommendation for everyone um we also um posted the link to the community platform where you can post the q a where you can see the blog series but also references to learning materials such as tutorials well let me have a look um real quick at the time and give you the opportunity to raise a question too. Yeah, sometimes... Maybe I don't... I don't... that's a tutorial. So the tutorial and the mission which we created is very similar to the blog post content, but then it's having this like a gamification dot in there, so customers, they can... anyone who are interested in this service, we can... click on the step by step they can buy as complete so they can show that the sense of achievement and when we talk about creating a ai project we sometimes we feel like it's super difficult to do and then it's very far away from our life but actually it's very near so we also provide sample data to start up with so people can download anyone can download the data and then to trigger that service for free and then is a super a good feeling that you have this achievement that's based on the data we provide. You can deploy your first AI project in your life. So yeah, welcome to have a try. Perfect, thanks. That I think is very important to remember if you want to have a try. Now I see a question from the community. Thanks, Varun. So is there a minimum threshold for the dataset volume for training to begin? Yeah, so of course we always have a thirsty for data. So the more data we have, the more accurate and everything we can have. So for now we think that the minimum threshold for the data item is 100 items because we believe that below 100 items we probably don't need a system to do recommendation. So for each of the item, otherwise to have 100 valid click streams for those items. Of course, for users, the more the better. So I think we have also set, we only set the 100 minimum for the item and then each of the item 100 valid clickstream interaction with items. And then for the upper limit, I believe we support more than 750. So we are trying to test the limit more because we recently have a query that wants to support, requests us to support. up to 1 million items. Thank you. I think that was pretty straightforward. Yeah, thank you both for your time and presenting today for the demo. And of course, thank you for everyone watching. You know where you can find the speakers, the experts, and where you can Raise your questions if you have any after this call. With that, have a good rest of day. I hope you enjoyed it. And we are always looking forward for feedback. So add your comments, give us a like, or just follow us in the community for further questions or following up on this topic. Hope to see you in one of the other community calls again. And yeah, take care. Bye everyone. Bye. Thank you.

And with me is Benjamin Tan, who is a full-stack machine learning developer, and Stephen Fu, who is product owner. both working in SAP artificial intelligence. Now before we start real quick, you already saw it, some of you already used it, we have a live chat.

You are very welcome to use it, chat with us, and of course already address your questions, raise your questions, and then as always in the community calls we will have time at the end to address them. We also have an additional Q&A support that's also working with the chat. So maybe you noticed the one or other expert that's joining.

But yeah, have fun and enjoy. The floor is yours, Benjamin and Stephen. Thank you.

So I will start first. So my name is Stephen. So I joined SAP eight years ago.

So I have been starting working with SAP AI for the past five years. So working on this topic about three years and more. So yeah, we're happy to be here to...

share our findings and share our new offerings, personalized recommendations with you. So let's spend a couple moments on the legal disclaimer and yeah, let's maybe we can start. So before we can jump start to talk about personalized recommendations, so we like to briefly touch on the background and the offering platform.

So Intelligent Enterprise uses innovative technologies such as artificial intelligence, machine learning and robotic automation to turn insights into action in across business in real time. Artificial intelligence is no longer the future for business is here and now so this fact has been validated by many statistics and many proved by the reports which we have been received daily so this shows us how urgently that's many experts on business are in need for artificial intelligence and the impact it could make. So SAP offers a wealth of business services that are based on SAP business technology platform. So this service combines our best practice from more than 30 industries with rapid time to value and flexibility to meet your business needs. So by combining SAP AI business services, with innovative AI technology from SAP, you can kickstart your AI projects.

So this is a high-level overview of SAP BDP. So put our service in perspective, we are located within the AI business services suite, which is part of the BDP, alongside with other services such as service ticket intelligence and document information extraction. So we have many internal stakeholders which have successfully integrated our service such as SAP Commerce Cloud, Intelligent Selling Services, and our service is not only for internal stakeholders but also publicly available since May, while CPA and Pay-as-you-go with free tier options. So now let's talk about recommendations.

So our objective is to provide recommendations or user from a long list of potential items in a given context. So there are many challenges along the way in these steps. So for the users, you have costar users, which have never seen before.

You have users who have shifting habits. Even if you have seen them before, they might not behave in the same way anymore. So for the items, you have new product launch, costar items with no history.

You have to adapt latest trend, seasonality. you have different marketing and promotion strategies, and you have to use recent instructions to get users intent in order to generate relevant and personalized reports. So you have to solve all these problems to deliver a good recommendation in a very short time, and then you have to keep it up to date most of the time, which is a very complicated problem.

So we have solved the problem to you while this UNIC introduced the personalized recommendation service. So we have worked with internal and external stakeholders to understand the problem, to apply the state-of-the-art machine learning approaches, master the ability to extract the insight from the customer data, to build the machine learning models to deliver highly personalized recommendations. So we're able to address all the problems which I mentioned in the previous slides, the costar users, shifting habits, costar items. is able to adopt, adapt latest trends, promotion strategies, use the given context in real time. So as a result, the user engagement is enhanced, the item discovery is improved, and our machine learning model is not a black box.

The model owner can still influence the recommendation output in many ways, such as filtering, boosting which we will show later to retrain business control to create relevance and to meet KPIs. So with a model trained from the training data which I shared in the previous slides that's customer training data, the service can be used in many diverse scenarios across business in all means such as next click, next share, affinity recommendation and smart search. So a next click scenario predicts the next item that the user is likely to click or view or purchase based on their historical clickstream. So for instance in an e-commerce setting after customer has viewed several products the model understands what the product the user is trying to find and recommend them other products within the same functionality.

In contrast the next scenario predicts the item the user is most likely to purchase based on the user's item which is recently purchased. So for instance, given user has recently bought a laptop and the model predicts the item and the user is likely to purchase next such as a mouse, keyboard, microphone, headset or other electronic accessories are related to the item which the user just bought. And another scenario is affinity recommendation, which can be used in multimedia platform and recommend user content based on their predicted attributes. So for instance, if the user has been viewing football related contents, we are able to detect the user's interest and recommend similar or relevant items based on the content.

Last but not least, smart search is also extended from the model which we built from the customer data. So we use natural language processing query to suggest an appropriate request with the help of the user's past behavior. This means that the search is more than just a simple query string match.

So for instance, if the user recently viewed stalker products and the recommendation which they will receive if they search for the keyword ball as shown below the screen, so we will show more stalker balls instead of other balls. in the context. So recommendation topic is the domain agnostic.

So it only depends on the user item interaction such as product purchase, cost learning, or job token. So in this case, it can be applied in multiple domains which can be seen like in retail which is adopted by SAP Commerce Cloud, human resources for learning such as SS. SAP SuccessFactors Learning, Career Path Explorer, and also SAP Limons Learning.

So this is all the embedded solutions which we have delivered together with other line of business within SAP. And it can also be used for content delivery such as SAP Community or any live streaming platforms which can be also adapted. So it's also available for public to find the next readable. domains which can be applied. Yeah, so stakeholders and customers always ask, what is the difference between your service and your competitors?

So the key features of this service can be seen here. So I will quickly run through the key features and later our AI developer Benjamin Tan will show you in detail with a demo UI. So firstly, we can provide sophisticated personalized recommendations. quick and relevant alternative recommendations, and user session scopes, affinity prediction, and then we can also explain to perform smart search. So for all these recommendations and search results, we can explain our results.

the ability to identify the confidence score for each of the predictions and the detailed contribution of each of the attributes from the relevant items or the user metadata. Next, for customizable strategy, so for new customers who have not enough clickstream, we can train models based on the item catalog only to kick start with our customers. So for the new items, new users, or coastal platforms, we can handle these scenarios very efficiently. We can also upload feature importance to help our customer to continuously improve training data quality and training efficiency because we know the garbage in, garbage out. So the more training data, the better of the training data, the better model performs.

So we support customer attribute boosting, affinity boosting for different promotion or marketing strategies. We support batch and inference modes, real-time inference modes, and then for real-time inference we can update the model for the data changes in between two trainings so that customer doesn't have to train very frequently so that the model can be still up to date almost at time. And the whole solution is easy to adopt and fully manage service with scalable architecture in SAP Cloud Platform. So there's a single set of API which manage trigger training, monitor training status, model serving status, making inference calls, and further recommendations. So here is the overview of this service from AI Business Services, personalized recommendation.

So if there's any burning questions, feel free to ask. We'll be very happy to answer before we show you a demo. Should we have a break for some questions?

I think we can move forward unless there's anyone who already has some questions in the chat. If not, they're probably excited to see the demo and then we can go to Q&A. Yeah sure, I better stop sharing now. Okay, let me share my screen. Okay, can everyone see my screen?

Yes. Okay, great. Hi, thank you, Stephen, for the introduction. And hi, everyone. I'm Benjamin, and I'm an AI developer in the same team.

I've been in SAP for two years. And for today's demo, I had a hand in integrating our service into the demo UI. So I can attest that our service is quite easy to integrate.

But yeah. So thank you very much for attending today's session. And today I'll be giving two short demos on our service. And through these demos, we hope to showcase some of the interesting capabilities of personalized recommendations, as well as how the recommendations can actually be used to elevate the user experience.

So for the first demo, as you can see over here, this demo is an instance developed by one of our internal stakeholders, Intelligent Selling Services, with our recommendations engine embedded into. this site to provide recommendations to our customers. So we will be emulating a customer's perspective of browsing an e-commerce site specialized in camera sales, as you can see here, but also highlighting the various personalized recommendation features that are used to improve the user experience as we go along. Yeah, okay.

So I'm a new user that's visiting this e-commerce site for the first time, and I'm familiar with the site and its products. So... The first thing I see here is actually two carousels showing trending products.

So before we begin, I'd like to point out that as I'm a new user, I do not have any context for recommendations. So where context refers to actually like my clickstream history, like Steven has mentioned previously. For example, click item clicks, item views, add to cart events and so on. So this is what we refer to as a co-start scenario. And in this co-start scenario, personalized recommendation service will actually leverage on the aggregated training data, contextual training data, to actually recommend the most popular or trending items to the new customers you can see over here.

So this is a great way to get our customers started. So now I'm interested in buying a camera, for example. So this camera looks nice.

This DSLR. So let me click on it. Yeah.

OK, great. So as you can see on this page, we can only see the camera item details. But we can actually see this carousel over here.

So this carousel is actually powered by personalized recommendations, similar item recommendations, where we try to predict, sorry, we try to recommend items that are most similar to the items that are currently being viewed. So this is very useful in scenarios that we like to retain the customer's attention during specific events. So for example, if the current item is out of stock, for example, we will recommend items similar to this item in hopes of retaining his purchasing intent and thereby increasing conversion rates. So let me click.

So I'm looking at the cameras now and I see this camera over here includes a lens in the package. So it's interesting to me. Let me add this in. Yeah, sorry, let me add this in.

Okay, let me view this item, sorry. Okay, so I'm satisfied with this item, so let me add this to the cart. Yeah, okay, great. Okay, so as we look below, we can actually see that the site also tracks my item views.

So you can see your recently viewed products in this carousel here. So this shows that we can actually use this contextual data to provide richer recommendations subsequently. Yeah, okay. So let's go back to our homepage.

Okay, great. Okay, so now that I'm back in the homepage, I actually noticed that there's a new carousel that we've never seen before, this one over here. And yeah, so this carousel is actually powered by personalized recommendations, next item recommendations, where we try to take in the customer's clickstream history and contacts to recommend items that the customer is likely to view or purchase next.

Yeah, so this is only possible because we have been collecting contextual information through our journey on this site. Yeah, okay. So it just so happens that after looking at cameras and buying a camera, I want to buy some accessories, right? So let me add this tripod. Let me build this tripod.

Yeah, okay. So I'm satisfied with this item. So let me add this item to the cart. Yeah, okay, great. So we can also see that similar item recommendations also recommend items that are similar to this.

So extra tripods over here. Okay so lastly, I know that I'll need a memory card for my camera. So let me go to flash memory, this category over here, to actually view memory cards. Okay so there's so many memory cards to choose from.

As a novice, I do not know what to choose right? Yeah so however, at the bottom we actually see a carousel that actually... shows some memory card recommendations. So this carousel is also powered by Next Item recommendations with an added twist to it.

So we apply filtering on top of these recommendations layer to actually filter by flash memory only, so that we can see what are the memory cards that other customers will like after purchasing a certain camera. So I see this 32 gig memory card over here, and I'm satisfied with it. So let me add this item to my cart as well.

Okay, so with this, we can actually see how recommendations can be used to elevate the user experience browsing through an e-commerce scenario. However, while this demo UI can actually show how we can apply recommendations, it does not showcase all of our features. So in addition to this customer persona that I was trying out in the demo earlier, we have actually another important persona called a merchandiser. So this merchandiser is actually responsible for managing and executing marketing and business and, sorry, marketing and sales strategies.

And so in this next UI, I'll actually show you how the merchandiser is able to sort of manage or execute marketing strategies. So you can see this over here. This is also provided by Intelligent Selling Services, our internal stakeholder. And, okay, let's click on.

this over here and we can click on influencers okay so over here we're able to see how uh the merchandiser or administrator can actually tweak some of the parameters to actually fine-tune the the results the search results so let's say i change something over here you can actually see the the preview being updated in real time so this is something merchandisers use to preview uh the selection the recommendations or the the item uh selection before going live. Yeah. However, we are unable to use this to showcase all of our features.

So actually, we have prepared another UI for that. Sorry, I think, yeah. OK.

Over here. OK. Yeah. So if we've had another UI to actually showcase our features in much more detail.

Yeah. So for this demo, we'll be using a movie catalog data set. So it's quite different from our previous demo.

But it just is also used to showcase how our personalized recommendation service can indeed be used across multiple LOBs, multiple scenarios. So as you can see in our demo app, we have similar components such as the clickstream history or context. and personalized recommendations, which is our next item recommendations, and our alternative or similar item recommendations over here. Okay, so let's do a quick recall. We can recall that next item recommendations will recommend items that the user is most likely to view next.

So for example, if I have item A and B in my clickstream history, these next item recommendations will try to recommend me item C, which is the next item in the sequence. Whereas for alternative recommendations, it tries to predict or recommend the items that are most similar to an item being viewed. So for example, if I'm looking at item A, so similar item recommendations, you try to recommend A prime, A double prime, and so on.

Yeah. OK, great. So I have prepared this demo with some items in my clickstream history or context, and we can see the recommendations over here.

Yeah, the movie recommendations. Okay, so the first feature that I'll be showing off today will be actually our boosting feature. So boosting is something similar to what we saw in the Merchandiser app earlier. So it allows us to boost certain items based on their features, based on their tags or categories and so on.

And it's actually very useful for merchandisers or administrators to actually execute their marketing campaigns by promoting certain items based on attributes. For example, in Christmas time, you want to promote Christmas related items. So you can actually promote them using this feature over here. Okay, so let's imagine that this week is Star Wars week, for example.

So May the 4th. May the 4th. So let me try to boost Star Wars.

So before that, we can actually take a look at our clickstream. While we have some Star Wars items in the clickstream, you can see that most of the items here are more related to Lord of the Rings, which is our currently viewed item. Okay, so let me start by boosting by Star Wars.

Okay, as you can see here, once we have added a boosting score for Star Wars, the results have actually changed. So as we tweak the scores a bit more, we can actually see Star Wars being pushed to the front. And that's intuitively how boosting is supposed to work, right?

So let's say I'm a merchandiser and it's Christmas week. I want to boost Christmas related items like decorations, chocolate sweets and things like that. I can actually boost using this tag. And this is how we preview the items for our recommendations.

Yeah. Okay. So, yeah, that's all for our boosting. So it's very useful for, let's say, not only merchandisers, but sorry, not only e-commerce, but for other LOBs as well, learning recommendations and things like that.

Okay. So the next. feature that we'll be showing today will actually be our explainability feature. So as Steven mentioned earlier, explainability aims to actually justify or explain the recommendations provided through our recommendation engine.

So there are three components to our ML explainability feature. So you can see over here. Okay, yeah, so the three components are the overall score, the confidence score, as well as our item at attribute contributions and lastly our context attention.

Yeah so these three scores together will actually provide justification to all our our recommendations and we can also compare how this accessibility shows differently based on different items. So for example in our first item over here the top rank item rank one you can see that a lot of the rings the two towers is shown next right and Under sequence attention, we can see that a lot of the rings, Fellowship of the Ring actually contributes 90% to the overall score. Whereas maybe let's take an item in our recommendations that's not related, maybe Men in Black. So this is rank number eight in our recommendations, right? And as you can see, a lot of the rings only contributes 55%, whereas the other two sci-fi movies actually contribute 45%.

So you can see how explainability gives us more clarity into why we recommend the items. Okay, great. Okay, so that's for explainability.

Yeah, so merchandisers can actually use this feature to understand and to actually fine-tune their parameters as they are preparing or reviewing the items for recommendation. And also for users, users can actually benefit from this feature as well in certain scenarios. For example, in learning recommendations, knowing the context behind recommended learning materials will actually provide... actually provide the user with a better understanding of his learning journey.

So yeah, that's for ML accountability for you. Okay, so the last feature that we will show today is actually the Co-Start feature. Yeah, so Co-Start, as I mentioned before, is a common problem with recommended services where we do not have enough information of this item or user, but we want to use them in recommendations anyway. So for example, I have a last-minute item addition to the catalog and I have no time to retrain the model.

As Stephen has mentioned earlier, we have no time to train the model. We actually still use this Co-Start feature to actually supplement. to ensure that there's no gap in our recommendations. So let's see this lot of the rings, Fellowship of the Ring, we have Adventure, Fantasy, right? And for text, maybe let's pinpoint some.

High Fantasy, Wizards, Trilogy. Okay, so as an example, let me remove this item from the clickstream. Okay, and let me try to add it as a co-start item.

Okay, co-start item, yeah. So let me add this item back as a co-start item. OK, something like this. OK, so the categories are adventure and fantasy.

And for the text, I remember wizards, trilogy, I think high fantasy maybe, and the.. OK. OK, these four texts. So with this, you're trying to emulate adding an actual movie item.

And just to reiterate, this item is not part of our training data. Yeah. And so as you can see, by adding this co-start item, we can actually see that the recommendations are actually quite accurate, although it's not identical to the one we see earlier. And this is because our recommendations solution, our process recommendations solution is able to extract key components of the coastal items metadata and approximate the embeddings to actually produce these recommendations, even though it's not part of the training data at all. So we can also actually view the explainability.

for these items as well. So this feature allows merchandisers the flexibility of adding new items without worrying about the visibility factor of these newly posted items. We're also streamlining the model lifecycle management so you don't have to keep retraining every time we add a new item into the catalog. And in addition to co-start items, we also handle co-start users as well. So this is a much more frequent occurrence where let's say I'm a new user and I register with the website.

I do not have any history or any context and definitely the model does not have enough time to train in between the time I register right but we are still able to use this costar user and item feature to actually onboard the user or item into the recommendations engine yeah so with this I guess we have come to the end of our demo so during this demo we have not only shown how hyper personalized recommendations can actually improve the overall customer UX and to drive sales of course, user experience. But we have also demonstrated many of our features that allow merchandisers to realize the sales and marketing strategies, such as the COSAT scenarios we have shown, filtering and boosting, and ML explainability. However, of course, that's not all we have in store. As Stephen has mentioned earlier as well, we do have affinity recommendations, smart search, and even real-time metadata updating that we have not shown yet.

But due to time constraints, we would like to... and our demo here. So please feel free to contact anyone from our team in the contact list.

Should you have any questions or would like to find out more about our service? Yeah, so with that, thank you very much for your kind attention. I'll pass it back to Maynita. Thank you very much. Thank you, Benjamin, for the demo.

You showed two use cases and I would be interested at first, like what other use cases exist for personalized recommendation that customers are already using maybe? Yeah, maybe I can take that. So we started from the project with Commerce Cloud.

So we kick off the collaboration with Commerce Cloud. The team which we work with is intelligent selling services, which we bring product recommendations for the Commerce Cloud customers. And then additionally, we work with SuccessFactors to bring learning recommendations for I think more than 100 customers for now.

And also we have also worked with SenseFactors for a new offering, which is called Career Path Recommender. So like I'm a developer, which the next step can be. So based on that, the drop taken history, we train the model and then it was serving also more than 100 customers for SenseFactors.

Similarly, we work with SAP DeepMos, which is another learning platform to bring learning content recommendation for e-sealers. Yeah, very, very interesting and good to know. Good to know the various use cases for personalized recommendation service. Question to the audience. Are you already using personalized recommendation service?

Let us know, share your experience. And of course, now that we have Steve, Stephen and Benjamin with us, raise your question on what you saw and what you heard. Let me continue.

Yeah, how about like... and I'm sure there are probably a few developers that are watching that are interested in this service. Can developers already test and like try out the service on their own?

Yes, so this service is publicly available since this May and then is offered with a free tier option so anyone is eligible to sign up account in SAP BTP they can this service for free and then there's a free tier and then there is a limitation that we support up to two trainings per month and then up to 1000 inferences per month as well. So it's a fully managed service which we only need training data from the customers and then after the training is triggered, after the training data is provided and then we take care of all the workflows including the... the data validation and training and then that's a model deployment and evaluation and then the model is deployed on AI core which is also running on the BDP platform so actually after this call maybe we can drop a link that we have actually posted a blog post on step by step how to do each of the step how to sign up the service how to get a token with the token how do I do trigger training how do I monitor the training status is very straightforward. So we have created multiple steps of step-by-step blog posts.

And then we also have a video of work through. So that's, I believe a customer can use that very easily, can follow the steps. And then if anyone have any questions, they can feel free to ask a question in the blog posts. And then we have the whole team watching, follow the channel so we can.

follow up visa customers directly. Awesome. Yes.

We've posted the link to the blog series on getting started with personalized recommendation, which you posted. And yeah, I think there are already great feedback and voices coming back to you. What was so, yeah, the highlight of this blog series, you would say, from getting started, the setup?

model training and serving and so on? I would say the highlight is that it's easiness to use. And then customer finds that they do not need to do any extra steps. So they do not need to remember any like new URLs or credentials. They do not need to worry about the data privacy and all that because everything we have is taken care of in the solution.

So the data which provided to... us, we will do the encryption and then no one can read the data in any way because the data there's an end-to-end encryption that is being put in place and then even our operation of developer team there's no way to decrypt it yet so that it's very secure and after the training is done and we destroy the data and because we have destroyed the whole training container as well so nobody is so it's like no there's no worry about the GDPR concern or personal data concern in any way and then yeah so we hear the feedback is that the most difficult part is the data preparation so because not many people they have the enough data for example that especially the collision data is not very easily available but we have this a kickstart of this support of costat and freezing start so even without a question which we majorly learn from we can also build a model and at least we can have a model so that the customer can serve recommendation and then from there they can collect more data for training and then that's a uh ethically we can improve the whole recommendation experience because we believe our delete perfection uh oh sorry uh into uh Continuous improvement is better than delayed perfection. So we want to help our customer to have recommendations as early as possible.

And that is why probably having the feedback and really comments on this coming from those that are using it, very crucial for this continuous improvement. Thank you for your response. Yeah, so you would say for anyone interested in learning. getting started you definitely refer to this blog series where you really outline the different steps and there's also tutorials i understand right yes so so yeah definitely recommendation for everyone um we also um posted the link to the community platform where you can post the q a where you can see the blog series but also references to learning materials such as tutorials well let me have a look um real quick at the time and give you the opportunity to raise a question too. Yeah, sometimes...

Maybe I don't... I don't... that's a tutorial. So the tutorial and the mission which we created is very similar to the blog post content, but then it's having this like a gamification dot in there, so customers, they can...

anyone who are interested in this service, we can... click on the step by step they can buy as complete so they can show that the sense of achievement and when we talk about creating a ai project we sometimes we feel like it's super difficult to do and then it's very far away from our life but actually it's very near so we also provide sample data to start up with so people can download anyone can download the data and then to trigger that service for free and then is a super a good feeling that you have this achievement that's based on the data we provide. You can deploy your first AI project in your life. So yeah, welcome to have a try.

Perfect, thanks. That I think is very important to remember if you want to have a try. Now I see a question from the community. Thanks, Varun. So is there a minimum threshold for the dataset volume for training to begin?

Yeah, so of course we always have a thirsty for data. So the more data we have, the more accurate and everything we can have. So for now we think that the minimum threshold for the data item is 100 items because we believe that below 100 items we probably don't need a system to do recommendation.

So for each of the item, otherwise to have 100 valid click streams for those items. Of course, for users, the more the better. So I think we have also set, we only set the 100 minimum for the item and then each of the item 100 valid clickstream interaction with items.

And then for the upper limit, I believe we support more than 750. So we are trying to test the limit more because we recently have a query that wants to support, requests us to support. up to 1 million items. Thank you. I think that was pretty straightforward. Yeah, thank you both for your time and presenting today for the demo.

And of course, thank you for everyone watching. You know where you can find the speakers, the experts, and where you can Raise your questions if you have any after this call. With that, have a good rest of day.

I hope you enjoyed it. And we are always looking forward for feedback. So add your comments, give us a like, or just follow us in the community for further questions or following up on this topic.

Hope to see you in one of the other community calls again. And yeah, take care. Bye everyone. Bye.

Thank you.

Transcript for:AI-Driven Personalized Recommendations in SAP

Transcript for:
AI-Driven Personalized Recommendations in SAP