influence of machine learning in search to improve the relevance has led to different types of machine learning search applications based upon the business use case as a result open search evolved a lot to adapt such machine learning advancements and released many features to make it easier for the users to build such machine learning search systems welcome to this demo session build the right machine learning search with Amazon open sech service I am Prine moan working as an analytic specialist and I have with me haer open Search solution architect who will do the demo let's first talk about the different search types supported by open search search or information retrieval can be broadly classified into two categories one is a sparse retrieval and the other one is the dense retrieval sparse retrieval is supported by algorithms like tfidf or bm25 the best mass 25 under sparse retrieval we have keyword search the basic match technique based upon the overlapping of keywords between the query and document next we have neural Spar search which is basically a way to expand the documents and queries with additional contextual terms and then perform the basic keyword search this way we will be able to avoid the vocabulary mismatch problem and then improve the relevance next dense retrieval is basically the similarity search performed by comparing dense features called vectors or embeddings these vectors are provided by the machine learning embedding models and are compared by algorithms such as K nearest neighbors or approximate nearest neighbors under dense retrieval we have Vector search which is similarity search using purpose-built models for text images audio or video on the other hand we have multimodel search which uses machine learning models that were trained to learn shared embedding space for multimodel elements here for example texted images will be sharing the high dimensional space next we have the hybrid search and as a name suggest here we combine the keyword search and the vector search scores of the documents for a query and then render results in a hybrid model the last one rack the retrieval augmented generative search comes from the application of large language models here we first retrieve documents using the vector search and then provide results as prompts to large language models to augment the response conversational search is nothing but an adding a memory element to a rag application and facilitating search in a QA conversational style we will be covering all the searches but rag in the demo for the demo we have a simple architecture where the client application is hosted on a e to machine which interacts with the back end that is Amazon open service through a Lambda function for all the search types that involve vectors we use remote uh models which is sitting in Amazon Sage maker and also the Bedrock models to provide vectors for your um text and images this feature of using remote machine learning services to provide vectors is available from open search version 2.9 internally Amazon open search service uses is the machine learning model serving framework facilitated by the ml Commons plugin this is where uh op search actually converts the provided text or image to its respective vector by triggering the remote models in the actual process to achieve this we first create uh machine learning connectors using dedicated machine learning blueprints that can connect to different third-party uh ml platforms these include uh Sage maker Bedrock CER and open AI as custom models for the demo we going to use the sage maker and the Bedrock connectors only once connectors are created we then register and deploy the models in open search using Rest apis by all these tips we provide open search all the details to reach out to the right uh machine learning service and then trigger the model and do the vector conversion for the provided text or images index pipelines and search pipelines actually facilitate the vector conversion during the indexing of documents and uh searching of search queries respectively hi in this demo you will search through a sample retail data set using sparse retrieval methods dance retrieval method search and hybrid search all powered by Amazon open search servers note that in this demo we use open search 2.11 now in the web application you can type in your search queries in the search query bar you can also upload an image you for the multimodel search and you can also select the type of search that you'd like to use from keyword search Vector search Hybrid search and multimodel search if You' like to enhance the keyword search using the sparse embedding features you can click on expand query and documents with sparse features at this demo we are leveraging open search pre-trained by enod model to create these parts embedding for both documents and query you can also fine-tune the hybrid search so you can give different ways to the subqueries Cann and um bm25 subqueries and you can also choose the normalization technique for each subquery then open open search will combines the document scores using one of the techniques that you can uh choose from as well uh like arithmetic mean geometric mean and harmonic mean last but not least you can also choose uh one machine learning model from uh Titan embedding model that is uh hosted on Amazon Bedrock or you can also uh use one of the machine learning models that you can um deploy or host on Amazon Sage maker note that in this solution uh open search invokes the uh embedding uh modeles in Amazon sagemaker and uh Amazon Bedrock using outof the Box AI connectors available in open sech 2.9 now let's try uh with first example looking for trendy footware for women click on go now for the uh first keyword search we have a list of results that have been displayed for us 10 documents as you can see the first uh two documents for example do not refer to uh a foot wear however it refers to a jacket or glasses why is that this is because the keyword search in itself will only match the keywords or terms in your search query with the terms that are available in the captions within each image here we see that there is a match between uh trendy and woman term so for keyword search this might be a relevant result now let's see how it works with uh Spar search let's click on expand query and documents with sparse features and we regenerate the results clicking on the blue button here so the results cre changed and now all the uh images are related to Footwear the sparse search here the sparse betting it will um generate terms that are similar or provides similar meanings as the terms available in your uh search query or even in your document um captions so for the query for example the terms that have been generated are uh Trend women food Etc and as you notice here every ter term is um attributed or uh has uh a weight that is attributed to that uh specific term same for the documents here so if we click on expanded document we see uh these um additional uh similar words like sandal for example uh which has a high weight of 223 and uh this what makes this image basically uh most relevant for this uh hyp for the uh sparse search uh method now that we improve the keyword search using the SP retrieval um methods let's move to Dan retrieval search type with uh Vector search for that I will select Vector search and regenerate the results now we see um different list of footware for uh women and in Vector search we don't necessarily have a matching keywords terms between the search query and the image captions however we might have um words that have similar meanings to what we are looking for for example style and comfort can be related to trendy um the same for the others for example here stylish black sneakers so it can be also related to trendy Etc moving to hybrid search now combination of bm25 and um knnn search and let's generate uh the results for hybrid search so we see uh new images that have been uh included into the search results we can also see nonrelevant uh search uh results such as the jacket here this is where we can actually F tune the hybrid search by giving more weight to the kyn uh subquery for example so uh we leave the uh normalization and combination technique as they are and let's regenerate the results click on the blue button here and now when looking at the uh results it's all about Footwear it's all about the trendy footware Etc and we no longer have that uh nonrelevant results because we gave more weight to the umn scoring than the bm25 scoring moving to the multimod search right now in multimodel search you have the option to search by text only by image only or by both text and image in this use case let's search by both text and image for that I will upload an image of a summer footware for women and click on go to generate results so I have my text trendy Footwear for women and the input image of a summer Footwear of women color black leather uh kind of specific type of uh style so as you can uh see the results here have changed so we have more leather examples and also even the style of the shoes um changed based on the image that we uploaded so we can see that the leather sandals here for for example it matches more the um image that we uploaded as an input now it's your turn to build your own machine learning search solution using Amazon open search service you can recreate the same uh web application that I shared with you in the demo using the first QR code you can also have a look uh or information on each type of search using the second uh QR code and in the third QR code you will learn more about the new features and enhancement that we have uh added to Amazon open seource service in 2023 thank you everyone