Integração de IA no Obsidian

artificial intelligence is all the rage these days and it's been integrated into just about every application that i use and i really wanted to dig into leveraging ai but doing it where you have your own private instance and installation of ai where it works in tandem with obsidian so using obsidian with ai also having it be able to work offline i want to explore this so Let's get into it. Hello, my name is Anton. And in this video, I'm going to talk about obsidian and AI and having a private setup for this here so that you can run your AI offline. Now, why would you want to do this? So let's get into some of the reasons why you would probably want to use an AI solution. One, because it can be offline. So you don't need the internet in order to do it. If you're sitting at on an airplane and you want to leverage this particular solution, it will work doing it that way. Your Your data has increased privacy because you're not sending it outside of your network. There's the low cost of utilizing your own private instance of artificial intelligence and it's dedicated to you so you don't have to worry about other people bogging down your performance when you're trying to do any Q&A or analyze any of the information that you're trying to get out of your artificial intelligence. So what are the different scenarios that you might want to use this? within the context of leveraging it with Obsidian. So there is the local note where you might just want to ask Q&A on a single note within your vault. There's the Q&A scenario where you're doing it on your entire vault. So when you do a question, it goes through your entire vault, all the information in your entire vault and leverages that for context. And then there's just using the default, your plain vanilla type of Q&A, where the... the model has all the information or as much information as it could have pulled from external sources like the internet and it has a lot of different facts and information within the model to where if you're trying to learn a certain topic or just get information like what's the distance from the earth to the moon you can easily answer those questions to get those questions answered leveraging ai so what can you use to run ai locally on your machine Now there are a few different solutions. I was actually surprised to see how many solutions there were out there that you could use. But I wanted to make sure I framed this and kind of kept this in a box to where they worked with Obsidian. So the plugins that I was looking at leveraging with Obsidian could easily be configured to work with these particular solutions that created local instances that could be leveraged for AI and you can run your models. And the ones that I found out there was Ollama. there was lm studio uh git for gtp for all and um another one like jan.io again there were the list could be a lot bigger but i just wanted to get a small list of ones that could be easily installed on your and set up on your local system i am on a mac so but these solutions here at least olama i know will work and is supported on mac windows and linux as long as you have the hardware to support it and that's the the pretty much the big key is that you will need to have hardware that can run just these local models on your system and actually be performant so if you're going to run this you don't want to have some old system that is going to take you know a long time to give you back responses and the ones that i saw are pretty much supported out of the boxes on a mac system it should be one of the M1 variants. So as long as you have an M series, I should say, not just the M1, but if you have one of the M series, M1, 2, or 3, then this should work no problem because they have really good GPUs in them. And then if you have a system like a Windows system or a Linux system that has GPUs in them, then those should work as well. So then there's the question of which ones are easy to install and get running. your machine on your local system and is supported with obsidian and the one that I've tested is Oh llama there are other ones like I've seen LM studio be one that comes up in some of these plugins as supported for those particular plugins so I chose to go with Oh llama I did not actually install LM studio so we're a little bit biased here and I have no feedback on the LM studio route now how many models are out there that you can actually use and after looking there are too many really there are a lot of different models that you can leverage for this particular use case so olama supports many different models that you can download and leverage with it as well as LM studio so really I think this we're at a time where if you have the hardware to support this the you know the world is your oyster you can pretty much go out there and play around with any of these models and see how they work for your particular use case now you might also be asking well which models should you actually use in order to do this and to be honest there because there are so many I kept my my test bed a little bit small so that i wasn't pulling down too many models and trying to test this so uh llama 2 llama 3 mistral those are really the ones that i focused on you know getting some time with those three different ones doing the indexing and somewhat seeing you know how how they responded to some of my questions how long it took to do some indexing i don't have legitimate like say performance metrics from one to the other but i will say the you know all of them were pretty much along the same line of a from a performance standpoint and yeah you can't really go wrong just know that the more information you have in your in your vault the longer it's going to take to index the the information and get it into the databases that uh the the models are going to use when you're doing your q a on these on on your information So as far as the different plugins that are out there and available, again there are a ton of different plugins that you could use. I stuck with just four of them and really three of them are were used during the test and are say comparable to each other. Cannoli is somewhat of a different beast where it's leveraged in the canvas and it can do a lot more things than the other plugins that you have that are really more Q&A type chat bots. So the copilot the bmo and the smart second brain those are all pretty much the same where you would pose a question or ask for information and kind of you know work with it the same way you would with chat gpt and it will give you responses based off what it understands or the information it has within the model now the plugins do support different different things here and with some of these plugins they some of them support local and remote And many of them do support both and others support whether you can just do your generalized chat with the model or chat with a single note or chat with the entire vault. So BMO is one where it will allow you to chat with a single note and also the you know, your general models that are on there for general information. But it will not do the entire vault. So you will have to this one here is a little bit different than the other. to which actually indexes your entire vault, pulls that information into its database, and then you can do Q&A on the actual information within your vault. And it should be able to respond and answer questions based on the information in your vault. And as I mentioned, Connolly is one where it it is mainly used within the canvas. So I'm not going to show that one at all in this particular video. So with that said, let's get to the first step that you're going to need to do in order to get any of this working is you're going to have to install a solution so that you can run your models locally as i mentioned i downloaded olama it's pretty easy to just go to the site olama.com and download the version that works for your particular system this is pretty easy so i just went and hit download download it for my mac it worked well with again the m series processor and i have an m1 pro in my mac macbook here download it you go through the install process if you take a different route you can also go to their github page and they have tons of information on what you can do to get it installed and working on the github page so once you have that service installed and running you can either work with it on the command line or if you go up to the like on the mac once you when you install it and get it set up you'll see that there's a little icon at the top and you can actually see that it's running from here. If it has quit, that means it's running. You can see in this particular case, I have an update to this here. So I'm going to actually just go ahead and run the update real quick and we'll get to the next steps. And also, if the service is running here for a llama, you can also go to the URL, your local host and then one one four three four. And if you see this a llama running, then you know it's running here. So let me go ahead and refresh and this one is running on this particular system. So the next step is really after you get this running, you just go into you can go into Obsidian. But I would suggest you you before you do that, maybe you just go ahead and you start pulling down your models that you're going to want to use. And the ones that I use were, as I mentioned, Ollama 2. Those seem to be the ones that are supported by most of the plugins that are out there. And I did Ollama 3 and then Mestral. So in order to do that, you can follow the instructions as they are on the GitHub page. Let's see if they have it up above, down below here. So you can just do an Ollama pull, Ollama pull, and then whatever the actual model you want to pull down. And that will work. If you want to see what you already have installed, you can just do Ollama list and that will show you the ones that you've already downloaded once they are downloaded. And yeah, it's pretty simple to get this stuff set up. It's amazing how how simple getting an AI solution installed on a commercial piece of hardware is these days. So, yeah, this part should be pretty easy here. OK, so once you get your models downloaded and pulled. Then let's go into Obsidian. And what we'll do is we'll go into the plugins area and what you're going to want to download the actual plugins. So I already have the plugins downloaded. Let me go to the smart. What I'm going to do is go to the smart one here, the smart second brain plugin. And I am going to go ahead and clear this one so that it's. starts out fresh here i want to start completely fresh with this one so you can see the full setup so i just cleared it it's set up and enabled when you do set this up you may want to exclude some files and folders so you do have the option to do that now by default it does not do its auto start when you leverage the smart second brain because it does that indexing it you will have to start it up in order for you to actually use it so it probably makes more sense to start this up automatically instead of having it be manual but unless you don't want to have it doing the indexing automatically when you use this then you can keep that to where it's off i'm going to go ahead and leave it on the plugin as i mentioned allows you to do local a local solution that you pointed to and that can be used completely offline which is what we're going to do or you can actually point this to a third-party service as well so we're going to go back and run this on our machine here i have it pointed to this local ur url as you saw when i went to that in the browser it showed that olama was running so it's pointing to that and then we can go ahead and see the list of models that it recommends so these are the ones that it recommends just because they're in this list up here does not mean that they're actually installed so if you don't have it installed when you do that olama list then you will have to install it now I have some additional ones installed that it can see here at the bottom. So if I come down here, I want me want to select, say, Lama three and use that one there. And then in here, this will also show the ones that you have downloaded. If you have anything different from what it recommends. Now, on the recommended one, I have both of these installed. There is one that apparently works best. I think it's in the GitHub page where they put some notes to plug in providers. But notes on which models work best during their testing but i think it's somewhat of a you know this is all somewhat of a guessing game when you set it up you will have to kind of play with it and see how things work for your particular use case so i'm going to leave it to be llama 3 and then we'll leave it as the embed text here and we'll keep the default settings for everything else now if we come back over here to the to obsidian and look at the plug-in here so we can see it shows up here on the side and it starts with a setup process this is i think this one is the only one that actually comes up and and has this little setup window outside of this setup that's already in the plug-in area but let's go ahead and walk through it anyway now when you go through they give you some instructions here on just setting some things up and the first thing you want to do is it says download the application we've already done that And then it says extract the zip and start Ollama. Ollama has already started. And then we can come in here and we can test and see if it's running. So if we hit test, it should see it running because we verified that earlier within the process. And it is running. I've already run some of these commands because I had it running before. But what you'll want to do is copy this particular piece here that it shows where you would set up this in your environment. And. Once you hit enter on this here will set up your environment the way that it needs So when you hit this when you start the restart the the olama service it will allow obsidian uh to leverage the service now let's go ahead and test it again it sees that this the test was had worked so we can actually move on to the next step where it says to install the embedding model now this is going to be used for the indexing of your your vault so as i mentioned before some of it's just you know trial and error go ahead and pick the one that it has in here for its recommended if you already have it installed you know you you're already at this step you're done but if you don't have it installed if you don't did not pre-do this then go ahead and install it from here or you can go back to the command prompt and do the install typing olama pull and then you can just copy paste this out if you want to do it that way now now that it's already installed we can come here and we can select the model that we want to use we'll keep it at this one here and then we'll go ahead and start the second brain and now it will actually start the indexing in here and we can see that it's going to take about 11 minutes from what it says there i'm going to delete some old information that it had in the chat now this varies the indexing of the vault you will see it go from 11 to 9. you might actually see this be higher depending on what model you chose to do the indexing here and use so be prepared for this to take a while and and wait for it to finish because it will not let you use this this plugin until the indexing is done. And to even try that out, let's come in here. We have the indexing going. It's chatting with the notes. Now I can turn this off, but if I turn this off, it's supposed to not have to worry about the the actual vault or any of the information within the indexing. I've had it where this did not work. Let's see if it will work here. Let's go ahead and. type in their high and now it says it's still indexing so it wants us to wait so we'll go ahead and wait and we will definitely speed through this here we'll let this go and i will fast forward through the video okay so it looks like the indexing is done and let's go ahead and turn it back on so it uses the the actual vault for q a and we can see here at the top here it shows olama it has the actual model we're using They have this comfy view setting where you can change the look of the chat window. And it doesn't really change it too much, but you can either have the bubble or just one stacked on top of the other. You can change the language. Looks like they support multiple languages there. And then you can set up what the creativity and the similarity of the information within your notes are. Let me tweak this down to change this to 50. There is... Let me see if there anything in the help. No, it has a question mark there, but it doesn't look like you can actually click on the help. Same for sensitivity. Yeah, it doesn't look like you can actually change any of these here. So I'm going to bump the creativity down to 50 and leave the similarity at 75. And first we can just do a general question like how far is the moon from the Earth? And this should want it does check. You can see it had a message where it doesn't have any notes that relates to that particular question that it could find. But this is information that should just be easily pulled from the the model. So what we came back with here. So we did ask about the how far is the moon from the Earth. So the moon's distance from the Earth. So according to our existing knowledge in Obsidian. And it gives you a link to a file and this file does not exist. And I've seen this a lot using this plug in where it just will randomly say that there's information within the vault and give you this link as if you can click that link and then go to that particular file. When this file does not exist at all. So if I was to just click on this, just do it just to show you it creates a new file similar to what a link would do if you did the same thing and I know and it's empty. So I'm not sure why that it does this, but yeah, it is a little bit confusing there. And it's obviously a response that is not completely true there. But it does go into giving you the information about the average distance from the center of the moon to the center of the Earth. And then it gives you the actual number and the distance, which is correct. So if we check with one of the other models when we run this we'll see that it is a similar or the same from one of these other models and we can actually even you know we could be running offline now i am not let's go to chat gpt And we will just ask the same question just to verify that the response is the same or similar. So as I mentioned before, it is the same response. So we are good at least there. But this extra information is not true. Now, if I want to pull something from information in my vault, you do have to ask these questions and be somewhat close to or detailed to what and help it. find what you're actually asking for. So like if I want to say how many, let me see, I'm not in the window here. So how many trips or vacations are planned this year? Let's see what it comes back with. Now I do have a note in there. that has information about travel and trips that me and my wife have taken or are planning to take and see what it comes back with. So it did not find anything related to what I wanted, but I know that the file is called, say, me and Kimberly's. And this is pretty small on the screen. So me and Kimberly. have trips planned this year what trips or vacations are and will even planned for this year or i'll even take off this year i'll just do planned on the question and let's see what comes back here One thing I don't like with this plugin is just particular this octopus that they have here I think it's in a way really think that this should just be put up top So it's out of the way here and we can see here. It did not Find anything related. So let me open up. I will actually open up the file Vacation All right, so let's see if, try this again. What trips or vacation trips, what vacation trips are planned for me and Kimberly? And I do want to put this year. Try not to be too vague. But it should be able to pick out. So we have the trips and vacations in here. I do have it where it says planned next to the actual trips that are planned. So I want to see if it'll recognize what I'm putting in here. And we can see it actually found something this time based on the question, the way I posed this question. And it came back with two planned and one tentative. So it. eventually did come back with the response that I wanted to see. But you again, you have to get that question right. And maybe if I come in here and I dial down the similarity a little bit, maybe I put it to 70. So if I put it down to 70 and maybe ask this question here again, let me see if it will, how it will respond. all right so it found the file again this time lowering the similarity a bit so yeah okay so the responses are coming back correctly now it would be nice too if we had like the the thumbs up or the thumbs down on this plugin so it knows whether a response was good or bad um and yeah i think that would that would really help now i can also prompt it and tell it that whether it was wrong or not but having that quality of life feature in there where you can do that would be nice so this is just you know a couple things here let me what i'm going to do in the next one let's take a look at copilot which is the next closest plugin to how this plugin works where it allows you to index your entire vault and then post q a to it we will take a look at copilot so let me so let's go back in we will do an install Browse Copilot, Copilot, we will install, we will enable. Let's go into our options here and let's go into one, the embedding one. I mentioned I'm going to change this to Ollama and let's go all the way down to the bottom. And I'm going to change this to use Ollama 3. all right so that is all set and let's come in here and let's start this plug-in again and we see that it is still not indexing let's choose we got gpt4 we got a llama local and what we want to do is do the vault qa so let's click on the vault qa and it is set to ask questions here so refresh the index button below got refresh a new chat but i want to what i want to do is reset everything now the settings have been reset and one one key thing i did forget here is that i forgot to to actually save and and um save and reload here when I made that change below. So let me come in here and I'm going to make these changes here. Save load. Okay, we got that there. We save and reload. Okay, now we can see that something's changed over here. All right. Default model. Let's go ahead and do we want that to be the local Olamar right here. And we should that should just be it right there. So save and reload. okay plugin reloaded successful settings have been saved okay so now if we come over here we have local olama local we have it set to chat let's do vault and it still is not refreshing the code product vault is up to date so it says it's already up to date so not entirely sure how to force it other than let's go in here and we change one of the models so instead of say three if i use two and we save and reload okay so everything has changed there come back in here then we come in here and default qa and we select it and no so it still has the two there oh um actually i've changed i want to change the embedding one so let me come in and let's put the one back to three there then the embedding there we go this one here and i'm going to put it where it was by default and let's see if that helps don't forget to save and reload from here and it defaults to chat so i'm going to have to change that vault one every time and it still is not indexing So it looks like everything is going. Yeah, it should actually come up when it does it index. It typically comes up and it has a little counter up here and then it will show that it's doing the index. But we can see right now that it's not actually doing that. So it has not done a re index on this here. So again, I'm not sure exactly how the forced is. So I guess we're going to have to skip that particular piece. but it does it will go through an indexing process similar to the the smart second brain plugin so expect that to happen and expect it to take some time for it to do it but let's go ahead and i want to just put this back to be the same as what we had on the other setup so we have that embedding text model and we have this one set to three okay so all that's set up so we've saved it and we have the default chat so what's the distance what i have in there what's the distance from the moon to the earth but it comes back with the correct answer here and let's Also let's give it this question here so first i'm going to give it the one you see i had yeah so this is the question that i gave it that if i remember it did not answer correctly the first time what trips are planned for me currently this year first time i ran this one yeah okay so it's going to copilot we'll run this one first we want to change this to the vault And let's run that question here. We'll send it and let's see if it comes back. Now I do have the page actually open this time where I did not have it open previously. So I'm not totally sure that that matters. It should not let this thing go and see what comes back. From my testing, I have seen that the responses that come back seem to be a bit better from Copilot. So let's see if it proves me right or wrong here. OK, so it did come back. And the first try, even with the question that I sent to to it the first time, the first time in the smart second brain where it did not answer it correctly. This time, the first time it responded back and it actually gave me a link to the source here. It gives me a couple of different sources here. I'm not even sure if any of these here are correct, but it has one here for 408 2021. But in either case, this is what I wanted to see. And it did respond back with that. And it did it on the first on the first try there. So again, it's this one here. You're still going to have to do a little bit of trial and error and making sure you're getting your questions detailed enough to get this stuff right. But it seems like out of the box, this one here is a little bit more home to find the and respond back with information that's not completely false or just completely out of the ballpark. Now, the other one that that I did test was BMO plugin, and it's more of just a chat bot, but it can actually talk to a single note. So there is no indexing on this one here. And yeah, it pretty much works. well out of the box for what it's for you would open up the actual note that you want to glean information from and have the AI kind of go over or you can just ask it you know just questions out of the box so like what's the distance between them All right, so it comes back with that response. And I'm not going to paste the same question that I did on the other models where I had to have a lot of information in it. I'm just going to ask it what trips are planned this year. And we'll see what it comes back with. So I'm a little more vague, but. because it's set up to look in this note, it should find what what I need without me having to be more detailed in my question. And we can see from this response, I was waiting to see if it would give more information here, but it did not come back with what I wanted. So we go ahead and we go ahead and paste exactly what I put in the previous prompts. And let's see what comes back here. So I see it's still not finding it. And I wonder if it's because this is not in Markdown. I'm going to do it, put this file in Markdown and let's give it this question we had up top. and try this again i think having it in that mind map format probably did not open out so the file does have a current file for reference because okay there we go so now at least coming back with some information from this file and based on information provided in the in the image which an actual image so i'm not sure why it says it's an image it what's funny is when it wasn't an image or it was a mind map it didn't respond back at all with anything related to this so it says planned trip hawaii planned trip but take a look at this here and we can see that these are not true so it's saying that in 2024 sea lakes that i have something planned where Cedar Lakes is not planned. We can see that Miami's planned. We got Wisconsin is planned. But so this is not correct here. So let's we'll clear this one and let's try this again. What trips are planned this year for me and Kimberly? Try it this way. Okay, so it did respond back and tell me that I had one scheduled to Wisconsin in July. But, yeah, it's... So you can see it's a little bit hit or miss here with this particular plugin as well. So trying to get these things to work as good as say an open AI API or chat GPT is a bit hard to get these things kind of trained on the information that you're trying to get back. I'm sure if I set up some prompts and say okay maybe tell it what the format of this file is what it's for and what plant means and so on and so forth that it will eventually make it easier for it to answer these questions but yeah this is just straight out of the box let's do a couple things here just to see how they work but they they're good for getting the general information out of the out of the model like the distance to the moon uh maybe what who was the president and you know 2020 so on and so forth those type of factual questions I think it's pretty easy it'll respond back and get those but taking information out of your vault where it may not be formatted in a way that makes it easy for it to understand the questions that you're asking it is something that I think needs to be worked on here and given time and repetition of me using one of these then I think it'll probably give me back responses a little bit better or I will get trained as well as it on how to Ask it better questions so that it can give me better answers. But out of these three plugins here I do like the copilot one. I think it's a bit faster on the index process, but that was a long video And all of this again could be done offline. I am connected right now We saw that I tested something with chat GPT from a question standpoint Just verifying to make sure the question was the same but technically I could uninstall or unplug myself from the Internet. And this will still work because everything is working locally on this particular laptop. And I think I'm going to make that something that I do for these particular use cases. I can have it to where I do. I have the offline setup running all the time, but in other use cases have the open AI set up to where I am reaching out to the the Internet and I'm actually getting stuff from. say the api that's out on the on the internet so what's really good here with these plugins is that i can kind of mix and match and set up and configure things the way i need it to be set up and i really love that flexibility so i'm going to keep playing around with this here even more there's cannoli which is another beast in itself which looks like it has a lot of potential use cases for so i'm going to dig into that and yeah probably have some feedback for you guys once i'm done working with some of this other these other um plugins like cannoli and also using and leveraging these local models a bit more with obsidian and with other use case other use cases all right well i hope you like the information that you found in this video if you did don't forget to like and subscribe to the channel and until the next time have a nice day

Now, why would you want to do this? So let's get into some of the reasons why you would probably want to use an AI solution. One, because it can be offline. So you don't need the internet in order to do it. If you're sitting at on an airplane and you want to leverage this particular solution, it will work doing it that way.

Your Your data has increased privacy because you're not sending it outside of your network. There's the low cost of utilizing your own private instance of artificial intelligence and it's dedicated to you so you don't have to worry about other people bogging down your performance when you're trying to do any Q&A or analyze any of the information that you're trying to get out of your artificial intelligence. So what are the different scenarios that you might want to use this? within the context of leveraging it with Obsidian. So there is the local note where you might just want to ask Q&A on a single note within your vault.

There's the Q&A scenario where you're doing it on your entire vault. So when you do a question, it goes through your entire vault, all the information in your entire vault and leverages that for context. And then there's just using the default, your plain vanilla type of Q&A, where the... the model has all the information or as much information as it could have pulled from external sources like the internet and it has a lot of different facts and information within the model to where if you're trying to learn a certain topic or just get information like what's the distance from the earth to the moon you can easily answer those questions to get those questions answered leveraging ai so what can you use to run ai locally on your machine Now there are a few different solutions.

I was actually surprised to see how many solutions there were out there that you could use. But I wanted to make sure I framed this and kind of kept this in a box to where they worked with Obsidian. So the plugins that I was looking at leveraging with Obsidian could easily be configured to work with these particular solutions that created local instances that could be leveraged for AI and you can run your models. And the ones that I found out there was Ollama.

there was lm studio uh git for gtp for all and um another one like jan.io again there were the list could be a lot bigger but i just wanted to get a small list of ones that could be easily installed on your and set up on your local system i am on a mac so but these solutions here at least olama i know will work and is supported on mac windows and linux as long as you have the hardware to support it and that's the the pretty much the big key is that you will need to have hardware that can run just these local models on your system and actually be performant so if you're going to run this you don't want to have some old system that is going to take you know a long time to give you back responses and the ones that i saw are pretty much supported out of the boxes on a mac system it should be one of the M1 variants. So as long as you have an M series, I should say, not just the M1, but if you have one of the M series, M1, 2, or 3, then this should work no problem because they have really good GPUs in them. And then if you have a system like a Windows system or a Linux system that has GPUs in them, then those should work as well.

So then there's the question of which ones are easy to install and get running. your machine on your local system and is supported with obsidian and the one that I've tested is Oh llama there are other ones like I've seen LM studio be one that comes up in some of these plugins as supported for those particular plugins so I chose to go with Oh llama I did not actually install LM studio so we're a little bit biased here and I have no feedback on the LM studio route now how many models are out there that you can actually use and after looking there are too many really there are a lot of different models that you can leverage for this particular use case so olama supports many different models that you can download and leverage with it as well as LM studio so really I think this we're at a time where if you have the hardware to support this the you know the world is your oyster you can pretty much go out there and play around with any of these models and see how they work for your particular use case now you might also be asking well which models should you actually use in order to do this and to be honest there because there are so many I kept my my test bed a little bit small so that i wasn't pulling down too many models and trying to test this so uh llama 2 llama 3 mistral those are really the ones that i focused on you know getting some time with those three different ones doing the indexing and somewhat seeing you know how how they responded to some of my questions how long it took to do some indexing i don't have legitimate like say performance metrics from one to the other but i will say the you know all of them were pretty much along the same line of a from a performance standpoint and yeah you can't really go wrong just know that the more information you have in your in your vault the longer it's going to take to index the the information and get it into the databases that uh the the models are going to use when you're doing your q a on these on on your information So as far as the different plugins that are out there and available, again there are a ton of different plugins that you could use. I stuck with just four of them and really three of them are were used during the test and are say comparable to each other. Cannoli is somewhat of a different beast where it's leveraged in the canvas and it can do a lot more things than the other plugins that you have that are really more Q&A type chat bots.

So the copilot the bmo and the smart second brain those are all pretty much the same where you would pose a question or ask for information and kind of you know work with it the same way you would with chat gpt and it will give you responses based off what it understands or the information it has within the model now the plugins do support different different things here and with some of these plugins they some of them support local and remote And many of them do support both and others support whether you can just do your generalized chat with the model or chat with a single note or chat with the entire vault. So BMO is one where it will allow you to chat with a single note and also the you know, your general models that are on there for general information. But it will not do the entire vault. So you will have to this one here is a little bit different than the other.

to which actually indexes your entire vault, pulls that information into its database, and then you can do Q&A on the actual information within your vault. And it should be able to respond and answer questions based on the information in your vault. And as I mentioned, Connolly is one where it it is mainly used within the canvas. So I'm not going to show that one at all in this particular video. So with that said, let's get to the first step that you're going to need to do in order to get any of this working is you're going to have to install a solution so that you can run your models locally as i mentioned i downloaded olama it's pretty easy to just go to the site olama.com and download the version that works for your particular system this is pretty easy so i just went and hit download download it for my mac it worked well with again the m series processor and i have an m1 pro in my mac macbook here download it you go through the install process if you take a different route you can also go to their github page and they have tons of information on what you can do to get it installed and working on the github page so once you have that service installed and running you can either work with it on the command line or if you go up to the like on the mac once you when you install it and get it set up you'll see that there's a little icon at the top and you can actually see that it's running from here.

If it has quit, that means it's running. You can see in this particular case, I have an update to this here. So I'm going to actually just go ahead and run the update real quick and we'll get to the next steps. And also, if the service is running here for a llama, you can also go to the URL, your local host and then one one four three four. And if you see this a llama running, then you know it's running here.

So let me go ahead and refresh and this one is running on this particular system. So the next step is really after you get this running, you just go into you can go into Obsidian. But I would suggest you you before you do that, maybe you just go ahead and you start pulling down your models that you're going to want to use. And the ones that I use were, as I mentioned, Ollama 2. Those seem to be the ones that are supported by most of the plugins that are out there.

And I did Ollama 3 and then Mestral. So in order to do that, you can follow the instructions as they are on the GitHub page. Let's see if they have it up above, down below here.

So you can just do an Ollama pull, Ollama pull, and then whatever the actual model you want to pull down. And that will work. If you want to see what you already have installed, you can just do Ollama list and that will show you the ones that you've already downloaded once they are downloaded.

And yeah, it's pretty simple to get this stuff set up. It's amazing how how simple getting an AI solution installed on a commercial piece of hardware is these days. So, yeah, this part should be pretty easy here.

OK, so once you get your models downloaded and pulled. Then let's go into Obsidian. And what we'll do is we'll go into the plugins area and what you're going to want to download the actual plugins. So I already have the plugins downloaded.

Let me go to the smart. What I'm going to do is go to the smart one here, the smart second brain plugin. And I am going to go ahead and clear this one so that it's.

starts out fresh here i want to start completely fresh with this one so you can see the full setup so i just cleared it it's set up and enabled when you do set this up you may want to exclude some files and folders so you do have the option to do that now by default it does not do its auto start when you leverage the smart second brain because it does that indexing it you will have to start it up in order for you to actually use it so it probably makes more sense to start this up automatically instead of having it be manual but unless you don't want to have it doing the indexing automatically when you use this then you can keep that to where it's off i'm going to go ahead and leave it on the plugin as i mentioned allows you to do local a local solution that you pointed to and that can be used completely offline which is what we're going to do or you can actually point this to a third-party service as well so we're going to go back and run this on our machine here i have it pointed to this local ur url as you saw when i went to that in the browser it showed that olama was running so it's pointing to that and then we can go ahead and see the list of models that it recommends so these are the ones that it recommends just because they're in this list up here does not mean that they're actually installed so if you don't have it installed when you do that olama list then you will have to install it now I have some additional ones installed that it can see here at the bottom. So if I come down here, I want me want to select, say, Lama three and use that one there. And then in here, this will also show the ones that you have downloaded.

If you have anything different from what it recommends. Now, on the recommended one, I have both of these installed. There is one that apparently works best. I think it's in the GitHub page where they put some notes to plug in providers. But notes on which models work best during their testing but i think it's somewhat of a you know this is all somewhat of a guessing game when you set it up you will have to kind of play with it and see how things work for your particular use case so i'm going to leave it to be llama 3 and then we'll leave it as the embed text here and we'll keep the default settings for everything else now if we come back over here to the to obsidian and look at the plug-in here so we can see it shows up here on the side and it starts with a setup process this is i think this one is the only one that actually comes up and and has this little setup window outside of this setup that's already in the plug-in area but let's go ahead and walk through it anyway now when you go through they give you some instructions here on just setting some things up and the first thing you want to do is it says download the application we've already done that And then it says extract the zip and start Ollama.

Ollama has already started. And then we can come in here and we can test and see if it's running. So if we hit test, it should see it running because we verified that earlier within the process. And it is running. I've already run some of these commands because I had it running before.

But what you'll want to do is copy this particular piece here that it shows where you would set up this in your environment. And. Once you hit enter on this here will set up your environment the way that it needs So when you hit this when you start the restart the the olama service it will allow obsidian uh to leverage the service now let's go ahead and test it again it sees that this the test was had worked so we can actually move on to the next step where it says to install the embedding model now this is going to be used for the indexing of your your vault so as i mentioned before some of it's just you know trial and error go ahead and pick the one that it has in here for its recommended if you already have it installed you know you you're already at this step you're done but if you don't have it installed if you don't did not pre-do this then go ahead and install it from here or you can go back to the command prompt and do the install typing olama pull and then you can just copy paste this out if you want to do it that way now now that it's already installed we can come here and we can select the model that we want to use we'll keep it at this one here and then we'll go ahead and start the second brain and now it will actually start the indexing in here and we can see that it's going to take about 11 minutes from what it says there i'm going to delete some old information that it had in the chat now this varies the indexing of the vault you will see it go from 11 to 9. you might actually see this be higher depending on what model you chose to do the indexing here and use so be prepared for this to take a while and and wait for it to finish because it will not let you use this this plugin until the indexing is done.

And to even try that out, let's come in here. We have the indexing going. It's chatting with the notes.

Now I can turn this off, but if I turn this off, it's supposed to not have to worry about the the actual vault or any of the information within the indexing. I've had it where this did not work. Let's see if it will work here.

Let's go ahead and. type in their high and now it says it's still indexing so it wants us to wait so we'll go ahead and wait and we will definitely speed through this here we'll let this go and i will fast forward through the video okay so it looks like the indexing is done and let's go ahead and turn it back on so it uses the the actual vault for q a and we can see here at the top here it shows olama it has the actual model we're using They have this comfy view setting where you can change the look of the chat window. And it doesn't really change it too much, but you can either have the bubble or just one stacked on top of the other.

You can change the language. Looks like they support multiple languages there. And then you can set up what the creativity and the similarity of the information within your notes are.

Let me tweak this down to change this to 50. There is... Let me see if there anything in the help. No, it has a question mark there, but it doesn't look like you can actually click on the help.

Same for sensitivity. Yeah, it doesn't look like you can actually change any of these here. So I'm going to bump the creativity down to 50 and leave the similarity at 75. And first we can just do a general question like how far is the moon from the Earth? And this should want it does check.

You can see it had a message where it doesn't have any notes that relates to that particular question that it could find. But this is information that should just be easily pulled from the the model. So what we came back with here. So we did ask about the how far is the moon from the Earth. So the moon's distance from the Earth.

So according to our existing knowledge in Obsidian. And it gives you a link to a file and this file does not exist. And I've seen this a lot using this plug in where it just will randomly say that there's information within the vault and give you this link as if you can click that link and then go to that particular file. When this file does not exist at all. So if I was to just click on this, just do it just to show you it creates a new file similar to what a link would do if you did the same thing and I know and it's empty.

So I'm not sure why that it does this, but yeah, it is a little bit confusing there. And it's obviously a response that is not completely true there. But it does go into giving you the information about the average distance from the center of the moon to the center of the Earth. And then it gives you the actual number and the distance, which is correct. So if we check with one of the other models when we run this we'll see that it is a similar or the same from one of these other models and we can actually even you know we could be running offline now i am not let's go to chat gpt And we will just ask the same question just to verify that the response is the same or similar.

So as I mentioned before, it is the same response. So we are good at least there. But this extra information is not true.

Now, if I want to pull something from information in my vault, you do have to ask these questions and be somewhat close to or detailed to what and help it. find what you're actually asking for. So like if I want to say how many, let me see, I'm not in the window here. So how many trips or vacations are planned this year?

Let's see what it comes back with. Now I do have a note in there. that has information about travel and trips that me and my wife have taken or are planning to take and see what it comes back with.

So it did not find anything related to what I wanted, but I know that the file is called, say, me and Kimberly's. And this is pretty small on the screen. So me and Kimberly.

have trips planned this year what trips or vacations are and will even planned for this year or i'll even take off this year i'll just do planned on the question and let's see what comes back here One thing I don't like with this plugin is just particular this octopus that they have here I think it's in a way really think that this should just be put up top So it's out of the way here and we can see here. It did not Find anything related. So let me open up. I will actually open up the file Vacation All right, so let's see if, try this again. What trips or vacation trips, what vacation trips are planned for me and Kimberly?

And I do want to put this year. Try not to be too vague. But it should be able to pick out.

So we have the trips and vacations in here. I do have it where it says planned next to the actual trips that are planned. So I want to see if it'll recognize what I'm putting in here. And we can see it actually found something this time based on the question, the way I posed this question. And it came back with two planned and one tentative.

So it. eventually did come back with the response that I wanted to see. But you again, you have to get that question right. And maybe if I come in here and I dial down the similarity a little bit, maybe I put it to 70. So if I put it down to 70 and maybe ask this question here again, let me see if it will, how it will respond. all right so it found the file again this time lowering the similarity a bit so yeah okay so the responses are coming back correctly now it would be nice too if we had like the the thumbs up or the thumbs down on this plugin so it knows whether a response was good or bad um and yeah i think that would that would really help now i can also prompt it and tell it that whether it was wrong or not but having that quality of life feature in there where you can do that would be nice so this is just you know a couple things here let me what i'm going to do in the next one let's take a look at copilot which is the next closest plugin to how this plugin works where it allows you to index your entire vault and then post q a to it we will take a look at copilot so let me so let's go back in we will do an install Browse Copilot, Copilot, we will install, we will enable.

Let's go into our options here and let's go into one, the embedding one. I mentioned I'm going to change this to Ollama and let's go all the way down to the bottom. And I'm going to change this to use Ollama 3. all right so that is all set and let's come in here and let's start this plug-in again and we see that it is still not indexing let's choose we got gpt4 we got a llama local and what we want to do is do the vault qa so let's click on the vault qa and it is set to ask questions here so refresh the index button below got refresh a new chat but i want to what i want to do is reset everything now the settings have been reset and one one key thing i did forget here is that i forgot to to actually save and and um save and reload here when I made that change below.

So let me come in here and I'm going to make these changes here. Save load. Okay, we got that there.

We save and reload. Okay, now we can see that something's changed over here. All right.

Default model. Let's go ahead and do we want that to be the local Olamar right here. And we should that should just be it right there. So save and reload.

okay plugin reloaded successful settings have been saved okay so now if we come over here we have local olama local we have it set to chat let's do vault and it still is not refreshing the code product vault is up to date so it says it's already up to date so not entirely sure how to force it other than let's go in here and we change one of the models so instead of say three if i use two and we save and reload okay so everything has changed there come back in here then we come in here and default qa and we select it and no so it still has the two there oh um actually i've changed i want to change the embedding one so let me come in and let's put the one back to three there then the embedding there we go this one here and i'm going to put it where it was by default and let's see if that helps don't forget to save and reload from here and it defaults to chat so i'm going to have to change that vault one every time and it still is not indexing So it looks like everything is going. Yeah, it should actually come up when it does it index. It typically comes up and it has a little counter up here and then it will show that it's doing the index. But we can see right now that it's not actually doing that.

So it has not done a re index on this here. So again, I'm not sure exactly how the forced is. So I guess we're going to have to skip that particular piece. but it does it will go through an indexing process similar to the the smart second brain plugin so expect that to happen and expect it to take some time for it to do it but let's go ahead and i want to just put this back to be the same as what we had on the other setup so we have that embedding text model and we have this one set to three okay so all that's set up so we've saved it and we have the default chat so what's the distance what i have in there what's the distance from the moon to the earth but it comes back with the correct answer here and let's Also let's give it this question here so first i'm going to give it the one you see i had yeah so this is the question that i gave it that if i remember it did not answer correctly the first time what trips are planned for me currently this year first time i ran this one yeah okay so it's going to copilot we'll run this one first we want to change this to the vault And let's run that question here.

We'll send it and let's see if it comes back. Now I do have the page actually open this time where I did not have it open previously. So I'm not totally sure that that matters. It should not let this thing go and see what comes back. From my testing, I have seen that the responses that come back seem to be a bit better from Copilot.

So let's see if it proves me right or wrong here. OK, so it did come back. And the first try, even with the question that I sent to to it the first time, the first time in the smart second brain where it did not answer it correctly. This time, the first time it responded back and it actually gave me a link to the source here. It gives me a couple of different sources here.

I'm not even sure if any of these here are correct, but it has one here for 408 2021. But in either case, this is what I wanted to see. And it did respond back with that. And it did it on the first on the first try there. So again, it's this one here.

You're still going to have to do a little bit of trial and error and making sure you're getting your questions detailed enough to get this stuff right. But it seems like out of the box, this one here is a little bit more home to find the and respond back with information that's not completely false or just completely out of the ballpark. Now, the other one that that I did test was BMO plugin, and it's more of just a chat bot, but it can actually talk to a single note. So there is no indexing on this one here. And yeah, it pretty much works.

well out of the box for what it's for you would open up the actual note that you want to glean information from and have the AI kind of go over or you can just ask it you know just questions out of the box so like what's the distance between them All right, so it comes back with that response. And I'm not going to paste the same question that I did on the other models where I had to have a lot of information in it. I'm just going to ask it what trips are planned this year.

And we'll see what it comes back with. So I'm a little more vague, but. because it's set up to look in this note, it should find what what I need without me having to be more detailed in my question. And we can see from this response, I was waiting to see if it would give more information here, but it did not come back with what I wanted. So we go ahead and we go ahead and paste exactly what I put in the previous prompts.

And let's see what comes back here. So I see it's still not finding it. And I wonder if it's because this is not in Markdown. I'm going to do it, put this file in Markdown and let's give it this question we had up top. and try this again i think having it in that mind map format probably did not open out so the file does have a current file for reference because okay there we go so now at least coming back with some information from this file and based on information provided in the in the image which an actual image so i'm not sure why it says it's an image it what's funny is when it wasn't an image or it was a mind map it didn't respond back at all with anything related to this so it says planned trip hawaii planned trip but take a look at this here and we can see that these are not true so it's saying that in 2024 sea lakes that i have something planned where Cedar Lakes is not planned.

We can see that Miami's planned. We got Wisconsin is planned. But so this is not correct here.

So let's we'll clear this one and let's try this again. What trips are planned this year for me and Kimberly? Try it this way. Okay, so it did respond back and tell me that I had one scheduled to Wisconsin in July.

But, yeah, it's... So you can see it's a little bit hit or miss here with this particular plugin as well. So trying to get these things to work as good as say an open AI API or chat GPT is a bit hard to get these things kind of trained on the information that you're trying to get back.

I'm sure if I set up some prompts and say okay maybe tell it what the format of this file is what it's for and what plant means and so on and so forth that it will eventually make it easier for it to answer these questions but yeah this is just straight out of the box let's do a couple things here just to see how they work but they they're good for getting the general information out of the out of the model like the distance to the moon uh maybe what who was the president and you know 2020 so on and so forth those type of factual questions I think it's pretty easy it'll respond back and get those but taking information out of your vault where it may not be formatted in a way that makes it easy for it to understand the questions that you're asking it is something that I think needs to be worked on here and given time and repetition of me using one of these then I think it'll probably give me back responses a little bit better or I will get trained as well as it on how to Ask it better questions so that it can give me better answers. But out of these three plugins here I do like the copilot one. I think it's a bit faster on the index process, but that was a long video And all of this again could be done offline.

I am connected right now We saw that I tested something with chat GPT from a question standpoint Just verifying to make sure the question was the same but technically I could uninstall or unplug myself from the Internet. And this will still work because everything is working locally on this particular laptop. And I think I'm going to make that something that I do for these particular use cases. I can have it to where I do. I have the offline setup running all the time, but in other use cases have the open AI set up to where I am reaching out to the the Internet and I'm actually getting stuff from.

say the api that's out on the on the internet so what's really good here with these plugins is that i can kind of mix and match and set up and configure things the way i need it to be set up and i really love that flexibility so i'm going to keep playing around with this here even more there's cannoli which is another beast in itself which looks like it has a lot of potential use cases for so i'm going to dig into that and yeah probably have some feedback for you guys once i'm done working with some of this other these other um plugins like cannoli and also using and leveraging these local models a bit more with obsidian and with other use case other use cases all right well i hope you like the information that you found in this video if you did don't forget to like and subscribe to the channel and until the next time have a nice day

Transcript for:Integração de IA no Obsidian

Transcript for:
Integração de IA no Obsidian