This unit on natural language processing seems like as good a time as any to talk about incorporating large language models into workflows. Now, this is something that um a lot of you have been talking about in the discussion forums. Um but I wanted to take a look at how this happens technically and maybe show you some of the school the tools that uh help you with the representation of the process and with the actual execution of the process. So this this um this discussion comes from a um a paper that I read recently on um on this very thing on on uh large language models and automatic workflow generation. So this is a paper in the um ACM digital library. ACM is the association for computing machinery. uh they have often you know fairly technical articles about you know issues having to do with management. So I read in there quite a bit. This is from 2023 um and it talks about using uh a specialurpose large language model to generate a workflow. Now the capabilities of large language models have gotten much more advanced in just the last couple of years. So we don't even need a specialurpose large language model now to do this very thing. And I'm going to show you uh how this works. So let's say you know every business regardless of the size has business processes. Um so here are some examples of just business processes. Depending on the size of the organization they'll have more or less level of you know u formality and documentation. Um but there are processes everywhere. So things like invoice processing, employee onboarding, customer support ticket routing, sales lead qualification, some kind of compliance reporting, marketing campaign approval or an IT access request. So these are these are processes and whether you know if you may not even have them very formalized. They're just things that you do. And nowadays, there are these tools, these so-called process mining tools that will go through your logs and say, "Wait, you actually do have a process here, and I'm going to go ahead and document it for you based on the events of what you've actually done." So, let's take a look. Let's just take an example, and we'll we'll do it very generically and say we would like to describe and formalize our process of how we process invoices. You know, of course, there multiple steps on this where you have to receive the invoice and that's kind of the, you know, the inciting event or the trigger event. And, you know, you have to uh, you know, check to make sure that this actually represents something that was approved. So, you have to access the purchase order. Uh, make sure that it was authorized. Uh, and then you have to set up the payment. So let's just take a look at this and say, well, can can an LLM give us a pretty good workflow? Now, a workflow is just a formal representation of a process. Um, okay. So, let's go to um I, you know, I I'm very fond of GPT, but for this particular task, I found that Claude works pretty well. So, I'm just going to say um generate a workflow for a um for invoice processing. Now, notice I I don't I didn't say what the invoices were or what process what steps I want, but let's just start off in the most generic way possible and see what it comes up with. Okay. So, all right. So, it it's giving me a text description of all of the steps. And this looks pretty good. This hits more than what I mentioned in the introduction. So, it talks about, okay, so there's the receipt. Uh, okay. So, you receive it a number of different possible ways. Um, you know, you have to use optical character recognition to pull the data off of it if it's a if it's a paper thing. Um, or you know, like a PDF, uh, that, you know, that doesn't have raw text in it. Um so then you have to verify that it's complete and legible. Okay. Step three is registration and coding. So if you have some kind of accounting system that you have to enter it into. Um so matches matching it with the purchase order and with a receipt to confirm that whatever it is you're paying for you've actually received those goods. U then this this approval workflow. Okay. So so Claude has laid out the steps of the process. So we have a you know workflow here. Okay. recordkeeping and reconciliation, reporting and analytics. So, so it it it this is probably more elaborate than most people need, but let's just say we're going to go with this. Um, I can now say, okay, I like this process, but I I want to have a a representation of it that's a little bit easier to read, kind of a flowchart. So, we use um so we're at this step here. So, we we've generated the steps of the workflow using the large language model. uh now we want to represent it using um a syntax which is called business process modeling notation. So this is something that we we teach to the the undergraduates as a as a way of of just visualizing a process so that it makes a lot of sense. And you've probably seen you know slightly different versions of this. The one that I like is what's called the swim lane diagram where each of these rows here is an agent. So some kind of functioning entity uh that's involved in the process. Uh and these activities are on these rectangles and the um the flows are actually these arrows. So you see that they so the customer buys the product and then this kind of wavy thing is uh some kind of document. Okay. So this is a nice way of visualizing the whole process. So you make sure you hit all the steps. So we can have this clawed large language model. Um um can you generate a business process modeling diagram of this workflow? Okay. Now the way cloud is set up is it when it when it creates artifacts it does it over here on the right. Um okay. So it it has created this kind of free text version of the okay and now it used that code to create the actual diagram. So right over here we have the the diagram itself. Okay. So we see that so the the initial event is the trigger. So invoice received um and then these you know so this this particular kind of process notation is just kind of a flowchart where the diamonds represent some choice. So the invoice is received uh and then you have to determine the invoice type and then based on that type you do certain things. Now now um okay so you know there are all these different decisions and activities and documents that have to be done. So, so if you find the purchase order, yes, then you have to make sure that it matches. So, in looking at a workflow like this, you can start to see, okay, anything that involves some kind of document processing. So, receiving an email and checking for something or writing an email or taking a document and summarizing it. So now we see these opportunities to have pieces of this workflow executed by the large language model. Now um so this is starting to get into what's called robotic process automation. That's kind of a strange term um because we often think of robots as something physical but in RPA the robots are really you know so if you think about a robot in a factory it's walking around and it's moving things and it's polishing things and it's picking things up instead of those activities you can think of robotic process automation as actually something virtual sitting at your machine doing the kinds of clicking and typing and sending ing and reading activities that you do. So let's say you receive an email based on that you query a database, you copy something out of that, put it into a spreadsheet, save the spreadsheet and then send it to somebody. A robotic process automation would do all those steps for you. You just have to set it up and then all of those steps will be done. They call it a robot. It's not really robot. It's really more like software scripting, but that's kind of the term that kind of won out in the in in the marketing space. So, so what we want to do then is to take this representation of the process and then have it executable. And the way we can do that now is to um you know once again we use claude we can say um please convert this process into uh BPMN XML. So this is a a tag based representation of the process that we can actually bring into an execution platform for this process. Okay? And so you wind up with something like this. So you don't necessarily have to know everything about about how XML works, but you have these tags and we have these um uh you know the name of the tag kind of describes what it is and then it has certain attributes. But notice that we have this tag here that is the start and the identifier you know the name of the start process is the invoice received. And what this is is a declaration of the steps of the process and then what has to be done. So notice that um okay so let's see if we can read some of this. Um so there's the start event. Um it's got something outgoing flow to invoice type. So, I guess what we're supposed to do is find, you know, the invoice type. Um, let me see. I think down here at the bottom it's got all the flow identifications and then what order they're supposed to go in. Uh, yeah. So, sequence flow. Um, so two invoice type and then it's got these. Okay. So, inputs, outputs. So, so basically all the inputs, outputs, and then the flows. Everything that was in that flowchart is now in this this XML document. Now this XML document can then be loaded into something like command which is you know they call it you know uh an agentic uh orchestrator. So really what this is is a workflow execution tool where you can take this XML document loaded into here and then the parser that's in this goes through that document and then executes certain software functions based on what's in that XML document. And one of the things that you can add into this is what's called an LLM connector. So if we say in our uh see if I can go back to that that physical representation um uh there's a document I want the diagram. Okay. So if you were looking at your process here and you see that you have to um check whether the data is complete and valid, you could um in that you know that agentic workflow execution platform you could add what's called an LLM connector here where you could say okay send the invoice to this large language model prompt the large language model in this way. Take the output of the large language model and if that output contains this then do this. So on these individual steps of the workflow you can actually automatically hook in connectors to the large language models. Okay. Um so uh okay. Okay. So, you know, a as we're as we're thinking about, you know, any kind of any kind of workflow that involves document processing. So, you receive an invoice, that's a document. Um, you have to send an email, you have to check, you have you have to check for similarities between two things. You have to access the ERP or the procurement system to find that purchase order and then make sure that document matches certain features of this document. All of these functions, all of these so-called document processing functions can be done with an LLM. So, you can have your whole workflow automated uh and have the LLM just kind of chime in where it needs to do any kind of document processing. So, I think this is how um you know, obviously not everybody is going to be using this. Um uh there there's a um well depending on the size of the business and the maturity of the processes and the formality of the processes, you know, you may may need something like this or you may not. But I do think that this is going to be a great way that um LLMs in particular and NLP in general are going to really um add a lot of value to businesses.