Hello and welcome to UI.Vision!
In this tutorial we will show how to automate all kinds of tasks and tests with computer vision.
Let's start with web browser automation. The sample task for our demo is to automate
a distance measurement on Google maps. We first record skeleton macro
with classic web automation. As we browse the website, the steps are
added to the macro automatically. Done. We stop the recording.
For the next automation steps, we use computer vision and native
mouse and keyboard simulation. The visual automation is not
created by recording. Instead, we create it by simply
adding one command after another. This is as easy as assembling Lego bricks.
First, we use VisualAssert to confirm visually that the map is loaded correctly.
Then we use the XMove command to position the mouse on top of the map.
We use the same image for this. Since our goal is to measure
the distance between Big Ben and Buckingham Palace, we
need to zoom out a bit. For this we simulate a mouse
wheel movement twice. Now we start the distance measurement. For that we use XClick to right
click on the Big Ben icon. Measure Distance is the
last item in the menu. So, to reach it, we simulate the cursor down key press 9 times, and then
simulate the ENTER key. To repeat the XType command 9 times, we create a loop with the TIMES command. Next, we need to click on the
target for the measurement. So, we use XClick to click on the
Buckingham Palace icon. This concludes the distance
measurement itself. As the last step we want to
extract the measured value. We could use OCR here, but it will be easier to
simply triple-click the line in the popup and then use CTRL-C to copy the
value into the clipboard. Let's do this. Inside the macro, we can now access the value
with the built-in !Clipboard variable. Ok, the macro is done. It took us less than five minutes
to create this automation! Now, let's test it. The macro opens google maps,
automates the search, simulates the right click,
selects the 9th menu item does the measurement
and finally, the distance is displayed in the macro. Everything worked fine. Since UI Vision is a cross-platform tool,
the same macro works on Linux and Mac without changes. To transfer the macro, let's export it. The ZIP archive files contains the macro
itself and all required images. Here we are on a Macbook.
As the first step we import the macro that we just created
on Windows a minute ago. Ok, the macro and its images are imported.
Let's run the automation. UI.Vision opens Google maps,
it automates the data entry and the search, it finds the Big Ben icon on
the map and right-clicks it and now it executes the key presses,
just like a real user would do it triggers the distance measurement
and extracts the result. All done. In the log file we can confirm that the Mac automation worked fine. Here we are on a Linux machine. And here is the portable macro
that we created previously Again, let's import the
macro and then run it. The import went fine. Macro and images are imported. Now let's run it.
The macro opens the website, fills in the search box, opens the context menu
does the measurement it copies the value and displays it in the
macro. All done.
The Linux automation worked fine. Previously we automated tasks
inside the web browser. As a demo for desktop automation, let's automate the Windows calculator
and read the output with OCR. To create the macro, let's turn
on desktop automation mode. In this mode, UI Vision looks at the desktop
instead of only the web browser. As before, we build the visual automation
by assembling steps like LEGO bricks. The first command we need is
a double click on the icon that starts the calculator.
Let's run the command. That worked.
The calculator is now open. Let's calculate 8 times 8. We use the XClick command
for each button press. The calculation part of
our macro is now done. For screen-scraping the result,
we use OCRExtractRelative. First we take a snapshot of
the overall result area. Then we open the image in
the built-in editor. Inside the image we first look
for a non-changing area. During replay, UI Vision will use this
area to find the correct region. This is also called the anchor image.
We mark this area with a green frame. The area with the text to be extracted
is marked with a pink frame. As the last step we output the OCR result. Done. Let's quickly mouse-over the XClick commands to verify the input images. Looks good, let's run the automation. All done. The calculator was automated and the
result extracted and verified. Last but not least let's dive into an advanced
topic, the UI Vision command line API. The UI Vision software can be
integrated with almost any other tool, programming or scripting language
due to its powerful command line API. In the previous parts of this video, we created
a web macro and a desktop automation macro. Now let's combine both tasks
and trigger them directly from within a 3rd party scripting language. As first example we use Powershell. Let's open the script and have a
quick look at the source code, just to confirm that the macro
names are in the script. Ok.
Let's run it. Everything runs automatically and the results
are returned back to the calling script. This worked well.
Now, let's do the same, but with a Node.JS script. Again, let s have a quick look and confirm
that the macro names are in the script. Ok.
Let's run it. Like before, everything runs
automatically and the results are returned back to the calling script.
While the automation is running, let me tell you that the full source code is available in our Github account.
There you find also many more sample scripts, for example Python, Linux shell
scripts and Windows batch files. Ok, now the Node.JS script is done. Everything worked well.
This concludes our tutorial. Thank you for watching. For more information, please visit our website
and the UI Vision user forum. *** Happy automation! ***