A vision system is an indispensable tool for most automated production processes. Dobot offers a Vision System for educational use, which can be used in conjunction with the Magician or M1 robot arms. The star of the show is the software included the package. The Vision Studio software has advanced image recognition features and a clean, easy to use visual programming interface based on flows.
Introduction
I received a review unit of the Dobot Vision Kit from a local Dobot supplier, dobot.nu. In this article I will take a look at the capabilities of the system, how to program it and hopefully provide some insight on its use cases. This is not intended as a full review, rather my first experiences with the kit. I quickly found that the software has so many features it is impossible for me to go through all of them here without making this article extremely long. Below are a couple of examples from the user guide and also made some programs myself to get familiar with the software. Hopefully this will provide enough insight on if the system is suitable for your application. It is mainly targeted at the educational market, but it might be interesting for other users as well like startups or in R&D environments.
Lets start with the hardware. The Vision Kit has a camera mounted to a stand. This allows you to use the camera even without a robot arm. Also, the camera can capture an image while the robot arm is out of the frame when handling parts, making this the most efficient option in terms of cycle time. The camera stand is mounted on a solid Aluminium plate, with a cut out for the robot arm. There is one for the Dobot Magician and also one for the Dobot M1.
The camera is a 5 Megapixel model by Hikvision with a fixed focal length. It has a 1/1.8” sensor and a lens with a 12 mm focal length. This should be comparable to a 58mm focal length for a full frame camera. It has the entire recessed area of the table in the frame when the camera is mounted at the top of the stand. The aperture and focus of the camera can be set manually and locked into place to prevent accidental changes.
Assembly
Assembly of the system can be done in under 10 minutes. It involves only screwing the stand into the bottom plate and assembling the camera fixture. The ring light is mounted onto the camera close to the front element. This does mean you have to rotate the entire ring light when focusing the camera. This works fine as long as you lock the focus ring when you are done adjusting it. I found the ring light to provide sufficient lighting across the entire visible area.
- Software
The Vision Kit comes with 2 software programs: MVS and Dobot Vision Studio. With the MVS software you can check the live view of the camera and save images and videos. However you will be spending most of your time in the Dobot Vision Studio Software. This is a fully featured machine vision programming environment. My first impression was that it is very user friendly. All programming can be done by creating flows. Simply put this is dragging function blocks into the working area and indicate the order in which they need to be processed by linking them together with connectors. I will show some examples of how this works later on in the video. For more advanced users there is also a script language available, but that is beyond the scope of this video.
- Manual & examples
Although the software is user friendly, for complete novices in the world of machine vision it can feel a bit overwhelming given the sheer amount of options available to you. The best way to get the hang of it is to go through the manual front to back, before you start playing with the software. Don’t worry, it is well written and loaded with pictures of examples, so it is quite pleasant to read.
The last section covers a number of examples that you can use to get started with the vision system and explore the possibilities before creating your own projects. In this article I will cover a couple of these examples and 2 of my own projects. In the limited time I had with the review unit I was able to test only a limited amount of the available functionality. I feel like I have only scratched the surface of what is possible, but regardless I hope this still gives you a good first impression of the Vision Kit.
- Visual programming by creating flows
Lets quickly go through the interface; On the left there is a task bar where you can find all of the available functions. In the center there is a working area where you create your flows. On the right side the camera image is displayed and below it the results for each function can be monitored.
A flow typically starts with an image captured by the connected camera. It makes sense to adjust exposure and focus on the object that is being captured before you proceed making a new flow. The Image can be zoomed in to verify if the object is in focus.
Color Sorting
The first example is a pick and place exercise with color identification, which is covered in detail in the manual. I am going to make a flow that detects only the blue blocks. After detecting a blue block the flow will guide the robot arm to the part, pick it up and drop it off at a specified location. I have a Magician robot arm, which I am using for this application, but you could also use a Dobot M1 robot arm if you are lucky enough to own one.
The flow starts with a camera block, which captures an image from the connected camera for processing.
The next block is a color extraction block, which can apply a filter to the image to only highlight a specified color. All of the blocks can be double-clicked to change settings specific to that function.
I am not going over all of the settings in detail, I will just discuss the ones needed for this project and why I am changing them. Make sure to set the camera to capture a color image, so we can actually detect colors.
The play button is used to capture an image or alternatively, the continuous cycle button to keep running the flow over and over again. This will allow you to change settings and immediately see the results of your changes.
Now we need to make sure that the software can actually identify these white regions as the parts and determine their position.
The next blocks Image Morph and Blob detection will create a rectangular shape from the white areas in the frame and find the center position of these rectangular areas.
The last part of the image processing is a calibration transformation. This will convert the coordinates the camera found into coordinates for the robot arm. This is done by loading a calibration file which holds all of the information needed for the software to convert a coordinate in the image to a real world coordinate. You have to create this file when you setup the camera for the first time and you can reuse it over and over again as long as you don’t change the position of the camera.
Calibration
The calibration procedure is quite straight forward. You load the calibration program and place the calibration sheet on the table. The camera recognizes 9 points in the pattern.
The procedure then needs you to move the arm manually to each of the calibration points in the correct order as accurately as possible and enter the coordinates in the software. When this is done you can create a calibration file. You can then use this file in any of your projects where you need to convert coordinates in the image to real world coordinates for pick and place operations, or for example to calculate if the dimensions of a part are within specification.
Back to the example:
Now that we have determined the real world coordinates of our parts, specific Dobot Magician blocks can be used to pick the parts and operate the suction cup.
The idea of connecting the blocks in the editor is that you can indicate the order of events, but this also allow blocks to reference output from previous blocks. For example in the move-block you can reference the X and Y output as calculated by the Calibration-transformation block.
For the Z-value, which is the depth to which the arm moves, I am not referencing anything, but instead I am moving the arm to a depth where the suction cup touches the part and I check the current Z-value. This value can then be entered in the move block to make sure the arm goes to the correct depth. In this case -30.
In order to complete the flow we need a block to make the arm go to the part, a block to activate the vacuum cup, a block to take the part to the specified location and lastly a block to deactivate the vacuum and drop the part.
Circle diameter measurement
In the next example we will try to identify circles, measure their diameter and based on some acceptance criteria put them in either a reject bin or a bin for good products.
Here we also start with a camera block, but this time with a monochromatic image.
A find-circle block, which automatically detects circles in the image based on the criteria you set, like minimum or maximum value, contrast, etcetera.
The next block is a scale transformation, that converts the radius of the part from pixels to a real world distance. Again this is based on the calibration file created earlier.
A Calibration-transformation is used to do a similar conversion, but this time for the center coordinates of the circle. So now we know both how large the circle is and where it is located. The next blocks are for moving to the part and to activate the suction cup.
We then need to decide whether the part is pass or fail. You can use an If-module to check the size of the radius found in the scale transformation and add some acceptance criteria, in this case a radius between 10 and 20 mm. So anything falling within this range will make the if-module output a 1 and anything else will generate a zero as output. In this case 1 means pass and 0 means reject.
With a branch-block linked to this, we can then go to different sub flows based on the results. One of the flows takes the part to the reject bin and the other takes the part to the location where the good parts need to be dropped.
Coaster Circle fit
Now for a more practical case. Just imagine you are a manufacturer of coasters in the shape of a flower. Sorry, this is the first thing I found when looking for examples, but please bear with me. I think this is quite an interesting example for computer vision.
Imagine we are trying to measure the diameter of the coaster to see if we don’t have any shrinkage problems or other issues that could cause a dimensional defect.
Again we start with a camera block to capture the image. This is followed by a fast match-block. This is a function that can identify patterns in the entire image, regardless of their position or orientation. This is a very powerful feature and it is easy to train. The idea is that you draw a rectangle around a feature that you would like to find in the image. If needed you can play around with the settings to create a tighter fit or something that is more forgiving to deviations.
This function will find all of the instances of the feature in the image, in this case all of the leafs of the coaster. We can then apply function to find the peak of the feature and create a coordinate out of it with the points assemble function. If we then drag a loop around this we can tell the software that we would like to do this for every leaf on the coaster. The amount of times the loop is executed can simply be referenced back to the amount of features found by the previous fast match-block.
All generated point are then used as input for the next block, a circle fit function. This even detects outliers and gives them a red color.
Lastly, we can do a scale transformation to convert the found radius from pixels to millimeters. This will give us the real world diameter of the coaster.
To have some indication of how repeatable the measurement I pick up the part and placed it back at a random position om the table. This is not a scientific experiment, so please take these numbers with a grain of salt. The minimum fitted diameter was 105.234 mm and the maximum 105.312 mm. The standard deviation in this case is only 22 microns, which is quite good actually. It has to be noted that the repeatability of the system is dependent on many factors such as contrast, lighting, how the system was trained to find features, etcetera…so your mileage may vary.
Sample part defect detection
So, now for the last experiment. In order to mimic a real world scenario I have drawn up a small part with some features that we can measure. The part has no purpose at all and is not meant to resemble any particular shape.
I picked some specific features that I wanted to measure and set specific acceptance criteria for them. Here they are:
I printed a couple of parts to the nominal size and some parts that are intentionally out of specification to see if the vision kit can correctly reject these out-of-spec parts. Also on one of the parts I intentionally omitted one of the holes to see how I could deal with this in the vision studio software.
Below is the program I created for verifying all of the specifications. It might seem like a big plate full of spaghetti, but it is just a combination of many separate checks which all feed into the end result.
At the top it starts again with the camera block to capture the image, followed by a “ Fast Match” block to recognize the part geometry. You have to show this block what the part looks like the first time you use it, similar to the feature recognition for the coaster example. Here, it finds 5 instances of the part. Even without measuring, we can already see that 2 of the parts seem to fit the signature quite well. On 3 of the parts there are some deviations visible which are marked in red. The fitting quality could also be used to reject parts, but we are actually going to measure according to the specifications.
I will go through the flow at a fairly high level. At the top of the flow there are several block that try to find either lines or circles in specific locations of the part. The green rectangle around the feature indicates the search area. The output data from these blocks is then used to perform the measurements.
Block 20 measures the angle between 2 of the found lines, in this case blocks 6 and 18.
Block 9 measures the distance between the line found in block 3 and the circle found in block 4. Block 12 does exactly the same thing, but now for the bottom hole.
Block 15 measures the distance between the lines from block 6 and 7, which represents the height of the part.
Block 8 find the bottom left circle and measures its diameter.
For all of the measurements the dimensions are scaled using the calibration file, similar to previous examples.
They all feed into the “ IF module” which checks if all of the dimensions are within specification.
Here are some examples of measured parts. This part is passed all criteria so it is marked as OK.
This next part fails the dimensional inspection. It is part 1b, which was modelled in CAD to be a reject by design. It misses the lower left hole, which is correctly picked up by the software. Since it cannot find the hole all values for this feature are registered as zero, which is outside the diameter specification for this hole.’
Also this third part fails to meet specifications. It is a printed part from design 1C, which has slight dimensional errors. The diameter of the bottom right hole is too small and furthermore, the top right hole is out of alignment, with a measured distance of 4.44 mm.
Other features
As mentioned earlier I have not been able to test all of the features of the vision Kit. These include for example a script function for advanced editing, multi flow communication, intensity detection, feature defect detection and pixel counting. It can read most of the common barcodes, like QR, 2D and 1D barcodes. It has optical character recognition for interpreting text and even deep a learning functionality which can be used to train the system on existing defects.
Conclusion
Time for some final thoughts: The Vision Kit system is primarily intended for educational use but I think it can also be used in an R&D environment or for small scale manufacturing. I think it is a powerful system with lots of functionality. Although the software is very user friendly, with the graphical flow system, the sheer amount of options could be a bit overwhelming for new users. However, there is a good manual available. If you follow it step by step and go through all of the examples, things start to make sense and you will soon be able to create your own flows from scratch. Just don’t expect to become an expert in a couple of hours.
Given the extensive use of vision techniques in modern production facilities, I think it is very useful as an education tool. The price for this system is around 1500 euros, for which you will get a camera with objective, a metal stand and the Vision Studio software. Personally, I think this is a reasonable price, but of course that is for everyone to decide for themselves. It does put the system out of reach for most hobbyists and makers, which is a shame because it is really awesome and provides many learning opportunities. For kids and students in technical educations, I hope they will be able to practice with a system like this to get an introduction into machine vision.
Thanks for taking the time to read this article. If you have any questions, please ask the in the comment section below.
Robin