Industrial applications are demanding on network robustness and reliability, and have requirements for low communication latency and guaranteed bandwidth. The Czech Institute of Informatics, Robotics and Cybernetics (CIIRC CTU) already has a Testbed for Industry 4.0 equipped with a state-of-the-art 5G SA campus network configured for industrial use and corresponding to real conditions in production practice.
Private Campus 5G Network for Industry 4.0 at CIIRC CTU
The complete architecture of the 5G SA campus network, the first of its kind in the Czech Republic, was put into operation in the Testbed for Industry 4.0 thanks to intensive cooperation with T-Mobile, which built it on new technology from Ericsson. Over the past two years, a number of applications requiring high speed, low latency and IoT device connectivity have been developed and tested. These applications have validated the suitability of the 5G network for tasks in industrial environments, as well as the network’s ability to consistently maintain the contracted quality of service.
The network architecture is designed in such a way that the radio part connects the end devices via 5G communication to the network controller, the so-called 5G core, to which the network segment for the application servers is already directly connected via a fixed connection. This allows applications to be run on the respective application server without additional delays caused, for example, by routing in larger networks.
Five-axis Delta Robot with Conveyor
A five-axis delta robot with a linear conveyor for pallet transport is installed in the Testbed for Industry 4.0. It is an example of a multi-axis positioning system with synchronised axes of motion. All axes are positionally controlled with feedback from absolute multi-turn encoders, while the interpolation of the robot endpoint position takes place in the control PLC. This configuration provides full control over the processes performed by the robot, whether they relate to inverse kinematics, inverse robot dynamics or the ability to run the robot simulation model, the so-called digital twin, directly in the PLC. It is thus possible to detect any deviations from the expected behaviour using machine learning algorithms, or “just” collect data on the operation and subject it to a more thorough subsequent analysis.
The robot is also equipped with a handle for so-called manual guidance, thanks to which the operator can demonstrate the desired movements to the robot in the learning mode by guiding it with a hand controller, the robot thus learns the trajectories to be performed during normal operations.
Through the campus 5G SA network, the robot connects to the application server (edge server) and thus can use its high computing power for applications deploying neural networks, computer vision and other functionalities. Thus, it is not necessary to use a powerful industrial computer in the robot space with a connected camera, but a low-cost single board computer such as rPi, which mediates the connection of a standard camera to the 5G network and the transfer of image analysis results from the server to the application, is sufficient. Such a computer is integrated in the hand gripper for manual guidance and, thanks to the above mentioned image analysis, helps in manual guidance with locating the target object and “snapping” the robot end effector to the object’s location. The use of a low-cost camera is possible, among other things, because it is part of the hand grip for manual guidance, which moves together with the robot end effector and the image quality is thus sufficient. For example, if the depth of field is insufficient, the camera can be placed at a suitable distance to be within the range of the available depth of field. In the sample application, these objects are the tires and wheel rims of a scale model car.
This concept confirms that with a 5G network, computing power can be transferred from the process technology to a server that is securely stored in an IT server room, which ultimately makes the entire deployment cheaper. The functionality itself is due to the application running on the server, offering a wide range of options for managing and updating applications in a single location. It also makes it cheaper for end devices, i.e. cameras, which do not need a computing unit to perform complex calculations but can take advantage of the high computing power of the server.
Quality Control of Assembled Wheels and Neural Network Training
Wheel and tyre localization also enables automatic wheel assembly without the need for operator intervention. Another task that the delta robot subsequently solves is to check that the wheel is correctly assembled on the conveyor pallet and to detect any type of defect arising during assembly.
For the neural network to behave correctly, it needs to be trained. This process is often done by manually annotating images describing particular situations, such as a poorly or well-fitted wheel. The process of annotating for the object detection task is tedious and time consuming, as it is necessary to have a relatively large amount of data in the database in order to properly train the robot and generalise the data well. Therefore, we proposed an algorithm to generate a synthetic dataset. This approach allows us to obtain an almost unlimited number of annotated images with high diversity and randomness.
The key idea is simple: photograph the background, take images of the objects from different viewpoints, select random coordinates within the background, and place the objects. The bounding box parameters can be easily obtained from the known coordinates and dimensions of the object image.
The proposed implementation handles cases where objects are few, very close together, and also handles perspective changes when objects are in the same place. It is possible to account for different distances from the camera and a large variety of locations due to random sampling of coordinates. It is possible to generate up to 100,000 images in one day and process the data stream at 15 frames per second in real time.
The dataset for this task was created by manually fitting wheels and simulating various negative examples. A total of 250 negative class images and 50 positive class images were used. This number of examples is quite small to successfully train a convolutional neural network (CNN) model.
Thus, for the purpose of better generalisation, we used data augmentation techniques such as vertical/horizontal flipping, rotation, brightness changes, camera noise, etc. No other regularisation techniques are used in our training process. This task has an unbalanced distribution of samples across classes, as there are many more possible variations of negative examples than positive ones. To address this problem, an oversampling technique was used, where one class is expanded more than the other, and a weighted loss function, where one class is given more weight than another.
The total number of trainable parameters is approximately 5.5 million. The model is trained on 100 thousand images, 15 thousand images are used for validation and 15 thousand images are in the test set.
The quality control algorithm again runs on the application edge server, but uses a different camera than the one in the manual control handle, in this case an industrial camera with a high quality lens. This goes back to the original idea of having multiple applications running on a single application server with different cameras placed at appropriate locations in the industrial process. This decentralises the image capture, but thanks to the robust transmission over the 5G SA network with defined and guaranteed quality of service, the computation can be centralised to a single location or server.
In the future, we foresee further extensions of the delta robot’s functions, for example by analysing force and torque waveforms from a force sensor located on the robot’s end effector in order to directly control the quality of tyre assembly, with the relevant algorithms for waveform analysis and control intervention in the assembly process running again on the application edge server.
