Deep learning is bridging the gap between the digital and the real world

We’re excited to bring Transform 2022 back to life on July 19th and virtually July 20-28. Join AI and data leaders for sensible conversations and exciting networking opportunities. Register today!


Algorithms are always at home in the digital world, where they are trained and developed in a fully simulated environment. The current wave of deep learning enables AI to jump from the digital to the physical world. Applications ranging from manufacturing to agriculture are endless, but barriers remain to be overcome.

For traditional AI experts, deep learning (DL) is the old hat. It had its breakthrough in 2012 when Alex Krievsky successfully deployed for the first time convoluted neural networks identifying deep learning technology with his Alexnet algorithm. It is the neural networks that allow the computer to see, hear, and speak. DL is the reason we can talk on our phones and write emails on our computers. Yet DL algorithms always play their part in a secure simulated environment of the digital world. Pioneer AI researchers are working hard to introduce deeper learning into our physical, three-dimensional world. Yes, the Real The world.

Even if you are a car manufacturer, chipmaker or farmer, deep study can do a lot to improve your business. As technology has matured, the leap from digital to the physical world has proved more challenging than many expected. This is the reason we have been talking about smart refrigerators for years, but in reality no one has yet. More than one challenge has to be overcome when algorithms abandon their cozy digital structures and stop for themselves in three extremely real and raw dimensions.

Automatic criticism

The first problem is accuracy. In the digital world, algorithms can be eliminated with almost 80% accuracy. It doesn’t quite cut it in the real world. “If a robot harvesting tomatoes sees only 80% of all tomatoes, the grower will miss 20% of its turnover,” says Albert van Bremen, a Dutch AI researcher who developed DL algorithms for agriculture and horticulture in the Netherlands. Its AI solutions include a robot that cuts the leaves of cucumber plants, a robot that harvests asparagus and predicts the harvest of strawberries. His company is also active in the medical manufacturing world, where his team created a model that optimizes the production of medical isotopes. “My clients are accustomed to 99.9% accuracy and they expect AI to do just that,” says Van Bremen. “Every cent of the loss of accuracy is going to cost them money.”

To achieve the desired levels, AI models must always be retrained, which requires a constant flow of updated data. Data aggregation is both expensive and time consuming, as all that data is recorded by humans. To solve that challenge, Van Bremen has equipped each of his robots with functionality that lets him know when he is performing well or badly. When making mistakes, robots will only upload certain data where they need to be corrected. That data is automatically collected throughout the robot fleet. So instead of receiving thousands of images, Van Bremen’s team only gets a hundred or more, which are then labeled and tagged and sent back to the robots for retraining. “A few years ago everyone was saying that data is gold,” he says. “Now we see that the data is really a huge haystack that hides the gold nugget. So the challenge is not just to collect a lot of data, but the right kind of data. ”

His team has developed software that automates retraining of new experiences. Their AI models can now train themselves for new environments, effectively pulling humans out of the loop. They have also found a way to automate the annotation process by training AI models to do most of the annotation work for them. Van Bremen: “It’s somewhat contradictory because you could argue that the model that can criticize photos is the only model I need for my application. But we train our annotation model with a much smaller data size than our goal model. The annotation model is less accurate and can still make mistakes, but it is good enough to create new data points that we can use to automate the critique process. “

The Dutch AI specialist sees great potential for in-depth education in the manufacturing industry, where AI can be used for applications such as defect detection and machine optimization. The global smart manufacturing industry is currently valued at 198 198 billion and is projected to grow at 11% by 2025. The Brainport area around Eindhoven, where Van Bremen’s company is headquartered, is full of world-class manufacturing corporates like Philips. And ASML. (Van Bremen has worked for both companies in the past.)

Sim-to-real gap

Another challenge to implementing AI in the real world is the fact that the physical environment is more diverse and complex than digital. Self-driving cars trained in the US will not automatically operate in Europe with its various traffic rules and signs. Van Bremen faced this challenge when he had to apply his DL model which cuts the leaves of the cucumber plant in a different manufacturer’s greenhouse. “If this happened in the digital world, I would take the same model and train it with data from a new manufacturer,” he says. “But this particular manufacturer ran its greenhouse with LED lighting, which gave all the images of cucumber a blue-purple glow that did not recognize our model. So we had to adapt the model to correct this real-world deviation. All of these unexpected things happen when you take your model out of the digital world and into the real world. “

Van Bremen calls this the “sim-to-real gap”, the inequality between predictable and unchanging simulated environments and unpredictable, ever-changing physical realities. Andrew NG, a well-known AI researcher at Stanford and co-founder of Google Brain who also seeks to apply in-depth learning in production, talks about ‘proof of concept for production gap’. There is a reason why 75% of all AI projects in manufacturing fail to launch. One way to solve this problem is to focus more on clearing your data set according to Ng. The traditional approach in AI was to focus on creating a good model and allowing the model to deal with noise in the data. However, data-centric views can be more useful in production, as the size of the data set is often small. Improving the data will immediately have an effect on improving the overall accuracy of the model.

In addition to cleaner data, another way to bridge the SIM-to-real gap is by using CycleGun, an image translation technique that combines two different domains that have become popular through older apps like FaceApp. Van Bremen’s team researched cycling for its use in the production environment. The team trained a model that optimizes the movement of the robotic arm in a simulated environment, where three simulated cameras observed the simulated robotic arm lifting a simulated object. They then developed a DL algorithm based on a cyclegun that translates from the real world (three real cameras observe a real robotic arm lifting a real object) into a simulated image, which can then be used to retrain the simulated model. Van Bremen: “The robotic arm has a lot of moving parts. Usually you have to program all those movements in advance. But if you give him a clearly defined goal, such as lifting an object, he will now first optimize the movements in the simulated world. By cycling you can use that optimization in the real world, which saves many people hours. ” Each individual factory using the same AI model to operate the robotic arm will have to train its own cyclegun to tweak the general model to suit its own specific real-world parameters.

Reinforcement education

The field of deep learning is constantly growing and evolving. Its new frontier is called Reinforcement Learning. This is where algorithms simply change from observers to decision makers, giving instructions to robots on how to operate more efficiently. Standard DL algorithms are programmed by software engineers to perform a specific task, such as moving a robotic arm to fold a box. Reinforcement algorithms may find that there are more efficient ways to fold boxes out of their pre-programmed range.

It was Reinforcement Learning (RL) that left the AI ​​system behind the world’s best go player in 2016. Now RL is also slowly entering production. The technology is not yet mature enough to be deployed, but according to experts, this will only be a matter of time.

With the help of RL, Albert van Bremen envisions optimizing the entire greenhouse. This is done by letting the AI ​​system determine how plants can maximize profits to the most efficient grower. The optimization process takes place in a simulated environment, where thousands of potential growth scenarios are tried. The simulation plays around different growth variables like temperature, humidity, lighting and fertilizer and then chooses the scenario where the plants grow best. The winning scene then translates back to the three-dimensional world of the real greenhouse. “The hurdle is the SIM-to-real gap,” explains Van Bremen. “But I really hope those problems are solved in the next five to ten years.”

As a trained psychologist I am fascinated by the transition that AI is making from digital to the physical world. It shows how complex our three-dimensional world really is and how much neurological and mechanical skills are required for simple tasks like cutting leaves or folding boxes. This transition is making us more aware of our own internal, brain-powered ‘algorithms’ that help us navigate the world and which has taken millennia of evolution. It will be interesting to see how AI competes with it. And if the AI ​​finally catches up, I’m sure my smart refrigerator will order champagne to celebrate.

Bert-Jan Wortman is the director of Microcentrum.

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including tech people working on data, can share data-related insights and innovations.

If you would like to read about the latest ideas and latest information, best practices and the future of data and data tech, join us at DataDecisionMakers.

You might even consider contributing to your own article!

Read more from DataDecisionMakers

Similar Posts

Leave a Reply

Your email address will not be published.