Soft Robotics grippers can move items that might be damaged by classic mechanical grippers. | Credit: Soft Robotics

Robots are finally getting a grip. 

Developers have been striving to close the gap on robotic gripping for the past several years, pursuing applications for multibillion-dollar industries. Securely gripping and transferring fast-moving items on conveyor belts holds vast promise for businesses. 

Soft Robotics, a Bedford, Mass. startup, is harnessing NVIDIA Isaac Sim to help close the sim-to-real gap for a handful of robotic gripping applications. One area is perfecting gripping for pick and placement of foods for packaging. 

Food packaging and processing companies are using the startup’s mGripAI system which combines soft grasping with 3D Vision and AI to grasp delicate foods such as proteins, produce, and bakery items without damage.

“We’re selling the hands, the eyes and the brains of the picking solution,” said David Weatherwax, senior director of software engineering at Soft Robotics. 

Unlike other industries that have adopted robotics, the $8 trillion food market has been slow to develop robots to handle variable items in unstructured environments, says Soft Robotics. 

The company, founded in 2013, recently landed $26 million in Series C funding from Tyson Ventures, Marel and Johnsonville Ventures.

Companies such as Tyson Foods and Johnsonville are betting on the adoption of robotic automation to help improve safety and increase production in their facilities. Both companies rely on Soft Robotics technologies. 

Soft Robotics is a member of the NVIDIA Inception program, which provides companies with GPU support and AI platform guidance. 

Getting a grip with synthetic data

Soft Robotics develops unique models for every one of its gripping applications, each requiring specific data sets. And picking from piles of wet, slippery chicken and other foods can be a tricky challenge. 

Utilizing Omniverse and Isaac Sim, the company can create 3D renderings of chicken parts with different backgrounds, like on conveyor belts or in bins and with different lighting scenarios. 

The company taps into Isaac Replicator to develop synthetic data, generating hundreds of thousands of images per model and distributing that among an array of instances in the cloud. Isaac Replicator is a set of tools, APIs, and workflows for generating synthetic data using Isaac Sim.

It also runs pose estimation models to help its gripping system see the angle of the item to pick. 

NVIDIA A100 GPUs on site enable Soft Robotics to run split-second inference with the unique models for each application in these food-processing facilities. Meanwhile, simulation and training in Isaac Sim offer access to NVIDIA A100s for scaling up workloads.

“Our current setup is fully synthetic, which allows us to rapidly deploy new applications. We’re all in on Omniverse and Isaac Sim, and that’s been working great for us,” said Weatherwax. 

Solving issues with occlusion, lighting 

A big challenge at Soft Robotics is solving issues with occlusion for an understanding of how different pieces of chicken stack up and overlap one another when dumped into a pile. “How those form can be pretty complex,” Weatherwax said.

Glares on wet chicken can potentially throw off detection models. “A key thing for us is the lighting, so the NVIDIA RTX-driven ray tracing is really important,” he said. 

The glares on wet chicken is a classic lighting and vision problem that requires a new approach for training machine learning vision models. | Credit: Soft Robotics

But where it really gets interesting is modeling it all in 3D and figuring out in a split second which item is the least obstructed in a pile and most accessible for a robot gripper to pick and place. 

Building synthetic data sets with physics-based accuracy, Omniverse enables Soft Robotics to create such environments. “One of the big challenges we have is how all these amorphous objects form into a pile,” Weatherwax said. 

Boosting production line pick accuracy

Production lines in food processing plants can move fast. But robots deployed with application-specific models promise to handle as many as 100 picks per minute. 

Still a work in progress, success in such tasks hinges on accurate representations of piles of items, supported by training data sets that consider every possible way items can fall into a pile. 

The objective is to provide the robot with the best available pick from a complex and dynamic environment. If food items fall off the conveyor belt or otherwise become damaged then it is considered waste, which directly impacts yield.

Driving production gains 

Meat-packing companies rely on lines of people for processing chicken, but like so many other industries they have faced employee shortages. Some that are building new plants for food processing can’t even attract enough workers at launch, said Weatherwax. 

“They are having a lot of staffing challenges, so there’s a push to automate,” he said.

The Omniverse-driven work for food processing companies has delivered a more than 10X increase in its simulation capacity, accelerating deployment times for AI picking systems from months to days. 

And that’s enabling Soft Robotics customers to get a grip on more than just deploying automated chicken-picking lines — it’s ensuring that they are covered for an employment challenge that has hit many industries, especially those with increased injury and health risks. 

“Handling raw chicken is a job better suited for a robot,” he said.

The post Soft Robotics mGripAI uses simulation to train in NVIDIA Isaac Sim appeared first on The Robot Report.

This week, we have a special video edition of The Robot Report podcast. This is the video feed from our recent interview with Tatum Robotics founder and CEO, Samantha Johnson. The video features American Sign Language (ASL) translation so that hearing-impaired individuals can also enjoy the content.

Tatum Robotics is building a robotic device shaped like a human hand and arm, that can mimic a human translator for deafblind individuals. Currently, deafblind individuals communicate by touching the hand of their translator. The human translator uses finger spelling and ASL signs to communicate.

Tatum Robotics is building a robotic analog to the human hand, designed to replicate the interaction between a translator and a deafblind user. Ultimately, Tatum Robotics wants to open up the world of ebooks for consumption by deafblind individuals. This will be followed by remote communication (i.e. over the web) between both hearing individuals and deafblind individuals, or even between two deafblind individuals.

As Samantha Johnson discusses in the video, until now, deafblind individuals are often isolated and bored for long periods of time, with no ability to communicate without a translator.

An early prototype of the Tatum Robotics communication robot for deafblind individuals. | Credit: Tatum Robotics

We want to thank the ASL translators on this project: Tymber Marsh and Sean Havas for their amazing translation skills. Tatum Robotics is currently recruiting additional ASL signers to contribute their unique ASL techniques to the robot design. If you are interested, contact Tatum Robotics directly for how you can contribute.

The post Video podcast episode featuring interview with Tatum Robotics founder appeared first on The Robot Report.

Soft Robotics’ mGrip soft gripper can handle even delicate foods, like cupcakes, without smashing them. | Source: Soft Robotics

Soft Robotics brought in $26 million in the first closing of its Series C funding round. This brings the robotic picking company’s total funding to $86 million, according to Crunchbase. 

Soft Robotics plans to use the latest round of funding to expand commercial deployments of mGripAI, its robotic picking product. mGripAI is an IP69K-rated automation package that uses 3D vision and artificial intelligence (AI) to allow industrial arms to perform automated bulk picking in food processes. 

mGripAI, originally brought to market in 2021, can perform over 90 picks per minute. The system includes perception modules that capture high-resolution, 3D images. These images are sent to an intelligence module, which translates them into action for the robotic arm and gripper. The mGrip soft gripper works in unison with the intelligence module to pick the product. 

The mGripAI system is able to track objects in real time for maximum pick accuracy. The system is also capable of grasp optimization, intelligent robot motion control and embedded object understanding. 

“We’re delighted that some of the world’s leaders in the food production and automation markets have decided to join existing investors in supporting SRI’s continuing growth journey,” Jeff Beck, CEO of Soft Robotics, said. “SRI’s technologies are increasingly crucial to enabling and scaling efficient and safe production of several food categories. This round of growth capital strengthens SRI’s ability to rapidly develop, deploy and support those technologies.”

Tyson Ventures, the venture capital arm of Tyson Foods and an existing Soft Robotics customer, led the funding round. Marel and Johnsonville, another Soft Robotics customer, also joined the funding round as new investors. The round also included participation from the company’s existing investors. 

“At Tyson, we are continually exploring new areas in automation that can enhance safety and increase the productivity of our team members,” Rahul Ray, Senior Director of Tyson Ventures, said. “Soft Robotics’ revolutionary robotic technology, computer vision and AI platform have the potential to transform the food industry and will play a key role in any company’s automation journey.”

While the company has primarily focused on automated bulk food processes, in May, Soft Robotics announced that it will be expanding the commercial focus of its mGripAI, a soft gripping solution for automating bulk food picking processes. The company plans to make the product available for order fulfillment, sortation, decanting and kitting. 

The post Soft Robotics picks up $26M appeared first on The Robot Report.

Harvard’s tentacle-like gripper wrapping around a succulent. | Source: Harvard Microrobotics Lab/Harvard SEAS

Researchers at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have created a tentacle-like gripper that can grasp irregularly shaped or soft objects without damaging them. 

The gripper is made up of many thin, soft tentacles that rely on inflation to wrap themselves around an object without any sensing, planning or feedback control. Individually, each tentacle is too weak to pick up many objects, but with many working together the gripper can gently lift heavy and oddly shaped objects.

Each tentacle is made up of a foot-long hollow, rubber rubes. The tubes are made with thicker plastic on one side, so that was the tube is pressurized it curls like a pigtail. As the tube curls, it wraps and entangles itself around an object. Each added tentacle increases the strength of this hold. The gripper releases the object by simply depressurizing the tentacles. 

When designing the gripper, the research team took inspiration from nature. The gripper’s tentacles act similarly to how a jellyfish stuns its prey. 

To test how effective the gripper was, the research team used simulation and experiments where the gripper was tasked with handling a range of objects, including different houseplants and toys. 

The team hopes that the gripper can be used to grasp fragile objects, like soft fruits and vegetables in agricultural production and distribution and delicate tissue in medical settings, as well as irregularly shaped objects, like glassware, in warehouses. The gripper could replace traditional grippers that rely on embedded sensors, complex feedback loops and advanced machine-learning algorithms to work. 

The team’s research was published in the Proceedings of the National Academy of Sciences (PNAS). It was co-authored by Clark Teeple, Nicholas Charles, Yeonsu Jung, Daniel Baum and James C. Weaver, and supported by the Office of Naval Research, the National Science Foundation, the Simons Foundation and the Henri Seydoux fund. 

The post Harvard researchers create soft, tentacle-like robot gripper appeared first on The Robot Report.

MIT researchers have created a 3D printed material with embedded sensors that can sense how its moving. | Source: MIT/CSAIL

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed programmable materials that can sense their own movements. The team created lattice materials with networks of air-filled channels, which allows researchers to measure changes in air pressure within the channels when the material is being moved or bent. 

The lattice structure created by the team is a kind of architected material, meaning when you change the geometry of the features in the material, its mechanical properties, like stiffness or toughness, are altered. For a lattice, the denser the network of cells making up the structure, the stiffer it is. 

It’s difficult to integrate sensors into these materials because of the sparse, complex shapes that make them up. Putting sensors outside the structure, however, doesn’t provide enough information to get a complete picture of how the material is deforming or moving. 

CSAIL’s team used digital light processing 3D printing to incorporate the air-filled channels into the struts that form the lattice structure of the team’s material. The researchers drew the structure out of a pool of resin and hardened it into a precise shape using projected light. In this method, an image is projected onto the wet resin, and areas struct by the light are cured. Researchers used pressurized air, a vacuum and intricate cleaning to remove any excess resin before it was cured.

When the resulting structure is moved or squeezed, the channels formed by the 3D printing are deformed, causing the volume of air inside to change. The team used an off-the-shelf pressure sensor to measure these changes in pressure and get feedback on how the material is deforming. 

The CSAIL team then built off of their results by building sensors into a class of materials developed for motorized soft robotics called handed shearing auxetics (HSAs). HSAs can be twisted or stretched, making them good for soft robotic actuators. Like architected materials, HSAs are difficult to embed sensors into because of their complex structure. 

The team ran the sensorized HSA material through a series of movements for over 18 hours, and used the sensor data they gathered to train a neural network to accurately predict the robot’s motion. 

In the future, the team hopes its technology could be used to create soft, flexible robots with embedded sensors. These robots could understand their own posture and movements. The CSAIL team also sees potential for their technology to be used to create wearable devices that provide feedback on how the user is moving or interacting with their environment. 

The team recently published the results of their study in Science Advances. Daniela Rus, the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science and director of CSAIL, was the lead author on the paper. Co-authors included Lillian Chin, a graduate student at MIT CSAIL, Ryan Truby, former CSAIL postdoc and now assistant professor at Northwestern University, and Annan Zhang, CSAIL graduate student. 

The post MIT CSAIL creates materials that can sense the way they move appeared first on The Robot Report.

The GelSight fin ray gripper was able to feel the pattern on Mason jars. | Source: CSAIL

A research team at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has developed a robotic gripper with Fin Ray fingers that are able to feel the objects it manipulates. 

The Perceptual Science Group at CSAIL, led by professor Edward Adelson and Sandra Liu, a mechanical engineering PhD student, created touch sensors for their gripper, allowing it to feel with the same or more sensitivity as human skin. 

The team’s gripper is made of two Fin Ray fingers. The fingers act similar to a fish’s tail, which will bend towards an applied force rather than away, and are 3D printed from a flexible plastic material. Typical Fin Ray grippers have cross-struts that run through the interior, but the CSAIL team decided to hollow out the interior to make room for their sensory components. 

The inside of the gripper is illuminated by LEDs. On one end of the hollowed-out gripper sits a camera mounted to a semi-rigid backing. The camera faces a layer of pads made of a silicone gel called GelSight. The layer of pads is glued to a thin sheet of acrylic material, which is attached to the opposite end of the inner cavity. 

The gripper is designed to fold seamlessly around the objects it grips. The camera determines how the silicone and acrylic sheets deform as it touches an object. From these observations, the camera, with computational algorithms, can figure out the general shape of the object, how rough its surface is, its orientation in space and the force being applied by, and imparted to, each finger. 

Using this method, the gripper was able to handle a variety of objects, including a mini-screwdriving, a plastic strawberry, an acrylic paint tube, a Ball Mason jar and a wine glass. 

While holding these objects, the gripper was able to detect fine details on their surfaces. For example, on the plastic strawberry, the gripper could identify individual seeds on its surface. The fingers could also feel the lettering on the Mason jar, something that vision-based robotics struggle with because of the way glass objects refract light. 

Additionally, the gripper could squeeze a paint tube without breaking the container and spilling its contents, and pick up and put down a wine glass. The gripper could sense when the base of the glass touched the tabletop, resulting in proper placement seven out of 10 times. 

The team hopes to improve on the sensor by making the fingers stronger. By removing the cross-struts, the team also removed much of the structural integrity, meaning the fingers have a tendency to twist while gripping things. The CSAIL team also want to create a three fingered gripper that could pick up fruits and vegetables and evaluate their ripeness. 

The team’s work was presented at the 2022 IEEE 5th International Conference on soft robotics. 

The post MIT CSAIL develops robotic gripper that can feel what it grabs appeared first on The Robot Report.

A simulated ball covered in tiny robots that can distort its shape. | Source: University of Bath

A research team from the Universities of Bath and Birmingham are hoping to reinvent the way we design robots. 

Typically, robots, like robotic arms, are controlled by a single central controller. The research team, however, is hoping to create robots that are made from many individual units that act individually but cooperatively to determine the machine’s movement. 

The researchers are doing this using active matter, collections of a large number of active agents that resist the way that ordinary soft materials move. Typical soft materials will shrink into a sphere, like water beading into droplets, to create the smallest surface area possible. 

Active matter could be used in many ways. For example, a layer of nanorobots wrapped around a rubber ball could distort the shape of the ball by working in unison. 

“Active matter makes us look at the familiar rules of nature – rules like the fact that surface tension has to be positive – in a new light,” Dr. Jack Binysh, the first author on the study, said. “Seeing what happens if we break these rules, and how we can harness the results, is an exciting place to be doing research.”

During the study, the researchers simulated a 3D soft solid with a surface that experiences active stresses. The stresses expand the surface of the material, pulling the solid underneath with it. This can cause the entire shape to change. This change could even be tailored by altering the elastic properties of the material. 

“This study is an important proof of concept and has many useful implications,” Dr. Anton Souslov, another author on the paper, said. “For instance, future technology could produce soft robots that are far squishier and better at picking up and manipulating delicate materials.”

Eventually, the scientists hope to develop machines with arms made of flexible material with tiny robots embedded in the surface. The technology could also be used to coat the surface of nanoparticles in a responsive, active material. This would allow them to customize the size and shape of drug delivery capsules. 

For now, the researchers have already started their next phase of work: applying the general principles they’ve learned to designing specific robots. The results of their study were published in Science Advances. 

The post Researchers aim to create soft, shape-changing robots appeared first on The Robot Report.

These nano drones use soft actuators to fly through the air. | Source: MIT

Researchers at MIT have created a tiny robot, weighing less than 1 gram, that can zip around with insect-like agility and resilience. The research team isn’t the first to take on the challenge of making tiny flying robots. After all, they do have a number of real-world applications such as pollinating a field of crops or searching for survivors in disaster areas.

Typically, these kinds of robots are made with rigid actuators built from piezoelectric ceramic materials. The rigid materials help the robot to fly, but they also make it fragile. These types of robots are unlikely to survive the number of collisions it would face in the real world.

Soft actuators can be much more resilient than rigid ones, but they present a different problem for tiny robots. They require higher voltages than rigid actuators, and more voltage means bigger power electronics that the robot isn’t capable of lifting.

Kevin Chen, an assistant professor in the department of electrical engineering and computer science, and senior author of the study, worked with his team to create a new fabrication technique for soft actuators. The produced actuators operate with 75% lower voltage than current versions and can carry 80% more payload.

The team’s rectangular robot has four sets of wings, each driven by an actuator, that beat nearly 500 times per second. The actuators work like artificial muscles, and are made up of layers of elastomer placed between two very thin electrodes and rolled into a squishy cylinder.

The wings flap by applying voltage to the actuator, which makes the electrodes squeeze the elastomer.

One defect after another

One challenge the team faced was creating more surface area for the robot. The actuator requires less voltage with more surface area, so the team tried to build as many thin layers of elastomer and electrode as possible. However, the thinner the elastomer layers are, the more unstable the robot is.

The team was able to create layers that were only 10 micrometers in thickness by rethinking the fabrication process. For example, during the spin coating process, where an elastomer is poured onto a flat surface and rapidly rotated to make thinner layers of film, the team had to contend with tiny air bubbles that the spinning process created.

“In this process, air comes back into the elastomer and creates a lot of microscopic air bubbles. The diameter of these air bubbles is barely 1 micrometer, so previously we just sort of ignored them,” Chen said. “But when you get thinner and thinner layers, the effect of the air bubbles becomes stronger and stronger. That is traditionally why people haven’t been able to make these very thin layers.”

To get around this, the team performed the vacuuming process after spin coating, before the elastomer had a chance to dry, and then baking the actuator to dry it. By solving this problem, the team was able to increase the power output of the actuator by more than 300% as well as increase the lifespan of the actuator.

Baking the elastomer layers also helped to reduce the curing time of the actuator.

The artificial muscles improve the robot’s payload and allow it to achieve better hovering performance. | Photo Credit: Kevin Chen

“The first time I asked my student to make a multilayer actuator, once he got to 12 layers, he had to wait two days for it to cure. That is totally not sustainable, especially if you want to scale up to more layers,” Chen said.

The team also ran into issues with the thin electrodes in the actuator. The electrodes are made up of carbon nanotubes that are about 1/50,000 the diameter of a human hair. The more nanotubes the actuator has the higher the actuator’s power output is.

The nanotubes, however, have sharp ends that could pierce the elastomer, so the team had to find the optimal number of nanotubes.

“Two years ago, we created the most power-dense actuator and it could barely fly. We started to wonder, can soft robots ever compete with rigid robots? We observed one defect after another, so we kept working and we solved one fabrication problem after another, and now the soft actuator’s performance is catching up,” Chen said. “They are even a little bit better than the state-of-the-art rigid ones. And there are still a number of fabrication processes in material science that we don’t understand. So, I am very excited to continue to reduce actuation voltage.”

The resulting robot could hover for 20 seconds. According to Chen, this is the longest ever recorded by a sub-gram robot. It also had a longer lifespan than other actuators, lasting more than two million cycles.

The team plans to continue testing fabrication techniques in a clean room where it won’t have to contend with dust in the air when creating the actuator layers. Chen hopes to make the layers as thin as 1 micrometer.

The post Soft actuators help MIT power nano drones appeared first on The Robot Report.

The unique bi-stable design of the actuator allows it to snap back and forth quickly with temperature changes. | Source: North Carolina State University

A research team at North Carolina State University (NCSU) has developed a quicker thermal actuator for soft robotic devices. Actuators create motion by converting energy into work.

“Using thermal actuation is not new for soft robots, but the biggest challenge for soft thermal actuators was that they were relatively slow – and we’ve made them fast,” Yong Zhu, corresponding author of the paper and the Andrew A. Adams Distinguished Professor of mechanical and aerospace engineering at NCSU, said.

The key to the team’s actuator’s quick movements is in its bi-stable design, and the shape it prefers depends on the temperature.

The research team put silver nanowires between two layers of different materials on top of each other. The two materials will heat and expand at different temperatures, as that happens the structure will bend.

At a certain point, once the structure reaches a critical temperature, it will snap into place. Once it snaps into place the structure is stable again. The same process occurs as the structure cools. The critical temperatures, however, will be different, with the heating critical temperature being higher.

“Think of a snap hair clip. It’s stable until you apply a certain amount of energy (by bending it over), and then it snaps into a different shape – which is also stable,” Shuang Wu, first author of the paper and Ph.D. student at NCSU, said.

The researchers created two prototypes to test their method. The first snaps shut or open, mimicking the movement of a venus flytrap. The second’s snapping movement helps it to crawl, moving at more than one body length per second.

The actuator has potential applications in a variety of industries, from biomedical applications to manufacturing.

Moving forward, the team hopes to automate the process by creating sensor and control mechanisms.

“We’re also interested in exploring other possible materials, so that we could fine-tune the thermal and mechanical properties,” Zhu said. “This could allow us to tailor both actuator speed and force.”

Editor’s Note: You can read the team’s entire research here.

The post NCSU researchers make faster thermal actuator appeared first on The Robot Report.

A hexapod robot with fin-ray inspired grippers that have altered crossbeams.

Many soft robotic grippers are inspired by the way a fish’s fin bends. They involve triangular components that bend inward, around an object to grip it. These components have transverse beams that sit at a 90° angle, which help the gripper to bend around an object.

These traditional grippers require a lot of force to pick up an object, and aren’t always capable of handling delicate objects without crushing them. Researchers from five different universities, led by Poramante Manoonpong from Syddansk University in Odense, are taking their inspiration from the way insects walk and grip things to develop a different way to make robotic grippers.

Instead of transverse beams sitting at a 90° angle, the international research team in Odense put the beams at 10° or 30° angles, similar to insect attachment pads. By changing the angle of these beams, researchers found that their grippers were able to bend more around objects and pick them up using 20% less force.

“We know that many animals are able to [grip or attach themselves] to surfaces using compliant pads or feet,” said Manoonpong. “We could see that the angle of insects’ attachment pads was different. Previous work has already investigated changing the angle of the gripper, but none have conducted detailed investigation on the effect of different crossbeam angles inside its frame.”

Using less force to pick up an object allows the gripper to handle more delicate items, like food items, without crushing them. When the researchers tested grippers with beams at 90° or 120°, it took much more force for the gripper to pick up an item.

The new grippers also improved the way that robots walk. When faced with uneven surfaces, like rocks, or round ones, like tubes to walk across, the robots were able to navigate the situation without complex control. The grippers resemble flippers, which can bend easily around the surface the robot walks across.

Inspired by nature

Manoonpong’s team of researchers were inspired by insect attachment pads, but they’re not the only researchers looking to nature to make advancements in robotics.

In December 2020, researchers at the University of Georgia College of Engineering developed a robotic gripper inspired by beanpoles. These soft grippers require only one pneumatic control, which simplified the operation. The gripper also had a fiber optic sensor in the robot’s elastic spine that can sense the item it’s picking up and any external disturbances.

In November 2020, researchers at UNSW Sydney created a gripper inspired by an elephant’s trunk. This soft fabric gripper was designed to handle heavy objects as well as fragile ones.

The Odense team tested their gripper model using only soft grippers. Their next challenge is to make one strong enough to handle harsher environments.

The Odense team’s entire research can be found here.

The post These robotic grippers are inspired by insects appeared first on The Robot Report.

The smart hand is soft and elastic, weighs about half a pound, and costs a fraction of comparable prosthetics.

For the more than 5 million people in the world who have undergone an upper-limb amputation, prosthetics have come a long way. Beyond traditional mannequin-like appendages, there is a growing number of commercial neuroprosthetics — highly articulated bionic limbs, engineered to sense a user’s residual muscle signals and robotically mimic their intended motions.

But this high-tech dexterity comes at a price. Neuroprosthetics can cost tens of thousands of dollars and are built around metal skeletons, with electrical motors that can be heavy and rigid.

Now engineers at MIT and Shanghai Jiao Tong University have designed a soft, lightweight, and potentially low-cost neuroprosthetic hand. Amputees who tested the artificial limb performed daily activities, such as zipping a suitcase, pouring a carton of juice, and petting a cat, just as well as — and in some cases better than — those with more rigid neuroprosthetics.

The researchers found the prosthetic, designed with a system for tactile feedback, restored some primitive sensation in a volunteer’s residual limb. The new design is also surprisingly durable, quickly recovering after being struck with a hammer or run over with a car.

The smart hand is soft and elastic, and weighs about half a pound. Its components total around $500 — a fraction of the weight and material cost associated with more rigid smart limbs.

“This is not a product yet, but the performance is already similar or superior to existing neuroprosthetics, which we’re excited about,” said Xuanhe Zhao, professor of mechanical engineering and of civil and environmental engineering at MIT. “There’s huge potential to make this soft prosthetic very low cost, for low-income families who have suffered from amputation.”

Zhao and his colleagues have published their work today in Nature Biomedical Engineering. Co-authors include MIT postdoc Shaoting Lin, along with Guoying Gu, Xiangyang Zhu, and collaborators at Shanghai Jiao Tong University in China.

Big Hero hand

The team’s pliable new design bears an uncanny resemblance to a certain inflatable robot in the animated film “Big Hero 6.” Like the squishy android, the team’s artificial hand is made from soft, stretchy material — in this case, the commercial elastomer EcoFlex. The prosthetic comprises five balloon-like fingers, each embedded with segments of fiber, similar to articulated bones in actual fingers. The bendy digits are connected to a 3-D-printed “palm,” shaped like a human hand.

Related: Watch a soft robotic hand play Mario Bros.

Rather than controlling each finger using mounted electrical motors, as most neuroprosthetics do, the researchers used a simple pneumatic system to precisely inflate fingers and bend them in specific positions. This system, including a small pump and valves, can be worn at the waist, significantly reducing the prosthetic’s weight.

Lin developed a computer model to relate a finger’s desired position to the corresponding pressure a pump would have to apply to achieve that position. Using this model, the team developed a controller that directs the pneumatic system to inflate the fingers, in positions that mimic five common grasps, including pinching two and three fingers together, making a balled-up fist, and cupping the palm.

The pneumatic system receives signals from EMG sensors — electromyography sensors that measure electrical signals generated by motor neurons to control muscles. The sensors are fitted at the prosthetic’s opening, where it attaches to a user’s limb. In this arrangement, the sensors can pick up signals from a residual limb, such as when an amputee imagines making a fist.

The team then used an existing algorithm that “decodes” muscle signals and relates them to common grasp types. They used this algorithm to program the controller for their pneumatic system. When an amputee imagines, for instance, holding a wine glass, the sensors pick up the residual muscle signals, which the controller then translates into corresponding pressures. The pump then applies those pressures to inflate each finger and produce the amputee’s intended grasp.

Going a step further in their design, the researchers looked to enable tactile feedback — a feature that is not incorporated in most commercial neuroprosthetics. To do this, they stitched to each fingertip a pressure sensor, which when touched or squeezed produces an electrical signal proportional to the sensed pressure. Each sensor is wired to a specific location on an amputee’s residual limb, so the user can “feel” when the prosthetic’s thumb is pressed, for example, versus the forefinger.

Good grip

To test the inflatable hand, the researchers enlisted two volunteers, each with upper-limb amputations. Once outfitted with the neuroprosthetic, the volunteers learned to use it by repeatedly contracting the muscles in their arm while imagining making five common grasps.

After completing this 15-minute training, the volunteers were asked to perform a number of standardized tests to demonstrate manual strength and dexterity. These tasks included stacking checkers, turning pages, writing with a pen, lifting heavy balls, and picking up fragile objects like strawberries and bread. They repeated the same tests using a more rigid, commercially available bionic hand and found that the inflatable prosthetic was as good, or even better, at most tasks, compared to its rigid counterpart.

One volunteer was also able to intuitively use the soft prosthetic in daily activities, for instance to eat food like crackers, cake, and apples, and to handle objects and tools, such as laptops, bottles, hammers, and pliers. This volunteer could also safely manipulate the squishy prosthetic, for instance to shake someone’s hand, touch a flower, and pet a cat.

In a particularly exciting exercise, the researchers blindfolded the volunteer and found he could discern which prosthetic finger they poked and brushed. He was also able to “feel” bottles of different sizes that were placed in the prosthetic hand, and lifted them in response. The team sees these experiments as a promising sign that amputees can regain a form of sensation and real-time control with the inflatable hand.

The team has filed a patent on the design, through MIT, and is working to improve its sensing and range of motion.

“We now have four grasp types. There can be more,” Zhao said. “This design can be improved, with better decoding technology, higher-density myoelectric arrays, and a more compact pump that could be worn on the wrist. We also want to customize the design for mass production, so we can translate soft robotic technology to benefit society.”

Editor’s Note: This article was republished from MIT News.

The post Inflatable robotic hand gives amputees real-time tactile control appeared first on The Robot Report.

A team of researchers from the University of Maryland has 3D printed a soft robotic hand that is agile enough to play Nintendo’s Super Mario Bros. – and win!

The feat demonstrates a promising innovation in the field of soft robotics, which centers on creating new types of flexible, inflatable robots that are powered using water or air rather than electricity. The inherent safety and adaptability of soft robots has sparked interest in their use for applications like prosthetics and biomedical devices. Unfortunately, controlling the fluids that make these soft robots bend and move has been especially difficult – until now.

The key breakthrough by the team, led by University of Maryland assistant professor of mechanical engineering Ryan D. Sochol, was the ability to 3D print fully assembled soft robots with integrated fluidic circuits in a single step.

“Previously, each finger of a soft robotic hand would typically need its own control line, which can limit portability and usefulness,” explains co-first author Joshua Hubbard, who performed the research during his time as an undergraduate researcher in Sochol’s Bioinspired Advanced Manufacturing (BAM) Laboratory at UMD. “But by 3D printing the soft robotic hand with our integrated fluidic transistors, it can play Nintendo based on just one pressure input.”

As a demonstration, the team designed an integrated fluidic circuit that allowed the hand to operate in response to the strength of a single control pressure. For example, applying a low pressure caused only the first finger to press the Nintendo controller to make Mario walk, while a high pressure led to Mario jumping. Guided by a set program that autonomously switched between off, low, medium, and high pressures, the robotic hand was able to press the buttons on the controller to successfully complete the first level of Super Mario Bros. in less than 90 seconds.

“Recently, several groups have tried to harness fluidic circuits to enhance the autonomy of soft robots,” said recent Ph.D. graduate and co-first author of the study Ruben Acevedo, “but the methods for building and integrating those fluidic circuits with the robots can take days to weeks, with a high degree of manual labor and technical skill.”

To overcome these barriers, the team turned to “PolyJet 3D Printing,” which is like using a color printer, but with many layers of multi-material ‘inks’ stacked on top of one another in 3D.

“Within the span of one day and with minor labor, researchers can now go from pressing start on a 3D printer to having complete soft robots – including all of the soft actuators, fluidic circuit elements, and body features – ready to use,” said study co-author Kristen Edwards.

The choice to validate their strategy by beating the first level of Super Mario Bros. in real time was motivated by science just as much as it was by fun. Because the video game’s timing and level make-up are established, and just a single mistake can lead to an immediate game over, playing Mario provided a new means for evaluating soft robot performance that is uniquely challenging in a manner not typically tackled in the field.

In addition to the Nintendo-playing robotic hand, Sochol’s team also reported terrapin turtle-inspired soft robots in their paper. The terrapin happens to be UMD’s official mascot, and all of the team’s soft robots were printed at UMD’s Terrapin Works 3D Printing Hub.

Another important benefit of the team’s strategy is that it’s open source, with the paper open access for anyone to read as well as a link in the supplementary materials to a GitHub with all of the electronic design files from their work.

“We are freely sharing all of our design files so that anyone can readily download, modify on demand, and 3D print – whether with their own printer or through a printing service like us – all of the soft robots and fluidic circuit elements from our work,” said Sochol. “It is our hope that this open-source 3D printing strategy will broaden accessibility, dissemination, reproducibility, and adoption of soft robots with integrated fluidic circuits and, in turn, accelerate advancement in the field.”

At present, the team is exploring the use of their technique for biomedical applications including rehabilitation devices, surgical tools, and customizable prosthetics. As Sochol is a faculty affiliate of the Fischell Department of Bioengineering as well as a member of both the Maryland Robotics Center and the Robert E. Fischell Institute for Biomedical Devices, the team has an exceptional environment to continue advancing their strategy to address pressing challenges in biomedical fields.

Editor’s Note: This article was republished from the University of Maryland.

The post Watch a soft robotic hand play Super Mario Bros. appeared first on The Robot Report.

Toyota Research Institute’s Soft Bubble Gripper. | Credit: Toyota Research Institute

As part of its effort to improve robotic manipulation, Toyota Research Institute (TRI) in September 2020 unveiled its Soft Bubble Gripper. It features two fingers that both combine the advantages of compliant gripping with real-time, real-world tactile sensing. TRI said the Soft Bubble Gripper can perform a range of tasks that would be difficult for rigid grippers to accomplish.

Now TRI is seeking your help to continue to advance the field of soft robotics. TRI is sharing the design source files and full build instruction so research institutions and robotics can build their own soft gripper. You can learn how to build its Soft Bubble Gripper here.

“The soft robotics community is small, and the visuo-tactile sensing community is even smaller,” said Alex Alspach, TRI’s robotics tactile team manager and the lead developer of the Punyo Soft Bubble Gripper. “By sharing the blueprints for this gripper with the world, we hope that our friends and colleagues can test our technology, improve upon it, and take us closer to building robotic assistants that help to provide independence, dignity and joy to those with disabilities or age-related challenges.”

Most robots are hard to the touch and use rigid grippers. TRI claims its air-filled, elastic bubble design allows robots greater flexibility to hold objects better. When combined with cameras on the inside, this shape and force sensing gripper enables robots to respond to and control an object when it slips or moves.

The Punyo bubbles employ state of the art visuo-tactile sensing techniques that allow a robot to recognize objects by shape, track their orientation in its grasp and sense forces as it interacts with the world. This feedback is critical as robots learn to push and pull on the world safely and robustly while assisting people by opening doors, putting things away, using household tools, and other domestic tasks.

TRI has said its Soft Bubble Gripper builds on its manipulation research toward making human-assist robots reliable and robust. Even without the sensing capability, the stretchy material or low stiffness makes for a superior gripper in comparison with standard soft grippers, said the researchers. It can conform to a wide variety of shapes and get a stable grasp.

To inspire collaboration and further the research on soft bubble grippers, TRI hosted VisuoTactile 2020, a workshop of thought-leaders in visuo-tactile technology.

The post TRI shares design of Soft Bubble Gripper to advance technology appeared first on The Robot Report.

mGrip is part of Soft Robotics’ gripper offerings. | Credit: Soft Robotics

Soft Robotics has added JMP Solutions to its Preferred System Integrator program. This partnership will help expand robotic adoption in the food sector enabled by Soft Robotics food-grade soft gripping, 3D vision, and AI technologies.

Soft Robotics Preferred System Integrator Program is an initiative to help integrators win more business with its industry-leading technologies, including the mGrip modular gripping system and SoftAI. SoftAI combined with 3D vision and mGrip, enables machine builders to deliver reliable singulated and bulk picking solutions for applications that couldn’t previously be automated due to challenges with delicate, variable, or easily damaged objects.

Through this program, integrators will benefit from personalized application support, product training, and growth opportunities with Soft Robotics far-reaching network in the food and beverage and consumer-packaged goods spaces. The program aims to solve difficult end-user automation problems by connecting members with trusted integrators that have a proven track record of success.

“Soft Robotics is excited to have JMP Solutions join our Preferred System Integrator Program,” said Harley Green, director of business development at Soft Robotics. “As a market leader in food and beverage, JMP offers an innovative approach to solving customer challenges and we’re excited to take this opportunity to further build our relationship to deliver on their customers’ needs. With a customer-first approach, and over 30 years experience, JMP welcomes new and innovative solutions to their toolbox to solve some of the toughest automation challenges in the food and consumer goods industries.”

“Our partnership with Soft Robotics has helped us to expand our scope along the path of working collaboratively with our customers and designing innovative robotic cells to solve their hardest to automate challenges,” said Mike Bannister, director of sales at JMP Solutions. “Hygienic, food packaging applications are a focused market segment for us, and the food-grade design of the Soft Robotics EOATs combined with their almost life-like dexterity has allowed us to take our capabilities in this area to the next level, further setting ourselves apart from our competitors. Our team is really looking forward to seeing the exciting developments that the future holds for this relationship.”

The post Soft Robotics adds JMP Solutions to integrator program appeared first on The Robot Report.

MIT built a deep learning neural network to aid the design of soft robots, such as these iterations of a robotic elephant. | Photo Credit: MIT

There are some tasks traditional robots – the rigid and metallic kind – simply aren’t cut out for. Soft-bodied robots, on the other hand, may be able to interact with people more safely or slip into tight spaces with ease. But for robots to reliably complete their programmed duties, they need to know the whereabouts of all their body parts. That’s a tall task for a soft robot that can deform in a virtually infinite number of ways.

MIT researchers developed an algorithm to help engineers design soft robots that collect more useful information about their surroundings. The deep learning algorithm suggests an optimized placement of sensors within the robot’s body, allowing it to better interact with its environment and complete assigned tasks. The advance is a step toward the automation of robot design.

“The system not only learns a given task, but also how to best design the robot to solve that task,” said Alexander Amini. “Sensor placement is a very difficult problem to solve. So, having this solution is extremely exciting.”

The research will be presented at the IEEE International Conference on Soft Robotics and will be published in the journal IEEE Robotics and Automation Letters. Co-lead authors are Amini and Andrew Spielberg, both PhD students in MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). Other co-authors include MIT PhD student Lillian Chin, and professors Wojciech Matusik and Daniela Rus.

Related: Designing a quadruped controlled & powered by pneumatics

Creating soft robots that complete real-world tasks has been a long-running challenge in robotics. Their rigid counterparts have a built-in advantage: a limited range of motion. Rigid robots’ finite array of joints and limbs usually makes for manageable calculations by the algorithms that control mapping and motion planning. Soft robots are not so tractable.

Soft robots are flexible and pliant — they generally feel more like a bouncy ball than a bowling ball. “The main problem with soft robots is that they are infinitely dimensional,” said Spielberg. “Any point on a soft-bodied robot can, in theory, deform in any way possible.” That makes it tough to design a soft robot that can map the location of its body parts. Past efforts have used an external camera to chart the robot’s position and feed that information back into the robot’s control program. But the researchers wanted to create a soft robot untethered from external aid.

“You can’t put an infinite number of sensors on the robot itself,” said Spielberg. “So, the question is: How many sensors do you have, and where do you put those sensors in order to get the most bang for your buck?” The team turned to deep learning for an answer.

The researchers developed a novel neural network architecture that both optimizes sensor placement and learns to efficiently complete tasks. First, the researchers divided the robot’s body into regions called “particles.” Each particle’s rate of strain was provided as an input to the neural network. Through a process of trial and error, the network “learns” the most efficient sequence of movements to complete tasks, like gripping objects of different sizes. At the same time, the network keeps track of which particles are used most often, and it culls the lesser-used particles from the set of inputs for the networks’ subsequent trials.

By optimizing the most important particles, the network also suggests where sensors should be placed on the robot to ensure efficient performance. For example, in a simulated robot with a grasping hand, the algorithm might suggest that sensors be concentrated in and around the fingers, where precisely controlled interactions with the environment are vital to the robot’s ability to manipulate objects. While that may seem obvious, it turns out the algorithm vastly outperformed humans’ intuition on where to site the sensors.

Related: Metamaterials could lead to transforming robots

The researchers pitted their algorithm against a series of expert predictions. For three different soft robot layouts, the team asked roboticists to manually select where sensors should be placed to enable the efficient completion of tasks like grasping various objects. Then they ran simulations comparing the human-sensorized robots to the algorithm-sensorized robots. And the results weren’t close.

“Our model vastly outperformed humans for each task, even though I looked at some of the robot bodies and felt very confident on where the sensors should go,” said Amini. “It turns out there are a lot more subtleties in this problem than we initially expected.”

Spielberg said their work could help to automate the process of robot design. In addition to developing algorithms to control a robot’s movements, “we also need to think about how we’re going to sensorize these robots, and how that will interplay with other components of that system,” he said. And better sensor placement could have industrial applications, especially where robots are used for fine tasks like gripping. “That’s something where you need a very robust, well-optimized sense of touch,” said Spielberg. “So, there’s potential for immediate impact.”

“Automating the design of sensorized soft robots is an important step toward rapidly creating intelligent tools that help people with physical tasks,” said Rus. “The sensors are an important aspect of the process, as they enable the soft robot to “see” and understand the world and its relationship with the world.”

Editor’s Note: This article was republished from MIT News.

The post Deep learning optimizes sensor placement for soft robots appeared first on The Robot Report.