As versatile, commonly used pieces of equipment, skid steers by default will be involved in accidents, colliding with objects, other pieces of equipment and even pedestrians. Regulators do not break down accidents to identify specifically those involving skid steers, but the equipment is common enough that accounts make the news--including the death of a two-year-old child in April of 2022.
But does it have to be this way?
The Occupational Safety and Health Administration (OSHA) cautions against mistakes that can lead to rollovers and crush injuries.Occupational Safety and Health Administration (OSHA)
There are common sense steps contractors can take to prevent injuries and deaths among both operators and those working near skid steers. The Occupational Safety and Health Administration (OSHA) cautions against mistakes that can lead to rollovers and crush injuries. OSHA also requires skid steer operators receive training under CFR 1926.21(b)(s) and CFR 1926.20(b)(4). But collisions with objects and pedestrians, including backover accidents, still occur.
Accident Prevention Technology Advances
Original equipment manufacturers (OEMs) have long included safety features in skid steers, including backup cameras and indicator tech for cross-slopes to prevent rolling and slope to prevent tips. But aftermarket technology is also available that enables a contractor to automate safety over and above what their OEM delivers. Taylor Sudden Service, for instance, offers its Vision Plus Integrated pedestrian detection system specifically on its Taylor X-Series forklifts and material handling equipment, but sells its Vision Plus Pro and Vision Plus Standard products aftermarket for most off-highway equipment, including skid steers. This technology provides a visual and audio alarm to alert a skid steer operator of pedestrians entering the machine path.
Sensata Technologies offers its PreView Sentry active radar system to detect objects and pedestrians in the skid steer’s large blind spot. This technology features a fully adjustable detection zone up to 98 feet, identifies velocity of up to 16 people and objects simultaneously and built-in failsafes. It can also be integrated with a skid steer’s braking system if an object is detected.
Sensata Technologies offers its PreView Sentry active radar system to detect objects and pedestrians in the skid steer’s large blind spot.Sensata Technologies
But emerging technologies are making skid steer pedestrian detection even smarter and more reliable while providing more effective interventions to avoid collisions.
Paris-based Arcure Blaxtair is offering its AI-driven pedestrian recognition technology through an aftermarket stereo camera installation that pairs with a cloud application. The technology is already proven on major infrastructure job sites around the world, most notably in France, Hong Kong and Turkey. The company claims the technology can reduce vehicle/pedestrian accidents by two thirds.
Arcure Blaxtair has been active in the United States with the Blaxtair pedestrian proximity detection product since 2019, and has fitted its system on machines used on the Ohio River Bridge—its first U.S. use case in construction, with most of their installations here having been on forklifts.
While the products are installed as an option by original equipment manufacturers (OEMs) in Europe, sales of the Blaxtair will for the time being be focused on selling units directly to equipment fleet owners as they work to build relationships with OEMs serving North America.
“This is new technology, so the first thing we want to do is convince the market it is valuable, and we are starting by demonstrating the product and selling to end users,” Arcure Blaxtair Founder/CEO Franck Gayraud said.
A Caterpillar 259D track loader with the Blaxtair pedestrian detection system installed. The stereo camera, combined with advanced machine learning algorithms, can sense people in the machine's path and either alert the operator or slow and stop the machine.Arcure Blaxtair
Slow to a stop
The Blaxtair system, which is comprised of a stereo camera that compares images in its field of vision to a sophisticated machine learning-generated algorithm to identify pedestrians, is now integrated with the Xwatch Safety Systems automated system to slow and even bring to a stop a piece of equipment on which it is installed if a pedestrian is detected. The on-equipment components are paired with a cloud application that provides monitoring, analytics and can update the firmware on the Blaxtair unit.
“We consider Blaxtair as an emergency sensor,” Gayraud said. “We don’t have increasing frequency of beep as a warning to a pedestrian. Because we are dealing with safety, we don’t want to add any distraction to the driver. First, there is a visual and sound alarm. Then, the Blaxtair will stop the movement of the vehicle—we cannot put on the brake. The Blaxtair sends an electrical signal to do a smart slowdown to avoid unbalancing the vehicle.”
Paris-based Arcure Blaxtair is offering its AI-driven pedestrian recognition technology through an aftermarket stereo camera installation that pairs with a cloud application.Arcure Blaxtair
In Europe, Blaxtair units have been factory-fitted on Liebherr equipment, which uses the intelligence from the system to slow down based on the current speed and load in the bucket. Manufaturers have used the Blaxtair in two different settings—to slow down when a pedestrian is close but not imminently in danger and then gradually slow to a stop as the pedestrian gets closer. The system has been installed widely on skid steers in Europe and Canada.
For aftermarket installations, the Blaxtair will generally send just one signal to slow or stop the vehicle, and will be less configurable than when the interface is designed with the hardware installed by an OEM. With OEM integration, it can also control the bucket or other moving parts.
Blaxtair Tech Stack
The Blaxtair uses a pure vision system rather than radar, laser or infrared sensors, functioning much like human eyes including the depth of field provided by stereo vision. Because of the system’s reliance on reflected light, some use cases add an artificial light to the vehicle to ensure the camera can see. The Blaxtair inputs using 3D imaging, and then applies a classification algorithm to identify pedestrians by shape and other criteria.
There are free pedestrian identification algorithms on the internet, which have been trained on a very small database but delivered pretty good performance until you change the shape, background or clothing. Then, detection drops dramatically, according to Gayraud. So the company developed a broad neural network that was trained for a period of 12 years using cameras installed on equipment at a recycling plant.
“After 12 years of exposure, we had a pretty good-sized database and good diversity,” Gayraud said. “Every machine had a Blaxtair, collecting data that was transferred to an internal server. A team of people then labelled the data, drawing a square to identify person, recording whether it was day or night and what they are wearing. At this point in time, we have millions of images collected and labeled so the algorithm can learn from them.”
But the human population is incredibly diverse, which means a neural network for pedestrian detection must keep expanding.
The Blaxtair uses a pure vision system rather than radar, laser or infrared sensors, functioning much like human eyes including the depth of field provided by stereo vision.Blaxtair
“We fitted the Blaxtair on an excavator in New Guinea,” Gayraud said. “The people are very small, and for the algorithm, this was very new. They are not the same shape as children, or the size of an adult. We had to change the algorithm to also detect also this type of person.”
Currently, the technology will sense a pedestrian from zero feet away to just over 26 feet away, and the company has a goal to double that range in future iterations.
As an option, a Blaxtair user can subscribe to the cloud application, which relies on an additional module on the hardware unit that relays image data and aggregated data on detections for analysis and to facilitate safety management processes. Data and images collected can influence things like separation of different disciplines or processes on a site or safety training. A Blaxtair cloud application user can also track reduction of accidents or close encounters with pedestrians for their own purposes or to share with stakeholders. In Europe, this data and installation of Blaxtair units has helped contractors reduce their insurance premiums, but this is not something that has happened yet in North America.
The Blaxtair currently communicates only over a cellular network, but a relay of information via satellite for use in areas with no cell service or wifi for when a piece of equipment returns to a depot may be available at some point, according to Gayraud.
The Blaxtair technology is designed for a broad spectrum of equipment, but according to Blaxtair Technical Expert Maxime Thevenin, delivers tremendous value on skid steers. Thevenin said skid steers should be equipped with pedestrian detection "every time" because they move quickly and are completely blind to the rear at close range.
“It can be a bit tricky to install as there's no space inside, but we never had issues to install it,” he said. "It is important to be careful with the passthrough of the cable between the processing unit and sensor head, as the cabin can be lifted completely to access the engine. But besides that, there is nothing very specific–everything is compact so an installer should just not hesitate to remove more panels."
Training a Machine to See People
Effective pedestrian detection requires a deep approach to machine learning, not only so technology can identify a person from other objects, but to gauge the direction a person is moving and the speed they are moving at. Accuracy is also important to avoid false positives—sending up proximity alarms or slowing or stopping a machine needlessly. Between the trajectories pedestrians take and their different appearances, this means pedestrian detection must rest on a deep neural network informed by millions of images and situations.
“We started with free algorithm on the internet, which would have been trained on a very small database and it means you would have good performance on various shapes,” Gayraud said. “But if you change the shape, background or clothing, accuracy, it drops dramatically … We fitted a Blaxtair on an excavator in New Guinea. The people there are very small, and at that point our algorithm was very new. It saw something that was not the same shape as children or the size of adult. We had to change the algorithm to detect this type of person. But after 12 years of experience, we have good size and good diversity.”
Towards an Autonomous Skid Steer
Applications like the Blaxtair that go beyond proximity alerts to actually slow or stop the skid steer are one step towards full autonomy. Remote control, as pioneered by companies like Torc Robotics in partnership with Caterpillar, has been added to skid steers—which has since yielded to the Cat Command product. Other OEMs ranging from Kanga to John Deere to Bobcat have followed suit. In May of 2022, Caterpillar then launched the 299D3 compact track loader with semi-autonomous load and haul capacity. A roof rack unit contains the cameras and LIDAR sensors required for the loader to collect data and take action based on its surroundings.
San Francisco-based Built Robotics has outfitted multiple equipment classes with its technology.Built Robotics
Meanwhile, San Francisco-based Built Robotics has outfitted multiple equipment classes with its automation exoskeleton, which includes pedestrian detection.
“If the robot sees a person, it will stop with plenty of range,” Built Robotics Vice President Gaurav Kikani said. “We have massive data repositories of footage from job sites and things we trained the machine learning model on to ensure low false positive and zero false negatives. We have a variety of sources to tune our algorithms. We also have external data sources to maximize the diversity in our sample. The short answer is we have trained our models on diverse environments and circumstances. We want our machine learning to detect at such a far range that personal details should be irrelevant.”
Like Blaxtair Built Robotics’ pedestrian detection has been thrown for a loop by geographic differences, as well.
“We have encountered a challenge in Australia where their personal protective equipment is different, that represents a different profile,” Kikani said. “We try to train the robot on the local environments. We encounter a lot more livestock in Australia—all of these are things our customer success teams are equipped to deal with.”
“Autonomy is the future,” Gayraud said. “We started with personnel detection because it was needed and it is an existing market. But we are already working on autonomy, and we see it now mostly in mining sites. The constraint is there cannot be a lot of people around.”
