Sufficient warning

The Mobileye system alerts drivers with audible and visual alert cues.

Only a test

Already tested out on the bustling streets of major metropolitan areas like New York, the Mobileye Shield+ system also has become the main component of TTI’s Automated and Connected Vehicle Test Bed to Improve Transit, Bicycle, and Pedestrian Safety.

“Ultimately, the goal of this project would be to come up with technologies to reduce the incidents between pedestrians and cyclists with buses,” Research Project Manager Wade Odell of TxDOT’s Research and Technology Innovation Office told TM&E. Odell also cited the number of traffic incidents that occur in a given year in relation to pedestrians and cyclists, as approximately 4,700 pedestrians were killed in traffic crashes in 2012, in addition to 700 cyclists. Another 49,000 traffic injuries among pedestrians and cyclists occurred that year as well, according to Odell.

Though not all traffic-incident casualties can be attributed to transit vehicles, the fact remains that there is room for safety improvements on buses, as 73 pedestrians were killed as a result of a collision with a bus on U.S. roadways in 2014, according to the National Highway Traffic Safety Administration. “If we can cut down on those numbers, then we’re certainly doing our part in terms of enhancing safety,” Odell said.

With this in mind, the TTI research team put together a test bed on the Texas A&M campus, using the university bus system to assess the effectiveness of the Mobileye technology. “We just used an existing route as our test route,” Shawn Turner, head of TTI’s Mobility Division, told TM&E. “Because of the fact that it was only a warning system, and it wasn’t taking over vehicle control, we felt pretty comfortable applying it on one of the existing bus routes.”

Overall, the TTI research team received positive results from the on-campus pilot, which has been in operation since March. “We got pretty much all positive feedback from the drivers we interviewed,” Turnbull said. “The drivers here—since they’re also college students—are one of the more challenging driver groups to work with because they do have less experience than a seasoned 20-year veteran out in some major urban transit system. So providing them with systems like this can provide, we think, the benefit.”

This initial stage of testing is the “concept of operations” stage for the TTI research team. According to Odell, the first phase is concluded, and the team is awaiting funding approval from the TxDOT administration to proceed with the next phases. “Phase two would be what’s called the ‘implementation or demonstration’ stage,” he said. “Ultimately, phase three would be implementation out in the field—actual open-course field studies for clarification.”

Creating a vision

Rosco Vision Systems began a relationship with Mobileye, a global pioneer in advanced collision avoidance systems, just under a decade ago to pursue pedestrian-sensing technology for buses. At the beginning of this partnership, the technology “hadn’t fully evolved to be able to handle the complex pedestrian environment around the surrounding areas of buses,” Ben Englander, vice president of engineering for Rosco Vision Systems, told TM&E.

That changed when Mobileye introduced its EyeQ2 chip, which according to Englander had a far greater capability to meet the demands of a pedestrian environment.

Aware of Rosco’s history with bus camera systems and other long-vehicle safety products, Mobileye reconnected with Rosco to develop a bus-specific application for its technology. “[Mobileye] had been looking at the bus market,” Englander said. “They were seeking a partner like Rosco that had an understanding of buses in general and the demands of vision around buses.”

Throughout 2014, Mobileye and Rosco Vision worked together to develop the Shield+ system, which was first demonstrated in December of that year at the Florida Autonomous Vehicles Conference in Orlando. According to Englander, Rosco specifically designed and built elements of the system such as the displays that provide the bus driver with a visual and audible alert to the presence of a vulnerable road user. “What Rosco did with Mobileye was develop the driver interface, and the connectivity between those driver interfaces and the cameras,” he said.

Since the development of the Shield+ system, Mobileye and Rosco Vision Systems have implemented tests for the technology, including within the New York City Department of Parks and the New York City Department of Sanitation, which have several sanitation trucks running the Shield+ system. Such tests demonstrate the system’s potential beyond just transit use, since the technology can assist drivers of a variety of long vehicles. “If a vehicle is a long vehicle and it’s going to be making turns, running the risk of blind-spot issues for the driver, this system has application,” Englander said.

A pedestrian sensor mounted in the cab of a test vehicle.

Fair warning

Combining the collision avoidance expertise of Mobileye and the vision safety capability of Rosco Vision Systems, Shield+ operates with an assemblage of vision sensors and cameras, with each found in different areas of the bus or long vehicle where the system is in place. The cameras/sensors are located on the outside of the bus, with two on the rear of the vehicle—one on the right and the other on the left—while the other two are positioned at the center of the windshield in the front of the vehicle and at the left corner of the windshield.

The sensors are accompanied by a warning system installed inside the vehicle in the driver’s line of vision, which includes visual and audible components. The visual display connected to the system is an icon in the shape of a pedestrian, which lights up yellow as an initial warning that a pedestrian or bicyclist is within range of collision with the bus. The next stage of warning is when the indicator turns red and provides an audible warning, communicating to the driver that collision is imminent.

“Essentially, the machine-vision system takes this video input, analyzes it, detects whenever there’s people within the scene, and it knows the speed of the bus,” Turner said. “It provides a warning to the bus driver when the estimated time to collision is just a few seconds. And that’s essential to give the driver enough time to react, and to make an evasive maneuver, either turning or breaking.”

Aside from providing a warning system, the Shield+ also is equipped with telematics capabilities. Mobileye implemented what it calls “Vision Zero mapping capability” within the technology, which allows data points to be stored every time the system detects either the presence of a pedestrian in a danger zone in relation to the moving bus or a pedestrian on a collision course with the vehicle. “These buses and other long vehicles that are running around are sending data points to their telematics cloud server, mapping those data points on the streets,” Englander explained. Each data point in the system has a status of either a pedestrian detection zone (PDZ), a place where a pedestrian or bicyclist is recognized as being in an area where the driver needs to be aware of them (a yellow detection); or where the system has determined that the cyclist or pedestrian is on a collision course with the vehicle (a red detection).

According to Englander, Mobileye and Rosco Vision Systems are accumulating these data points in a cloud server in order to help city planners, departments of transportation and transit authorities visualize the hot spots for potential negative interaction between buses and pedestrians.

“Fortunately, we don’t get a lot of reds,” Englander said. “The system is designed in most situations to be fairly quiet and not to have very many collision warnings, because they don’t happen very often.”

Exterior-mounted cameras work with the technology embedded in the bus cab to alert drivers of potential hazards or obstructions.

Towards autonomy

A major aspect of the implementation of the Shield+ system is the potential for the technology to move toward an autonomous vehicle system. Automated buses are nothing particularly new, as companies like Mercedes-Benz have been testing such technology as recently as this past summer in the Netherlands. Similar in certain ways to the Mobileye Shield+ system, Mercedes’ CityPilot technology also is outfitted with multiple cameras as well as a pedestrian detection system. In other words, the Shield+ system possesses the foundations of an automated vehicle system.

“What you would do with an autonomous bus is you would combine this system [Shield+] with several other machine-vision systems that would essentially allow a bus to follow a given path,” Turner said. The only thing needed, he added, would be a navigation system to make the bus follow such a path. “But then the issue is what happens if somebody steps out in front of that bus on its navigation path?” he said, which is where he indicated the Shield+ technology comes in. “The bus has to be able to respond instantaneously to different objects that might come in its path.” The Shield+ technology provides the aspect of detection and warning needed in order to alert either the driver or the autonomous system itself to the presence of an obstacle or the potential for collision.

The active components of the Shield+ system would still need further development to become more autonomous, according to Englander. “In buses, it’s a very different world than the active nature of technology in cars,” he said. “Active breaking exists in cars already; but in buses, in particular where you have standing passengers, there’s a need for more work to be done to manage their safety.”

For now, Englander emphasized, the technology as it is adequately stands alone as a useful implement for pedestrian and cyclist detection on long vehicles. Going beyond the test bed at Texas A&M and the pilot in New York, the system will continue to move forward as more pilots are created to test this technology in other transit systems around the country.

About The Author: Bruns is associate editor of TM&E.