The depth sensing capabilities of the Apple car may not be limited only to LiDAR, Apple suggesting that a version using an infrared camera and light pulses could also be used to detect road obstructions.
Of the many challenges associated with creating an autonomous vehicle, perhaps the most important is getting the autonomous systems of the car to actually get data from the road itself. There are a myriad of sensors and systems available, but with considerable variations in terms of accuracy, cost and physical requirements.
For example, LiDAR is a useful technology for depth sensing, but it remains relatively expensive to use due to the parts involved. By taking advantage of cheaper components in a different system, Apple could get closer to the accuracy of LiDAR without much cost.
In a patent granted to Apple by the US Patent and Trademark Office on Tuesday entitled “Remote sensing for the detection and range of objects”, Apple suggests such a system, effectively composed of a controllable light source, a synchronization circuit and a circuit or processor to determine the range.
For Apple’s proposal, the light source could be a vertical cavity surface emission laser (VCSEL), but the most interesting version is its use of near infrared light. A camera is also used to create a video stream of a monitored area, which could be an infrared camera in the latter case.
Both versions would depend on synchronization circuits to emit pulses of light into the environment, which are then reflected by nearby objects and returned to the camera. In short, by measuring the time it takes for the light pulses to return to the camera, the range finding circuit could determine how far a detected object is, with the results being able to be supplied to other computer systems used to conduct.
An example of how pulses can align with exposure windows to determine the distances of an object.
The range determination would be calculated using a window synchronization circuit, generating multiple exposure windows corresponding to the light pulses. The presence of a light pulse in a window would inform the distance determination circuit of the time it took for the pulse to travel the entire distance to the object and vice versa.
For example, a pulse detected in a time window could suggest a short distance, while the next window would set this distance as greater, and with longer distances with the following windows.
When a camera is used, it can cover a wide field of vision, allowing a large area to be monitored by the system rather than by individual points, as LiDAR generally covers. In addition, it also improves the use of standard video cameras, which use context, contrast and color to determine the placement of objects in the context of image processing, since the proposed system would only have to use the detected presence of infrared light pulses to determine the distance.
It is proposed that the system can be used in conjunction with an existing LiDAR configuration, to create a hybrid system that detects objects at different levels of accuracy. The configuration of the lower precision light pulse camera detailed in the patent could be used to generate areas of interest for the autonomous system of the vehicle, which can then indicate where the LiDAR element should concentrate its scanning efforts.
The IR system could be used in tandem with another, such as LiDAR.
Indeed, this allows complete coverage of the scene, but with a higher degree of detection accuracy when detecting objects and telemetry in selected areas, if applicable.
The use of detection windows can also be used to improve the accuracy of the proposed system. After a first wave of pulses and exposure windows, the system processor may then decide that a second wave may require greater resolution of the monitored distances.
To do this, time windows can be shortened, increasing the number of ranges in which the distance can be calculated to be inside.
The patent lists its inventors like Richards E. Bills, Micah P. Kalscheur, Evan Cull and Ryan A. Gibbs.
Apple files numerous patent applications on a weekly basis, but although the presence of a patent filing indicates areas of interest for the company’s research and development efforts, there can be no assurance that the concepts described will appear in a future product or service.
It is generally believed that the “Apple car” has some form of autonomous driving technology. The company has publicly tested its driving systems on special test benches in California for a number of years, with vehicles largely relying on LiDAR units attached to the outside of the car.
In 2017, Apple released a research paper explaining how its LiDAR-based 3D object recognition works for autonomous vehicles.
As with its extensive research on the subject, Apple has also obtained numerous patents and filed more applications over time for the technology it has developed.
In 2016, he developed a method to “avoid collisions with arbitrary polyonal obstacles, usable by an autonomous driving system, as well as a new form of LiDAR mapping the same year. A “trust” system has also been proposed to allocate resources to specific data sources, rather than wasting the processing on a giant mountain of data generated by on-board sensors.
The idea of using the latest system proposed with other sensors was also raised, with the patent “Sensor data shared on sensor processing pipelines” of June 2020, suggesting how processing systems for different sources of data could be stacked in a single pipeline.
As for how this plethora of sensors could appear on the latest Apple car, Apple might be able to hide it all. Another patent granted in October 2019 proposed the idea of placing sensors in the car body, such as his research on the manufacture of compact and inexpensive radar systems.