FMCW LIDAR is coming, what does it mean?

Topic: 
Demo point cloud from Blackmore LIDAR with speed shown as "redshift" colour.

There are lots of LIDARs out there. The vast majority use near-infrared light pulses, or sometimes a flash. They steer the beam either with rotating parts, mirrors, MEMs or in some cases solid state approaches. All those differences are important, but they come down to the same thing in the end -- scanning to get a distance from time-of-flight, along with a reflection intensity.

There are also longer wave LIDARs in the 1500nm band. These can have more power and range due to eye safety, but cost more because you can't do it on silicon. But again, they all work by measuring the time for a light pulse to make the return trip with a timer.

A new breed of LIDAR is coming which uses a different technique, a method that is the standard method for automotive radar today, known as frequency modulated continuous wave (FMCW.) If you send out a wave but constantly vary the frequency, you can look at the frequency of the return. Based on what you get back, you can say, "Oh, that's the frequency I put out 300 nanoseconds ago, so this signal did a 100m round trip."

It's not quite that simple, because the Doppler effect changes the frequency of waves that bounce off moving targets. That's a great blessing and a minor curse. The minor curse is you have to play some tricks to figure out what's happened when the frequency changes from both your variation and Doppler, but once you do, you now learn not just how far away your target is, but how fast it is moving relative to you.

All the LIDARs currently used in robocars just get back distance and intensity. The radars get distance, intensity and velocity but they get it with very poor resolution and with lots of confusion from bounced waves.

Recently, startup Blackmore LIDAR released a demo of their FMCW LIDAR in action. A point cloud is visible above. They are not the only ones working on this, another company called AEVA has a project and I have heard whispers of a few other stealthed projects.

What's not to love?

Getting velocity in your LIDAR return is obviously a plus, as long as it doesn't come with a major cost in range, price or other important components. It means the vehicle can immediately tell most non-stationary obstacles from stationary ones with no computation needed. Today, a robocar gathers multiple frames of LIDAR point cloud, typically 100ms apart. Using these frames, it finds groupings of points, and notices when they move from one frame to another. To do this it must reliably group the points and understand that a grouping in frame 2 is the same as a grouping from frame 1, even though it has moved slightly and may have changed shape and orientation. Once it does this, it can see how far it moved, and deduce a velocity. If it needs 3 or 4 frames to do this, that can mean up to 400ms of latency while figuring it out. Knowing right away is sweet.

This technique can also make it easier to distinguish close things. A motorcycle passing close to a truck might seem like one object, but if they are at different speeds, Doppler can make that immediately apparent with no ambiguity.

Finally, some objects consist of things moving at different speeds. Most famously, the legs of a cyclist. As they spin, they keep going back and forth, and that shows up very clearly on accurate Doppler. (In fact, it shows up nicely on radar too.)

Combining coherent beams is also reportedly more immune to both crosstalk from other LIDARs and interference from the sun and other external light sources.

The result is faster, more accurate detection.

It should be noted though, that things moving horizontally, perpendicular to you, show almost no Doppler signature. That's why you see problems like the Tesla fatality in Florida, where Tesla's system could not identify a giant truck crossing the road in front of it. With no Doppler, the radar could not tell it from the regular radar returns that come from stationary objects, which are always present. (LIDAR would have of course seen the truck and braked, but taken time to figure out how it was moving.)

Another famous crosser was the victim of Uber's fatality. She also would have had minimal Doppler on radar or this type of LIDAR. In those cases, you still have to follow the technique of tracking an object in the view from frame to frame and figuring out speed. Sadly, many of the "nightmare" scenarios for robocars involve things that are crossing your path rather than moving along with it or towards you. That's why people continue to want more resolution, more range, and to fuse the results from multiple sensors to get the best appraisal of what everything they detect is and what it's doing.

In effect, in certain situations, FMCW effectively gives you a bit more range. As a car drives, obstacles will become visible to its LIDAR at the fringes of range. FMCW will immediately return their speed, while otherwise it may take 2-3 frames to learn that. At 70mph, 3 frames is about 10 meters of travel. For oncoming vehicles, the distance can drop 20 meters in that time. The biggest concern is stalled or braking vehicles, and you will immediately know them from regular traffic. In addition in just 2 frames you will know their rate of braking. On the other hand, your radar also tells you this, and has been telling it to you before the LIDAR saw anything.

Blackmore is now shipping in low volumes. Robocars will almost surely be on the road long before any FMCW LIDAR gets into volume production for automotive use. (Though there are hints that Strobe, which Cruise bought, may be working on this and thus be closer to production as part of GM, but who knows?) Blackmore is optimistic that it will be making production volumes when the market arrives.

I hope to hear more about these units and their specs, and forecast prices. Early suggestions I had heard that they did not get intensity returns back are not true, and in fact the dynamic range is significantly greater, according to Blackmore.

Note: I received an update that this is a steerable beam device with over 200m range (It is a 1550nm with similar range characteristics to others in this high power zone, though it runs at only 100mw of laser power and modest input power.) It can do up to 1.2M points/second which is very nice.

Comments

One other important benefit is that FMCW lidars tend to be more robust to interference:

"CWFM lidar sensors "are pretty much totally immune to background light," says Paul Suni, a researcher at Lockheed Martin who has worked on the technology. A conventional time-of-flight lidar can get confused if there are other light sources transmitting at the same frequency. A CWFM system is more nimble, and Suni says it can continue functioning even in the face of glare from the Sun."

Some of the people behind Strobe, the lidar startup GM bought last year, have done work on FMCW lidars. And when Cruise CEO Kyle Vogt announced the Strobe acquisition, he touted their robustness to interference and their ability to capture velocity as well as distance information. So I strongly suspect they're working on FMCW lidar as well.

Why is each frame duration 100ms? That seems like the limiting factor to traditional LIDAR usefulness? Is it just limited by processing time, or is large gap useful to avoid interference?

Depending on your LIDAR technology, you have a limit on the number of points you can see per second per emitter/detector. That's because for a target 150m away, it takes one microsecond for the light to go there and back. So the most you can get is 1 million points per second, and actually less because you want more range and you want some gap. So at 10hz it's 100K points per second, which is actually not that much, compared to camera images. Of course you can have multiple emitters in different directions etc.

And then, yes, it does take time to process a frame, and so there is not a lot of value in getting frames faster than you can process them. 100ms seems to be enough for now, though there are arguments for doing it faster as the systems become capable of it.

Ah interesting, thanks for the explanation! Does flash LIDAR solve this (at a cost of range), or is it also limited to a small number of points per 'flash'?

Flash lidar is limited only by the number of detectors you can have, but it also has eye safety limits. I don't know them but there is a limit to how much energy you can put into the eye in a unit of time, and so if you flashed too fast you would exceed it, I think. I've not studied flash lidar as much. In the past it has been just too expensive for car applications because of the vast array of sensors you need.

Add new comment