Robust and Efficient Depth-based Obstacle Avoidance for Autonomous Miniaturized UAVs

Unmanned aerial vehicles (UAVs) are nowadays used for monitoring, inspection, surveillance, transportation, logistics and many other fields [shakhatreh2019unmanned]. In several scenarios, a small form factor brings advantages - smaller drones are more agile and can traverse complex environments ranging from cluttered offices to industrial facilities, allowing safe operation close to humans in locations otherwise inaccessible [gyagenda2022review, miiller2021funfiiber]. Nano-UAVs [ps2020mini] that weigh a few tens of grams mostly rely on off-board computation due to highly restricted on-board capabilities, typically milliwatt-power microcontrollers (MCU), strongly limited by power and size constraints [miiller2021funfiiber]. MCUs are not powerful enough to run state-of-the-art solutions such as complex navigation models and simultaneous localization and mapping (SLAM) [song2021autonomous, loquercio2021learning]. On the other hand, relying only on on-board sensing and computation brings many advantages - higher reliability when wireless links fail (or are jammed), lower latency in control actions, and reduced bandwidth requirements if external control is limited to high-level commands. Until now, autonomous exploration with the same agility and safety as an expert human pilot has been confined to low speed due to the lack of compact integrated low-power sensors, and resource-constrained navigation strategies [gyagenda2022review, song2021autonomous].

The major challenge for nano-UAVs is achieving autonomous navigation through a reliable and universal obstacle avoidance policy and trajectory planning in real-world applications [rezwan2022artificial]. To enable on-board decisions based on the nano-UAV surroundings, the processing should use only a minor fraction, i.e., 10%, of the overall energy envelope. For instance, in nano-UAV platforms like the Crazyflie 2.1, in which the total power is around $10W$ including also the motors, the maximum processing power needs to be in the orders of hundreds of mW to do not substantially affect the flying time [elkunchwar2021toward]. This power budget is compatible with a simple MCU, such as a general-purpose ARM Cortex-M4 core, commonly used on nano-UAVs, featuring a clock speed of just a few hundred $MHz$ and just $\sim$$200kB$ of RAM.

The motivation to enable local and lightweight navigation policies on highly constrained platforms pushes the research to explore alternative solutions that are not a direct downscale of their bigger and more powerful counterparts [mcguire2019comparative], reconsidering the whole perception pipeline, from the sensor to the navigation strategy [mcguire2019minimal, coppola2020survey]. Moreover, the on-board pipeline has to be robust against disturbances, such as sensor noise, illumination conditions and motion blur [loquercio2021learning]. Obstacle avoidance is commonly vision-based, exploiting deep neural network (DNN) approaches to extract the navigation context from the scene [niculescu2021improving, loquercio2021learning, wang2020uav], but also solutions with laser range finders or even radars exist [schouten2019biomimetic, yasin2020unmanned, zhao20203, duisterhof2019learning]. On nano-UAVs, approaches with cameras, mono or stereo, have been investigated [bahnam2021stereo, niculescu2021improving]. However, they suffer from a fundamental drawback of a high number of input pixels and, therefore, a high computational load, which can absorb a significant fraction of the computational capabilities of an MCU, even a powerful multicore one [garofalo2020pulp]. Moreover, they are dataset-dependent, meaning that in general, the on-board network needs to be retrained if the operational environment changes [niculescu2021improving, loquercio2018dronet, li2020visual]. This effect is further exacerbated by the limited amount of on-board memory, pushing researchers to optimize model size at the cost of decreased generalization.

This work focuses on infrastructure less and autonomous exploration, where the nano-UAV can move through an unknown environment without a local/remote infrastructure for supporting positioning or distributed perception. Existing algorithms often rely on global map planning with known obstacles or are not suitable for position-blind contexts [wang2019autonomous]. Moreover, they offer only a limited set of local decisions, like the wall following approach in the bug algorithm literature that generally exploits single point laser-beam sensors [mcguire2019comparative]. Single-beam rangers have the drawback of only returning a one dimensional measurement, making it impossible to acquire a depth map with a single sensor at a high frequency. Placing multiple sensors or using previously available multi-zone sensors is too power-hungry and heavy for operation on nano-drones. Engineering a more complex policy than wall following can be challenging, requiring tailored heuristics to find dataset-dependent architectures [loquercio2021learning, coppola2020survey], motivating the researchers to implement model-free methods [mcguire2019minimal], which have the potential to ease the engineering process, providing also a robust solution.

This work proposes a complete depth-based perception system optimized for lightweight and low-power flying robots and targeted to support obstacle avoidance and enable autonomous indoor navigation of nano-UAVs. Our solution exploits a novel commercial multi-zone ranger to automatically extracts complex obstacle geometry from the background. The decision policy, together with the preprocessing, is a model-free solution; it replaces sophisticated frameworks targeted at high-end platforms with an algorithm that fits commercial MCU specs. We employed the VL53LC5CX, an 8x8 or 4x4 pixel ToF sensor from STMicroelectronics, which can generate a depth map and a pixel validity matrix at zero-computational cost and a frame rate up to $60fps$. We empirically characterized the VL53LC5CX field of view (FoV), which ranges between $20cm$ and $4m$. The pixel validity indicator simplifies filtering outliers or out-of-range measurements, resulting in highly computationally efficient navigation for a low latency response.

As a reference platform, we selected the Crazyflie 2.1 from Bitcraze, on which a $2.49g$ custom expansion board was designed to support two VL53LC5CX in opposite directions, front and back-facing w.r.t the flying direction. The obstacle avoidance policy, developed on the reported dataset, consists of a decision tree fed by the depth 8x8 matrix. It runs in real-time, with a frame processing time of just $210\mu s$, on the Crazyflie MCU, an ARM Cortex-M4f, using a mere 0.31% of its computational capacity. The depth information, pre-filtered by removing invalid pixels, is categorized into four zones to control the 3D spatial movements of the nano-UAV and the flying speed.

Although our perception solution requires only a fraction (9.4%) of the total power budget, results show best-in-class performance, with virtually zero crash rate flying at $0.5m/s$, the possibility to avoid moving obstacles at up to $2m/s$ and to randomly explore complex unknown environments characterized by narrow pipes ($65cm$) and thin reflective objects with standard ambient light and in complete darkness. Our lightweight model-free perception system successfully demonstrates its potential in real-world experiments characterized by outdoor and indoor environments other than controlled mazes.

On average, the speed of $1m/s$ appears to be the best balance between crashing probability and flight distance, respectively below 20% and $100m$ in a variety of complex real case studies. The main scientific contributions of this work are listed below. (i) We designed a lightweight ($2.49g$) perception board for nano-UAVs with up two VL53LC5CX sensors. With the Crazyflie 2.1 it can be used as a plug & play expansion board. (ii) We leverage a compact integrated multi-region (8x8) ToF sensor to extract depth information without the support of standard vision-based frameworks and complex model-based approaches. (iii) We empirically characterized the ToF sensor in nano-UAV flying conditions, demonstrating the possibility of using this SoC solution to extract precise and reliable depth information from the scene. (iv) We collected a dataset containing 43 records showing time-synchronized data from a grey-scale CMOS camera, one depth matrix extracted from the VL53LC5CX, the internal Crazyflie state, and an absolute 3D position measured by a mocap system. (v) We developed a lightweight obstacle avoidance and random exploration algorithm that can be executed in real-time on a resource-constrained microcontroller. It interactively reacts to complex obstacle geometries, commanding the escape maneuver to the internal state controller. It is based on a model-free decision tree that groups objects at different distances and locations, which is easily extendable by adding path planning and mapping capabilities. (vi) We carried out real-world assessment and performance evaluation in multiple operating conditions, such as mazes, indoor/outdoor environments, narrow passages, and moving obstacles. Evaluation metrics are based on the maximum flight velocity, cumulative and individual success rate, flying time, distance, perception artifacts and latency and, lastly, the flight trajectory. (vii) The whole project, together with the hardware, the dataset, and the obstacle avoidance policy, is released open-source¹¹1https://github.com/ETH-PBL/Matrix_ToF_Drones.

UAVs are massively adopted in real-life scenarios, from civilian applications [mohamed2020unmanned] – such as surveillance, transportation, environmental and industrial monitoring, agriculture services, and first aid – to military services [shakeri2019design]. In particular, indoor navigation enables smart buildings and drone-machine or drone-human interaction, opening new research frontiers [polonelli2020flexible]. In this area, nano-drones have great potential [rezwan2022artificial], which is why nowadays there are numerous research projects aiming to address open challenges for enabling autonomous nano-UAV navigation, mapping, automation, distributed computing and swarm formations [shakhatreh2019unmanned, rezwan2022artificial, miiller2021funfiiber, mohamed2020unmanned].

State-of-the-art exploration solutions that proved to work well on conventional drones, such as SLAM, are still too resource-demanding for nano-UAVs [niculescu2021improving, foehn2021alphapilot]. While other works proved that mapping is also possible with nano-drones [chathurangasensor], their approach relies on off-board computing, which introduces the need of having a computer in the loop, limited range, and communication overhead.

Perceiving the 2-D/3-D structure of the environment is vital for the functionality of UAVs or robotic systems in general [loquercio2021learning], as it enables path planning and autonomous navigation through mapping and obstacle avoidance [rovira2022review]. High-end UAV platforms extract the 3-D environmental information relying on complex, specialized neural networks. By sensing the environment and interacting with other agents [rovira2022review], they learn to generate a depth map estimating distances to objects using a variety of active sensors like monocular and stereo cameras [loquercio2021learning, schilling2019learning], asynchronous event cameras [gehrig2021combining], structured light [muglikar2021event], lidar [yasuda2020autonomous], and ToF sensors [mcguire2019minimal] sampling the scene at a fixed scan rate.

Scaramuzza et. al demonstrated the possibility to fly at high speed in complex environments, such as forests, exploiting a stereo camera and a neural network trained only on synthetic data sets [loquercio2021learning]. To remove the context bias from the simulator environment, and then train the algorithms to work on a generalized scenarios, the authors do not directly process RGB images, but a depth map is extracted from the Intel RealSense 435 stereo pairs. Using the depth matrix not only for obstacle avoidance but also for mapping stages, they achieved a maximum fly speed of $10m/s$ in real scenarios, in which the drones featured a success rate above 50% - 100% below $8m/s$ - in various and unknown environments.

In [loquercio2021learning], the authors present a state-of-the-art approach to enable autonomous navigation in indoor and outdoor environments, based on depth estimation. However, their methodology cannot be applied on nano-UAV platforms as it demands high memory (i.e., gigabytes) and computational requirements. Furthermore, their approach also relies on high-resolution sensing, which is an uncrossable technological barrier for nano-UAVs [miiller2021funfiiber], leaving de facto the nano-UAV autonomous exploration still an open challenge.

Due to recent technological advancements, miniature depth sensors are becoming a reality, being already incorporated in commercial devices such as smartphones or top-range quadrotors. The SONY DepthSense IMX556PLR back-illuminated ToF image sensor²²2www.sony-depthsensing.com features a resolution of 640 x 480 pixels with up to $8.3m$ working distance, while the TeraRanger Evo 64px³³3www.terabee.com proposed a compact 64 pixel and $12g$ solution for robotic application. The aforementioned commercial sensors feature depth extraction and filtering, directly providing a pixel-by-pixel confidence flag on the sensor, moving relevant computation effort from the computing core. However, they are still not compatible with nano-UAVs platforms due to their weight or incompatible digital interfaces with a commercial MCU, e.g., the IMX556PLR is targeted for high-end cellular processors. Nevertheless, there is a clear research trend in this area, which aims to replace the traditional control framework by using an optimized solution able to sense and extract the depth map with a single SoC commercial component, simplifying the mapping and planning processing latency. Moreover, this increases the system’s robustness against unexplored flying scenarios, in which dataset-based solutions show limitations or are more error-prone [niculescu2021improving, yasuda2020autonomous].

Table I presents a comparison between the most recent SoA works on perception-based navigation with nano-drones. In [niculescu2021improving], the authors present an automatic deployment flow of a convolutional neural network (CNN) that runs on-board a nano-drone and enables autonomous navigation and obstacle avoidance capabilities. However, despite its effectiveness with static and dynamic obstacles, the general performance seems poorer in unfamiliar environments (i.e., not present in the training dataset). Furthermore, the CNN can reliably detect the presence of an obstacle and reduce the drone’s forward velocity, or it can adjust the drone’s heading when following a lane. However, due to a dataset limitation, it is often unable to steer around an unknown obstacle to avoid collision, especially in narrow corridors. In contrast to our approach, where all algorithms run in a single SoC (i.e., STM32), their system takes advantage of an additional multi-core SoC which is in charge of running the CNN and transmitting the inference result to the main MCU, increasing the total mass by $4.4g$.

The authors of [mcguire2019minimal] introduce a swarm gradient bug algorithm (SGBA) for enabling autonomous exploration relying on an array of four ToF sensors. However, the drones can only follow walls and do not perform any localization or surrounding detection. The proposed scenario is to find ”victims” in an office environment. However, the video data is stored on SD-cards and has to be read off-board after the flying mission, which introduces a delay in obtaining the information about the environment. Furthermore, despite the excellent measurement accuracy of the ToF sensors (i.e., 3% of the full scale), a vision-based solution was necessary to compensate for the poor spatial coverage of the four distance ”pixels” at low distances [mcguire2019minimal].

Similarly, [duisterhof2021tiny] relies on four ToF sensors and a light sensor and presents an approach based on deep reinforcement learning that enables a nano-drone to seek and find a light source while avoiding obstacles. However, they do not provide any results on dealing with dynamic obstacles, and the maximum speed they report during the testing phase is $1m/s$, which is significantly slower than our system.

The work in [li2020visual] proposes a vision-based system for drone racing, whose goal is to detect ”gates” and fly through them as fast as possible. While effective in passing through gates, their system is tuned to work with a particular type of obstacle and does not deal with general objects in the trajectory. Although their algorithms run entirely on-board, they use a power-hungry SoC (i.e., Cortex A7 plus a dual-core GPU), resulting in a drone that weighs twice as much as our solution.

Lastly, [chathurangasensor] demonstrates the capabilities of creating a 2-D map of the environment, relying on a particle filter that fuses information from 12 ToF sensors. They create a custom deck to accommodate the 12 sensors, which they use in combination with a nano-drone, but the whole computation necessary to run their algorithms is carried off-board.

This work presents a complete system description of an obstacle avoidance system for nano-UAVs, from the hardware design to in-field evaluations. In this application scenario, design optimization and weight minimization are essential to enable longer flight times. We used the commercial Crazyflie 2.1 platform from Bitcraze, extending its functionality with a custom expansion board and new sensors, such as the VL53L5CX from STMicroelectronics. All used components are commercially available, and our design is released as open-source.

This section provides an empirical evaluation of the sensor to assess its effectiveness in measuring the distance to various objects in different lighting and flying conditions. Throughout this evaluation, the sensor is mounted at the height of $1m$ from the ground on static support (i.e., a tripod). The whole setup is positioned such that the sensor is parallel to a wall, and the whole area covered in the sensor’s field of view is flat. Using this setup, we sweep our sensor within the range of $0.2m$ - $3m$ from the wall while maintaining its orientation. The movement is performed with a step of $0.2m$, and at each step, we acquire 1000 distance frames from the sensor using the 8x8 configuration. In addition to the distance matrix, we also store the measurement validity matrix provided by the sensor, which reports which entries in the distance matrix are trustful. We repeat this acquisition procedure for the following four configurations: i) white wall, ambient indoor light (i.e., $\sim$ $500lx$) ii) white wall, darkness (i.e., $<$ $10lx$) iii) white wall, ambient indoor light iv) white wall, darkness. The data stored for these scenarios represent the foundation of our characterization.

First, we evaluate the error of the distance measurements in terms of mean and standard deviation. Figure 4 shows these metrics for the case of a white background with normal ambient light when the sensor is positioned $1m$ far from the wall. The error statistics are computed over 1000 samples for each individual pixel in the matrix. We note that the highest mean errors in the corners, being about $1cm$ – $2cm$ higher than errors associated with the rest of the pixels. The mean error takes values in the range of $19mm$ – $42mm$ and we remark a gradient in the mean error from left to right, which is most likely because of the imperfections in the sensor alignment. The standard deviation of the distance error takes values in the range of $3.4mm$ – $7.3mm$, while the highest values are again encountered in the corners.

Second, we extend the previous investigation to analyze the statistics of the distance error in all the four configurations introduced at the beginning of this section. Figure 5 shows the distance error as a function of the distance to the wall for each scenario, considering only one pixel of the matrix (i.e., one of the four inner pixels). We highlight a pairwise similarity (i.e., median error $<$ $0.5cm$) between the white wall scenarios. Furthermore, the same pattern applies to the brown wall scenarios, which in terms of median error, seem to lead to better results than the white wall case. However, this difference seems to be a constant offset, while the min -– max range and inter-quartile difference are about the same for each scenario at a given distance. The min – max range spans up to $1cm$ for an absolute distance of $20cm$ and up to $8cm$ for an absolute distance of $3m$.

Lastly, we evaluate how the sensor measurement validity decreases with the absolute distance. The validity depends on the amount of reflected light but also external disturbances, such as ambient lighting. Figure 6 shows the measurement validity curves for each of the four considered scenarios. We point out that the validity is higher than 95% in all scenarios, given an absolute distance to the wall smaller than $2m$ and higher than 50% given an absolute distance of $2.6m$. Overall, the measurement validity is higher for the scenarios with a white wall due to a higher surface reflectivity.

To conclude the sensor characterization, we can claim that the sensor does not require any calibration phase, as its accuracy is very good within the operating range that we target (i.e., a few meters). Furthermore, we also observed that the reliability is high for absolute distances of up to $2m$, which is sufficient for enabling obstacle avoidance on nano-drones.

After static tests and the VL53L5CX empirical assessment, a dynamic dataset was collected in different configurations while maneuvering in indoor environments. Tests were performed in controlled and open spaces, with the support of a motion capture system (mocap) Vicon Vero 2.2⁵⁵5https://www.vicon.com/hardware/cameras/vero/ at a rate of $50Hz$. A human pilot manually steered the Crazyflie. Initially, the dataset was used to develop and test the obstacle avoidance algorithm presented in Section VI. However, other researchers can also use it to improve our system by integrating the multi-zone ToF data with processed information from a CNN and the grayscale camera [niculescu2021improving] or by applying a more general DNN algorithm to enhance on-board intelligence [liu2022adaptive]. For this reason, we release the acquired data as open source\@footnotemark. We collected (a) internal state estimation (attitude, velocity, position) of the Crazyflie, (b) multi-zone ToF array in 8x8 pixel configuration, (c) camera images (QVGA greyscale), (d) Vicon data (attitude, position) in a time series format with a millisecond accuracy. The dataset consists of three main groups: object approach moving the drone on a single axis, yaw rotations around the Z-axis, and a general-purpose set of flying tests approaching various obstacles and narrow holes. The first dataset group, named Linear Movements, consists of 10 recordings of flights with (a), (b), (c), and (d) data, approaching a wood panel at different speeds and stopping and flying back always on the same axis, rotations and altitude variations are disabled. The total test time is $216s$ with an average of $22s$ per acquisition. The next group, Yaw Rotations, consists of 3 recordings with (a), (b), (c), and (d) data, rotating on a single axis (yaw) at $1m$ from an obstacle. Recorded data reach a total of $94s$. The third and final group, named Obstacle Avoidance is composed of 30 recordings with a mixed combination of (a), (b), (c), (d) - 14 acquisitions - and (a), (b), (c) - 16 acquisitions. In total, for the third group, $17min$ of flight maneuvers are present in the GitHub\@footnotemark repository, with an average of $35s$ per acquisition.

For each of the 43 released records, a pair of a .csv and .dat file format are present for (a), (b), and (d). Whereas, for (c), a series of .jpg files are present, named with the acquisition frame time in milliseconds. To combine images and decode the time-series files, we also provide a Python script named ”Flight_visualizer.py”, which generates a 3D visualization of the drone attitude and spatial position from the internal state estimator and the Vicon system. Moreover, images and the 8x8 ToF matrix are time-aligned and plotted together with the drone state. The script offers the possibility to test the control algorithm on the collected data. We provide an example in object_detection and decision_making functions that can be used as a reference point for future work. Figure 7 and Figure 8 are respectively two representative examples from O16 and O4 recordings\@footnotemark, reporting the grayscale image and the depth matrix. In Figure 7, the drone is hovering in a fixed position, $V_x,V_y,V_z\approx 0$ and $(yaw,pitch,roll)\approx (0,0,0)$, at $1m$ from the ground while a person is walking perpendicularly to the VL53L5CX FoV. In Figure 6(b), the 8x8 depth matrix shows the foreground distance from the nano-drone, which is reported within a centimeter precision.

Thanks to the sensor’s ability to automatically detect invalid pixels, the background (out of range) is automatically subtracted, and the moving object, the foreground, is then extracted from the scene at zero-computational cost. Despite Figure 6(a) supporting the reader in understanding the test setup and the 8x8 matrix, one can already notice that the HM01B0 is saturating due to the ambient lighting. This condition could decrease the integrity of algorithms fully based on vision-based sensing. Note that the legs of the person are right at the pixel border, leading to only one pixel for them in column 4, and the person is stepping forward, leading to the right knee (pixel 5/5) being out of range. On the other hand, Figure 8 gives an example of a real flight controlled by a human pilot, in which the multi-zone ToF sensor does not correctly extract the object shape. The pilot took off and then swerved by $180°$ from the starting position and is approaching the chair at $0.35m/s$. Despite in Figure 7(a) an office chair is correctly visible (note that pixels in row 7 belong to the ground), the depth map in Figure 7(b) does not fully extract the foreground detail. Indeed, the chair sitting and backrest are identified and measured to be at $83cm$, but the metallic support between the two is completely invisible to the multi-zone ToF sensor, which then wrongly identifies a possible safe passage between rows 1 and 2. In this scenario, the chromed and thin metallic support reflects the majority of the $940nm$ laser beam, being visible only from certain angles or at very short distances, i.e., below $50cm$. This peculiar behavior motivates the technical choice to use an image segmentation approach instead of a pixel-granular cost function to avoid obstacles.

This section describes the whole pipeline used to implement the obstacle avoidance onboard the Crazyflie. In Figure 9 we show how the proposed algorithm is integrated with the existing open-source Crazyflie firmware. The blocks in green belong to the base Crazyflie firmware and are used without modification, while the blocks in red represent our contribution and implement the obstacle avoidance algorithm. The base firmware performs state estimation relying on the information from the onboard IMU and the two sensors found on the Flow-deck v2: the downward facing one-dimensional ToF sensor used for height estimation and the optical flow sensor used for horizontal velocity estimation. The sensor data is fed into the eKF implemented in the base firmware, which produces the state estimate – position, velocity, attitude. The “Obstacle Avoidance” block exploits the information from the multi-zone ToF sensor, producing a forward target velocity and a steering rate that enable the drone to avoid collision with the frontal obstacles. These commands are sent to the onboard controller implemented on the base firmware, which actuates the drone accordingly. In our obstacle avoidance pipeline, we firstly perform feature extraction to identify the objects in the ToF frame and then use a decision tree to determine the forward velocity and steering rate.

This section provides a power and computational requirements analysis of our approach. Furthermore, it presents an evaluation of our system, demonstrating the obstacle avoidance and exploration capabilities in real-world experiments. We evaluate the system’s functionality with both static and dynamic obstacles in various environments.

The key element of our system is a lightweight and reliable obstacle avoidance algorithm that leaves enough resources (computationally and energy-related) for other tasks. Thus, we foresee our work as a base for many future applications since reliable obstacle avoidance is only the first task towards accomplishing autonomous flying. This section provides an open discussion about the future work that can extend our perception system: either by integrating additional hardware (i.e., sensors) or by using more advanced data processing techniques.

The primary constraint in adding more sensors comes from the additional weight, so we aimed to design the custom ToF deck as light as possible, leaving enough weight budget for additional sensors. We showed that it is possible to fly with both the multi-zone ToF deck and the AI-deck during the dataset acquisition. Developing a sensor fusion algorithm that can use the camera and ToF sensor information could improve the obstacle avoidance robustness and enable new capabilities, such as reliable object recognition. Adding a rear and side-facing ToF sensors on our custom deck would extend the overall FoV and, therefore, the system awareness, enabling the system to avoid rear/side approaching dynamic obstacles. One of the main limitations of our system is dealing with highly reflective materials from extreme angles. Even if such material also poses challenges for traditional cameras, novel miniature radars have the potential to mitigate these issues and complement the ToF sensors.

Incorporating more sensors would result in more information to be processed and, therefore, an increased need for computational resources. Our algorithm works with a relatively low dimensional input (i.e., 64-pixel depth map) and requires about $0.31\%$ load from the STM32F405 microcontroller on board the Crazyflie. This not only leaves a large computational budget for developing more complex algorithms, but also enables the system to deal with larger dimensional inputs. The multi-zone ToF sensor released by STMicroelectronics is the first of its kind in terms of precision, form factor and pixel number. However, further releases could come with improved performance, such as a higher measurement range, number of pixels or data rate. Since higher dimensional outputs would not be as straightforward to process and interpret as in our case (i.e., 8x8), more complex algorithms such as CNNs could be a good candidate for dealing with a larger input and therefore enabling new functionalities, such as flying in uneven terrains (e.g., stairs) or detecting narrow passages. Even if CNNs usually require large amounts of memory and computational resource, novel parallel system-on-chips – such as the GAP8 (PULP-based) on-board the AI-deck – proved to be very effective in running such models [niculescu2021improving] given the real-time constraints of autonomous navigation.

The paper presented an on-board obstacle avoidance perception system to enable autonomous navigation with nano-UAVs. It allows nano-UAWs to autonomously explore office environments reliably, only using on-board computing. We used a Crazyflie 2.1 that already features an IMU, extended by a flow deck and our multi-zone ToF deck, featuring a forward-facing 64 pixels ranger sensor. All our processing is done on-board, easily fitting on an STM32F405 microcontroller next to the flight controller, only using up $0.31\%$ of the computational power and featuring a $210\mu s$ latency. The power to lift the additional sensor with all accompanying electronics as well as the supply of it totals in less than $10\%$ of the whole drone, making a flight time of around 7 minutes possible. We tested our system in various challenging environments, achieving autonomous flights with distances up to $212m$. The 100% reliability and high agility at a low speed in an office environment provide a base for many more complex future applications. We also provide a dataset with ToF, state estimation and camera data to learn or simulate future applications.

The authors thank STMicroelectronics for the support provided during the development of this work. Moreover, this work was partially supported by Politecnico di Torino outgoing mobility program and EDISU international mobility grant. Thanks to Iman Ostovar for his work and professor Ernesto Sanchez for his guidance and support. The authors would also like to thank armasuisse Science & Technology for partially funding this research.