Atmospheric Image Formation Model | Detection Methods and Collision Avoidance
Detection Methods and Collision Avoidance:
We tested a number of different approaches to aircraft detection and for each we evaluated their performance characteristics. We settled on an approach based on morphological image processing and machine learning techniques to reduce false positives. The code base at the time of writing totals over 17,000 source lines of code (SLOC).
Vision based Detection Methods:
We evaluated the performance of the algorithm in terms of true positive percentage and false positive rate measured in number of false positives per frame. We also broke down performance by range. As of this writing the approach yields detection rates of 99.7% up to 3.5 miles and 96.1% up to 4.5 miles with a false positive rate of 0.02 per frame ( using an IPX-4MP5 and Zeiss 85 mm lens thereby yielding 0.005 false positives per megapixel per frame ). Furthermore almost all of the false positives that occur are due to objects that may be relevant to collision avoidance: birds, radio towers. See the Results section for more details in this regard.
Main Stages of the algorithm:
Stage 1: Morphological Filtering
In the first stage we detect the horizon in the image and apply the morphological filter to the region of the image above the horizon in order to detect the points which are most likely to be aircraft. The red boxes in the video represent the points of interest detected by the morphological filter. The true position of the aircraft is shown in a green box. The first stage produces a large number of false positives which are eliminated in the subsequent stages
Stage 2: Classification of interest points
In the second stage a pretrained Support Vector Machine (SVM) classifier is applied to each interest point generated in the first stage. This stage eliminates most of the false positives while still retaining the true aircraft in the image.
Stage 3: Non-maximum supression
In the third stage non-maximum suppression is carried out in the neighbourhoods of the most likely interest points remaining from Stage 2 to ensure that same points do not show up as multiple detections in the image.
Stage 4: Tracking
In the fourth stage the interest points from Stage 3 are tracked over subsequent images which further eliminates false positives. False positives being random in nature do not persist and hence are thrown out by the tracking stage of the algorithm. We also detect dead pixels and dust on the sensor which remain static over long periods of time.
The video shows in the inset the aircraft being tracked along with a bird and a dead pixel. The groundtruth position of the aircraft has been manually picked out and is being shown in a green box in the video. The track history of the aircraft and bird are overlaid with a black trailing line. In this particular video the aircraft is tracked reliably out to 4.5 miles.
Stage 5: Dead pixel and dust detection
In stage 5 we detect the dead pixels if any in the sensor and also dust particles which deposit on the lens. This process ensures that the Sense and Avoid system works even in the presence of dust and dead pixels. The video below shows the dead pixels and dust particle on the sensor in blue.
Collision Avoidance software:
We have developed and tested through software in the loop a collision avoidance system that given the state of the intruder aircraft would be able to come up with an avoidance trajectory in finite computational time. Any system that performs satisfactorily for this purpose must satisfy a number of perormance criteria. It must be able to take into account the uncertainty in the state estimation of the intruder aircraft, utilize the dynamics of the ownship as completely as possible in evading the intruder while at the same time come up with a plan that is feasible to execute for the ownship within finite time. Conventional approaches have the drawback of being extremely slow as a consequence of trying to solve the search problem in a high dimensional state space. The resulting trajectory has the least possible execution cost but takes a long time to calculate. Unmanned Aerial Vehicles must be able to aggressively maneuver to avoid possible threats and obstacles if required while not breaching the flight envelope which can result in possible instability and uncontrollability.
We have developed a collision avoidance system that can avoid another aircraft traveling up to 250 knots and making lateral maneuvers no more than 1g as long as the separation between the ownship and the intruder aircraft is at least 2.1 km. The system is based on an approach called rapidly-exploring random tree and maneuver automata. Details of the performance of the system are given in the Results section.
The following video demonstrates the working of the collision avoidance system. The software-in-loop simulation is done with the Piccolo Autopilot simulator which simulates an aircraft with similar dynamic constraints as a Cessna Stationair. This can be easily swapped with parameters of any UAV or other aircraft to evaluate the limits of the planining system.
It must also be noted that the performance limits of the collisio avoidance system along with the regulatory constraints impose the minimum detection ranges for the Sense and Avoid system.