Terrain classification using ToF sensors for the enhancement of agricultural machinery traversability

Abstract

Excessive slipping, skidding or trapping situations can compromise the vehicle or other elements in the workspace. Detecting soil surface characteristics is an important issue for performing different activities in an efficient, safe and satisfactory manner. In agricultural applications, activities such as seeding, fertilizing, or ploughing are carried on within off-road landscapes which contain a diversity of terrains. Thus, the machinery requires to understand the surrounding terrain type or its characteristics to take the proper control actions. This work is focused on the soil surface characterization by implementing a visual system capable of distinguish between six types of off-road terrains. Computer vision and machine learning techniques are applied to characterize the texture and color of images acquired with a Microsoft Kinect V2 sensor. In a first stage, the development tests showed that only infra-red and RGB streams are useful to obtain satisfactory accuracy rates (above 85%). The second stage included field trials with the sensor mounted on a mobile robot driving through various agricultural landscapes. These scenarios did not present illumination restrictions nor ideal driving roads hence, conditions can resemble real agricultural operations. In such circumstances, the proposed approach showed robustness and reliability, obtaining an average of 80% of successful classifications.