Abstract
The detection of defects and contamination on wind turbine blades (WTBs) using neural networks is a rapidly growing research area. One problem with publicly available datasets of wind turbine blade images is that the image size is quite large, which results in slower training of object detectors. A new pre-processing pipeline is proposed in this research, in which the Blade30 dataset is used to create a new masked version of the dataset that effectively takes about 50% less disk space. To provide a thorough comparative analysis and to prove the robustness of the proposed approach, the masked version of the dataset is trained with different YOLO object detectors, namely YOLOv5, YOLOv6, YOLOv7 and YOLOv8. The obtained results reflect that the time taken for the proposed pre-processing approach and completing 300 epochs of training with YOLO object detectors helps in cutting down around 1-2 h compared to training with the original version of the dataset. Validation results using the proposed technique demonstrate a gain in mean average precision (mAP) scores ranging from 0.8-7.9% across different versions of YOLO compared with the results on the original dataset, and a gain in mAP scores ranging from 1.1-22.7% compared with different existing methods for WTB defect detection. Testing with the trained weights obtained through the masked version of the dataset shows a significant gain in mAP scores, ranging from 27.3-33.7%.
Introduction
The recent advancements in research in computer vision have opened doors to numerous new possibilities. Object detection continues to be one of the most valuable applications in computer vision. It refers to identifying objects of interest in an image and drawing a bounding box around them to localize them. Object detection has been effectively used to solve many real-world problems, such as person detection in challenging scenarios [1], brain tumour diagnosis [2], crowd counting [3], disease detection on crops [4], and video surveillance [5], to name a few. One such area in which object detection is gaining the interest of computer vision researchers is damage detection on wind turbine blades (WTB).
Wind turbines are a source of clean and renewable energy. Since wind turbines are subject to strong winds and adverse weather conditions, they require routine maintenance for continued efficient performance. Different methods are used to perform non-destructive testing of wind turbine blades. Some of these methods include ultrasonic testing [6], tap testing [7], and infrared thermography [8]. Visual inspection is one of the most common methods to identify structural damage on wind turbines [9]. The use of drones has become a popular method for visually inspecting wind turbines [10]. WTB inspection using drones can significantly reduce turbine downtime from 1.5 h (using conventional methods) to an average of just 20 min [11]. Using drones for WTB inspection also mitigates the risks of human injury or even death. A study suggests that most deaths occur during the construction or maintenance of wind turbines [12]. Furthermore, since drones can capture and store images of WTBs, they also help maintain a better record of the condition of the turbine, which can be revisited at any time.
Researchers have used various image processing techniques to get a better representation of wind turbine images captured using drones. Experiments performed in [13] reconstruct a 3D model of a WTB using multiple images. A technique was proposed in [14] to measure strain on wind turbines using drone images. In another work presented in [15], images of WTBs were synthetically generated to address the problem of motion blur when drones are used. With continued research in computer vision, captured drone images have been used to train computer vision algorithms to identify damages on WTBs automatically [16]. However, the availability of WTB image datasets in the public domain remains a problem for researchers. Recently, the Blade30 dataset [11] was made publicly available, which offers a collection of annotated images for the purpose of identifying different types of defects and contamination on WTB. As shown in Fig. 1, Blade30 consists of images of WTBs and binary segmentation masks of corresponding images. One major problem with WTB images captured using drones is the size of the images. Each image present in the Blade30 dataset is roughly of the size 5400 × 3600. Images of a higher resolution result in slow training of object detection algorithms [17]. Moreover, datasets with images of higher resolution also need a good amount of storage space on the hard drive. It is also worth mentioning that the presence of a challenging background can increase the possibility of false positive detections by the object detector algorithm. One such case for this is shown in Fig. 2.
This research proposes a novel pre-processing pipeline that helps in creating a new version of WTB images. The Blade30 dataset has been used in this research to demonstrate the effectiveness of the proposed approach. Through different experiments, it is proven in this work that the new version of the Blade30 dataset prepared using the proposed approach significantly reduces the storage space required, takes less time to complete training, and achieves better training results compared to the original version of the Blade30 dataset. In this work, a comparative analysis of the performance of different You Only Look Once (YOLO) models for defect detection on WTB surfaces is also presented.
The main contributions of this research are summarized below:
- 1.A novel pre-processing pipeline using binary masks and edges of WTBs to generate a new version of WTB image dataset that optimizes storage space, training speed, and accuracy.
- 2.A comparative analysis using different versions of YOLO to demonstrate the performance of YOLO object detectors on WTB images.
The next sections in this paper are organized as follows. Section 2 discusses the related work performed in the domains of object detection and WTB defect detection. Section 3 presents the complete methodology. Section 4 details the experimental design, and Section 5 discusses the obtained results. Finally, Section 6 constitutes the conclusion.
Section snippets
Related work
The detection of defects on WTB using computer vision techniques falls under the category of object detection. For this specific problem, objects are classified as different types of damages, defects, or contaminations on surfaces of WTBs. Reviewing the literature on object detection algorithms to understand their development with time is vital. Due to this reason, this section is divided into two parts. The first part gives a brief overview of the literature in the object detection domain, and
Methodology
An overview of the complete training process is given in Fig. 5. The method proposed in this research can be divided into three simple steps — generation of binary segmentation masks using Pixel U-Net, extraction of relevant WTB area, and object detection using YOLO. In the proposed pipeline for preparing the dataset for training, the captured drone images of WTBs are passed through a segmentation model called Pixel U-Net to generate binary segmentation masks. Using the binary segmentation
Pre-processing of dataset
The Blade30 dataset consists of a total of 1302 high-resolution images, along with their binary segmentation masks. For the first step, Pixel U-Net was trained to generate binary segmentation masks. A total of 1054 image-mask pairs were used for this purpose, out of which 15% images were used for validation. The discarded images were removed from the dataset based on inaccurate ground truth binary segmentation masks. To achieve quicker training, the images were resized to 608 × 416, and a
Analysis of results
In the tables in this manuscript, a down arrow represents that a lower value of the metric is better, whereas an up arrow suggests that a higher value of the metric is better. From Table 2, it can be observed that preparing the WTB image dataset using the proposed methodology results in a significant reduction in the required storage space, which leads to quicker training. The total time taken by the pre-processing steps required for this methodology is given in Table 1, which equates to 0.182
Conclusion
A novel pre-processing technique to aid with identifying defects and contamination on the surface of WTB is proposed in this work. The binary segmentation masks of WTB images are used to create edge images in this work, which are then collectively used to extract only the area of interest from the WTB image, resulting in a mapped version of the original dataset. Through extensive experiments with different versions of YOLO, namely YOLOv5, YOLOv6, YOLOv7 and YOLOv8, it is proven that the