Siemens and GGS: Revolutionizing Powerline Inspection with Multi-Sensor Systems and Artificial Intelligence
The Sieaero System, implemented by Siemens AG, is a multi-sensor system that records and analyzes all required data in one go using multiple sensors – including five Phase One 100MP RGB sensors – innovative AI, and machine learning. Designed and built by GGS, the combined multi-sensor system along with novel analysis methods, integrates of all observations to create Digital Twins of infrastructure.
The helicopter mounted system also optimizes inspection workflow, and also significantly saves time and money.
Powerline infrastructure provides the backbone for the electricity supply of industrial, administrative and private sectors. Its maintenance requires regular inspections, which are still largely carried out manually and are thus very time consuming and labor intense tasks.
Today, powerline inspection is done mostly by aerial photography missions, capturing data with Lidar, RGB cameras as well as specific thermal infrared and UV sensitive sensors. During flights, results are reported by visual observation and later analyzed and verified using the captured data. Typically, inspection teams only operate one or two sensor modalities per flight, and additional missions must be planned for the other methods. This adds not only time but also significant costs to keep the infrastructure running smoothly. In addition, human visual checks and verification are inaccurate and faults or findings might be overlooked. On top of that, missions are often associated with significant risks.
Sieaero - Smooth Integration
GGS has developed a new and innovative approach to powerline inspection for Siemens AG by designing an automated inspection system based on a multi-modal sensor solution. It is then combined with Siemens' artificial intelligence to fully automate their inspection tasks. High-level organization of the sensor data and inspection results are utilized to form Digital Twins of powerlines. This allows for a structured, automated analysis of the recorded data to review the infrastructure status in less time and with significantly less effort than subjective human interpretation.
“Helicopter missions are expensive and manual data analysis takes time. Sieaero technology captures 100km of powerlines a day and the data analysis takes around 10 days, which significantly optimizes workflow.”
Dr. Gerhard KemperOwner GGS GmbH
One System Does it All
The newly developed sensor-head, mountable on helicopters, covers all the required methods. It includes LiDAR to scan and record point-clouds with more than 300pt/m², five 100MP RGB Phase One cameras providing object resolutions of 1.5 and 4mm (dependent on what part of infrastructure is observed) and coverage of all components from various viewpoints, four 640x512pix thermal infrared sensors to measure temperature profiles along powerlines, and a UV sensitive camera to record corona discharges at 50Hz.
Additionally, an integrated high-grade GNSS-INS unit is used for precise geo-referencing of all recorded data, and thorough sensor calibration is performed on-the-fly.
For the safety of the inspection flight crew and protection of the infrastructure and equipment, the entire sensor head is designed to operate from 40m away from the powerlines. Flying at 30km/h this setup enables the operator to survey 100 kilometers of powerlines a day, recording tens of terabytes of raw data along the way.
The sensor-head setup in detail:
Five Phase One cameras - Oblique forward/backward:
- Upper cameras: Phase One 100MP with 90mm RS lens
- 1.5-2.5mm GSD,
- Lower cameras: Phase One 100MP with 110mm RS lenses
- 1.5-2.5 mm GSD
- Nadir, slightly oblique camera: Phase One 100MP with 50mm RS lenses - 4mm GSD
- Four Flir Thermal Microbolumeters 640x512pix, 50mm lenses
- 10-25mm GSD,
- Twinsetup of a UV and Pan Camera with 50Hz to detect Corona effects and a Riegl VUX LR Laserscanner to generate the pointcloud with 200-400p/m² in two MTA Zones in two flights.
To operate all sensors in a synchronized environment, the GNSS position, pps pulse and timestamp was used as reference for all devices.
- Upper cameras: Phase One 100MP with 90mm RS lens
Fully Digitized Interaction
The Sieaero System is fully digitalized and allows easy data integration. All sensors are integrated in one multi-sensor system to record all required data in one go. The highly accurate geo-referenced 3D localization of all sensor data allows to automatically identify and classify components, such as insulators, spacers, dampers etc. As a result, artificial intelligence plus machine learning applications enables quick generation of data models (2D and 3D) and inspection reports. Easy data integration provides easy access via web interface and full integration into customer systems.
- Automatic Detection and Evaluation
- More precise
- Cost effective
Multi-sensor Inspection Covers all Criteria
Powerline inspection involves monitoring for findings that could have a significant and detrimental impact on the smooth running of the infrastructure. Issues include hot spots, anomalies, bird nests, and vegetation growing too close to the powerline (right-of-way violations). Standard inventory checks are also conducted of aerial markers, balls, and – as one of the most critical parts – powerline insulators, etc.
To solve all these inspection tasks, Siemens relied on the whole range of established sensor modalities by GGS for powerline inspection. . Vegetation distance and right-of-way violations are handled using 3D Lidar data. Components mounted on the infrastructure such as insulators are detected, localized and inspected based on high resolution RGB images. Thermal signatures of various components such as clamps are further analyzed using thermal infrared sensors. Finally, corona discharges occurring along the high voltage powerlines are found using a dedicated UV sensor.
Mission Planning and Surveying
Mission planning for powerline inspection is a specific task especially if the cameras are positioned at oblique and slightly forward angels to the cables and pylons. The position of the traverses and the layout of the poles have to be taken into account in order to adjust the viewing angle and the flight path for capturing the infrastructure in an optimal way. Combining views with the flight path indicates optimal trigger points that are planned for the flight management software (FMS).
The first step in surveying powerlines, involves flying left and right of the powerlines and directly over them. The data is used in mapping the corridor, typically 50m left and right of the powerline. The next step is generating a Digital Surface Model (DSM), followed by extracting a Digital Terrain Model (DTM). The DTM is used to derive vegetation information for further analysis, and for getting the Orthophotos from the high-resolution Phase One cameras.
Automated Defect Detection with Artificial Intelligence (AI)
Due to the sheer amount of data recorded with the sensor-head, it is mandatory to utilize automatic analysis methods to handle inspections efficiently. The results of these automatic methods are consolidated in a digital twin representation, which is a very powerful way to deal with inspection results. AI Software can detect more than 90% of the findings that are often not visible to the human eye using the high-resolution images from the 100MP Phase Cameras.