Automatic Annotation of Subsea Pipelines using Deep Learning

Anastasios Stamoulakatos, Javier Cardona, Chris McCaig, David Murray, Hein Filius, Robert Atkinson, Xavier Bellekens, Craig Michie, Ivan Andonovic, Pavlos Lazaridis, Andrew Hamilton, Md Moinul Hossain, Gaetano Di Caterina, Christos Tachtatzis

Research output: Contribution to journalArticle

Abstract

Regulatory requirements for sub-sea oil and gas operators mandates the frequent inspection of pipeline assets to ensure that their degradation and damage are maintained at acceptable levels. The inspection process is usually sub-contracted to surveyors who utilize sub-sea Remotely Operated Vehicles (ROVs), launched from a surface vessel and piloted over the pipeline. ROVs capture data from various sensors/instruments which are subsequently reviewed and interpreted by human operators, creating a log of event annotations; a slow, labor-intensive and costly process. The paper presents an automatic image annotation framework that identifies/classifies key events of interest in the video footage viz. exposure, burial, field joints, anodes, and free spans. The reported methodology utilizes transfer learning with a Deep Convolutional Neural Network (ResNet-50), fine-tuned on real-life, representative data from challenging sub-sea environments with low lighting conditions, sand agitation, sea-life and vegetation. The network outputs are configured to perform multi-label image classifications for critical events. The annotation performance varies between 95.1% and 99.7% in terms of accuracy and 90.4% and 99.4% in terms of F1-Score depending on event type. The performance results are on a per-frame basis and corroborate the potential of the algorithm to be the foundation for an intelligent decision support framework that automates the annotation process. The solution can execute annotations in real-time and is significantly more cost-effective than human-only approaches.

Original languageEnglish
Article number674
Number of pages15
JournalSensors (Basel, Switzerland)
Volume20
Issue number3
Early online date26 Jan 2020
DOIs
Publication statusPublished - 1 Feb 2020

Fingerprint

annotations
Remotely operated vehicles
Oceans and Seas
learning
Pipelines
Inspection
Learning
Image classification
Labels
Data acquisition
Anodes
Oils
Sand
Lighting
Gases
Personnel
inspection
Neural networks
vehicles
Burial

Cite this

Stamoulakatos, A., Cardona, J., McCaig, C., Murray, D., Filius, H., Atkinson, R., ... Tachtatzis, C. (2020). Automatic Annotation of Subsea Pipelines using Deep Learning. Sensors (Basel, Switzerland), 20(3), [674]. https://doi.org/10.3390/s20030674
Stamoulakatos, Anastasios ; Cardona, Javier ; McCaig, Chris ; Murray, David ; Filius, Hein ; Atkinson, Robert ; Bellekens, Xavier ; Michie, Craig ; Andonovic, Ivan ; Lazaridis, Pavlos ; Hamilton, Andrew ; Hossain, Md Moinul ; Caterina, Gaetano Di ; Tachtatzis, Christos. / Automatic Annotation of Subsea Pipelines using Deep Learning. In: Sensors (Basel, Switzerland). 2020 ; Vol. 20, No. 3.
@article{f67b47377f274c46a725a8530ece205a,
title = "Automatic Annotation of Subsea Pipelines using Deep Learning",
abstract = "Regulatory requirements for sub-sea oil and gas operators mandates the frequent inspection of pipeline assets to ensure that their degradation and damage are maintained at acceptable levels. The inspection process is usually sub-contracted to surveyors who utilize sub-sea Remotely Operated Vehicles (ROVs), launched from a surface vessel and piloted over the pipeline. ROVs capture data from various sensors/instruments which are subsequently reviewed and interpreted by human operators, creating a log of event annotations; a slow, labor-intensive and costly process. The paper presents an automatic image annotation framework that identifies/classifies key events of interest in the video footage viz. exposure, burial, field joints, anodes, and free spans. The reported methodology utilizes transfer learning with a Deep Convolutional Neural Network (ResNet-50), fine-tuned on real-life, representative data from challenging sub-sea environments with low lighting conditions, sand agitation, sea-life and vegetation. The network outputs are configured to perform multi-label image classifications for critical events. The annotation performance varies between 95.1{\%} and 99.7{\%} in terms of accuracy and 90.4{\%} and 99.4{\%} in terms of F1-Score depending on event type. The performance results are on a per-frame basis and corroborate the potential of the algorithm to be the foundation for an intelligent decision support framework that automates the annotation process. The solution can execute annotations in real-time and is significantly more cost-effective than human-only approaches.",
keywords = "deep learning, multi-label image classification, sub-sea pipeline survey, transfer learning, visual inspection",
author = "Anastasios Stamoulakatos and Javier Cardona and Chris McCaig and David Murray and Hein Filius and Robert Atkinson and Xavier Bellekens and Craig Michie and Ivan Andonovic and Pavlos Lazaridis and Andrew Hamilton and Hossain, {Md Moinul} and Caterina, {Gaetano Di} and Christos Tachtatzis",
year = "2020",
month = "2",
day = "1",
doi = "10.3390/s20030674",
language = "English",
volume = "20",
journal = "Sensors",
issn = "1424-3210",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "3",

}

Stamoulakatos, A, Cardona, J, McCaig, C, Murray, D, Filius, H, Atkinson, R, Bellekens, X, Michie, C, Andonovic, I, Lazaridis, P, Hamilton, A, Hossain, MM, Caterina, GD & Tachtatzis, C 2020, 'Automatic Annotation of Subsea Pipelines using Deep Learning', Sensors (Basel, Switzerland), vol. 20, no. 3, 674. https://doi.org/10.3390/s20030674

Automatic Annotation of Subsea Pipelines using Deep Learning. / Stamoulakatos, Anastasios; Cardona, Javier; McCaig, Chris; Murray, David; Filius, Hein; Atkinson, Robert; Bellekens, Xavier; Michie, Craig; Andonovic, Ivan; Lazaridis, Pavlos; Hamilton, Andrew; Hossain, Md Moinul; Caterina, Gaetano Di; Tachtatzis, Christos.

In: Sensors (Basel, Switzerland), Vol. 20, No. 3, 674, 01.02.2020.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Automatic Annotation of Subsea Pipelines using Deep Learning

AU - Stamoulakatos, Anastasios

AU - Cardona, Javier

AU - McCaig, Chris

AU - Murray, David

AU - Filius, Hein

AU - Atkinson, Robert

AU - Bellekens, Xavier

AU - Michie, Craig

AU - Andonovic, Ivan

AU - Lazaridis, Pavlos

AU - Hamilton, Andrew

AU - Hossain, Md Moinul

AU - Caterina, Gaetano Di

AU - Tachtatzis, Christos

PY - 2020/2/1

Y1 - 2020/2/1

N2 - Regulatory requirements for sub-sea oil and gas operators mandates the frequent inspection of pipeline assets to ensure that their degradation and damage are maintained at acceptable levels. The inspection process is usually sub-contracted to surveyors who utilize sub-sea Remotely Operated Vehicles (ROVs), launched from a surface vessel and piloted over the pipeline. ROVs capture data from various sensors/instruments which are subsequently reviewed and interpreted by human operators, creating a log of event annotations; a slow, labor-intensive and costly process. The paper presents an automatic image annotation framework that identifies/classifies key events of interest in the video footage viz. exposure, burial, field joints, anodes, and free spans. The reported methodology utilizes transfer learning with a Deep Convolutional Neural Network (ResNet-50), fine-tuned on real-life, representative data from challenging sub-sea environments with low lighting conditions, sand agitation, sea-life and vegetation. The network outputs are configured to perform multi-label image classifications for critical events. The annotation performance varies between 95.1% and 99.7% in terms of accuracy and 90.4% and 99.4% in terms of F1-Score depending on event type. The performance results are on a per-frame basis and corroborate the potential of the algorithm to be the foundation for an intelligent decision support framework that automates the annotation process. The solution can execute annotations in real-time and is significantly more cost-effective than human-only approaches.

AB - Regulatory requirements for sub-sea oil and gas operators mandates the frequent inspection of pipeline assets to ensure that their degradation and damage are maintained at acceptable levels. The inspection process is usually sub-contracted to surveyors who utilize sub-sea Remotely Operated Vehicles (ROVs), launched from a surface vessel and piloted over the pipeline. ROVs capture data from various sensors/instruments which are subsequently reviewed and interpreted by human operators, creating a log of event annotations; a slow, labor-intensive and costly process. The paper presents an automatic image annotation framework that identifies/classifies key events of interest in the video footage viz. exposure, burial, field joints, anodes, and free spans. The reported methodology utilizes transfer learning with a Deep Convolutional Neural Network (ResNet-50), fine-tuned on real-life, representative data from challenging sub-sea environments with low lighting conditions, sand agitation, sea-life and vegetation. The network outputs are configured to perform multi-label image classifications for critical events. The annotation performance varies between 95.1% and 99.7% in terms of accuracy and 90.4% and 99.4% in terms of F1-Score depending on event type. The performance results are on a per-frame basis and corroborate the potential of the algorithm to be the foundation for an intelligent decision support framework that automates the annotation process. The solution can execute annotations in real-time and is significantly more cost-effective than human-only approaches.

KW - deep learning

KW - multi-label image classification

KW - sub-sea pipeline survey

KW - transfer learning

KW - visual inspection

UR - http://www.scopus.com/inward/record.url?scp=85078688207&partnerID=8YFLogxK

U2 - 10.3390/s20030674

DO - 10.3390/s20030674

M3 - Article

C2 - 31991872

AN - SCOPUS:85078688207

VL - 20

JO - Sensors

JF - Sensors

SN - 1424-3210

IS - 3

M1 - 674

ER -

Stamoulakatos A, Cardona J, McCaig C, Murray D, Filius H, Atkinson R et al. Automatic Annotation of Subsea Pipelines using Deep Learning. Sensors (Basel, Switzerland). 2020 Feb 1;20(3). 674. https://doi.org/10.3390/s20030674