Background

Pipelines are an essential lifeline today  silently transporting critical  resources like water, oil, and gas to power our homes and fuel our vehicles. Acting as the backbone of the global energy industry, pipelines play a crucial role in transporting crude oil to be refined into various products such as fuel and chemicals. Beyond energy, they also facilitate the distribution of clean water and manage sewage in an environmentally friendly manner. It is critical that these supply systems be well maintained and cracks and leakages be detected and reported immediately.

Pipeline Failures

The industry specially the Energy Industry now has years of experience in operating and maintaining pipelines the safety of today’s pipelines is dependent on not only their design and operation, but also their maintenance, and management. Hence, it needs to be emphasised that pipelines are not dangerous or unsafe, but their design, operation, and maintenance and management can make them unsafe. Numerous incidents have been reported of oil and gas pipelines bursting and resulting in property and casualty loss [1]. Big accidents like the Deepwater Horizon [2] and Exxon Valdez spills [3]  have created headlines and show just how critical it is to ensure the safety of such supply systems.

Motivation

Maintaining pipelines in optimal condition is paramount to ensuring safety, efficiency, and affordability. These robust pipelines are engineered to prevent leaks and mitigate environmental risks, ultimately safeguarding public health. Furthermore, well-functioning pipelines conserve resources and require fewer repairs, leading to smoother operations and prolonged lifespan. Conversely, aging or damaged pipelines pose significant hazards, ranging from environmental contamination to disruptions in water supply, transportation, and energy distribution networks. Therefore, prioritizing pipeline maintenance is essential for ensuring the safety of communities and the uninterrupted flow of essential resources.

Traditional Pipeline Inspection Methods and its limitations

To make sure pipelines are in good condition, there are some routine checks and maintenance work done. This involves looking at the outside of pipelines for any signs of damage, like rust or leaks. Sometimes, special tools like drones or cameras are used to check parts that are hard to reach or hidden underground. Also, there are tests that can be done without causing any damage, like using ultrasound or magnets to see if there are any problems inside the pipes. These checks help make sure the pipelines are strong and safe to use. However, traditional methods like manual inspection have drawbacks. They are time-consuming and prone to human error, potentially resulting in overlooked issues or mistakes. Also the dangerous and rugged terrain of some of these pipelines result in danger to the human inspector working on checking these supply pipelines.

How AI enabled technologies are pitching in

Imagine employing robots and drones equipped with advanced cameras to inspect pipelines, ensuring safety in hazardous environments and reaching inaccessible areas effortlessly. Images taken by drones then can be processed with Advanced Computer Vision methods to determine if certain sections of pipelines are damaged and then action maybe realised.

Introduction to the Automated Pipeline Quality Checking & Maintenance Project

In the current work we demonstrate a POC  by leveraging generative AI a solution to the automated pipeline maintenance problem at hand. For the current work we have developed an application using streamlit which accesses a provided repository of pipeline images and feeds them with a custom crafted prompt as a multimodal input to “Gemini Vision Pro” a Multimodal LLM from google to analyse the picture. We then post process the analysis to extract the condition of the pipes in the images and generate a statistical summary of the same presented to the user via streamlit. The Key components of this POC are

1)     A streamlit based GUI which accepts the image repo and presents the summarized output

2)     A custom crafted prompt which is used for inferencing Gemini Vision Pro for analysing the raw pipeline images.

3)     A post process step which further analyses the response from Gemini and generates a statistical summary of the conditions for all the images

4)     A graphical display of the summarized results using streamlit which acts as a status summary dashboard for the user, presented via streamlit

Interface

The front end interface using streamlit allows users to simply upload a compressed zip file containing images of pipelines for analysis(see Fig. 4). The application notifies the user on successful upload and also displays the count of the files in the zipped repo uploaded (see Fig 5)

Processing

The process flow for the application is schematically depicted in Fig. 6. Once the application is invoked the user is asked  to upload the zipped file of the images. Post upload the file is processed and a subsequent success message on read completion is displayed along with the count of files in the zipped upload. The images are then passed sequentially with a custom  iterative prompt engineering employed to inference the gemini pro LLM twice. A first pass extracts the description of the image with respect to the health of the pipe. Subsequent passes extract statistics on the count and condition of the pipes in the image. This is performed for all the images uploaded in the zipped folder. The Aggregated results are then displayed as bar chart and also a descriptive summary is saved as a CSV file. The schematic process flow is shown in Fig. 6

Output Interfaces

The application provides for a multifaceted overview of the results, accessible through three distinct tabs within the interface.

1)     Dataframe

2)     Chart

3)     Images

We will describe each of these tabs below

Output Interface – Dataframe Tab

The "Dataframe" tab serves as a pivotal resource, presenting users with a structured summary of the analysis findings. Here, row wise details for each image is presented with information such as total pipes in the image, how many are good , how many bad , the overall quality of the pipes in the image and the description of the image as a crafted response from Gemini. This will enable users to get valuable insights into the overall condition of the inspected pipelines. (See Fig. 7)

Output Interface – Charts Tab

In the adjacent "Charts" tab, (see Fig. 8) users are offered a dynamic platform for visualizing the analyzed data through interactive charts. This immersive experience facilitates deeper exploration and analysis, allowing users to gain nuanced insights into the distribution and trends of pipeline quality metrics. Additionally, the option to download the charts in various formats ensures seamless integration into presentations, reports, or other documentation, facilitating effective communication of the analysis results.

Complementing the structured data and visual representations, the "Image" tab(see Fig. 9)  provides users with a rich visual narrative of the analyzed imagery. Here, users can explore detailed descriptions and quality assessments accompanying each image, offering contextual insights into the observed pipeline conditions. Leveraging interactive features, users can zoom in on specific areas of interest within the images, enabling closer inspection and analysis.

From POC to Product

The work presented in this study is open for advancement and collaboration with interested parties specially in Energy Sector and Utilities. We will be interested to take up such collaborative opportunities to convert the study to a product.

Future work

Looking ahead, the project holds immense potential for further augmentation and expansion. Real time Drone imagery maybe augmented with Computer Vision  techniques coupled with Multi Modal Transformer models to analyse pipe images in real time and raising automatic alerts. The same maybe extended to Structural Engineering problems like maintenance of Bridges and communication towers etc.

REFERENCES

[1] https://www.bloomberg.com/news/articles/2016-11-30/30-years-of-oil-and-gas-pipeline-spills-mapped

[2] https://www.britannica.com/event/Deepwater-Horizon-oil-spill

[3] https://darrp.noaa.gov/oil-spills/exxon-valdez

[4] https://www.bloomberg.com/news/articles/2016-11-30/30-years-of-oil-and-gas-pipeline-spills-mapped

[5] https://www.shutterstock.com/image-photo/male-worker-inspection-visual-pipeline-260nw-1442316368.jpg

[6] https://www.dronefromchina.com/Uploads/image/20201201/20201201033903_52929.jpg

Author Details

Shubh Mehta

Shubh is a versatile Data Scientist with experience across Data Analysis, Engineering, and Machine Learning roles. He holds a Bachelor's degree in Data Science from SP Jain School of Global Management, Sydney, Australia. Shubh has authored papers, holds a patent, and has undertaken projects ranging from Statistical Analysis to Sentiment Analysis to Computer Vision and now in GenAI. He is proficient in Python, R, SQL, and various cloud platforms

Rajmohan Bajaj

Raj is currently pursuing his Masters in AI for Business from SP Jain School of Global Management, Sydney, Australia.  He also holds a Bachelors in Computer Science Engineering from Pune University. He has three years of experience as a data engineer, working with Python , SQL Server, and AWS technologies. He is passionate about entrepreneurship and productizing ideas from Academia to Industry for leveraging technology to solve real-world challenges and create value.

Dr Anish Roychowdhury

Anish is a Data Science Consultant and Educator with a total career experience   of 20 plus  years across industry and academia. Having both taught in full time and part time roles at leading B Schools and Technology Schools, including SP Jain School of Global Management,  and Plaksha University , Chandigarh. He has also held leadership roles in multiple organizations He holds a Ph.D., in computational Micro Systems from  IISc Bangalore),  with a Master’s Thesis in the area of Microfabrication, (Louisiana State University, USA) and an Undergraduate degree from  NIT Durgapur with several peer reviewed publications and awarded conference presentations