Researchers from the University of Engineering and Technology (UET) in Pakistan, led by Muhayy Ud Din, have developed a novel framework that leverages advanced artificial intelligence models to enhance autonomous maritime port inspections. Their work, published in the IEEE Xplore Digital Library, focuses on improving the efficiency and accuracy of port inspections using unmanned aerial vehicles (UAVs) and unmanned surface vehicles (USVs).
The study introduces a framework that integrates Large Language Models (LLMs) and Vision Language Models (VLMs) to enable autonomous and context-aware maritime port inspections. Traditional inspection methods often rely on manual operations and conventional computer vision techniques, which can lack scalability and contextual understanding. The proposed framework aims to address these limitations by utilizing the synergy between LLMs and VLMs.
The LLM module in the framework translates natural language mission instructions into executable symbolic plans. These plans include dependency graphs that encode operational constraints, ensuring safe and coordinated operations between UAVs and USVs. This approach replaces traditional state-machine mission planners, allowing for more flexible and adaptive mission planning.
The VLM module performs real-time semantic inspection and compliance assessment. It generates structured reports with contextual reasoning, providing detailed and actionable insights for port operators. This enhances the overall efficiency and accuracy of the inspection process.
The framework was validated using the extended MBZIRC Maritime Simulator, which features realistic port infrastructure. Additionally, real-world robotic inspection trials were conducted to further assess the framework’s performance. The lightweight on-board design ensures that the system is suitable for resource-constrained maritime platforms, making it practical for real-world applications.
This research represents a significant advancement in the development of intelligent, autonomous inspection systems for the maritime industry. By improving the efficiency and accuracy of port inspections, the framework can help ensure safety, regulatory compliance, and operational efficiency in complex maritime environments. The project resources, including code and videos, are available on the researchers’ GitHub page for further exploration and implementation.
This article is based on research available at arXiv.