NOAA to test AI electronic monitoring in New England clam survey

NOAA Fisheries wants to improve its electronic monitoring of clam harvesting with artificial intelligence, and it hopes to begin collecting imagery to develop the technology this August.

The Northeast Fisheries Science Center is looking for a contractor to install video cameras on a commercial clamming vessel, the ESS Pursuit, develop a machine learning algorithm that can automatically process that video, and produce a large library of images for future artificial intelligence development.

NOAA has long relied on an army of human observers to physically go out with fishermen on their vessels to collect important catch data an ensure regulatory compliance. Human observers, however, present their own challenges. With rates of up to USD 800 (EUR 736) per day, paying for monitors can be expensive for fishermen operating on low margins. And once out to sea, fishermen often find observers getting in the way in the constrained operating space of a fishing vessel.

NOAA has turned to electronic monitoring – where vessels are outfitted with cameras to track commercial fishing activities – to help reduce the physical and financial burdens of monitoring. However, electronic monitoring still relies on an army of human observers who, instead of being posted on a vessel, play back the extensive recordings for review. According to a white paper put out by Pew Charitable Trusts in February, the resources needed to manually review millions of hours of footage collected via electronic monitoring programs is a huge impediment to wider adoption.

Automating the review process with artificial intelligence is one solution that NOAA is considering. The agency believes a machine learning tool would reduce the burden of reporting on fishermen, enhance the accuracy of data collection, and reduce the overall cost of monitoring.

Still, there remain significant hurdles to building an effective artificial intelligence tool, the biggest of which is developing the necessary training data. Machine learning algorithms have to be trained on massive data sets of images in order to effectively scan video and pull out desired information. NOAA has identified building a large library of images as a regional priority and an agency-wide strategic initiative, but current data sets are not robust enough for machine learning, according to NOAA.

The Northeast Fisheries Science Center plans to use its 2023 bottom trawl clam survey in Woods Hole, Massachusetts, U.S.A. this August to create that data and begin developing an algorithm. The center is looking for a contractor to set up a video recording system on the ESS Pursuit, the commercial vessel participating in the clam survey. The contractor will then use open-source software and existing imagery to begin developing a machine learning algorithm and determine what types of images and training data is needed to improve it.

One of the key tasks for the algorithm will be differentiating between ocean quahogs and Atlantic surfclams, which NOAA claims fishermen are encountering in mixed catches more frequently.

“Training an AI to distinguish the primary commercial clam species should be relatively simple in comparison to fish identification. The species are morphometrically distinguishable … and are also typically … different colors,” NOAA said in its contract solicitation.

Measurement of shell length will likely be more challenging, as the orientation of the shellfish on the conveyer belt “can be suboptimal.”

Both the data and the algorithms produced through this project will be open source, meaning future endeavors can easily build off the clam survey to continue developing artificial intelligence tools.

NOAA has been working to better incorporate artificial intelligence into its operations since 2019, when the agency listed artificial intelligence as one its new science and technology focus areas. NOAA adopted an artificial intelligence strategy in 2020, and toward the end of that year the agency officially established the NOAA Center for Artificial Intelligence to help foster adoption of the technology across the agency.

“This development aligns with NMFS’s longer term goal of implementing machine learning in fisheries monitoring to aid in the development of resources with which to enhance the efficiency and accuracy of automated species identification,” NEFSC said.

Photo courtesy of NOAA Fisheries