Abstract:

Generative AI (Gen AI) has undertaken rapid development in recent years with applications in commerce, science, research among other fields. One known design flaw, pertaining to rigorous research, is Gen AI’s tendency to hallucinate responses. A new paradigm has emerged as a response, Retrieval-Augmented Generation (RAG) which serves as the bridge between Gen AI and the external database. We propose a novel RAG tool as an interface to be used in conjunction with scientific papers related to oceans. A key objective was to create a dedicated agentic pipeline that reads the chunked text files, generates multiple choice questions, conducts quality assurance and finally attempts to answer the generated questions using various types of agents. Benchmarking was conducted to verify accurate results. To compare, our RAG to a no-context and perfect context agent was also tested. It was found that our RAG agent is effective at retrieval of accurate data with accuracy being greater than 80%. The no context agent was highly limited with best runs only achieving 70% accuracy. Limitations of the study include the token limit with the API tools used and the limited number of scientific papers. Further research may continue to build the framework of this tool; expanding it to encompass more scientific papers. The tool could be connected to a private LLM and have the user load data in. Hence, when our RAG agent achieves greater accuracy, it becomes usable as a tool for interpreting oceanic scientific papers at a greater level of efficiency.

The results presented are from the NOAA EPP CSC NERTO graduate internship project conducted under the mentorship of Andrew Ross, Geophysical Fluid Dynamics Laboratory, GFDL. This NERTO experience aligns with the NOAA Cooperative Science Center in CESSRST-II, supporting the Center’s goal of collaborative research to understand and predict changes in our oceans and that impact on marine life. This project, titled Testing artificial intelligence tools for understanding and predicting oceanographic changes and their effects on marine ecosystems addressed the research question: can we use Agentic AI systems with RAG tools to reliably understand changes in the ocean within the Northeastern U.S shelf region and can that system extrapolate conditions via data ingested. The work provides value to the scientific community and program stakeholders by expanding the knowledge NOAA has on AI tools and allowing researchers to utilize this Agentic RAG combination to accelerate their own research pipelines. Through this internship, the student deepened their understanding of NOAA mission areas and gained enhanced skills in utilization of complex agentic AI systems and an increased understanding in oceanography. Additionally, gained experience using cloud computing platforms and NVIDIA APIs.