Using GenAI to accelerate public comment analysis

Federal agencies are required to comply with the congressionally mandated rulemaking process to develop new or amend existing regulations—from issuing a notice of proposed rulemaking, to reviewing and analyzing public comments, to publishing a final rule in the Federal Register.

Agencies need to consider each comment—and with millions of comments submitted each year and often thousands of pages of comments analyzed per rule, this is an intensive process. But despite the volume of comments, agencies are under more pressure than ever to analyze them in weeks or days after submission.

Clearly, innovation is in order.

A challenging step in the federal rulemaking process

Today, when a proposed rule is issued, the public is provided 30-90 days, on average, to submit comments, typically through Regulations.gov. Agencies assign appropriate resources to ensure every comment is considered and prepare responses to significant issues raised in the comments. The process can be complex and laborious, and when the volume of public comments spikes, agencies may need surge capacity to process them.

In collaboration with our federal clients, we have been testing whether generative AI (GenAI) can supplement the analysis and triage of public comments. Can it be used to improve efficiency, accuracy, and certainty in comment characterization—and do it at scale? We are bringing together experts in federal rulemaking, data science, and user experience to address this question through a variety of solutions.

Exploring GenAI solutions with our policy experts

ICF’s policy experts have provided federal agencies with regulatory development support and public comment analysis for more than 40 years. ICF’s cloud-based comment analysis software, CommentWorks, plays an essential role—in fact, federal agencies have relied on CommentWorks to help analyze more than 20% of the public comments posted across all federal dockets over the past ten years. With help from our data and AI teams, we are incorporating new GenAI features into CommentWorks to accelerate consideration of comments and elevate significant comments earlier for agency review.

Another fruitful area of exploration for us has been in developing a GenAI pilot to help a federal agency client solve a specific regulatory challenge.

To find success with GenAI solutions, it’s important to start with a focused, well-defined use case. The regulatory commenting pilot we selected was an ideal candidate for GenAI exploration because the data was curated and the focus was narrow—which allowed us to move quickly.

We built a fusion team of ICF and federal rulemaking experts, data science and AI practitioners, and user experience (UX) experts, and started by doing an analysis of alternatives. Our goal was to understand our federal client’s existing environment and infrastructure so we could leverage economies of scale. The agency’s environment was not GenAI-ready on day one, so we leveraged our secure cloud infrastructure and a FedRAMP-compliant Generative LLM service endpoint to allow the pilot to advance without delay.

Our team spent significant time up front with the models to assess their performance. Working with our federal policy experts, we pressure-tested the model efficacy to ensure the results were accurate, ethical, unbiased, and aligned to how human regulatory experts would classify the comments.

Once we proved the accuracy of the models, we turned our attention to the user interface. How could we make it easy and intuitive for agency regulatory experts to use? Our UX experts stepped in to optimize the experience for system users, applying product management best practices to create a streamlined user interface that encouraged adoption by internal system users.

The result? Our federal client is enthusiastic and working to scale the solution to other areas of the business. In less than three months from inception to delivery, we proved the viability of using GenAI to produce faster insights into numerous regulatory comments.

Harnessing the power of industry expertise + AI

GenAI models are powerful efficiency boosters, but it’s important to apply the proper checks and balances to ensure the accuracy of the results. We follow Responsible AI principles and rely on ICF’s data science and regulatory experts to evaluate the model’s output to keep it aligned with how human experts would classify and summarize comments—and we partner closely with our clients to provide transparency throughout the process.

Streamlining the federal rulemaking process is a goal shared by many agencies, and the capabilities we offer through CommentWorks and GenAI pilots can be applied to other regulatory contexts—allowing the federal government to expedite the process further and reduce federal employee labor on public comment analysis without sacrificing citizen confidence.