The urgent need to reinvent America’s public health enterprise

Full transcript below: 

Dave: Hi, and welcome to what we hope will be the first of many episodes in a series of podcasts on America's public health enterprise. I'm Dave Speiser, executive vice president of corporate strategy here at ICF. Here at ICF, we've been trying to understand--at a systems level and at an enterprise level--just how big the challenge will be to re-engineer the U.S. public health enterprise, as we discovered just how challenged it's been by the COVID-19 pandemic, and as we all admit to ourselves how poor the country's performance has been on many important public health metrics even before this emergency. To that end, we've published our first overarching assessment of the public health enterprise which came out in "Fortune Magazine" on July 16th. I'd encourage anyone interested to check that article out--and you can find a link at fortune.com if you search for Coronavirus Public Health Reform--but that article only scratched the surface. So, to delve a bit deeper into a few of the issues we've been intrigued by, we've gathered a few of ICF's leading public health experts and leaders to share some of their thoughts and expertise. And I'd like to introduce them now. Christine Walrath, why don't we start with you?

Christine: Hi, I'm Christine. I serve as the chief science officer at ICF with a primary focus on public health. I provide oversight and support to public health research and evaluation capacity, making sure that our thought leadership and our quality underscores everything that we do in this area.

Dave: Thanks. And Jen Welham, over to you.

Jen: Thanks, David. I'm Jen. I lead ICF's business and health and human services and my specialty is helping clients understand and leverage research and data to make decisions that protect and improve public health. Dave: Thanks, Jen. And last on the list today, Mary Schwarz.

Mary: Hey, David, thank you for having me today. I lead ICF Next’s government division. ICF Next is the arm of ICF that focuses on citizen facing digital and engagement. So, I've spent the bulk of my career interestingly enough at these crossroads, designing and implementing public awareness, public engagement, and behavior change tools and programs.

Dave: Thanks, Mary. I'm looking forward to learning a lot from all of you today. To start with, I guess I'll direct this to Jen. One of the underlying issues or constraints that the team has outlined is the very uneven state of our public health information systems. And within that, kind of the fragmentation of the systems architecture. Can you describe what we mean by fragmentation?

What a fragmented public health system looks like

Jen: Yeah, sure. Thanks, David. Well, essentially, just to start off, there are hundreds of public health information systems for many, many different types of data. And for the most part, they lack interoperability and interconnectedness. So that's what we mean by fragmentation. Interoperability is the ability of computer systems or software to exchange and make use of information. And interconnectedness is the extent to which data systems are linked, allowing for easier sharing of the information.

Dave: So, if there are that many different systems they must fall into a bunch of various categories. What are those various categories? And how does the fragmentation affect them?

Jen: Oh, my goodness, there are so many different categories. How about if I just focus on a few that are really important in the context of the current pandemic. With that, I’ll start with the data that track healthcare facilities. This includes reports about the number of COVID-19 patients each hospital is treating, the number of available beds and ventilators and other information that we need to track the pandemic. Until recently, it was done through CDC's National Healthcare Safety Network. As of July 15th, however, hospitals are now reporting this information directly to a new system called HHS TeleTracking. The National Healthcare Safety Network is continuing to collect data from nursing homes and long-term care facilities. So, as you can see, there's multiple different systems tracking this information already.

Another category that's really important in the COVID-19 pandemic is test result data. This is primarily the results of individuals' test for the presence of the virus that causes COVID-19, and whether they're positive or negative. But it could also include antibody tests, and that shows whether an individual had COVID-19 in the past and now has the antibodies against the virus. This one's much more diverse. Most state and local health departments have their own disease reporting systems, and they're not necessarily designed to be interconnected or interoperable. CDC works with the state and local health departments to facilitate reporting of their data. And this goes to CDC's National Notifiable Diseases Surveillance System.

A third category is those that track immunization data-- registries of all vaccine doses administered by participating providers, like physicians and healthcare facilities, to individuals in a given geographic area. So, while we don't yet have a COVID-19 vaccine, these systems are likely to take on an increasing importance if and when we do. Again, most states have their own immunization information systems and they're not necessarily designed to be interconnected or interoperable. Another category, it's a little bit different, is syndromic data. This is information about clinical symptoms that individuals have like fever, or nausea, or headaches. Because these symptoms are often discernible before a formal diagnosis is made, they can be used as an early indicator of disease outbreaks. And this was particularly important and developed when bioterrorism was a huge risk to try and detect bioterrorist attacks. Most state and local health departments have their own systems, and they voluntarily share their data with CDC. CDC's system is called BioSense. And that's actually part of the National Emergency Preparedness System.

BioSense receives 6 million to 8 million electronic health records every day from over 5,800 facilities. And that represents about 75% of hospitals nationwide. So, as you can see, that system is very interconnected and interoperable. It's all going to a common source. And data is available within 24 hours of patient visits. So very rapid availability of the data. And I guess, one that I feel compelled to mention, it sort of defies any kind of categorization because it really brings a lot of disparate data sets together over 200 actually disparate data sources is called HHS Protect. And this is the system that the White House Coronavirus Task Force uses to inform its decision making. It was designed to address fragmentation. And it was built within just a few weeks at the beginning of the COVID-19 pandemic. It integrates data across federal, state, and local governments and the healthcare industry. Although it provides a very robust picture of the pandemic, it's only as up-to-date and accurate as the data that are feeding into it. And that varies widely by the source.

How you (attempt to) put the fragments together

Dave: Wow, Jen, that description of a veritable, I guess I would call it a zoo of different types of systems, certainly it doesn't reflect any kind of what you would call kind of rational system design, right, or rational enterprise design. I'm assuming, therefore, that these systems that operate differently in different states and local jurisdictions can have that, they gather data kind of in different ways, right? Can you describe a little bit about what some of those differences are and how that contributes to the fragmentation?

Jen: Yeah, you're right, David, zoo is a good term for it. There are so many, and they're all very different. And how they collect data is really often dependent on the data source itself. For example, some COVID-19 test results may arrive via electronic data feeds. That's great. They're electronic, they're easier to incorporate. But many still come by phone, by email, by physical mail, snail mail, or even by fax. And you probably are as surprised as I was to know how popular faxes in the medical community and the reason is, is because it's a technology that complies with digital privacy standards for health information.

So, this creates a lot of fragmentation itself. It's the result of many different systems used to collect and analyze the data. And there's no standard software or system. I've noticed, and I think others would agree, that the patchwork of systems is really not up to the challenge of the current pandemic state. We've got hundreds of laboratories, thousands of test results and very little is interoperable.

Dave: Yeah, I think we've seen just how everybody's scrambling to get decent information in the current emergency. So, at the source, where does all this heterogeneity and systems come from? Is it from federal legislation or the lack of federal legislation? Is it an intentional approach by specific states to tailor their efforts to some sense of their own unique priorities? Or is it just a result of some jurisdictions' desire to kind of limit their own level of effort? I mean, where does all this heterogeneity arise from?

Jen: I'm going to say all of the above, David. First, there is no legislative mandate that standardizes electronic health records or requires public health surveillance systems to be interoperable. Without that, for example, testing labs and physicians, they just do what works best for them. State and local health departments do the best they can. They have many competing demands for funding and funding has significantly diminished over the past decade. But that said, there have been some coordinated attempts to modernize. For example, in early 2010, the federal government spent billions to encourage doctors to replace fax machines with electronic records. That program was called the HITECH Act. But it did not include funding for public health departments to help them automatically digitize faxes and other non-standard types of results. It also did not require hospitals and doctors' offices to build technology that would automatically send the relevant test results to the local health officials.

So public health departments again, as I said their budgets have been cut back significantly over the past decade, and they were unable to finance the digital upgrades themselves. Since then CDC has continued efforts to modernize public health reporting, but on a much smaller scale. For example, in the mid 2010s, the agency spent about $30 million from the Affordable Care Act funds to help state and city health departments go digital. That program made some progress, but it didn't really move the country to a completely digital public health reporting system. Since then, there have been some smaller one-time grant programs over the last three years, but there's really no long-term funding source for digitization. So, we still have a lot of work to do in this area.

Dave: It certainly sounds like the lack of funding--and one might even say neglect of kind of local public health systems--is a topic we might explore in future episodes, because so much of the action obviously takes place at the local level.

Jen: Yeah, I think that would be a great idea, David. That's certainly an important topic to consider. Dave: In light of all this heterogeneity then, I'm assuming that there's a whole additional set of work that's required in the back end of these systems to ensure they're generating useful data. Jen: Yeah, and that goes back to what I said in the beginning about HHS Protect. It's great to have a system with so many different data feeds. But it requires the data feeds to be current and accurate. And there's a lot of variety based again on how the data are received. It's safe to say that most hospitals and state and local health departments are really overwhelmed currently by the pandemic. When data aren't available electronically, they have to be entered into data systems manually. And this can be really time consuming, and it also could introduce transcription errors. Given the volume of test data that's currently coming in to state and local health departments, a lot of them just can't keep up. For example, Washington State recently had to bring in 25 members of the National Guard to assist with manual data entry for the results that weren't reported electronically. So yeah, electronic data, it definitely makes public health surveillance easier and quicker, but it's not a one-size-fits-all and it's not a perfect solution. Data still need to be checked for import errors. And they also need to be harmonized, which accounts for varying formats and different naming conventions. There's a lot of work that goes on on the back end.

In the context of COVID-19 test data, specifically an example, the absence of a standard digital process is hampering case reporting and contact tracing efforts, which are crucial to slowing the spread of the disease. Data often come to public health authorities using only the information that laboratories need to track the record, not the details that public health officials need to track the outbreaks and conduct contact tracing. Reports often come in duplicate, they might go to the wrong health department. They might be missing crucial information such as the patient's phone number or address. I read in one article that nationally about 80% of coronavirus test results are missing demographic information and half don't have addresses. So, health department staff have to spend hours searching databases like LexisNexis to find phone numbers and addresses that were really already collected by the clinic that ordered the test in the first place.

Public perceptions, with an equity lens

Dave: Thanks, Jen. That's an amazing environment. I wish to be working in it. It sounds like, frankly, we've only scratched the surface. A real enterprise approach to designing public health information systems would surely result in some different outcome than the one you're describing. And maybe that's something we can talk about again in the future. Mary, I'd like to turn to you on a different topic. Information is essential in responding to public health threats and emergencies. But of course, it's we as citizens and residents that, ultimately, make most of the difference in our behaviors and in how we respond as individuals and as communities to the threats that we face and the guidance that we get from public health officials. I know that ICF has been conducting surveys to better understand public attitudes during the COVID-19 pandemic. Can you describe the questions that you're asking, kind of who you're asking them of and what you're trying to uncover?

Mary: Sure. We're issuing these on a, roughly, a monthly basis. We've had, I believe fielded 4 at this point. And at our core, we're looking to better understand kind of a couple of core topics. Risk perception, so questions along the lines of what's the individual's likelihood of getting sick? What do they feel about that? Are they worried about the availability of hospital resources? And do they perceive that coronavirus is a threat? Impact on their lives, so has there been an effect on the individual's employment or their income? And then gauging their opinion on the reopening of non-essential business. Likewise, we then are also looking to understand trust and information seeking behaviors. And this is, of course, one that's very dear to me. And we have a number of questions related to trust, where they're getting information, how well governments are responding, and what their perception is about the stopping the speed of the virus.

And then another critical one is personal responsibility. What can they as an individual do to help stop the spread, and how likely are they to be vaccinated? And we're working to get a representative sample as much as possible and are working through panels. So, we're really looking for a way to have as broad-based study as possible, where we can start to drill down into gender and race as the numbers permit within the data.

Dave: So, if you can tell us at this point, what have been the most kind of surprising or interesting findings at this point?

Mary: Well, it's been interesting because a lot of the findings, we're not necessarily seeing a difference in race and ethnicity by the different groups. And what's been really interesting for me is that broad consensus around preventative measures. When we look at the number of people who responded, the adults... Obviously we're looking at adults in the survey. We're not studying just stuff that wasn't clear before, teens or youth. But the number who thought how important it was to wash your hands after being in a public place, we were seeing 9 out of 10 responded positively that it was very important in March and April, and then it was 85% in May and June, so very high numbers.

And on a similar note, that same 90% of the adults surveyed felt that it was very important to stay home if they had a cough or a fever in the first few months of the survey, and then 85%, so it dropped a little bit, but not much in May and June. And then lastly, the same was also true for self-quarantining if someone was exposed. These are all really great signs and indicate a sense of personal responsibility and accountability to your community. And when we think about this from an engagement or a public education point of view, it gives us a wonderful frame to craft messaging. You can look at it both as a degree of personal responsibility, but then your ability to impact others and the health of others.

Dave: Well, that'll be good news if it continues. Now, you mentioned looking for differences among different racial and ethnic groups. Obviously, equity in public health and healthcare in general is a critical issue. Have we learned anything at all about how different minority populations have experienced the pandemic or how their attitudes and levels of engagement might differ? Not necessarily looking for problems, right? But obviously, equity is important, and we'd like to understand all we can.

Mary: Absolutely. And interestingly, we aren't seeing many significant differences in the data that we've collected when we cut it by race. For example, looking at the May data, Black adults who responded did feel that they were no more likely than others to contract COVID-19. Now we do see a difference in Hispanic adults who indicated they are slightly more likely, about 43%, than non-Hispanic, 30%, to contact COVID-19. Although vaccine rates have historically differed across race and ethnicity, the likelihood of getting--or the perceived likelihood of getting--a COVID vaccine between whites, Blacks and Hispanics in our data set...we weren't seeing a large enough difference to be statistically significant.

Now with that said, Black and Brown populations have a history of medical mistreatment by the medical field. Surveys conducted in May and June by Pew and ABC-Washington Post do show that African Americans were both less likely to want the vaccine and then less likely to support experimental interventions. This has a lot of implications for how we approach vaccine research and education around them both from a participation in the study but then also adoption and use once the products are brought to market. So on that vaccine adoption front, we did learn from our June fielding of the survey that there's a positive correlation between believing that the information about vaccines from the government is reliable and trustworthy with a willingness to try a COVID vaccine within six months. Now, unfortunately, fewer than two thirds of respondents agree that information about vaccines from these same government health officials is reliable and trustworthy.

What that tells us is that as outreach and vaccine education programs are developed, we really need to look at the context of the community. What are the individual communication preferences? Is healthcare readily available? How do these communities access services? This isn't a one-size-fits-all solution. We really have to look at the context and the tailoring of messaging and the context of that community as we're thinking about how to engage and how to work with those communities to better understand the disease and the vaccine, and to help the communities move forward. And interestingly, to address this, the Department of Health and Human Services Office of Minority Health is already working with Morehouse School of Medicine.

They just announced a couple of weeks ago a three-year project to work with community-based organizations to deliver education and information on resources to help fight the pandemic. This is a great sign that the government is tuned in to the individual needs, and the need to tailor and really engage those communities in the process in often non-traditional ways. ICF is very fortunate, we are teamed with Morehouse on this and we're thrilled to be part of this project.

Addressing disparities in public health research

Dave: Yeah, I read that we were a part of the project. It's very exciting. Mary, you and I, and Jen, and members of your teams have talked for a long time about how it has gotten more difficult to get broad public participation in long term scientific studies. How does that issue kind of connect with addressing disparities in the context of COVID-19?

Mary: Yeah, this is such a critical issue. And so little is really known about COVID-19. We're all learning as we go. And if you think about it, six months ago, we didn't even have a name for the disease. The awareness of the need for research and the need for volunteers to participate is incredibly high. And that's great because people are tuned in and more open and willing to participate. But the challenge here is that we have to make sure that we have really solid representation of people of different ages, races and genders participating in that research. So now, we need to make sure that we're engaging communities throughout the research process. And what I was talking about, in general, we're talking about health adoption and behaviors, it's really getting them involved as early as possible, so they understand and potentially are partnering on that trial design.

We're clearly laying out what types of questions researchers are looking to answer, and how they'll use the data collected to answer those questions, making sure that the actual research teams have representation within them of people of different, again, races, ages, genders, so that we're not just looking at it as heterogeneous a group as possible. And then kind of what's critically important because as these studies are moving incredibly fast, realistically setting expectations for not just what we're going to see out of this initial round of studies, but for the data and the findings that we're going to see over time. You can't really rush that community engagement part, or you risk communities feeling pushed into participating without really understanding why they should--or that you go with the volunteers who were right there. And again, there's been a tremendous response. But then you have that risk of not having really sufficient representation of the data so you can't necessarily get down into population segments, as you may want to.

Dave: Yeah, that makes sense. Certainly, people self-selecting because of their interests could skew the people participating in the study.

Mary: Exactly. And there's a history of a lack of representation, especially among Black populations. And we do see correlation between awareness and participation in the research. This has been seen in the HIV community, and then use and adoption of treatments. The more we can engage communities, the earlier on, the better results we'll see.

Dave: Well, I mean, obviously, that makes sense. And it's a super important issue. Obviously, when you mentioned treatments, then your mind naturally goes to prevention, and specifically vaccines. I know we've all been developing a new hobby, tracking the progress of the various leading vaccine candidates. I know I spend a good bit of my spare time doing that. But the distribution and acceptance of vaccines are the only things that really protect people, right? Vaccines sitting in the wild don't help anybody. Do we know anything about the differences and acceptance of coming COVID-19 vaccines among various segments of the population? And how do public health authorities influencing and convince people to do the right thing? We know that this has been an issue even before COVID-19. And in the current information environment, it seems like it would be even more of a challenge.

Mary: Yeah. You know, this is a great question and one that has a lot to unpack. So first, if we start with the trials that are in development, I'm a big fan of clinicaltrials.gov. It's a wonderful resource run by the National Library of Medicine that provides information on both public and privately supported trials on a wide range of diseases, including COVID-19. When I checked earlier this week, there were 30 plus vaccine trials out of the hundreds, if not thousands, of COVID-related trials on clinicaltrials.gov. So now, when you drill into the data a little bit, of those 30 plus vaccine trials, seven are active in the U.S. And several of these are companion trials. So, they're actually testing the same product, just potentially with different study designs. And that left me with about four possible vaccine candidates in the U.S. Interestingly, a lot of these vaccine candidates are actually novel technologies and haven't necessarily been approved by the FDA yet.

That's a lot of analysis without going into the specifics of the vaccine mechanics itself. And that's a lot of research that many people won't necessarily go into before you actually get into the details of the vaccine. The way I've been seeing this is that I anticipate that people won't necessarily differentiate between the type of vaccine but rather, they'll look for the safety profile once it had been tested with sufficient populations and sufficient numbers. And again, I expect where members of their community participated in that research, so where they have some type of personal identity or connection back to that. I think we're going to see the need for long term study--safety studies over time--and that's really going to have a lot of influence on people's willingness to participate or to vaccinate themselves. And I believe our data validated this where there's hesitancy to use and to adopt vaccines, and really, that need to prove it out and make sure that the data is safe, and then we're going to see rounds, people who will be up front willing and then say six months in and then potentially longer term.

Dave: Thanks, Mary. Obviously, it's wise to avoid the discussion of specific vaccine technologies or you would never get me to be quiet as a recovering biochemist.

Mary: Yes.

When and how to plan an effective program evaluation

Dave: So, Christine, until now, we've talked about two seemingly very different topics, the "hard topic" of information systems and data collection and aggregation and processing, and the "soft topic" of public perception and attitudes. But any effort by public health authorities to kind of invoke or revamp programs to address any of these issues, they're going to have to prove their effectiveness, as Mary was just saying. And that gets us to what I know is your professional passion, the topic of evaluation. How do we think about the evaluation of disparate efforts like new information systems or strategic communication campaigns? And how do we approach the question of whether things work, and we get value for the money?

Christine: Well, that's a great question. Before I answer it, I do want to underscore one thing you just mentioned, Dave, and that really is that evaluation is absolutely essential. Just as you said, any effort that addresses an issue, tries to change the situation, improve a set of circumstances, it really must prove its effectiveness. But your question about disparate efforts and evaluating them, like new information systems and communication campaigns. The evaluation of efforts--like new information systems and communication campaigns--while disparate on the surface, are quite interdependent. I'll start with the information systems. Given the complexities of the current state of the systems and the data contained within them, demonstrating effectiveness is really a tall order. But the way you would do that would really be to show that those systems achieve sufficient levels of high quality data--and are interconnected and interoperable just as Jen talked about earlier.

From a practical perspective, you might say that the litmus test for the effectiveness of a new system would really be if the data output or the data that's generated in those systems could be reliably used, used for surveillance, used for response, used for resource deployment, and used for evaluation--evaluations of programs and campaigns--which takes me to the second part of your question. To do educational awareness, communication campaigns, you ultimately need high quality data. That's really the essential fuel for the evaluation. And that's the type of data that would ideally be available as a result of a new information system. If you don't have that data effectiveness, even the lack of effectiveness really can't be understood and solid decisions can't be made. Mid-course corrections are really challenging.

Dave: It sounds like, therefore, that what you do at the beginning of a program really can affect whether it can be evaluated later. Are there things that programs can do in their startup phase to make it easier to establish their efficacy?

Christine: Absolutely. There's a whole lot of things but I'm going to focus on three. The first is that all evaluation efforts really should be integrated as part of the program while it's being planned. Evaluation is really not an afterthought. It's not something that you undertake retrospectively after the program starts or even after it ends. As you're planning a program or planning a campaign, you should be planning an evaluation. As your campaign or your program is rolling out, your evaluation should be rolling out. That's the first thing.

The second is that you really up front in your planning phase need to understand the intended outcomes of that campaign or that program, making sure that those intended outcomes are realistic, you can achieve them and they're logical. If you know those intended outcomes, you can immediately map those to high quality available data sources. And then finally, you need to make sure that you have pre-campaign or pre-program data available or baseline data. That's what an evaluator would call it.

To demonstrate effectiveness, typically, you have to demonstrate positive change in a behavior or an outcome, and you need at least two points in time to demonstrate change. If you want to say that a campaign or a program influences that change, you need data before and after the program at minimum. There are a lot of other things you could integrate. But if you think about those three things as the key things to integrate into your up-front planning, you'd be in a great position to determine effectiveness of your program later.

Dave: Well really, it sounds to this recovering scientist, just like good experimental design. So you need some way of measuring the data and you need a way to demonstrate that whatever you did during the experiment actually had the effect you're hoping for.

Christine: Absolutely.

The pandemic’s effects on mental health

Dave: So, in our discussions leading up to this, something came up that I really wanted to get out. And before we wrap up, I know that your teams have been finding some very interesting and concerning information on comorbidities in U.S. residents who've been impacted by COVID-19, and all of the societal impacts that surround it whether it's shutdowns and job losses and quarantines. What have you learned? And are there any efforts underway to address these impacts?

Christine: I'm really glad that you raised that issue Dave. Mary mentioned some of the findings earlier from the ICF survey. She talked about some of the findings related to risk perception and trust and information seeking and even personal responsibility. There are another set of findings that have been generated from the same survey. And these are related to mental health and well being in the context of the COVID pandemic. The findings in the survey are consistent with other information that's being published in the media, published in the academic journals, and also in the trade journals. And really, what they're suggesting is that while the pandemic focuses centrally on physical health and it's taking an enormous toll there, it's also potentially fueling a mental health epidemic in the country. So some of the data from the survey, just as examples, across that four waves of the survey, there's about 70% of respondents who are saying that they experienced anxiety or depression in the two weeks prior to the survey being administered.

A lot of the times that's associated with a COVID diagnosis, worrying about getting a test, getting a positive result, worrying about getting sick. There's also a notable relationship between poor mental health or experiencing more bad mental health days and the loss of a job or reduced work hours or being furloughed. And another notable relationship between these poor mental health days and increasing use of substances like alcohol and cigarettes. So, these are really concerning findings. In and of themselves they're concerning, but in addition, depression, anxiety, job loss, substance use, all of these are risk factors for suicide. And some of the known protective factors against suicide like social connectedness, connecting to your community, connecting to your religious institutions are really dampened by stay at home orders and social distancing requirements.

Then you add on top of this, the ever-present stigma associated with seeking mental health treatment, and now there's even more barriers to access and to getting the treatment you need. Really a perfect storm. And this mental health and wellness challenge may last well beyond the end of the actual COVID-19 pandemic. But there is progress and there is some good news. I mean, as I mentioned, there's been an increasing awareness of these issues over the last few months. And progress is definitely, definitely being made.

For example, they've redirected emergency response funds to mental health issues, streamlined application processes so states and local jurisdictions can get quick access to the dollars. And if those dollars can be used to invest in early detection of mental health challenges, ensure that therapy and medication treatments are continuing during the pandemic and not being interrupted, mitigate financial challenges or physical challenges or even better, the stigma associated with getting help, that really bolsters the mental health system in the country. It both provides supports to people who need it now and helps us get ahead of any mounting needs that might continue as the pandemic continues.

Dave: Wow, thanks for outlining that Christine. There's obviously a lot of issues that are being faced by our fellow Americans and there's no expectation that those are going to disappear on their own for sure. I want to thank Jen, Mary, and Christine for helping us dive into some of the very complex and interrelated issues that are faced by the very broad and fragmented public health enterprise. I hope you've enjoyed listening as much as I have enjoyed being a part of this discussion. I hope you'll join us again when we bring together another unique group of participants from across the public health enterprise. And with that, I bid you a great day. Take care.