The pandemic caused by 2009 H1N1 had a substantial impact leading to an estimated 43 to 89 million cases and 8,870 to 18,300 deaths in the United States from April 2009 to April 2010 [1]. Because antivirals are most effective if given early in the course of infection, there was a need to quickly evaluate patients, particularly those with underlying health conditions at risk for more severe disease. In addition, there were concerns about spread of influenza in slow-moving, overcrowded waiting rooms. Many healthcare facilities struggled to respond to the increased number of patients with symptoms of influenza and to those who sought care out of concern for influenza but did not have symptoms (worried well). In the state of Minnesota, influenza activity peaked first in June 2009, abated during July and August, and increased in activity once again in September 2009 coinciding with the return of students to school [2], [3]. As healthcare utilization began to increase, based on reports to Minnesota Department of Health (MDH) from hospitals, clinics, and nurse triage telephone service providers, there were gaps in meeting the specific needs of those with influenza-like illness (ILI; defined as fever >100°F (37.8°C) with cough and/or sore throat) or exposure to someone with ILI, in particular those who were uninsured or underinsured.

To meet these needs, the MDH and Minnesota healthcare systems (defined as healthcare organizations that provide healthcare insurance coverage as well as healthcare delivery organizations) sought a mechanism to support the existing healthcare delivery system, improve access to antivirals if needed, and minimize unnecessary healthcare interactions. Telephone-based evaluation and triage have been utilized in many areas of medicine including management of chronic disease and acute illnesses with similar outcomes compared to in-person care [4]–[7]. A telephone-based nurse triage line (NTL) service was designed and implemented by MDH in collaboration with all Minnesota-based healthcare systems with existing NTLs. This service was called the MN FluLine and began service on October 21, 2009.

Other NTLs were utilized during the response to 2009 H1N1 [8]–[12]. We have previously described how the MN FluLine reached over 27,391 callers preventing up to an estimated 10,998 in-person healthcare contacts [13]. However to date there has not been an assessment of how NTLs were perceived by the callers nor an assessment of the costs associated with such programs. In order to evaluate this intervention, a program evaluation of the MN FluLine was conducted to further assess who utilized the service, determine caller reported outcomes, caller levels of satisfaction, and estimate costs associated with this type of intervention.

There was a total of 27,391 calls to the MN FluLine, of which 13,958 (51%) reported symptoms of ILI or being exposed to someone with ILI. Of these 3,691 (26%) were information only/non-flu-related/duplicate calls, 3,799 (27%) were transferred to a partner NTL, and 6,468 (46%) had the nurse protocol administered by the contract NTL of which 374 were prescribed oseltamivir by the NTL [13]. Of the 6,468 calls managed by the contract NTL, 38 calls occurred in March and were ineligible to be included as randomization for the survey occurred in March prior to the completion of MN FluLine operations, leaving 6,430 eligible to be included in the survey (Figure 1).

A total of 1,221 people were called to conduct the evaluation survey including those who completed nurse protocol and those who did not complete nurse protocol groups; 325 agreed to participate for an overall response rate of 26% (Figure 1). Among those who completed the nurse protocol, there were no differences in regards to caller demographics, disposition recommended or oseltamivir prescriptions. Call times to complete the nurse protocol were shorter among persons who responded to the survey compared to persons who did not respond (12.2 vs. 13.3 minutes, p = 0.02). Among those who did not complete the nurse protocol, there were no statistically significant demographic differences between survey responders and non-responders.

309 (27%) callers who completed nurse protocol and 16 (20%) of callers who did not complete the nurse protocol participated in the survey (Table 2). Among those who completed the nurse protocol, 154 (50%) reported an in-person visit with a healthcare professional for their illness, and 58 patients (19%) were prescribed oseltamivir. When asked what they would have done without the MN FluLine, 73% would have sought care (26% would have gone to clinic, 9% to urgent care, 13% to emergency department (ED) and 25% would have called their clinic among those who completed the nurse protocol. Similar responses were observed among those who did not complete the nurse protocol (Table 2).

Of those completing the nurse protocol, 274 (89%) reported that the MN FluLine was moderately helpful or very helpful and 282 (91%) were satisfied or very satisfied with their MN FluLine experience (Figure 2). No significant differences were found in reported helpfulness or satisfaction by month of call despite extremely high call volume and longer wait times in the opening weeks of the MN FluLine. No significant differences in reported helpfulness or satisfaction were found between those who were recommended to seek in-person healthcare compared to those who were recommended for home care. Among those who did not complete the nurse protocol, 8 (50%) still found the MN FluLine was moderately helpful or very helpful and 8 (50%) reported being satisfied or very satisfied (Figure 2).

No statistically significant differences in satisfaction or helpfulness of the MN FluLine among those who completed the nurse protocol were found with regard to age of patient, gender of patient, call length or insurance status. In addition, no significant differences were found with regard to visiting a doctor for their illness, receiving oseltamivir, or what they reported they would have done without the MN FluLine. Those who completed the nurse protocol were significantly more likely to report being satisfied or very satisfied compared to those in the group who did not complete the nurse protocol (p<0.001). Similarly, those who completed the nurse protocol felt the service was moderately or very helpful compared to those who did not complete the protocol (p<0.001).

Qualitative data were collected via three open ended questions. A sample of the range in types of statements collected from participants is provided in Table 3. When asked how helpful the MN FluLine was, 78 people provided comments; 57 (73%) were categorized as being positive and included general positive feelings towards the service, appreciating not having to go in to the clinic or ED, and feeling calmed or reassured. Common themes among the 13 (17%) negative comments were not getting the answer they were looking for, too long a wait to get a callback, having H1N1 “hyped” too much by the media, and not having the same person initially answering the phone and administering the triage protocol. Eight (10%) of the comments were classified as neither positive nor negative.

When asked about their satisfaction with the MN FluLine, 181 people provided comments. Of these comments, 131 were positive (72%), 19 neutral (11%) and 31 were negative (17%). Common themes among positive comments were positive feedback about the service, receiving information that was helpful, and thankfulness at not having to go in to the ED or clinic. Among negative comments, common themes were frustration over the lag in callback time, receiving information that was not helpful, and not agreeing with the information given.

Finally, participants were asked if there was anything else they’d like to comment on and 134 callers provided comments of which 95 (71%) were positive, 21 (16%) neutral, and 18 (13%) negative. Common themes for positive comments included general positive feedback, information received was good and useful, and nurses being helpful. Common themes from negative comments included receiving information that was perceived to be wrong or not helpful and complaints over waiting too long for a callback.

The contract cost paid by MDH for the operation of the MN FluLine was $331,226. This cost included procurement and operations of the toll-free phone number and associated infrastructure needs. This also included transferring callers to their healthcare system nurse triage line if they had insurance and providing nurse triage services to those who did not have insurance or their healthcare system did not have a nurse triage line. The 27,391 calls were managed with an average cost of $12.09 per call.

The MN FluLine was a statewide NTL intervention implemented to respond to the public health threat of 2009 H1N1 influenza and included prescribing of antivirals. Investing in a toll-free number and agreeing to operate in a coordinated fashion, MDH and the healthcare systems in Minnesota with nurse triage lines were able to augment the current healthcare system at a time when it was highly strained. The inclusion of prescribing oseltamivir allowed callers to be served without the need for an in-person healthcare visit if antiviral treatment was indicated. This service had an immense impact on the MN population and healthcare community reaching over 27,000 callers in need of services [13]. The program evaluation based on a random sample of callers had an overall response rate of 26%, similar to findings from other evaluation studies conducted among nurse triage line callers that had 16% and 25% response rates [14], [15]. Costs associated with utilizing the MN FluLine were estimated at slightly more than $12 for each call. This is noteworthy, especially when compared with estimates of in-person healthcare delivery in the US Midwest (typically not including medication costs) at $876 per emergency department visit, $269 per urgent care visit, and $192 per clinic visit for upper respiratory infection and fever visit [16].

For callers with ILI who received the full MN FluLine intervention (i.e. completed nurse protocol), the vast majority reported that they were either very satisfied or satisfied and felt the service was very or moderately helpful. Although the numbers are small, and may not be representative, this is similar to findings regarding satisfaction from other telephone triage services [17], [18]. It was surprising to find that half of those who did not complete the nurse protocol reported being very satisfied or satisfied since this group did not receive the full intervention as it was intended, but it is possible that callers felt the existence of the service was valuable and satisfactory even if they themselves did not fully benefit from it. Qualitative comments provided by callers were very useful in providing clarity in areas that were successful and for learning about ways to improve the service during future applications.

The program evaluation identified several key aspects for improvement if a similar nurse triage line will be utilized in the future. First, the need was substantially underestimated and planning for far greater phone line and personnel resources initially which could be scaled back, would have improved the service. Second, callers expected to be connected to a nurse right away and greater public education about the process (i.e. talking to a medical screener first and then a nurse) would have improved this experience. Third, advance development of triage protocols instead of creating them de novo would have decreased the development time. Finally, advance planning with healthcare system, including healthcare plans, and healthcare delivery organizations and pharmacy representatives would have allowed for a more rapid implementation.

Fundamentally the MN FluLine was designed and executed rapidly to meet a major public health need. Had it been designed as a pilot project, it would have been useful to collect additional data that would have enabled additional assessment including a thorough cost-analysis. The evaluation survey was conducted several months after the launch of the MN FluLine creating the possibility for recall bias. We did not, however, see a difference in satisfaction or helpfulness by month of call which would have suggested recall bias. We kept the survey brief to increase participation, but this also meant we were limited in the data we were able to collect from callers. We utilized a cross-sectional post-test-only study design which does not allow us to factor in baseline measures. We did, however, utilize a random sample of callers to minimize sampling bias. Evaluation data were based on self-report. Since callers reported their survey responses to a nurse there is the potential for social desirability bias in their responses. Finally, wide implementation of NTLs may be limited by state and local laws. In MN, prescribing by a nurse protocol under physician supervision is allowed but this may not occur elsewhere. An assessment of federal and state law could provide additional insight into potential limitations to broad implementation.

Phone triage lines represent a powerful tool for public health. It provides the opportunity for flexible implementation of healthcare delivery during an emergency and a portal to consistent information. Creating a system to allow for early assessment and intervention likely prevented influenza-associated morbidity and possibly additional influenza illnesses. Further, it provides a system of clinical evaluation that is accessible in places where access to the internet or distance may limit rapid deployment of healthcare services. Such a service is likely to be cost savings to our entire healthcare system and allows the leveraging of existing call center infrastructure. Under the current United States healthcare system the financial benefits provided by such a system are not easily quantified due to the fractionated healthcare delivery and the fiscal division of the agents that bear the costs from those that receive benefits. The private-public partnership shared the cost (financial, infrastructure, and personnel). Federal, state, and local planners should consider phone triage lines as an important tool in response to future emergency response.