How to Deal with Uninformed and Poorly Informed Opinions of Citizens? A Critical Approach to Online Public Engagement

Public engagement is increasingly recognized as a mutual learning of perspectives between lay and expert stakeholders. Still, the intention to educate citizens sometimes prevails over an open and honest exchange. Because of this overemphasis on education, researchers may more easily label lay opinions invalid if uninformed or based on distorted beliefs. Our experience with uninformed and poorly informed (UPI) opinions in an online public engagement initiative (the DNA Debate) has taught us to think differently. First, UPI opinions might be ethically instructive, provided that one searches for the implicit message participants try to convey related to their values, fears, or needs. Since there will always be less informed or uninformed citizens, that would avoid rejecting their voices. Second, UPI opinions may highlight misconceptions in the general population for which more targeted education is necessary. This article is an invitation to reflect ethically on why and how researchers should deal with UPI opinions, illustrated by quotes from the DNA Debate.


INTRODUCTION
The need for public engagement on ethical, legal, and societal issues of genomic technologies is now well recognized.These technologies offer unprecedented benefits but could also impact individuals and society profoundly (Middleton et al. 2023;Pezzullo et al. 2021;Dryzek et al. 2020;Wirz, Scheufele, and Brossard 2020).They engender wicked problems such as balancing massive data sharing with privacy protection, equity in access to care with expensive personalized treatments, redefining health and its priorities, eugenics, and the risk of discrimination and stigmatization.The scientific community may not manage sustainably these new complex challenges if ignoring citizens ' values, needs, and concerns (Levitt 2003;MacDonald et al. 2020;Wirz, Scheufele, and Brossard 2020).The fundamental rationale for engaging citizens is to learn from their perspectives and interact with them in an open and honest dialogue (Pasgaard et al. 2023).That conception of engagement is gradually gaining ground as the norm to follow, in opposition to the outdated model of strategic education (Levitt 2003;Wynne 2006;Pytlik Zillig and Tomkins 2011;Stilgoe, Lock, and Wilsdon 2014;Woolley et al. 2016;Bell et al. 2018;Saunders 2018;Samuel and Farsides 2018;Dryzek et al. 2020;Wirz, Scheufele, and Brossard 2020;Boon et al. 2022;Das et al. 2022).
Still, public engagement remains sometimes used as a one-way education to gain trust and support from citizens (Levitt 2003;Wynne 2006;Stilgoe, Lock, and Wilsdon 2014;Woolley et al. 2016;Bell et al. 2018;Samuel and Farsides 2018;MacDonald et al. 2020;Wirz, Scheufele, and Brossard 2020;Boon et al. 2022;Das et al. 2022).Although education is an essential element of well-conducted engagement, it should not be pursued as its final aim, apart from initiatives honestly labeled as sensitization or educational campaigns.Because the overemphasis on education remains anchored in engagement practices, researchers may more easily label lay opinions invalid if uninformed or based on distorted beliefs.From our experience within a public engagement project called the DNA Debate, we have learned to stay alert for the unjustified dismissal of insufficiently educated perspectives, as these can also contribute meaningful insights to the societal debate.This article invites researchers to reflect on the risk of labeling lay opinions invalid if uninformed or poorly informed and formulates recommendations for a more suitable approach to public engagement.

THE DNA DEBATE AS A CASE STUDY
The DNA Debate was a broad online public engagement initiative surrounding the ethical, legal, and societal issues of genomic information, organized by Sciensano, the Belgian Scientific Institute of Public Health.It aimed to formulate recommendations for policymakers, experts, and relevant stakeholders based on the contributions of Belgian citizens because genomic technologies will impact their lives broadly and in multiple ways.
The debate was designed as an interactive platform where citizens could voluntarily contribute by posting their opinions under a pseudonymized profile.Participants were confronted with various educational materials before entering the platform.These included an interactive test featuring 15 ethical dilemmas on genomic technologies, a video explaining how genomic information can impact everyone's life (Sciensano 2019), a booklet gathering nine case studies and additional literature (King Baudouin Foundation and Sciensano 2018), and a pedagogical dossier for secondary schools to enable young generations to participate.An independent and multidisciplinary experts committee validated these materials to ensure they were as neutral and accessible as possible to every level of education (Raeymaekers and Teller 2019).They illustrated diversified perspectives, balanced the benefits and risks of genomic technologies, and articulated values at individual, familial, and societal levels.
Participants could contribute by sharing their thoughts about five open questions: 1.What encourages me to learn more about my DNA or dissuades me from doing it?Why?; 2. What motivates me to share my DNA data or stops me from doing so?Why?; 3. A genomic passport for all: a good idea or not?Why?; 4. How would DNA be used in my ideal society?What applications should be avoided?Why?; 5. Do you want to share another idea about how we should deal with DNA in society?A short description contextualized each question within the broader debate on current and future genomic data usage.For instance, the genomic passport illustrated the increased collection of genomic data, potentially combined with other health data, to provide citizens with personalized medicine for preventive, diagnostic, and therapeutic purposes.In total, 1,127 citizens voiced 1,258 opinions.For a more detailed description of the methods and the qualitative analysis of all contributions, see Mayeur, Saelaert, and Van Hoof (2021) and Mayeur, Mertes, and Van Hoof (2023).
During the inductive thematic analysis, certain opinions left us in doubt about whether we should or should not discard them.First, they indicated misconceptions about genomics while revealing relevant underlying values, concerns, or needs.Second, they were harder to interpret because the origin of the misconceptions was not always clear.Third, those participants could not be asked for clarification because their pseudonymized profile protected their identity.We decided to take them as case studies to open the reflection on why and how researchers should deal with uninformed and poorly informed (UPI) opinions when engaging citizens.
The acronym UPI covers distorted, exaggerated, or unrealistic beliefs that one could easily reject as invalid based on empirical evidence.Researchers unconsciously approach citizens' opinions based on their thought patterns, usually focused on the verifiability of facts, rigorous observation, and scientific objectivity.Yet, a mismatch can arise between the experts' and the citizens' ways of thinking.Citizens generally reason in values, common sense, and subjective experience (Wirz, Scheufele, and Brossard 2020), often based on fragmented knowledge and sometimes flawed beliefs.Despite the correct information they received, some participants will stick to their perceptions and intuitions, which some experts could label as far-fetched and consequently disregard.Especially in online settings engaging a broad audience, it might be hard to transform the subjective experience of all uninformed and poorly informed participants into wellinformed and reasoned opinions.

WHY RESEARCHERS SHOULD CONSIDER THE UNINFORMED AND POORLY INFORMED OPINIONS OF CITIZENS
Authentic public engagement requires researchers to stay open-minded towards the thinking patterns of citizens, from which they could learn more than expected (Stilgoe, Lock, and Wilsdon 2014;Saunders 2018;Wirz, Scheufele, and Brossard 2020).UPI opinions of citizens within the DNA Debate are of interest for three reasons.First, analyzing them allowed us to identify underlying values, fears, and needs, which are still legitimate from a normative perspective, although not scientifically well informed.Second, they highlight misconceptions in the general population for which more targeted education is necessary.Third, considering UPI opinions would avoid rejecting the voices of less educated participants, acknowledging that public engagement is more than solely education.

NORMATIVE INPUTS GAINED FROM UNINFORMED AND POORLY INFORMED OPINIONS
Some UPI participants overestimated the influence of genomics on their identity, thereby establishing a profound connection between their genome and their true self.
It seems like a pretty good idea to me, if used properly and with clear rules attached.I can well imagine the idea.So much data are already scattered about you everywhere through the internet and the like.But there are no data about the real person you are on the inside.When every citizen has a genomic passport, people should worry less about being different because it proves that everyone is different and that no DNA is identical.People sometimes have an identity crisis because they no longer know who they are.A genomic passport would make all doubts disappear because everyone would have the truth about who we really are, next to the thoughts about who they are or who others are.(Contribution n°597, Question 3) I would not like to know anything about my DNA.DNA contains many surprises that I would like to discover myself.Each DNA has its own information.It is like a book: you can first read the end or start from the beginning.If you read from the beginning, there are no spoilers; everything is a surprise, and you have no idea what will happen.What is left if we know everything about our lives and talents in advance?There are no more challenges to finding your talents; no more surprises if you know from the start that you are a math genius or a creative person.We are supposed to find these things out ourselves and make mistakes because we learn from our mistakes.The only exception I would make is for checking serious illnesses because if I had a fatal disease, I would want to know about it.Then I will enjoy more things that are really important to me, like friends, family, traveling, etc. (Contribution n°1015, Question 1) The strong link between DNA and identity aligns with results from previous public engagement initiatives in which citizens were well educated (Metcalfe et al. 2018;Voigt et al. 2020;Hopkins, Kinsella, and Evans 2021;Mayeur and Van Hoof 2021).The perception of DNA as all-powerful may generate disproportionate hopes and expectations or, to the contrary, exaggerated fears and mistrust (Carver et al. 2017).
I am NOT in favor of this.Knowing your DNA and that of others will bias all the choices in your life and make life completely emotionless.Knowing your DNA is a bit like knowing the date of your death.What would become of a society whose individuals knew what their future would be like?What place will be left for the trials of life resulting from its randomness, which are sources of individual and collective enrichment?All forms of altruism are likely to disappear in the long run.The randomness of encounters will disappear: why would anyone want to build a life with someone whose DNA is at-risk?That could lead to the fragmentation of society, with the good DNAs together and the bad ones put aside.It could also lead to ideological movements of DNA purification… (Contribution n°64, Question 1) These data limit your freedom and your future.I would not like to share my DNA data with the outside world.If I were to apply for a job, for example, they would know straight away whether I was suitable for the job or not because everything can be read in DNA.It would be terrible if someone were to finish their medical studies and then be told when they apply for a job: "Ah, but your DNA shows that you are not stress-resistant enough for this job.Sorry, we can›t hire you."I don›t think anyone is waiting for such a moment.If the sharing of DNA data were mandatory, a privacy law should be enacted.Otherwise, there will be incidents like this in the future.(Contribution n°598, Question 2) Although the genetic determinism reflected in these contributions is scientifically incorrect, some actors can still abuse genomic information in a deterministic manner.Those participants feared that the massive use of genomic data would create a power imbalance, rendering them vulnerable.Discrimination by for-profit actors is one of the most frequent concerns pinpointed in engagement initiatives (Middleton et al. 2016;Haeusermann et al. 2018;Dheensa et al. 2019;Middleton et al. 2019;Hopkins, Kinsella, and Evans 2021;Mayeur and Van Hoof 2021;Rivas Velarde et al. 2021), showing that distrust is widespread among the public, whether scientifically educated or not.
The scientific culture has been criticized repeatedly for its tendency to attribute public mistrust to poor literacy about science, explaining the overemphasis on education in engagement activities to gain support from citizens (Wynne 2006;Stilgoe, Lock, and Wilsdon 2014;Samuel and Farsides 2018;MacDonald et al. 2020).That can be seen as a form of epistemic injustice in which experts discredit lay knowledge when it does not align with the scientific way of thinking.Two injustices conceptualized by Fricker ( 2007) are here at play: the testimonial injustice that discredits someone's word based on the prejudices attributed to that person (mistrust comes from ignorance) and the hermeneutical injustice discrediting someone's views because the person lacks adequate vocabulary and interpretative concepts held by the dominant discourse.The science-driven approach to public engagement may severely damage the relationship between science and citizens because they do not feel taken seriously as legitimate stakeholders, offering a valuable perspective with divergent concerns, interests, needs, and values (Wynne 2006;Woolley et al. 2016;MacDonald et al. 2020;Wirz, Scheufele, and Brossard 2020;Das et al. 2022).Conversely, a more open-minded approach would enable researchers to understand the causes behind participants' mistrust.
One question remains unanswered: Can we assume that participants would have expressed the same values, needs, or concerns if better informed?Since we could not measure the impact of the educational materials on participants' opinions within the DNA Debate, we searched for relevant answers in the literature studying the influence of information on attitudes, values, and perceptions.Different empirical studies reached similar conclusions.Carver et al. (2017) 2017) stated that citizens with a higher comprehension of genetics can display higher hostility towards genetic technologies.The factors shaping perceptions and attitudes toward science are individual characteristics (such as morality, experience, trust in science, personal knowledge), information climate (media, marketing, television), and sociopolitical and cultural contexts (religion, institutions, economics) (Wirz, Scheufele, and Brossard 2020).People tend to search for and trust facts confirming and illustrating their values and worldviews, a phenomenon called confirmation bias (Ecker et al. 2022).Kuklinski et al. (2000) indicated that participants receiving nuanced information (supporting and running counter to their opinion) tend to revert to their initial position illustrating their values.Accordingly, education in public engagement has limited influence on participants' perceptions and values, although it may impact how they express them, for instance, through relevant and correct information.

UNINFORMED AND POORLY INFORMED OPINIONS HIGHLIGHT MISCONCEPTIONS ABOUT GENETICS
The UPI opinions from the DNA Debate highlighted miscomprehensions among the participants for which more targeted education is necessary.The most worrisome one relates to their deterministic view of DNA.I wouldn't want my DNA to be known.I don't want anyone to have access to it.I don't want DNA to influence my life.I don't want to know that I will die on October 6 th , 2069.(Contribution n°270, Question 1) Everyone has a particular future ahead of them, which can drastically change through one specific act.If DNA determined that you would die at 37, OK, nobody wants that of course.But I think you have to accept your future and this can be made more difficult if suddenly everybody already knows how and when you will die or get sick.That's why I believe it's best to leave your DNA for what it is and not do any research into it because this will drastically change your future and maybe even the future of others.(Contribution n°422, Question 1) Their deterministic conception of DNA was linked to the fear that genomic testing would render them prisoners of their genome, psychologically or socially speaking.A systematic review of the public attitudes, knowledge, and educational needs (Calabrò et al. 2020) pointed out the low level of awareness and education about genomics in general, including the influence of genetic and nongenetic risk factors on disease developments and the possibilities offered by genomic testing and research.According to a study investigating deterministic beliefs about genomics among the general public, one source of the problem lies in schools (teachers, educational material) and media that disseminate a non-negligible discourse about genetics with a deterministic background (Carver et al. 2017).
DNA is not a fixed destiny because the translation of genotype to phenotype evolves over a lifetime, highlighting the significance of prevention.Additionally, the scientific interpretation of testing results is constantly progressing, stressing the importance of conveying the uncertainties inherent in genomics, such as the possibility of receiving results of unknown significance.Education on genetics could, for instance, clarify that DNA tests provide mainly probabilistic information and that many diseases are multifactorial, even with a genetic component (Carver et al. 2017).Tackling the incomprehension of the general population would avoid disproportionate concerns or hopes, resulting in more evidence-based decision-making regarding the introduction of genomic technologies in healthcare (Carver et al. 2017;Calabrò et al. 2020;Williams et al. 2020).Education can be effective only if targeted to the needs of citizens while considering their concerns, values, and interests.The systematic review mentioned above (Calabrò et al. 2020) identified the internet, TV, newspapers, and magazines as the principal channels that citizens used to be informed about genomics, and suggested that more specific topics, such as ethical issues, could be addressed through public engagement initiatives to stimulate debates based on trusted information.

ENGAGEMENT CANNOT BE REDUCED TO EDUCATION
Alongside the enriching elements we have learned by analyzing UPI opinions from the DNA Debate, a more fundamental reason justifies why researchers must consider these opinions when engaging citizens.Engagement cannot be reduced to education, as Burgess already warned us against in 2003: Overemphasis on the need to become "expert" in a particular application neglects the expertise and responsibility we all have as citizens to consider the effects of our actions on others, and to participate and respect the stakes of others in the kind of society we become.On the other hand, wholly uninformed discussion are unhelpful and at worst, misleading.So public engagement should make technical information available, but it must not do so at the expense of representation of the interests and perspectives of all members of society (p.15).
Ultimately, the reason for engaging citizens lies not in their level of knowledge but in the unique input and perspective they can provide as lay stakeholders on a normative rather than epistemic level.One should be careful about drawing a sharp distinction between experts and nonexperts but consider different groups of people to have complementary expertise (Levitt 2003;MacDonald et al. 2020).In the public engagement context, citizens might be seen as experts in their lived experiences and values (Werner et al. 2020), enriching the scientific knowledge provided by genomic experts and the ethical expertise of philosophers and bioethicists.Three years of research investigating the usefulness of public engagement in the ethics of genomics governance (Tansey and Burgess 2006) underlined that the values and norms of citizens differ from those of policymakers and experts.Hence, citizens cannot be ignored even on topics for which it is difficult to establish a thorough understanding for lay people (Levitt 2003; Wirz, Scheufele, and Brossard 2020).In the context of advanced new and complex technologies like genomics, it is unsurprising that some participants remain insufficiently knowledgeable despite receiving accessible information.Considering UPI opinions would avoid rejecting the voices of those participants.It is also important to note that panels of experts do not consistently formulate guidelines for the best clinical practice based on facts and evidence but often use their personal experience and interests (Tsay 2018).Intuitive thinking is universally human (MacDonald et al. 2020).
Importantly, our findings do not state that every opinion is valuable at the same level, whether well informed, poorly informed, or uninformed.Researchers should always pay attention to conducting public engagement in a way that spreads scientifically valid, as neutral as possible, but easily understandable information.While it would not be wise to base policy on opinions that one can rebut on a factual level, it can be instructive to analyze the information citizens lack to express their thoughts correctly or the beliefs they constructed to make sense of their values, needs, and fears.The Council of Europe (2020) recently applauded public engagement that enlightens the beliefs on which citizens build their views:

HOW RESEARCHERS MUST TAKE UNINFORMED AND POORLY INFORMED OPINIONS INTO ACCOUNT
First and foremost, researchers should not take UPI opinions literally as they are deformed by poorly informed or scientifically inaccurate beliefs, contrary to wellinformed ones that can provide direct relevant input for the qualitative analysis.UPI opinions require more interpretation to reveal implicit ethical messages.Two independent studies interviewing participants from public engagement initiatives about their expectations reached the same conclusion.Citizens primarily acknowledge their role as consultative and do not expect decisionmakers to implement their recommendations literally.They asked authorities to recognize their concerns and follow broad directions and changes emerging from their recommendations (Jacquet 2019;Werner et al. 2020).Given this, there is sufficient margin to include the UPI opinions in the general qualitative analysis.We advise carefully separating them from the well-informed opinions and incorporating the former as a separate dataset (see Supplementary File 1: Dataset of citizens' uninformed and poorly informed opinions in the DNA Debate) when constructing the final recommendations.When publishing results mixing well-informed and UPI opinions, researchers should pay attention to transparently reporting how they interpreted UPI opinions and how these influenced the conclusions.That process avoids giving equal credit to wellinformed opinions and UPI ones, among others, because researchers had to take a wider margin of interpretation to translate the latter into ethical outcomes.
The question remains how far researchers should interpret the UPI opinions.The absence of mutual interaction in online consultations complicates the task of researchers in interpreting participants' contributions.When trying to understand what matters to UPI participants, researchers should always pay attention to the context of the opinion, such as the question answered, the information received that seems to feed their misunderstanding, and the significance of the words used.For instance, participants of the DNA Debate, who worried that their genomic passport would allow the government to track their movements, potentially conflated what a genomic passport could be with current travel passports.
Making a genomic passport for every citizen greatly reduces freedom, in my view.That way, the government can use DNA to see where every citizen has been.(Contribution n°561, Question 3) The healthcare benefit is an asset.But if this information falls into the wrong hands or the government decides to do something else with it, then you no longer have privacy and everyone knows where you have been by collecting DNA samples.(Contribution n°1103, Question 3) Instead of dismissing these contributions, researchers could extract privacy and individual freedom as meaningful values and distrust in governments as an impediment to government-initiated genomic information sharing.
A potential pitfall is overinterpretation, which can be as paternalistic as excluding the opinion from the analysis.For instance, contribution n°211 strongly opposes genomic data collection at a population level but lacks justifications to understand its categorical statement.
The creation of a permanent database containing the DNA of the whole population constitutes a crime against humanity.(Contribution n°211, Question 4) Trying to analyze the exact meaning of a crime against humanity would have required overinterpreting it at the risk of identifying inaccurate values or fears.This example illustrates the limitations of collecting public attitudes without the option of engaging in a conversation with participants to explore their views further.However, it does not mean the engagement was worth nothing for those participants because they reflected more deeply and heard divergent points of view and alternative values and experiences.

CONCLUSION
It is increasingly acknowledged that public engagement is more about listening to and learning from citizens' perspectives than educating them.Yet, some initiatives are conducted strategically as a one-way education to gain trust and support from citizens.The overemphasis on education may incite researchers to disregard lay opinions if they are UPI.Reflecting on our experience within the DNA Debate project, we argue that researchers should develop strategies to deal with the UPI opinions of participants for several reasons.First, they are still legitimate expressions of people's values, fears, and needs, although not scientifically well informed.Hence, we encourage researchers to search for the implicit message participants try to convey related to their values, fears, or needs.Second, UPI opinions may highlight misconceptions in the general population for which more targeted education is necessary.Finally, considering UPI opinions would avoid rejecting the voices of less educated participants.
The reflection and conclusions drawn from this article are most relevant in the context of types of public engagement in which researchers cannot intervene during the process to discuss and correct the flawed beliefs of participants, which would otherwise be the preferred approach.They are nonetheless interesting for public engagement in general, as they underline researchers' duty to consider the voice of all participants, including those whose contributions require a more careful interpretative approach.Thereby, researchers would respect the participation of all citizens who devote their time, energy, and trust to their engagement project.We hope to inspire public engagement initiatives to think outside the box in developing methods to include the valuable inputs of participants who are not well informed in a scientifically rigorous way.
and MacDonald et al. (2020) deconstructed the belief that more education leads to positive attitudes toward science.MacDonald et al. (2020) demonstrated that focusing on technical facts polarizes participants' concerns, [OK] and confirmed that values significantly influence opinions about new technologies.On their side, Carver et al. ( It is important to know to what extent the evidence of public views will represent informed conclusions and to what extent it is the expression of underlying beliefs or values.It may also be important to know what informs these views (for example, technical knowledge or religious faith), how entrenched they are, and in what ways and for what reasons they might alter over time (p.17).