Composition Vendor Newgen KnowledgeWorks (P) Ltd. PLoS One plos plosone PLOS One 1932-6203 Public Library of Science San Francisco, CA USA 10.1371/journal.pone.0347757 PONE-D-25-15257 Research Article Biology and life sciencesNeuroscienceCognitive science Research and analysis methodsResearch designSurvey researchSurveys Physical sciencesMathematicsStatisticsStatistical data Biology and life sciencesPsychologyBehaviorImitation Social sciencesPsychologyBehaviorImitation Physical sciencesMathematicsProbability theoryProbability distribution Biology and life sciencesNeuroscienceCognitive scienceCognitive psychologyLanguage Biology and life sciencesPsychologyCognitive psychologyLanguage Social sciencesPsychologyCognitive psychologyLanguage Social sciencesLinguisticsSociolinguistics Physical sciencesMathematicsProbability theoryProbability density LLM-impersonated debate contributions are more authentic, relevant and coherent than their original: A representative study using BBC1’s Question Time LLM-impersonated debate contributions are more authentic, relevant and coherent than their original https://orcid.org/0000-0001-9765-2803 Herbold Steffen Conceptualization Data curation Investigation Methodology Project administration Resources Visualization Writing – original draft Writing – review & editing * Trautsch Alexander Data curation Methodology Software Writing – review & editing Kikteva Zlata Data curation Investigation Methodology Writing – original draft Writing – review & editing https://orcid.org/0000-0002-5901-9633 Hautli-Janisz Annette Conceptualization Methodology Project administration Resources Writing – review & editing Department of Computer Science and Mathematics, University of Passau, Passau, Germany Hassan Mohammad Salah Editor A Sharqiyah University, OMAN * E-mail: steffen.herbold@uni-passau.de

The authors have declared that no competing interests exist.

 172026 2026 21 7 e0347757 2432025 142026 2026 Herbold et al This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Generative AI has the potential to pollute the public information sphere with made-up content, posing a significant threat to the cohesion of societies at large. This paper offers the first large-scale and systematic study of how authentic, relevant and coherent impersonated content from Large Language Models (LLMs) is perceived by the general public. Based on a cross-section of British society, we show that LLM-generated responses to questions drawn from a broadcast political debate programme in the UK are judged to be more authentic and relevant than the original responses given by the panel members who were impersonated. We also show that stylistic differences do not influence these judgments, meaning that the distinction of original and generated content is challenging for the general public. Taken together, this means that LLMs can be made to deceive the public regarding the nature of statements in the political domain, with the consequence that there is a dire need to inform the general public of the potential harm this can have on society.

 http://dx.doi.org/10.13039/501100001663 Volkswagen Foundation 98544 https://orcid.org/0000-0002-5901-9633 Hautli-JaniszAnnette A.H. work was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’ URL: https://www.volkswagenstiftung.de/ The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Data Availability All materials are available online in the form of a replication package that contains the data and the analysis code at https://github.com/aieng-lab/replication-kit-qtgpt-study and https://doi.org/10.5281/zenodo.12698363. Introduction

Modern Generative Artificial Intelligence models (GenAI) like GPT [1], Claude [2], and Gemini [3] have been shown to generate high-quality textual content including source code [4], persuasive student essays [5], and legal analysis [6]. They also seem (at least partly) to be able to role-play [7], to mimic the linguistic pattern of authors [8], to generate content that reflects general political identity [9,10] and to assume the role of a domain expert who responds accordingly [11]. Whereas developments like these are already questionable in terms of moral and ethical responsibility, Large Language Models (LLMs) influencing the political opinion of humans [12], purporting political bias [13] and successfully generating targeted persuasive communication [14,15] clearly signals that the technological advancement is reaching a critical stage where harm to societies is only a prompt away. This is solidly supported by the findings of this paper.

In our study we condition an LLM in such a way that it impersonates well-known political and societal personalities in a political talk show on UK national television. Specifically, the LLM is tasked to respond to the audience questions in the show as an impersonation of the original speakers on the show. Based on a representative cross-section of British society (n = 948), we study how UK citizens rate the actual response of the person in comparison to the impersonated response along three axes: authenticity (the likelihood that the impersonated response comes from the actual person), coherence (the logical flow of the response), and relevance (the extent to which the response is relevant to the question). We also compare how content and linguistic properties differ between original and impersonated statements and elicit the openness of citizens to using AI technology for generating contribution to public debates. These three lines of research answer the following research questions:

RQ1: To what extent do UK citizens rate the authenticity, coherence, and relevance of impersonated debate responses differently from actual debate responses by that person?

RQ2: To what extent does a difference in content and linguistic style between actual and impersonated debate responses impact their authenticity ratings?

RQ3: What is the general public’s view on using AI in public debates and is this view affected by exposure to technology?

The data underlying our study originates from 30 episodes of BBC1’s Question Time, one of the most viewed political debate programmes in the UK, which were broadcast from 2020 to 2022 [16]. The survey participants are asked to (i) attribute both actual and impersonated responses to public persons; (ii) evaluate how coherent and relevant both actual and impersonated responses are; and (iii) express their opinion regarding the use of AI in public debates. The last task is split into two sub-tasks: First, the participants give their opinions on AI unaware of the source of the material they just rated (actual vs. impersonated). In the next step, they are shown the source of their rated content, and they express their opinion on the technology again.

 Materials and methods Data Original debate content.

The original debate data is from ‘Question Time’ (QT), one of the most-viewed political talk shows on UK television. QT30 [16] is a collection of 30 episodes of QT aired on BBC1 between June 2020 and November 2021, currently the largest dataset of analysed broadcast political debate. QT features a moderated panel format, where well-known members of politics and society sit on a panel and respond to questions from the audience on the current topics of the week. The panellists are directed by the moderator and are asked to respond to the questions independent of a prior conversation on the topic and the initial statements by other panellists. The panel members featured in the dataset belong to one of six categories: politicians (50%), business people (16.67%), journalists (14.17%), medical experts (6.67%), writers (5.83%), and other well-known members of UK society (activists, actors, political experts and sports personalities – 6.67%). The use of this data for research is covered by Section 29 (1) of the British Copyright, Designs and Patents Act (CDPA). The data cannot be shared publicly and will be shared with other researchers for the sole use of research upon request (see data availability statement).

From QT30 we manually extract a total of 119 unique questions with 555 responses from 119 different speakers. We discard the responses of seven speakers who do not have a Wikipedia page, a requirement for generating the impersonated debate responses. At the same time, this criterion serves as a filter to determine whether the panel members are well-known personalities in the public sphere. We also discard one response where the corpus data does not contain information about the speaker. This yields a set of 527 valid question/response pairs from 112 different speakers. We randomly drop seven responses to achieve a final count of 520 question/response pairs to facilitate easier sampling. A manual check of the resulting data ensures that the responses are understandable without context and do not refer to other panel members’ previous contributions, as this could affect judgments of authenticity and relevance.

 Impersonated debate content.

To generate the impersonated responses, we use GPT-4 Turbo [1]. While more recent models, e.g., Opus Claude [2], seem to be slightly better at logical tasks like mathematics, we are not aware of any benchmark where GPT was significantly outscored in tasks that involve common knowledge (as shown, for instance, in [17]). For prompting, we use a complex emulation protocol similar to Bhandarkar et al. [8] with the following prompts:

System prompt: You are an expert at mimicking different persons in debates. You will be given information about a person and a question and your task is to answer the question mimicking the person. You only answer as the person you are asked to mimic. Do not say the name of the person you are mimicking. Do not introduce yourself. Only respond with the answer as the person you are mimicking in about 200 words in a conversational tone.

User prompt: Please only answer this question: [QUESTION] as this person: [SPEAKER_WIKIPEDIA]. Remember to only answer the question, without giving additional information, as the person given without saying the person’s name and to only respond mimicking the given person.

The system prompt defines the behaviour we expect from the model, i.e., it is tasked to mimic a well-known person, to be brief in the response to the question and to use a conversational tone. The user prompt defines the task, provides the question and adds the short biography of the speaker that we obtain from the first paragraph of their Wikipedia article (this paragraph provides a summary of the information on their origin, career, party affiliation, political offices etc.) The user prompt also repeats the task for the model.

After receiving the responses, we conduct a manual sanity check to ensure that the impersonated responses adhere to the guidelines given in the prompt, i.e., do not contain the name of the speaker, any information that the response was generated by an LLM or a reason why no response was possible (for instance, due to lack of access to real-time data or for ethical reasons). This check did not flag problematic content, meaning that all responses were used in the subsequent study.

 Study participants

Since the debate content we study originates from one of the most popular British topical debate programs, we recruit a representative sample of British citizens above the age of 18 using the online platform Prolific. The recruitment time frame ran from June 18th until June 26th, 2024. Participants are informed about the purpose of the study, consent to participate and receive a participation fee for compensation which lies above UK minimum wage. We recruit a total of 948 participants, who are distributed randomly across the different tracks that the study comprises, resulting in at least two judgments for each of the 1560 question/response pairs (520 original question/original response pairs, 520 original question/impersonated response pairs, and 520 pairs of original question/original responses with random speaker) in each track.

 Study design Measures and variables.

To evaluate the perception of original and impersonated debate content, we elicit judgments on the following measures:

Authenticity: The likelihood that the response is an actual and real utterance by the speaker in a debate, i.e., the degree to which this utterance could have been contributed by that speaker in a debate. This variable measures the core aspect of our study, i.e., if people believe that a statement is genuine.

Coherence: The logical flow of the response. This variable measures the internal reasoning structure of the response.

Relevance: The extent to which the response addresses the question. This variable measures if the response stays on topic and conveys relevant information.

Content: The extent to which the overall meaning of original and impersonated response is identical. This variable allows us to understand if LLM-generated responses differ from the actual responses.

Confidence: The confidence in judging whether the response was given by a specific speaker. This variable is used as a control variable to understand if the certainty in judging debate content is affected by whether it is original or impersonated.

Familiarity: The knowledge on a panel member based on their previous public appearances. This variable is used as a control variable to understand if familiarity with a speaker has an impact on the authenticity judgments.

For all measures we use a five-point Likert scale (for instance, ‘not authentic’ to ‘very authentic’) such that the middle point of the scale is neutral. The supplementary material contains the full description of the measures. The judgments are collected using an online survey, the design of which we present in the following.

 Study procedure.

Our custom-made online survey starts with the collection of demographic data on the participants, i.e., their age, gender, country of residence within the United Kingdom, and political preference. At this stage, the participants are only informed that the debate questions and responses are taken from the BBC1 show ‘Question Time’. They are not aware that some of the responses are generated by an LLM (this information is disclosed only at a later stage of the survey). This deceptive design prompts participants to believe they are judging actual debate content.

The participants are then randomly sampled into three tracks. Track1 measures the perception of the authenticity, coherence, and relevance of a single debate response given a question. The participants are shown the question, one response, and the name of the speaker. The response is either the original response by the speaker or an LLM-generated, impersonated response, as described above. Track2 augments this setting by showing original and impersonated response side-by-side: the participants see the question, the name of the speaker and both responses at the same time. Their task is to compare the responses in terms of authenticity, coherence, and relevance to the question. We remove position bias by randomly positioning original and impersonated responses on the left or the right side of the page. Track2 also measures whether the content of the impersonated responses is the same as that of the actual responses.

With Track3 we try to understand better the factors that lead to differences in authenticity: Here, participants are shown a question, a response, the name of the speaker, and the speaker’s short biography. The biography is the same one that we provide to the LLM as part of the user prompt. For statistical reasons, we create three populations of participants. The first population sees the question, the actual speaker, their biography and the actual response. The second population sees the question, the actual speaker, their biography and the impersonated response. And the third population sees the question, the actual response from the actual speaker, but a randomly selected different public person from our data set, plus that person’s biography. All participants are asked to judge the authenticity of the responses, to rate their confidence in the judgment and to rate their familiarity with the speaker.

Once the participants complete their respective tracks, they participate in an exit poll. Here we ask questions regarding their familiarity with AI and chatbots, their opinion on the use of AI in public debates, and the perceived need for transparency and regulation in this setting (see more details in the supplemental material S1 Appendix). Only after the exit poll is completed, we reveal to the participants which of their judged debate responses were generated by an LLM, together with their ratings. In light of this new information, we repeat the exit poll for all participants. This allows us to see whether more insight into the quality of the generated responses affects their opinions regarding the use of AI in public debates. As a last step, the participants are then invited to provide an (optional) free-text comment regarding their answers in the exit poll.

Overall, each participant judges eight different responses. For Track1 and Track3 this means that each participant judges eight question/response pairs (we use rejection sampling to ensure each question/speaker pair only appears once, i.e., it is not possible for a participant to judge both the actual and the impersonated responses from a speaker to a question). For Track2 this means that each participant judges four pairs of question/original plus impersonated response. Additional details about the survey, including the exact wording of the questions, are provided in supplemental material S1 Appendix.

 Stylistic comparison

We also measure stylistic differences between the original and the impersonated responses by comparing discourse-related linguistic patterns. This allows us (1) to understand if the responses share properties on the linguistic surface and (2) whether the language is related to human judgments in terms of authenticity, coherence, and relevance. The study is based on the following linguistic properties:

Syntactic complexity: Syntactic complexity in terms of the mean number of conjuncts, clausal modifiers of nouns, adverbial clause modifiers, clausal complements, clausal subjects and parataxis per sentence as an approximation of language complexity. [18]

Nominalisations: The number of nominalisations per sentence as an approximation of the of abstractness of the language.

Modals: The number of modal constructions (e.g., ‘definitely’, ‘potentially’) per sentence as a signal of the stance of the speakers towards their utterances. [19]

Discourse markers: The number of discourse markers (e.g., ‘first’, ‘moreover’) per sentence as an approximation of the coherence of text and the use of explicit argumentative structure. [20]

Epistemic markers: The number of epistemic markers (e.g., ‘I think’, ‘in my opinion’) as an indication of the commitment of a speaker to the message they convey.

Lexical diversity: Lexical diversity measured with MTLD [21] as an approximation of the diversity of the used vocabulary.

Lexical overlap: The percentage of words in the question (excluding stop words) that also appear in the response as an approximation of the influence of the question on the response.

These stylistic features are automatically extracted from the original and the generated responses using a combination of rule-based, stochastic and neural models for natural language processing. Modals, discourse markers, epistemic markers, nominalisations, and the lexical markers are normalised by the number of sentences within a response.

 Statistical analysis

The inter-rater reliability between the two judgments for authenticity, coherence, relevance and content is measured with Cronbach’s α [22]. Additionally, we report pair-wise differences between the judgments to quantify the disagreement between participants. We exclude confidence and familiarity because we cannot expect agreement regarding a subjective self-reflection. For the subsequent statistical analysis, we map the Likert scales to the integers [−2, −1, 0, 1, 2] and compute the average rating between the two judgments for the same data point. Since the variables from our survey are based on Likert scales, we use non-parametric rank-based statistical tests.

In Track1 we assess the difference in authenticity, coherence, and relevance between the original responses and the impersonated responses. The track has a between-subjects design (i.e., actual and impersonated responses are rated by different participants) with data that is paired by the question and the speaker. Consequently, we use a two-sided Wilcoxon signed rank test [23] to determine if the difference between both populations (actual versus impersonated responses) is significant.

Track2 uses a within-subjects design (i.e., one participant judges both the actual and the impersonated response). We conduct a two-sided one-sample Wilcoxon signed rank test to determine if the judgments regarding authenticity, coherence, relevance, and content are significantly different from zero. For authenticity, coherence, and relevance, a significant tendency towards negative values means that the participants favour the original responses; a significant tendency towards positive values means that the generated, impersonated responses are favoured. Regarding content, a significant positive value means that the content between original and impersonated responses is similar; a negative value means that the impersonated content is different from the actual content by the speaker. We post-process the data such that the original response is always on the left and the impersonated response is always on the right.

With the data from Track3 we assess if the authenticity and the confidence in the rating depend on whether the speaker is real, random, or impersonated. This results in three populations. The track has a between-subjects design where the populations are paired by the question and the actual speaker. To elicit whether there is any difference between the three populations, we use a Friedman test [24] with a Bonferroni-Dunn post-hoc test based on pair-wise two-sided Wilcoxon signed rank tests. This determines which differences between pairs are significant. Additionally, we use familiarity judgments to understand how this affects authenticity. For this, we conduct a subgroup analysis where we split the ratings into those where the familiarity is less than 0 (i.e., ratings where the participants are not/to a limited extent/fairly familiar with the speaker) and judgments with a familiarity greater than or equal to 0 (speakers are somewhat familiar/familiar with the speaker). For the latter subgroup, we do not have paired data anymore, because we have independent raters for the three populations. For instance, the raters for the responses attributed to the actual speakers may be familiar with different speakers than the raters for the impersonated responses, leading to different subgroups. We therefore use a Kruskal-Wallis test [25] with Bonferroni-Dunn post-hoc tests based on pair-wise two-sided Wilcoxon–Mann–Whitney tests [26].

The statistical analysis of the linguistic surface markers is similar to the analysis of Track1 since we also have two populations for each linguistic marker (actual versus impersonated responses). Since the data from the linguistic markers does not follow a normal distribution (visual analysis of the distribution in Fig 5 shows, for instance, long tails), we also use non-parametric tests, namely two-sided Wilcoxon signed rank tests to determine if differences for each of the seven linguistic markers are significant.

10.1371/journal.pone.0347757.g001Judgments when a debate question, the name of the speaker, and either the GPT-generated or the actual response by the actual speaker are shown.

Violins show a kernel density estimation of the probability distribution, the miniature box-plots depict the median, upper and lower quartiles, and the whiskers the largest/smallest value observed within 1.5 times the interquartile range of the upper/lower quartile. The statistical markers reported are the p-value of two-sided Wilcoxon signed rank tests, the effect size r, the sample sizes n, mean values M and standard deviations SD.

 10.1371/journal.pone.0347757.g002Judgments when a debate question, the name of the speaker, and both the actual and GPT-generated responses are shown side-by-side.

The stacked bar chart reports the percentages of the ratings that we observed. The statistical markers reported are the p-value of a two-sided one-sample Wilcoxon signed rank tests for a difference from zero, the effect size r, the sample sizes n, mean values M and standard deviations SD.

 10.1371/journal.pone.0347757.g003Judgments when a debate question with either the response and biography from the actual speaker, the GPT-generated response and the biography of the actual speaker, or the response from the actual speaker but the name and biography of a random public person are shown.

Violins show a kernel density estimation of the probability distribution, the miniature box-plots depict the median, upper and lower quartiles, and the whiskers the largest/smallest value observed within 1.5 times the interquartile range of the upper/lower quartile. The statistical markers reported are the p-value of the omnibus test for differences and pair-wise Bonferroni-Dunn correct two-sided post-hoc tests, the effect size r, the sample sizes n, mean values M and standard deviations SD.

 10.1371/journal.pone.0347757.g004Judgments whether the content of the actual response and the GPT-generated response are the same.

The actual and impersonated responses are shown side-by-side. The stacked bar chart reports the percentages of the ratings that we observed. The statistical markers reported are the p-value of a two-sided one-sample Wilcoxon signed rank tests for a difference from zero, the effect size r, the sample sizes n, mean values M and standard deviations SD.

Thus, we conduct three statistical tests with the data of Track1, four with the data of Track2, three with the data of Track3 and seven statistical tests for the linguistic markers, i.e., a total of 17 tests. We use a conservative approach based on Bonferroni correction [27] to account for multiple tests and consider results as significant if the p-value of a test is less than α=0.0517≈0.003. Based on the large size of our populations with 520 question/response pairs and assuming that we observe differences of 0.5 points (i.e., half a step on the Likert scales), we compute the expected statistical power as β=1. Consequently, differences of 0.5 or larger in judgment should always be picked up by our tests and if there are no differences in judgments, there is only a 5% chance that we find a difference that is not there.

We report the arithmetic mean (M) and standard deviation (SD) as statistical markers for the populations. While our data is not perfectly normal, it also does not have severe outliers or multimodalities, so we prefer the clear interpretation of the arithmetic mean (M) and standard deviation (SD) to report statistical markers for populations. We use the biserial rank correlation r [28] to measure the effect size. Violin plots visualise the distribution of the data based on a kernel density estimation of the underlying probability distribution. The violins include miniature box plots that depict the median, upper and lower quartiles and the whiskers defined as the largest/smallest observed value at most 1.5 times the inter-quartile range away from the upper/lower quartile. Additionally, we use stacked bar charts to depict ratios of Likert scale items, where appropriate.

Supplemental material S2 Appendix provides details regarding the results, e.g., the demographic information of the participants. Supplemental material S3 Appendix reports on the results of additional quantitative analysis of possible confounding factors, i.e., randomisation of speakers assigned in Track3, the lengths of responses, and the influence of spelling errors. The results for all these factors are negative, i.e., it is highly unlikely that these aspects serve as alternative explanations of our findings.

The statistical analysis of the data is mostly implemented in Python. We use pandas 2.2.2 and numpy 1.26.4 for processing the data, pingouin 0.5.4 for the calculation of Cronbach’s α, effect sizes r and pair-wise tests, scipy 1.13.0 for the omnibus tests, and seaborn 0.13.2 for the generation of plots. We compute the statistical power with the R package mkpower 0.9.

 Qualitative analysis

The qualitative analysis in this study is two-fold: On the one hand, we code the free-text answers from the exit poll of the survey using inductive coding [29] and have one author assign one or more codes to each answer. The codes are aimed to capture the intent of the free-text answer, e.g., convey the reason for changes in the exit poll or observations regarding the impersonated content that the participants found striking (details on the codes are in supplemental material). This is initially done for twenty answers, at which point the coding is checked by and discussed with a second author, resulting in an agreed-upon coding taxonomy. The first author then continues to code the remainder of the data. Upon completion of this coding, the second author again checks all codes and discusses the coding to achieve agreement in the same manner as for the initial set of codes. We then conduct one round of axial coding [30] to group related codes into categories. Same as above, the axial coding is initially conducted by one author, then checked by and discussed with a second author to achieve agreement. We do not report inter-rater agreements for this data, because all data is fully checked by two authors and all disagreements are discussed and resolved, which results in final coding on which both annotators perfectly agree.

On the other hand, we conduct a qualitative analysis to better understand the human judgments regarding the differences in content between actual and impersonated responses, with a specific focus on whether the stance between actual and impersonated responses differs. For this, we randomly sample 50 pairs of actual and generated responses that were judged to be different in content by the study participants. We use deductive coding to determine if (a) the two responses have the same stance, i.e., arrive at the same conclusion, argue for the same points or take the same side, (b) express a different stance, i.e., draw different conclusions or argue for another position, or (c) whether this distinction is not applicable, e.g., because the question does not require the hearers to take a stance. Additionally, we determine for each response whether it addresses the question. Similar to the coding procedure above, the coding was first done by the one author and then checked by a second author. Disagreement between the two annotators was minimal and adjudicated to achieve agreement.

 Ethics statement

Our study used human subjects as participants in a survey. The ethics committee of the University of Passau approved this study (ref. III/Herbold.I-07.5095/240229). The study makes use of an online survey that obtained informed written consent prior to participation in the study.

 Results Impersonated responses are perceived as more authentic, coherent and relevant

The results clearly show that LLM-generated, impersonated content is judged as more authentic, coherent, and relevant than the actual debate responses. When the participants only see one question and its response (either actual or impersonated) (see Fig 1), we observe a significant difference across all three dimensions with a large effect size for authenticity (r = −0.55), relevance (r = −0.82) and coherence (r = −0.84) in the responses. When the participants directly compare an impersonated response with an original response along these dimensions (see Fig 2), the results are supported: The effect sizes for relevance (r = 0.79) and coherence (r = 0.87) remain large, but the difference in authenticity decreases to a weak effect (r = 0.28), so the gap between impersonated responses and actual responses is smaller in this setting (but it is still statistically significant). If the participants see the biographies of the speakers during rating (see Fig 3, top-left), we observe a similar effect size for the authenticity when comparing actual and generated responses than when the questions are shown side-by-side (r = 0.25). This means that the authenticity of the impersonated response is still higher than that of the actual response, even if biographies are provided.

An important control in our study is whether the debate content is just generally assumed to be authentic, instead of only when the response matches the common knowledge that the public has about the speaker. The data in Fig 3 shows that when deliberately assigning an actual response to a randomly picked (wrong) speaker, the authenticity is significantly lower compared to when the response is assigned to the actual speaker with the actual response (r = 0.39) or the actual speaker with the impersonated response (r = 0.59). When we take into account the confidence of the participants in their rating, we do not find any significant differences in the certainty of attributing original or impersonated responses to the actual speaker, or attributing an actual response to a random speaker. If we only consider a subgroup of data where the participants are highly familiar with the speakers, this again neither affects the authenticity nor the confidence: While the significance tests for differences between actual and impersonated responses, as well as actual speakers versus random speakers, are not significant anymore, the distributions are almost exactly the same as for the full dataset. This indicates that there is no shift in distributions, but rather the effects are too small to be detected with the smaller subgroups.

 Original content is different from impersonated content

If the authenticity is rated high, but the content of the original response differs from the content of the impersonated response, i.e., the impersonated content is not in line with the actual statements of the person, we are faced with a situation where GenAI can be employed for targeted misinformation about the speaker’s point of view. Our results show that a significant majority of actual responses are judged to be different in content from the impersonated counterparts, though the spread in the distribution is fairly large (see Fig 4). About half of the responses are considered to be dissimilar in comparison to only about one-third of responses that are considered similar. We observe no notable pattern or correlation between the similarity of the content and the authenticity of the responses (ρ=−0.16).

For the qualitative analysis of the differences, we sample 50 of the total 255 pairs of actual and generated responses for which the content is judged as being dissimilar. Table 1 summarizes the results of the analysis. Three categories emerge: The first category comprises cases where the differences in content are about how arguments are presented, but where both the actual and the generated response arrive at the same conclusion, while also addressing the question (n = 16 for responses with a stance, n = 5 without a stance, 42%). The second category comprises cases where only the actual speaker (n = 2, 4%) or only GPT (n = 14, 28%) responds to the question and the other source dodges the question. We do not annotate these instances as exhibiting a difference in stance – one side does not elaborate on the topic of the question and therefore does not communicate a stance towards it at all, so the identification of the difference in stance is not applicable. Overall, this means in 32% of cases, the question is dodged by one source. The third (and most important category) captures cases where the stance between the actual response and the generated response is notably different (n = 13, 26%). If we extrapolated this to the full dataset, we would expect that around 13% of generated responses communicate a different stance than the actual response.

10.1371/journal.pone.0347757.t001Results of qualitative analysis of the differences in content.
Content similarity Count
Same stance, both address question 16
Stance not required, only generated response addresses question 14
Different stance, both address question 13
Stance not required, both address question 5
Stance not required, only actual response addresses question 2
Human judgment is reliable

We measure the inter-rater reliability with Cronbach’s α [22] for the two judgments on authenticity, coherence, relevance, and content that we get for each question/response pair. The ratings are based on a five-point Likert scale. Additionally, we report the pair-wise differences between the two participants to understand which disagreements our participants have. We exclude confidence and familiarity because we cannot expect agreement regarding a subjective self-reflection. For the subsequent statistical analysis, we map the Likert scales to the integers [−2, −1, 0, 1, 2] and compute the average rating between the two judgments for the same data point. Since the measures from our survey are based on Likert scales, we use non-parametric rank-based statistical tests.

In all variables, we observe an overall modest agreement when measured with Cronbach’s α, with values of at most α=0.55. We analyse the data to understand which combinations of different judgments we observed and find that regarding authenticity (across all variants) there are relatively few polar differences, i.e., one participant rating an item as authentic and the other as not authentic. For relevance, coherence and content the differences are in how positive a judgment is, with small differences of a single point (e.g., ‘neutral’ instead of ‘agreement’), again showing that while the absolute ratings have some variance, the tendency regarding the judgment is typically the same for both participants. Overall, the tendency towards positive or negative judgments about a variable is fairly consistent, especially given our large sample size which is a representative cross-section of the British society.

We note that our method neither allows us to observe participants directly nor measures underlying aspects regarding their capabilities (e.g., their reading comprehension). In principle, if participants with different capabilities are not evenly assigned to tracks, this could lead to unobserved confounding effects. However, given our large sample size, a strong influence of such confounding effects is unlikely.

 Linguistic style is different

To provide a perspective different from the human evaluation, we augment our results with a comparison of the linguistic properties of the original and the generated responses (see Fig 5). This experiment yields a number of interesting insights: First, the complexity of the sentences in terms of the number of conjuncts, clausal modifiers, clausal complements, clausal subjects and parataxes as well as the use of modals such as ‘should’ and ‘must’ are not significantly different between the actual responses and the impersonated ones. Secondly, the actual responses contain more discourse markers (e.g., ‘because’, ‘therefore’) than the impersonated responses, even though with a small effect size (r = 0.24). The reason for the statistically significant difference is that there is a long tail of actual responses that contain many discourse markers, even though the peak of the distributions is the same for actual and impersonated responses.

Epistemic markers like ‘I think’ are used substantially more often in original responses – they are only rarely found in the impersonated statements, leading to a large effect size (r = 0.87). The rarity in generated responses is not surprising, giving that these markers indicate that the speaker has a stance on some issue, which we expect to be strongly the case in a broadcast political debate and much less so in the case of an LLM.

Furthermore, the impersonated responses contain more nominalisations (r = −0.88) and have a higher lexical diversity (r = −0.92), both with large effect sizes. The overlap between the words from the question and the response is higher for impersonated than actual responses, with a large effect size (r = −0.98). In fact, the distribution shows that it is not uncommon for all words from the question to appear in the impersonated responses, while this is only rarely true for the actual responses.

 Public opinion

Public opinion on the use of AI technology for public debates is collected in two rounds in the exit poll: First, the participants respond without prior knowledge of the data source they just rated (actual versus generated). In the second step, the source of the data is revealed and the participants are asked the same questions on the use of AI technology again.

The results of our exit poll prior to revealing the use of AI (see Fig 6) paint a clear picture: The participants mostly state that they are familiar with AI. Interestingly, while they mostly believe that AI cannot provide valuable contributions to public debates, they simultaneously claim that they support the use of AI if it is made explicit and if it is known how the system was developed. When it comes to the matter of regulating AI, the participants’ opinions are rather mixed, with roughly equal-sized groups favoring regulation, opposing regulation, and being undecided. After revealing the use of AI, over 90% of the participants do not change their opinion. For those who do change their opinion, we see a clear trend: The participants realize they are less familiar with AI than they thought, but also have a more favorable opinion on the use of AI in debates, while at the same time seeing a bigger need for regulation.

10.1371/journal.pone.0347757.g005Linguistic surface of actual debate responses versus impersonated debate responses.

Violins show a kernel density estimation of the probability distribution, the miniature box-plots depict the median, upper and lower quartiles, and the whiskers the largest/smallest value observed within 1.5 times the interquartile range of the upper/lower quartile. The statistical markers reported are the p-value of two-sided Wilcoxon signed rank tests, the effect size r, the sample sizes n, mean values M and standard deviations SD.

 10.1371/journal.pone.0347757.g006Results of the exit poll on the opinion of the participants.

The stacked bar chart reports the percentages of the ratings that we observed. The statistical markers reported are the sample sizes n, mean values M and standard deviations SD. The bar chart depicts the counts for each topic that was addressed in the free-text answers.

The optional free-text answers (n = 248) further corroborate these results. Many participants explicitly note that they did not change their results (n = 107). However, the other free-text answers indicate that the changes in opinions are caused by the confrontation with the capabilities of AI through the survey. Participants often mention that the impersonated responses are better than the human responses (n = 66) or that the quality of the impersonated responses is higher (n = 32). A few participants noted that the high coherence in the impersonated responses made them sceptical, leading them to believe that AI was possibly used in the survey (n = 7) and that this advantage over the humans can be explained by the live debate setting where the actual panellists do not have time to carefully prepare their responses (n = 5). One participant even notes that this advantage of AI means that AI could be used to train humans for debates. Some participants note that they are not able to distinguish between AI and humans at all (n = 26). There are also a few comments noting negative aspects regarding the quality of the generated responses (n = 4) or that AI was worse than the humans (n = 1), but these are outliers.

Another aspect that is stressed in the comments is the requirement to regulate the use of AI (n = 62), especially with respect to transparency. Particularly, many participants (n = 39) express concerns regarding the potential for deceptive use of AI in debates and the associated risks, some even note feelings of fear, shock, and worry (n = 17). However, some participants express positive emotions like surprise and amazement given the strong capabilities of AI (n = 16). When it comes to the use of AI technology in debates, some participants argue that AI’s convincing performance indicates potential for its use in debates (n = 36), while others question the general concept of AI debaters (n = 23). For instance, participants are uncertain about AI’s ability to represent party opinions and are concerned that involvement of AI will undermine the value of debates as forums for meaningful discussion between people.

 Discussion and conclusion

Our results demonstrate that modern AI based on LLMs is able to provide high-quality impersonated debate content that is perceived as authentic when attributed to actual people. We also find indications that people in general rate the impersonated content to be slightly more authentic than the actual human debate responses. In addition, the impersonated responses are judged as more coherent and relevant than actual responses. While the lower coherence can be attributed to the panellists being under scrutiny in a nationally broadcast political debate programme, the higher relevance of the LLM-generated responses indicates that the LLM tends to stay more on-topic than human speakers. Our analysis of responses judged as different in content supports this, as we find that the panel members do not address the question more often than the LLM. While we rule out length and grammatical errors as possible sources for differences in authenticity, we cannot rule out that there are unobserved factors introduced by our experiment design. Interestingly, the authenticity is not negatively affected by the notable differences in the linguistic surface of the responses. GPT clearly has its own unique style defined by a diverse vocabulary and an avoidance of epistemic markers, which is, however, not noticed by our study participants. There does not seem to be a problem with an uncanny valley [31], which would make the participants feel uncomfortable with the impersonated responses.

Even though most of our participants stated that they are familiar with AI, they did not expect AI to have generated these responses and therefore underestimated the capabilities of modern generative AI. Being confronted with the AI’s ability to generate convincing debate contributions elicited different reactions from the participants including evidence-driven discussions of the merits of AI, negative emotional responses driven by concerns over the potential for misuse, and positive emotional responses to technological progress demonstrated by the AI’s capabilities. Overall, the participants’ encounter with AI via our survey seemed to increase their appreciation for generative technology in some ways, while also highlighting the need for regulation of such tools when used in a debate context.

When questioned about the merits of AI, the participants expressed a strong belief that AI can be a valuable tool, but they have heterogeneous views on the need for regulation and restrictions on use. However, on the matters of transparency, the public perspective is clear: Over 85% of participants think that AI use has to be made explicit and that information on how the AI was developed needs to be shared.

In general, the risks that are implied by our findings are severe. While it has been previously established that LLMs are capable of generating persuasive misinformation [32] and that the automated and human detection of such misinformation is unreliable [33], the results of our study add another layer of concern: We demonstrate that LLMs can generate authentic information by impersonating specific people, meaning that LLM-powered misinformation campaigns can go beyond targeting general topics and societal group, but can be made to target individual people by imitating their contributions to public discourse. Furthermore, the potential for AI to generate responses that do not only successfully imitate a politician, but also push a specific political agenda, warrants a larger-scale further exploration and an assessment of the associated risks. We note that our setting did not study this targeted misinformation, i.e., we did not prescribe the position the LLM should express. Future work needs to study if LLMs are still perceived as authentic when used in such a targeted manner. Since the dissemination of excerpts from political statements via social networks is a common form of political communication [34], it is easy to spread such generated statements at scale. Therefore, content moderation to identify and remove undisclosed AI-generated statements will be crucial [35]. Our own results suggest that a current model [36] can be used for such content moderation (accuracy of 89% on the task of classifying responses into impersonated or actual). However, more sophisticated approaches may be able to fool such detectors [37].

Overall, the implications of our results for the communication of political content are devastating: Threat actors can easily use LLMs to pollute public information spheres with fake, but authentic-sounding, political statements, for instance, to sow confusion about the original statement and to invent talking points. If this is further combined with deep fakes that are already known to be able to generate reliable authentic voices and videos of public people [38], the risk for society is enormous.

 Supporting information Additional details for the survey.

(PDF)

 Additional details for the results.

(PDF)

 Additional analysis of alternative explanations for results.

(PDF)

 References OpenAI. GPT-4 Technical Report; 2023. arXiv:2303.08774. Anthropic AI. The Claude 3 Model Family: Opus, Sonnet, Haiku. Technical Report; 2024. Team G, Anil R, Borgeaud S, Wu Y, Alayrac JB, Yu J, et al. Gemini: a family of highly capable multimodal models. arXiv:231211805 [Preprint]. 2023. Ziegler A, Kalliamvakou E, Li XA, Rice A, Rifkin D, Simister S, et al. Measuring GitHub Copilot’s impact on productivity. Commun ACM. 2024;67(3):54–63. doi: 10.1145/3633453 Herbold S, Hautli-Janisz A, Heuer U, Kikteva Z, Trautsch A. A large-scale comparison of human-written versus ChatGPT-generated essays. Sci Rep. 2023;13(1):18617. doi: 10.1038/s41598-023-45644-9 37903836 Katz DM, Bommarito MJ, Gao S, Arredondo P. GPT-4 passes the bar exam. Philos Trans A Math Phys Eng Sci. 2024;382(2270):20230254. doi: 10.1098/rsta.2023.0254 38403056 Shanahan M, McDonell K, Reynolds L. Role play with large language models. Nature. 2023;623(7987):493–8. doi: 10.1038/s41586-023-06647-8 37938776 Bhandarkar A, Wilson R, Swarup A, Woodard D. Emulating author style: a feasibility study of prompt-enabled text stylization with off-the-shelf LLMs. In: Deshpande A, Hwang E, Murahari V, Park JS, Yang D, Sabharwal A, et al., editors. Proceedings of the 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024). St. Julians, Malta: Association for Computational Linguistics; 2024. p. 76–82. Available from: https://aclanthology.org/2024.personalize-1.6 Simmons G. Moral mimicry: large language models produce moral rationalizations tailored to political identity. arXiv:220912106 [Preprint]. 2022. Hackenburg K, Ibrahim L, Tappin BM, Tsakiris M. Comparing the persuasiveness of role-playing large language models and human experts on polarized US political issues. OSF Preprints. 2023;10. Salewski L, Alaniz S, Rio-Torto I, Schulz E, Akata Z. In-context impersonation reveals Large Language Models’ strengths and biases. Adv Neural Inf Process Syst. 2024;36. Bai H, Voelkel J, Eichstaedt J, Willer R. Artificial intelligence can persuade humans on political issues. Preprint. 2023. Available from: http://dx.doi.org/10.21203/rs.3.rs-3238396/v1 Santurkar S, Durmus E, Ladhak F, Lee C, Liang P, Hashimoto T. Whose opinions do language models reflect? 2023. Available from: https://arxiv.org/abs/2303.17548. arXiv:2303.17548. Matz SC, Teeny JD, Vaid SS, Peters H, Harari GM, Cerf M. The potential of generative AI for personalized persuasion at scale. Sci Rep. 2024;14(1):4692. doi: 10.1038/s41598-024-53755-0 38409168 Simchon A, Edwards M, Lewandowsky S. The persuasive effects of political microtargeting in the age of generative artificial intelligence. PNAS Nexus. 2024;3(2):pgae035. doi: 10.1093/pnasnexus/pgae035 38328785 Hautli-Janisz A, Kikteva Z, Siskou W, Gorska K, Becker R, Reed C. QT30: a corpus of argument and conflict in broadcast debate. In: Calzolari N, Béchet F, Blache P, Choukri K, Cieri C, Declerck T, et al., editors. Proceedings of the Thirteenth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association; 2022. p. 3291–300. Available from: https://aclanthology.org/2022.lrec-1.352 Zellers R, Holtzman A, Bisk Y, Farhadi A, Choi Y. HellaSwag: can a machine really finish your sentence? In: Korhonen A, Traum D, Màrquez L, editors. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics; 2019. p. 4791–800. Available from: https://aclanthology.org/P19-1472 https://doi.org/10.18653/v1/P19-1472 Weiss Z, Riemenschneider A, Schröter P, Meurers D. Computationally modeling the impact of task-appropriate language complexity and accuracy on human grading of German essays. Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications. Florence, Italy: Association for Computational Linguistics; 2019. p. 30–45. Available from: https://aclanthology.org/W19-4404 https://doi.org/10.18653/v1/W19-4404 Siskou W, Friedrich L, Eckhard S, Espinoza I, Hautli-Janisz A. Measuring plain language in public service encounters. Proceedings of the 2nd Workshop on Computational Linguistics for Political Text Analysis (CPSS-2022). Potsdam, Germany; 2022. Lenk U. Discourse markers and global coherence in conversation. J Pragmat. 1998;30(2):245–57. Available from: https://www.sciencedirect.com/science/article/pii/S0378216698000277 https://doi.org/10.1016/S0378-2166(98)00027-7 McCarthy PM, Jarvis S. MTLD, vocd-D, and HD-D: a validation study of sophisticated approaches to lexical diversity assessment. Behav Res Methods. 2010;42(2):381–92. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. doi: 10.1007/bf02310555 Wilcoxon F. Individual comparisons by ranking methods. Biom Bull. 1945;1(6):80–3. Friedman M. A Correction: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc. 1939;34(205):109. doi: 10.2307/2279169 Kruskal WH, Wallis WA. Errata: use of ranks in one-criterion variance analysis. J Am Stat Assoc. 1953;48(264):907. doi: 10.2307/2281082 Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Statist. 1947;18(1):50–60. doi: 10.1214/aoms/1177730491 Bonferroni C. Teoria statistica delle classi e calcolo delle probabilita. Pubbl R Ist Sup Sci Econ Com Firenze. 1936;8:3–62. Cureton EE. Rank-biserial correlation. Psychometrika. 1956;21(3):287–90. doi: 10.1007/BF02289138 Thomas DR. A General inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006;27(2):237–46. doi: 10.1177/1098214005283748 Corbin J, Strauss A. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage publications; 2014. Mori M. Bukimi no tani [The uncanny valley]. Energy. 1970;7:33. Zhou J, Zhang Y, Luo Q, Parker AG, De Choudhury M. Synthetic lies: understanding AI-generated misinformation and evaluating algorithmic and human solutions. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23. New York, NY, USA: Association for Computing Machinery; 2023. p. 1–20. Available from: doi: 10.1145/3544548.3581318 Jiang B, Tan Z, Nirmal A, Liu H. Disinformation detection: an evolving challenge in the age of LLMs. Proceedings of the 2024 SIAM International Conference on Data Mining (SDM). SIAM Publications Library; 2024. p. 427–35. Available from: https://epubs.siam.org/doi/abs/10.1137/1.9781611978032.50. arXiv: https://epubs.siam.org/doi/pdf/10.1137/1.9781611978032.50 Osatuyi B. Information sharing on social media sites. Comput Hum Behav. 2013;29(6):2622–31. doi: 10.1016/j.chb.2013.07.001 Clark CJ, Jussim L, Frey K, Stevens ST, Al-Gharbi M, Aquino K, et al. Prosocial motives underlie scientific censorship by scientists: a perspective and research agenda. Proc Natl Acad Sci U S A. 2023;120(48):e2301642120. doi: 10.1073/pnas.2301642120 37983511 Guo B, Zhang X, Wang Z, Jiang M, Nie J, Ding Y, et al. How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection; 2023. Available from: https://arxiv.org/abs/2301.07597. arXiv:2301.07597. Sun Y, He J, Cui L, Lei S, Lu CT. Exploring the deceptive power of LLM-generated fake news: a study of real-world detection challenges; 2024. Available from: https://arxiv.org/abs/2403.18249. arXiv:2403.18249. Farid H. Creating, using, misusing, and detecting deep fakes. J Online Trust Saf. 2022;1(4). 10.1371/journal.pone.0347757.r001 Decision Letter 0 Carrasco-Farré Carlos Academic Editor 2026 Carlos Carrasco-FarréCarlos Carrasco-Farré This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version 0

19 Oct 2025

-->PONE-D-25-15257-->-->LLM-impersonated debate contributions are more authentic, relevant and coherent than their original: A representative study using BBC1’s Question Time-->-->PLOS ONE

Dear Dr. Herbold,

Thank you for submitting your manuscript to PLOS ONE   . I am pleased to inform you that, following peer review, the reviewers and I found your study to be original, clearly written, and addressing a timely and important topic. Both reviewers recognized the relevance and ambition of your work and appreciated its methodological clarity and potential contribution.. I am pleased to inform you that, following peer review, the reviewers and I found your study to be original, clearly written, and addressing a timely and important topic. Both reviewers recognized the relevance and ambition of your work and appreciated its methodological clarity and potential contribution.

==============================

However, they also identified several issues that need to be addressed before the manuscript can be considered for publication. Based on their feedback, I am inviting you to submit a major revision   . Below, I summarize the main points raised by the reviewers and provide some guidance on how to proceed.. Below, I summarize the main points raised by the reviewers and provide some guidance on how to proceed.

<h3 data-end="1014" data-start="953">1. Contextual and interpretive alignment (Reviewer 1)   </h3></h3>

Reviewer 1 commends the design and analysis but raises concerns about the contextual asymmetry between the original human responses and the LLM-generated ones. Since the human content was transcribed from a live television programme (“Question Time”), it may include disfluencies, references, or transcription artifacts that affect perceptions differently than the directly generated LLM text. The reviewer suggests adding a post-hoc robustness check or qualitative analysis to demonstrate that these contextual or transcription-related differences do not drive the main effects.

Relatedly, Reviewer 1 encourages you to expand the discussion of content differences between human and impersonated responses. Specifically, the paper would benefit from showing how    these differ substantively; for example, by analyzing whether impersonations diverge in stance or argumentation, rather than merely in surface-level content. This would clarify whether the framing of “misrepresentation” is warranted.these differ substantively; for example, by analyzing whether impersonations diverge in stance or argumentation, rather than merely in surface-level content. This would clarify whether the framing of “misrepresentation” is warranted.

Finally, the reviewer notes that the reliability of human ratings is modest and recommends greater transparency about inter-rater variability, or, if feasible, additional averaging or collection of ratings to reduce subjective noise.

Minor notes include adding a brief explanation that Question Time    is a television programme, clarifying “random speaker” in the methods, and checking for potential answer length differences that could influence results.is a television programme, clarifying “random speaker” in the methods, and checking for potential answer length differences that could influence results.

<h3 data-end="2561" data-start="2496">2. Conceptual and methodological refinements (Reviewer 2)   </h3></h3>

Reviewer 2 finds the study compelling but identifies several areas where methodological clarification is needed:

Construct clarity:    The meaning of The meaning of authenticity    should be defined more precisely, whether it captures authorship likelihood, realism, or plausibility. Reviewers recommend clarifying how this construct differs from related ones (e.g., coherence or content similarity) and ensuring that participants understood it consistently.should be defined more precisely, whether it captures authorship likelihood, realism, or plausibility. Reviewers recommend clarifying how this construct differs from related ones (e.g., coherence or content similarity) and ensuring that participants understood it consistently.

Experimental design:    Provide more detail on the randomization and balancing of stimuli across conditions. If possible, include or reference any inter-rater checks used during the manual screening of generated responses.Provide more detail on the randomization and balancing of stimuli across conditions. If possible, include or reference any inter-rater checks used during the manual screening of generated responses.

Measurement approach:    Consider complementing subjective Likert judgments with an objective or behavioral indicator (e.g., response times or accuracy), or at least acknowledge the limitations of relying solely on self-reported data.Consider complementing subjective Likert judgments with an objective or behavioral indicator (e.g., response times or accuracy), or at least acknowledge the limitations of relying solely on self-reported data.

Statistical analysis:    The reviewer suggests using non-parametric or mixed-effects models better suited to ordinal and nested data, reporting medians and interquartile ranges, and revising effect size measures accordingly. You might also re-evaluate inter-rater reliability using intraclass correlation coefficients.The reviewer suggests using non-parametric or mixed-effects models better suited to ordinal and nested data, reporting medians and interquartile ranges, and revising effect size measures accordingly. You might also re-evaluate inter-rater reliability using intraclass correlation coefficients.

Interpretation:    Finally, temper the strength of the claims about deception or indistinguishability between AI and human responses. Reviewer 2 encourages reframing your findings as perceptual rather than deceptive effects, potentially driven by linguistic fluency or stylistic uniformity.Finally, temper the strength of the claims about deception or indistinguishability between AI and human responses. Reviewer 2 encourages reframing your findings as perceptual rather than deceptive effects, potentially driven by linguistic fluency or stylistic uniformity.

Both reviewers found your study promising and well written. Their feedback is intended to help you enhance the rigor, clarity, and interpretive precision of your work. I encourage you to take this as a positive step toward publication; since the paper already has a strong foundation and, with the recommended revisions, could make a meaningful contribution to ongoing debates on authenticity, impersonation, and LLM-mediated communication.

==============================

Please submit your revised manuscript by Dec 03 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

-->

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols..

We look forward to receiving your revised manuscript.

Kind regards,

Carlos Carrasco-Farré

Academic Editor

PLOS ONE

Journal Requirements:

-->1. When submitting your revision, we need you to address these additional requirements.-->--> -->-->Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at -->-->https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and -->-->https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf-->--> -->-->2. Please update your submission to use the PLOS LaTeX template. The template and more information on our requirements for LaTeX submissions can be found at http://journals.plos.org/plosone/s/latex.-->--> -->-->3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. -->--> -->-->When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.-->--> -->-->4. Thank you for stating the following financial disclosure: -->-->A.H. work was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’-->-->URL: https://www.volkswagenstiftung.de/-->-->The funders had no role in the study whatsoever.  -->--> -->-->Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." -->-->If this statement is not correct you must amend it as needed. -->-->Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.-->--> -->-->5. Thank you for stating the following in the Acknowledgments Section of your manuscript: -->-->The work reported on in this paper was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’-->--> -->-->We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. -->-->Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: -->-->A.H. work was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’-->-->URL: https://www.volkswagenstiftung.de/-->-->The funders had no role in the study whatsoever. -->--> -->-->Please include your amended statements within your cover letter; we will change the online submission form on your behalf.-->--> -->-->6. Thank you for uploading your study's underlying data set. Unfortunately, the repository you have noted in your Data Availability statement does not qualify as an acceptable data repository according to PLOS's standards.-->--> -->-->At this time, please upload the minimal data set necessary to replicate your study's findings to a stable, public repository (such as figshare or Dryad) and provide us with the relevant URLs, DOIs, or accession numbers that may be used to access these data. For a list of recommended repositories and additional information on PLOS standards for data deposition, please see https://journals.plos.org/plosone/s/recommended-repositories.-->--> -->-->7. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.-->--> -->-->8. If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions-->

-->Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.-->

Reviewer #1: Yes

Reviewer #2: Yes

**********

-->2. Has the statistical analysis been performed appropriately and rigorously?-->

Reviewer #1: Yes

Reviewer #2: Yes

**********

-->3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.-->

Reviewer #1: Yes

Reviewer #2: No

**********

-->4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.-->

Reviewer #1: Yes

Reviewer #2: Yes

**********

-->5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)-->

Reviewer #1:    I found this paper interesting to read and easy to follow. The experimental design and methods were well motivated and clearly explained. The statistical analysis is sound. The results were generally well presented.I found this paper interesting to read and easy to follow. The experimental design and methods were well motivated and clearly explained. The statistical analysis is sound. The results were generally well presented.

While my overall opinion on the paper leans positive, I have three main concerns, which I believe could at least be partly addressed in revisions:

1) QT is a television programme, where panellists give live answers to audience questions. This means that the real human responses that form the basis of this paper are transcribed from speech that occurred in a specific context. The LLM, on the other hand, generated text directly in a decontextualised setting, as provided by the system prompt (lines 84ff).

The authors acknowledge this discrepancy (e.g. line 443) but do not meaningfully engage with it. Fundamentally, human responses are not being evaluated in the setting and modality for which they were intended.

The paper would be stronger if it demonstrated that this discrepancy does not affect results. For example, human responses may include markers (such as references to other panellists names, disfluencies) that are perceived as inauthentic, incoherent, etc. when transcribed and decontextualised, but not when spoken on the programme. There may also be transcription errors, which LLMs would not have. This seems in scope for post-hoc analysis.

2) The paper finds that original content is different from impersonated content and concludes from this that AI may be used to generate targeted misinformation about the speaker’s point of view (lines 339ff). However, there is no analysis into *how* content differs in human vs LLM-impersonated content, and whether this difference does in fact correspond to “misrepresentation”. For this, it would be necessary to show that the LLM-impersonated answers differ from the human answers in their issue-specific stance.

It seems very plausible that LLMs follow different lines of argumentation but ultimately make similar points, which may (hypothetically) even be endorsed by the original speaker. Asking only about “difference in content”, which may be explained in many ways, cannot capture this. Consequently, the framing of the paper as relating to misinformation and deception seems only partly warranted.

3) The paper claims that human judgments are reliable but does not provide strong evidence for this. Agreement is modest at best (lines 359 following), and the supplementary variables show low agreement on several variables. Clearly, there is substantial noise in the data from subjective rating differences, which could be addressed by collecting and averaging across more ratings. If not, it would still be useful to be more open and specific about the (lack of) human agreement in the main body of the paper.

Minor notes:

- I found no mention of answer length in the paper. It seems very plausible that there is a difference in length between human and LLM answers, which may in turn affect at least some of the analyses. Some normalisation would likely be necessary?

- It may help the (non-UK) reader to note in the Data section that Question Time is a TV programme. The only mention of TV / television right now, I believe, is in the intro.

- On first reading, I was a bit confused by lines 117-119: “random speaker” may need brief explanation. Of course, this becomes clear later.

Reviewer #2:    The manuscript explores the capacity of large language models (LLMs) to impersonate public figures in political debates. The topic is timely and uses an interesting dataset. The paper is well written and methodologically ambitious. However, the strength of the conclusions is weakened by shortcomings in the experimental design and in the statistical analysis. The findings are interesting but should be interpreted with greater caution, given the limitations discussed below.The manuscript explores the capacity of large language models (LLMs) to impersonate public figures in political debates. The topic is timely and uses an interesting dataset. The paper is well written and methodologically ambitious. However, the strength of the conclusions is weakened by shortcomings in the experimental design and in the statistical analysis. The findings are interesting but should be interpreted with greater caution, given the limitations discussed below.

1. Experimental Design

The overall three-track design is conceptually sound allowing the author to assess perceptions under different conditions. Nonetheless, the construct of authenticity is insufficiently defined. It remains unclear whether participants interpreted it as authorship likelihood, realism, or plausibility, which may conflate distinct psychological dimensions. Moreover, each participant evaluated only a few items, which limits the reliability of individual responses. Besides, more details about the randomization of questions and speakers is required to ensure data is balanced. The manual inspection of generated responses lacks formal coding criteria or inter-rater checks, reducing transparency and reproducibility.

Suggestions for improvement:

Clarify the exact wording and intended meaning of each measure, particularly “authenticity.” Provide evidence that the randomization of stimuli was balanced across conditions. Include or reference inter-rater checks for the manual screening of generated responses.

2. Measures and Data Collection

The study relies entirely on self-reported Likert-scale judgments. This approach is valid for perceptual studies, but complementary behavioral or comprehension-based measures (e.g., response time, detection accuracy) would have strengthened construct validity. Some measures—especially “content similarity”—may overlap conceptually with “authenticity,” potentially introducing redundancy.

Suggestions for improvement:

Consider clarifying the independence of constructs and, if possible, provide an additional objective indicator of perceptual discrimination between real and impersonated responses. Provide information about possible moderator factors.

3. Statistical Analysis

The paper presents a detailed description of the statistical workflow and justifies the use of non-parametric tests. However, several inconsistencies remain. The analysis averages ordinal Likert data and reports means, standard deviations, and Cohen’s d, assuming interval properties. This is not strictly correct for ranked data and can inflate effect sizes. Inter-rater reliability (reported Cronbach’s α) indicates moderate agreement at best, suggesting considerable noise in the judgments. The treatment of each outcome variable (authenticity, coherence, relevance, content) as independent is also problematic, as they are conceptually and statistically correlated. Finally, while some corrections are applied, the overall analytical structure would benefit from mixed-effects or ordinal regression models that account for the nested and repeated nature of the data.

Suggestions for improvement:

Report medians and interquartile ranges alongside or instead of means. Use effect size measures suited for ordinal data (e.g., rank-biserial correlation or other relevant measure.) Reassess reliability using intraclass correlation or a mixed-model framework. Consider a multivariate or mixed-effects approach to handle correlated measures and participant/item dependencies.

4. Interpretation of Results

The finding that impersonated responses are perceived as more authentic, coherent, and relevant than the originals is statistically supported but likely overstated. Given the modest reliability and limited item sampling, the conclusion that participants “cannot discern” AI-generated content should be presented more cautiously. It is plausible that judgments reflect linguistic fluency rather than genuine attribution of authorship.

Suggestions for improvement:

Moderate the interpretive claims and emphasize the perceptual, rather than deceptive, nature of the results. Discuss alternative explanations such as stylistic consistency or linguistic polish in LLM outputs.

**********

-->6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review?    For information about this choice, including consent withdrawal, please see our For information about this choice, including consent withdrawal, please see our Privacy Policy..-->

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.. Please note that Supporting Information files do not need this step.

 10.1371/journal.pone.0347757.r002 Author response to Decision Letter 1 Submission Version 1

25 Nov 2025

Response to Reviews

We thank the two reviewers, as well as the editor, for their feedback. Based on the comments, we prepared a revision of our manuscript for your consideration. To aid the reviewers, all changes are marked in blue within the updated manuscript.

Editor

Comment E.1: Reviewer 1 comments the design and analysis but raises concerns about the contextual asymmetry between the original human responses and the LLM-generated ones. Since the human content was transcribed from a live television programme (“Question Time”), it may include disfluencies, references, or transcription artifacts that affect perceptions differently than the directly generated LLM text. The reviewer suggests adding a post-hoc robustness check or qualitative analysis to demonstrate that these contextual or transcription-related differences do not drive the main effects.

Response E.1: We address this issue by conducting additional checks such as automated spell checking and a manual analysis of the responses to rule out such effects. The additional results are reported in the new supplemental material S3.

For more details, we refer to Response R1.1.

Comment E.2: Relatedly, Reviewer 1 encourages you to expand the discussion of content differences between human and impersonated responses. Specifically, the paper would benefit from showing how these differ substantively; for example, by analyzing whether impersonations diverge in stance or argumentation, rather than merely in surface-level content. This would clarify whether the framing of “misrepresentation” is warranted.

Response E.2: Based on these understandable concerns, we conduct additional manual analysis, in which we compare the stance conveyed by the actual and the generated response. While there are indeed cases in which the differences are rather in how a stance is conveyed, we also find a non-negligible number (26%) of the samples that we checked, where the differences in content were indeed differences in the stance that was conveyed.

For more details, we refer to Response 1.2.

Comment E.3: Finally, the reviewer notes that the reliability of human ratings is modest and recommends greater transparency about inter-rater variability, or, if feasible, additional averaging or collection of ratings to reduce subjective noise.

Response E.3: While we acknowledge this concern, we want to highlight that the differences in the judgments are neither random (i.e., completely unreliable) nor polar (i.e., opposite judgments), but rather on neighboring items on the Likert scale. We discuss these concerns in detail in the “Human judgment is reliable” section, which dedicates a whole paragraph to this issue. While we acknowledge that additional data collection could possibly help, we do not believe that there will be substantial changes, given our already very large sample size of 520 question/response pairs.

For more details, we refer to Response 1.3.

Comment E.4: Minor notes include adding a brief explanation that Question Time is a television programme, clarifying “random speaker” in the methods, and checking for potential answer length differences that could influence results.

Response E.4: We now address all these concerns explicitly in the paper, including an additional analysis on the random speaker assignments and the influence of answer lengths.

For more details, we refer to Responses 1.4, 1.5, and 1.6.

Comment E.5: 2. Conceptual and methodological refinements (Reviewer 2)

Reviewer 2 finds the study compelling but identifies several areas where methodological clarification is needed:

Construct clarity: The meaning of authenticity should be defined more precisely, whether it captures authorship likelihood, realism, or plausibility. Reviewers recommend clarifying how this construct differs from related ones (e.g., coherence or content similarity) and ensuring that participants understood it consistently.

Response E.5: We clarify our definition for authenticity as requested.

For more details, refer to Response 2.1.

Comment E.6: Experimental design: Provide more detail on the randomization and balancing of stimuli across conditions. If possible, include or reference any inter-rater checks used during the manual screening of generated responses.

Response E.6: We acknowledge that we should have been clearer regarding these issues. We almost always use the full data set and avoid sampling. The only case where we sample is the random assignment of speakers, as well as the new qualitative analysis of answers we already discussed above. We provide additional analysis for this aspect. We also failed to clarify why no inter-rater agreements are reported for our own manual analysis: all cases in which the second rater disagreed with the original judgment by the first rater were discussed and agreement was reached in all cases.

For more details, we refer to Responses 2.3 and 2.4.

Comment E.7: Measurement approach: Consider complementing subjective Likert judgments with an objective or behavioral indicator (e.g., response times or accuracy), or at least acknowledge the limitations of relying solely on self-reported data.

Response E.7: While this would yield insightful additional data, such data cannot be reliably obtained with the only survey method we used. While there is possibly an impact because of unobserved confounding effects, this is highly unlikely given our sample size. Still, we acknowledge this limitation in the “Human judgments are reliable” section.

For more details we refer to Response 2.5.

Comment E.8: Statistical analysis: The reviewer suggests using non-parametric or mixed-effects models better suited to ordinal and nested data, reporting medians and interquartile ranges, and revising effect size measures accordingly. You might also re-evaluate inter-rater reliability using intraclass correlation coefficients.

Response E.8: We carefully considered all suggestions by the reviewer and partially updated our methods. We revised the effect sizes to be non-parametric (almost no changes, one increase in in effect strength). All non-parametric markers are now reported next to the parametric mean and standard deviation in the appendix and we explain why we kept the mean and standard deviation in the main body of the paper. We did not switch to a different statistical model (i.e., no variant of a linear model), because we directly want to observe how the variables change and not how the combination of variables explains a change in dependent variable.

For more details, we refer to Response 2.6.

Comment E.9: Interpretation: Finally, temper the strength of the claims about deception or indistinguishability between AI and human responses. Reviewer 2 encourages reframing your findings as perceptual rather than deceptive effects, potentially driven by linguistic fluency or stylistic uniformity.

Response E.9: While our additional analysis further supports our results, we acknowledge that some wordings were too strong in the initial submission as our results are limited to the perception of our participants. We consequently edited the wording in key locations of the paper such as the abstract and the results section.

For more details, we refer to Response 2.7.

Journal Requirements:

Comment J.1: When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response J.1: We updated all file names and types to match the requirements.

Comment J.2: Please update your submission to use the PLOS LaTeX template. The template and more information on our requirements for LaTeX submissions can be found at http://journals.plos.org/plosone/s/latex.

Response J.2: We updated our submission to use the latest version of the LaTeX style file (Version 3.7 from Aug 2025)

Comment J.3: We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

Response J.3: Thank you for reporting this. We now clarify the matter in the cover letter.

Comment J.4: Thank you for stating the following financial disclosure:

A.H. work was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’

URL: https://www.volkswagenstiftung.de/

The funders had no role in the study whatsoever.

Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

Response J.4: The funders had no role in the study and we clarify this in the cover letter.

Comment J.5: Thank you for stating the following in the Acknowledgments Section of your manuscript:

The work reported on in this paper was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

A.H. work was partially funded by the VolkswagenStiftung under grant Az. 98544 ‘Deliberation Laboratory’

URL: https://www.volkswagenstiftung.de/

The funders had no role in the study whatsoever.

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

Response J.5: Thank you for this clarification. We provide the amended statements in our cover letter and drop the acknowledgements section from the manuscript.

Comment J.6: Thank you for uploading your study's underlying data set. Unfortunately, the repository you have noted in your Data Availability statement does not qualify as an acceptable data repository according to PLOS's standards.

At this time, please upload the minimal data set necessary to replicate your study's findings to a stable, public repository (such as figshare or Dryad) and provide us with the relevant URLs, DOIs, or accession numbers that may be used to access these data. For a list of recommended repositories and additional information on PLOS standards for data deposition, please see https://journals.plos.org/plosone/s/recommended-repositories.

Response J.6: We noticed that the DOI-reference to our long-term archive at Zenodo was missing the last character. This is now fixed and the new Data Availability statement is:

“All materials are available online in the form of a replication package that contains the data: https://doi.org/10.5281/zenodo.12698364”

Comment J.7: Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

Response J.7: We now have an additional subsection at the end of the Methods section with the full ethics statement. We extend the ethics statement to clarify that the obtained consent was written.

Comment J.8: If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise.

Response J.8: Thank you for this comment. There were no recommendations for additional references in the reviews.

Reviewer 1

Comment R1.1: QT is a television programme, where panellists give live answers to audience questions. This means that the real human responses that form the basis of this paper are transcribed from speech that occurred in a specific context. The LLM, on the other hand, generated text directly in a decontextualised setting, as provided by the system prompt (lines 84ff).

The authors acknowledge this discrepancy (e.g. line 443) but do not meaningfully engage with it. Fundamentally, human responses are not being evaluated in the setting and modality for which they were intended.

The paper would be stronger if it demonstrated that this discrepancy does not affect results. For example, human responses may include markers (such as references to other panellists names, disfluencies) that are perceived as inauthentic, incoherent, etc. when transcribed and decontextualised, but not when spoken on the programme. There may also be transcription errors, which LLMs would not have. This seems in scope for post-hoc analysis.

Response R1.1: This is a good point and would be a viable alternative explanation for our results. We add additional analysis regarding this (and other) alternative hypothesis to the supplemental material in Appendix S3. We decided to exclude this from the main body of the paper in order to keep the presentation crisp and only add a reference to this new supplemental material to the end of the Section “Statistical analysis”. In summary, errors have no correlation with judgments. While transcriptions have more errors (2.4 on average vs. 0.26 for generated responses), most errors are minor, e.g., missing commas between clauses of compound sentences. In 10% of the original responses, we manually checked if actual responses contain references to other panelists and we do not find such cases. We add this information in Section “Original Debate Content.”

Comment R1.2: The paper finds that original content is different from impersonated content and concludes from this that AI may be used to generate targeted misinformation about the speaker’s point of view (lines 339ff). However, there is no analysis into *how* content differs in human vs LLM-impersonated content, and whether this difference does in fact correspond to “misrepresentation”. For this, it would be necessary to show that the LLM-impersonated answers differ from the human answers in their issue-specific stance.

It seems very plausible that LLMs follow different lines of argumentation but ultimately make similar points, which may (hypothetically) even be endorsed by the original speaker. Asking only about “difference in content”, which may be explained in many ways, cannot capture this. Consequently, the framing of the paper as relating to misinformation and deception seems only partly warranted.

Response R1.2: Thank you for this comment. We acknowledge that our definition of the term “content” as “difference in overall meaning” is too broad to directly justify these statements. To mitigate this, we include additional qualitative analysis on the share of data that was judged as different in content, indicated by a negative mean value of the “content” judgments by the study participants. The goal of this qualitative analysis is to establish whether the differences in content are indeed about misrepresentation. For this, we focus on the stance, i.e., if the response arrives at the same conclusion, argues for the same points or takes the same stance. This focus excludes the use of rhetorical devices as other means to convey a point, which may be used to make misrepresentations authentic. Additionally, we check if the respons

Submitted filename: PLOS-One-QT-GPT-Response.pdf

 10.1371/journal.pone.0347757.r003 Decision Letter 1 Hassan Mohammad Salah Academic Editor 2026 Mohammad Salah HassanMohammad Salah Hassan This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version 1

23 Mar 2026

-->PONE-D-25-15257R1-->-->LLM-impersonated debate contributions are more authentic, relevant and coherent than their original: A representative study using BBC1’s Question Time-->-->PLOS One

Dear Dr. Herbold,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.-->--> -->-->

Please submit your revised manuscript by May 07 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:-->

A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

-->If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols..

We look forward to receiving your revised manuscript.

Kind regards,

Mohammad Salah Hassan, Ph.D

Academic Editor

PLOS One

Journal Requirements:

1. If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise.

2. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments :

Dear Authors,

The revised manuscript, PONE-D-25-15257R1, entitled “LLM-impersonated debate contributions are more authentic, relevant and coherent than their original: A representative study using BBC1’s Question Time,” has now been reviewed.

Please accept my sincere apologies for the delay in reaching this decision, as one of the invited reviewers did not reply and additional time was therefore needed to secure a further review.

Reviewer 1 recommends acceptance and confirms that the authors have adequately addressed the comments raised in the previous round. The reviewer considers the manuscript technically sound, the statistical analysis appropriate, the data availability satisfactory, and the language clear and intelligible.

A few very minor points remain, but these should be treated as final editorial clean-up rather than substantive revisions. In line with the reviewer’s observations, the authors should clarify the limitation concerning the mismatch in context and modality between the original human responses and the LLM-generated responses, and moderate the “threat” framing where the wording may still appear stronger than warranted. In addition, a few minor editorial inconsistencies should be corrected in the final files, including residual typographical and spacing issues, consistency of the manuscript title across all submitted documents, and the final Data Availability link/DOI.

These minor matters do not affect the overall positive recommendation. Based on the reviewer’s report and my own assessment, I recommend acceptance of the manuscript subject to final minor editorial polishing.

Kind regards,

Mohammed Salah Alazzawi, PhD

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

-->Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.-->

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

-->2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.-->

Reviewer #1: Yes

Reviewer #3: Yes

**********

-->3. Has the statistical analysis been performed appropriately and rigorously? -->

Reviewer #1: Yes

Reviewer #3: Yes

**********

-->4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.-->

Reviewer #1: Yes

Reviewer #3: Yes

**********

-->5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.-->

Reviewer #1: Yes

Reviewer #3: Yes

**********

-->6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)-->

Reviewer #1: I thank the authors for their detailed response to my first review. My concerns have largely been addressed, and I am happy to recommend acceptance. However, I would encourage the authors to engage with the following points:

1) I still believe that the mismatch in context and modality between the original human responses and the LLM-generated responses cannot be ruled out as at least a partial explanation of results. Length and prevalence of spelling errors may not individually have strong correlations with perceptions of authenticity etc., but their combination (and other unobserved factors) may still partially explain human judgments. A true like-for-like comparison would compare human-written textual responses to LLM-written responses. Since this comparison cannot be made, I think this should be clearly acknowledged as a limitation in the main body discussion.

2) I would encourage the authors to be clearer in their “threat model”: What exactly is the new threat created by LLM impersonation? And to what extent do the experiments in this paper measure this new threat? For example, I would argue that not all “made-up content” is equally problematic – LLM impersonation poses a threat only if the LLM-generated content *misrepresents* the speaker’s views *while still being perceived as authentic*. This is not something the paper tests for! The small qualitative analysis shows that there are 13 of 50 cases where the LLM-generated response misaligns in stance, but this is not enough to make claims about general LLM tendency to misrepresent, or how this misrepresentation impacts authenticity judgments. Personally, I would therefore tone down the “threat” language throughout the paper and simply provide a sober description of results related to authenticity, and pointers to open questions for future work.

Minor notes:

- line 18: typo and missing whitespace

- line 184: missing whitespace

Reviewer #3: This paper has been significantly improved. Thank you very much for your efforts in making the manuscript stronger and more polished after the revision.

**********

-->7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review?    For information about this choice, including consent withdrawal, please see our For information about this choice, including consent withdrawal, please see our Privacy Policy..-->

Reviewer #1: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

To ensure your figures meet our technical requirements, please review our figure guidelines: https://journals.plos.org/plosone/s/figures

You may also use PLOS’s free figure tool, NAAS, to help you prepare publication quality figures: https://journals.plos.org/plosone/s/figures#loc-tools-for-figure-preparation.

NAAS will assess whether your figures meet our technical requirements by comparing each figure against our figure specifications.

-->

 10.1371/journal.pone.0347757.r004 Author response to Decision Letter 2 Submission Version 2

25 Mar 2026

We fixed all reported minor issues. For the two comments from R1, we added the following to the main body of the manuscript in the Section "Discussion and conclusion" (1) to highlight the limitation due to possible unobservable factors affecting our results and (2) to frame the generalizability more carefully:

(1) "While we rule out length and grammatical errors as possible sources for differences in authenticity, we cannot rule out that there are unobserved factors introduced by our experiment design."

(2) "We note that our setting did not study this targeted misinformation, i.e., we did not prescribe the position the LLM should express. Future work needs to study if LLMs are still perceived as authentic when used in such a targeted manner."

 10.1371/journal.pone.0347757.r005 Decision Letter 2 Hassan Mohammad Salah Academic Editor 2026 Mohammad Salah HassanMohammad Salah Hassan This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version 2

7 Apr 2026

LLM-impersonated debate contributions are more authentic, relevant and coherent than their original: A representative study using BBC1’s Question Time

PONE-D-25-15257R2

Dear Dr. Authors,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact  and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact billing support..

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Mohammad Salah Hassan, Ph.D

Academic Editor

PLOS One

Additional Editor Comments (optional):

Dear Authors,

Thank you for submitting the revised version of your manuscript, “LLM-impersonated debate contributions are more authentic, relevant and coherent than their original: A representative study using BBC1’s Question Time.”

The manuscript has improved substantially, and the concerns raised in the previous round have been adequately addressed.

I am pleased to inform you that I recommend acceptance of the manuscript for publication in PLOS ONE.

Kind regards,

Reviewers' comments:

 10.1371/journal.pone.0347757.r006 Acceptance letter Hassan Mohammad Salah Academic Editor 2026 Mohammad Salah HassanMohammad Salah Hassan This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited., which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

PONE-D-25-15257R2

PLOS One

Dear Dr. Herbold,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS One. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

You will receive further instructions from the production team, including instructions on how to review your proof when it is ready. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few days to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

You will receive an invoice from PLOS for your publication fee after your manuscript has reached the completed accept phase. If you receive an email requesting payment before acceptance or for any other service, this may be a phishing scheme. Learn how to identify phishing emails and protect your accounts at https://explore.plos.org/phishing.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Mohammad Salah Hassan

Academic Editor

PLOS One