PLoS Comput Biol plos ploscomp PLOS Computational Biology 1553-734X 1553-7358 Public Library of Science San Francisco, CA USA PCOMPBIOL-D-24-00534 10.1371/journal.pcbi.1012037 Research Article Engineering and technologyMechanical engineeringRoboticsRobots Physical sciencesPhysicsClassical mechanicsMotionTorque Biology and life sciencesAnatomyBody limbsArmsWrist Medicine and health sciencesAnatomyBody limbsArmsWrist Biology and life sciencesNeuroscienceCognitive scienceCognitive psychologyPerceptionSensory perception Biology and life sciencesPsychologyCognitive psychologyPerceptionSensory perception Social sciencesPsychologyCognitive psychologyPerceptionSensory perception Biology and life sciencesNeuroscienceSensory perception Biology and life sciencesAnatomyNervous systemCentral nervous system Medicine and health sciencesAnatomyNervous systemCentral nervous system Engineering and technologyMechanical engineeringRobotics Research and analysis methodsSimulation and modeling Computer and information sciencesComputer visionTarget detection During haptic communication, the central nervous system compensates distinctly for delay and noise Delay and noise are distinctly compensated for during haptic communication https://orcid.org/0000-0003-0733-265X Eden Jonathan Conceptualization Formal analysis Methodology Software Visualization Writing – original draft Writing – review & editing 1 2 * Ivanova Ekaterina Conceptualization Formal analysis Investigation Methodology Visualization Writing – original draft Writing – review & editing 1 3 Burdet Etienne Conceptualization Funding acquisition Visualization Writing – original draft Writing – review & editing 1 Department of Bioengineering, Imperial College of Science, Technology and Medicine, London, United Kingdom Department of Mechanical Engineering, the University of Melbourne, Victoria, Australia School of Electronic Engineering and Computer Science, Queen Mary University of London, United Kingdom Haith Adrian M Editor Johns Hopkins University, UNITED STATES OF AMERICA

The authors have declared that no competing interests exist.

 * E-mail: eden.j@unimelb.edu.au 11 2024 6 11 2024 20 11 e1012037 28 3 2024 18 10 2024 2024 Eden et al This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Physically connected humans have been shown to exploit the exchange of haptic forces and tactile information to improve their performance in joint action tasks. As human interactions are increasingly mediated through robots and networks it is important to understand the impact that network features such as lag and noise may have on human behaviour. In this paper, we investigated interaction with a human-like robot controller that provides similar haptic communication behaviour as human-human interaction and examined the influence and compensation mechanisms for delay and noise on haptic communication. The results of our experiments show that participants can perceive a difference between noise and delay, and make use of compensation mechanisms to preserve performance in both cases. However, while noise is compensated for by increasing co-contraction, delay compensation could not be explained by this strategy. Instead, computational modelling suggested that a distinct mechanism is used to compensate for the delay and yield an efficient haptic communication.

 Author summary

Increasingly humans are making use of networks and robots to coordinate haptic interactions through teleoperation. However, with networks there comes delays and noise that can change both the force that is transmitted and how we perceive that force. The haptic communication involved in joint actions, such as when moving a piano or performing a pair spin, has been shown to improve performance, but how does delay affect this behaviour? We tested how participants tracked a moving target with their right hand while connected to a human-like robotic partner, when perturbed by delay or noise. Through a comparison between noise and delay perturbation, in experimental performance and in simulation with a computational model, we found that participants could from small values of perturbation identify if the perturbation was from delay or noise and that they adopted different adaptation strategies in each case.

 http://dx.doi.org/10.13039/100010663 H2020 European Research Council ICT-871803 Burdet Etienne http://dx.doi.org/10.13039/501100000266 Engineering and Physical Sciences Research Council EP/R026092/1 Burdet Etienne This work was supported in part by the EU H2020 grant ICT-871803 CONBOTS and by the UK EPSRC EP/R026092/1 FAIRSPACE program (all to EB). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. PLOS Publication Stage vor-update-to-uncorrected-proof Publication Update 2024-11-18 Data Availability The experiment data and simulation code are available on Zenodo at link https://doi.org/10.5281/zenodo.10811993. Introduction

How do humans succeed in carrying out skilled motor tasks together, such as when moving a piano or when skaters perform a pair spin? The results of recent studies on joint tracking tasks [1–3] suggest that these collaborations are supported by the partners exchanging their motion plan via the haptic channel [4, 5]. Importantly, performance benefits arise between connected partners of different skills [1], where each partner must integrate information from their visual and haptic channels, and where there is evidence that connected partners can have better task learning than when in a solo configuration [6, 7]. The ability to coordinate incoming sensory information from different modalities and with different time lags has been proposed to be critical for the central nervous system (CNS) to make sense of interactions with the environment [8, 9]. Here, the response to temporal delays is also of practical importance due to the transmission delays present when partners are connected by robotic systems, e.g. teleoperation in space applications with one partner on Earth and the other on a space station [10], or during remote physical training [11]. However, despite the physiological importance of sensorimotor delays and its noted affect on some sensory modalities [12] and the perception of haptic interaction [13, 14], how it influences haptic communication is not yet known. Therefore, we designed an experiment to investigate how haptic communication is affected by temporal delays present over digital connection [15].

This paper examines what mechanism physically connected individuals use to collaborate despite delayed haptic feedback. One possibility is that the CNS ignores the delay (no compensation strategy, Fig 1A), in which case haptic communication (that can be modelled as in [4]) takes place as normal, such that performance should degrade with increasing delay. However, the CNS is known to inconspicuously fuse signals with different timing information [8], e.g., recognising the relationship between the visual signal of lightning and the delayed audio of thunder. This ability may extend to haptic communication, where sensor fusion occurs across partners [4]. Here, the evidence suggests that haptic communication arises due to the CNS identifying the interaction with the partner as task-relevant and optimally combining their own and the partner’s sensory information by considering the respective noise of each information source [4]. Corresponding to these results, the mismatch of the delayed haptic information from the partner may instead be compensated as if it was an additional noise on the information derived from the partner (compensation as noise strategy, Fig 1B). Alternatively, the CNS may be able to identify that the haptic information has been delayed and actively compensate for it, using either a temporal [16] or state based [17] mechanism, thus enabling the extraction of more specific information from this signal (compensation by delay prediction strategy, Fig 1C).

10.1371/journal.pcbi.1012037.g001 Possible neural mechanisms to deal with a delayed interaction with a partner.

A: Haptic communication with no compensation, where the CNS understands that the haptic feedback is related to the visual task and uses it (without adjusting for delay) to infer the partner’s motion plan that is then combined with their own motion plan. B: Compensation as noise mechanism in which the CNS, additionally to A, considers the delay as an additional noise to be filtered. C: Compensation by delay prediction mechanism in which the CNS, additionally to A, identifies the temporal delay and uses this knowledge to make a prediction (dashed line) of the partner’s motion using the delayed haptic feedback. This prediction may deviate from the true trajectory if the partner model does not match their behaviour in the time after the delayed feedback.

To study how temporal delay affects haptic communication and to test these three possible approaches, we investigated how participants (connected to a human-like robot controller [5]) tracked a target moving along a multi-sine function with their dominant arm’s wrist flexion/extension. The participants moved an individual handle of the Hi5 dual robotic interface [18] to track a target with a cursor on their own monitor (Fig 2A). They were physically connected by a virtual spring to the robotic partner of [4, 5], which has been shown to induce similar interaction perception, behaviour and learning to a human partner [6, 19]. This enabled the systematic investigation of the effect of temporal delay and noise in haptic communication. In this paper, we first reanalysed the 20 participant’s data from [15] to investigate the effect of temporal delays up to 540 ms. To analyse whether the CNS compensates for delay as if it was noise (compensate as noise), we further tested the effect of noise in the haptic connection through a new group of another 20 participants. As muscle co-activation control is a known strategy to deal with noise [20, 21], the activity of a wrist flexor/extensor muscle pair was recorded. A questionnaire was used to compare perception of both types of perturbations. Our new experimental findings showed that while the noise data displayed a trend consistent with co-activation control, the delay data showed a different tendency. These findings were also validated against a computational model for haptic communication [5], where by extending the model we investigated the response to different possible mechanisms used by the CNS to compensate for temporal delays and reproduce the observed experimental findings.

10.1371/journal.pcbi.1012037.g002 Experiment description.

A: Participants tracked a randomly moving target with their wrist flexion/extension movement while being connected to a reactive robot partner (RP). B: The experimental protocol for the delay group [15] and noise group. The grey boxes represent the familiarization/washout trials and the colourful blocks are the experimental conditions. Both groups started with the solo condition (i.e. without interacting with a RP). They then were connected with a RP without delay/noise. Subsequently, the delay/noise was increased in every block. After each block, participants were asked to fill in a questionnaire. The order of the blocks was the same for all participants.

 Experimental results

The experiment consisted of six blocks of target tracking (Fig 2B) conducted on the Hi5 robot with or without interaction with a robot partner (RP) (see Methods). Participants were split evenly into two groups of 20, where each group was defined by the perturbation added to the interaction: delay; or noise. For each group, the initial block was a solo condition in which the participants tracked the target without any connection. This was followed by five additional blocks in which a connection to the robot was provided and a perturbation on that connection was increased after each block. Delays of {0, 20, 60, 180, 540} ms were added to the haptic feedback in the delay group and random noise torques with standard deviation {0, 8, 22, 67, 184} mNm were added in the noise group.

Perception of delay and noise

After each experimental block, we asked participants to provide information about their perception of the haptic interaction by answering a questionnaire (see Supporting information). In both the delay and noise groups, participants clearly identified the presence of forces in all interaction blocks (Item 3: “During the task it seemed like I felt haptic forces”, Friedman test, delay: χ2(5) = 59.644, p < 0.0001; noise: χ2(5) = 57.847, p < 0.0001), such that the solo condition was distinguished from all other conditions (p < 0.05 for all pairwise comparisons to solo condition with post-hoc Wilcoxon tests).

The questionnaire also asked participants if they perceived delay (Item 10) and noise (Item 6) during each interaction block through a 5-point Likert scale from “strongly disagree” to “strongly agree” (see Fig 3). In the delay group, participants appeared to clearly perceive delay (Item 10, Fig 3A) even at small applied delay values, where the perception was always different to that of the no-delay condition (χ2(5) = 34.121, p < 0.0001; p < 0.05 for all pairwise comparisons between 0 ms and 20–540 ms delay groups). Moreover, participants disagreed that they perceived a delay in the 0 ms group (p = 0.8281 for the one-group comparison with the “disagree” value of Item 10). The delay group’s perception of noise (Item 6, Fig 3C) also changed depending on the applied delay (χ2(5) = 34.217, p < 0.0001). However, in contrast to their delay perception, the participants only had a higher perception of noise (relative to the no-delay condition) at the two highest applied delay levels, 180 and 540 ms (both p < 0.05).

10.1371/journal.pcbi.1012037.g003 Perception during the interaction with a RP perturbed by delay or noise.

(A) and (C) show the perception of delay and noise, respectively, for the delay group, while (B) and (D) show this perception for the noise group. Note the 5-point Likert scale goes from strongly disagree (‘low’) to strongly agree (‘high’).

Participants in the noise group clearly perceived noise (χ2(5) = 66.411, p < 0.0001) as shown in Fig 3D. In contrast to the perception of noise in the delay group, noise group participants perceived the presence of noise from the smallest applied noise level (p < 0.01 for all pairwise comparisons between 0 mNm and 8–184 mNm noise conditions). In this group, the perception of the delay (Fig 3B) also changed with the applied noise level (χ2(5) = 23.956, p = 0.00022), however, none of the conditions were found to be clearly different from each other (all p > 0.05).

In summary, in both delay and noise groups, participants were able to recognise the presence of their respective perturbation from its smallest value. There was also some increase in perception of the non-adjusted factor in each group. However, there was limited confusion with a clear perception of the incorrect factor (compared to the solo condition) only reported in the delay group for its two highest delay levels. This indicates that the participants were able to perceive and distinguish delay from noise even without any knowledge of the experiment perturbation types.

 Different behaviours induced by delay and noise

We then analysed the participant’s performance (Fig 4A) to determine if the participants benefited from the haptic interaction (as has been previously observed in [4, 5]) and if they were able to compensate for the applied disturbances. Here, since they were instructed to track the target “as accurately as possible”, performance was assessed through the root mean squared error relative to this target. In both groups, the robot partner condition affected the performance (delay: χ2(5) = 63.143, p < 0.0001; noise: χ2(5) = 62.257, p < 0.0001). Here, as in [4, 5], for both groups the addition of the robot partner (and the potential for haptic communication) improved the participant performance (solo vs. 0 for delay: W = 210, Z = 3.9199, p < 0.0001; and noise: W = 210, Z = 3.9199, p < 0.0001). However, while the participants were always able to compensate for the perturbation in the noise group such that the performance was always better to that of the solo condition (all p < 0.04) and not distinguishable from the condition working with a RP without perturbation (p > 0.05 for pairwise comparisons between 0-noise and 8, 22, 67 mNm), this was not the case for the delay group. Instead, here although the participants’ performance was not clearly affected by small applied delays (0 vs. 20 ms: W = 82, Z = −0.85865, p < 0.4091; 0 vs. 60 ms: W = 61, Z = −1.6426, p = 0.3162), they were not able to compensate for the larger applied delays (0 vs. 180 ms: W = 23, Z = −3.0613, p = 0.0073; 0 vs. 540 ms: W = 3, Z = −3.8079, p < 0.0001) such that at 540 ms their performance was worse than that of the solo condition (solo vs. 540 ms: W = 31, Z = −2.7626, p = 0.0211). This shows that while both groups improved from the addition of haptic communication, participants were only able to compensate for small delay values while they could compensate for all tested noise disturbances.

10.1371/journal.pcbi.1012037.g004 Performance and effort during the interaction with the robotic partner with delay (left) or noise (right) perturbation: Tracking accuracy (A) and co-contraction (B).

Each dot represents the average value in each block for one participant.

To understand if participants were using the previously observed co-activation noise compensation strategy [22] to deal with both types of applied perturbation, we analysed their co-contraction (Fig 4B) computed as the minimum measured absolute muscle activity of the flexor and extensor muscles over a trial (see Methods). Participants changed their co-contraction in response to both applied perturbation types (delay: χ2(5) = 40.829, p < 0.0001; noise: χ2(5) = 58.6, p < 0.0001). However, while the noise group displayed the previously observed noise compensation trend of increasing the median co-contraction with each subsequent applied perturbation level after the 8 mNm condition such that the co-contraction at the highest applied noise level was clearly larger than the no delay co-contraction (0–184 mNm: W = 0, Z = −3.9199, p < 0.0001), this was different in the delay group. Instead here, the participants held their co-contraction levels roughly constant for all small applied delay values (p > 0.05 for all pairwise comparisons between solo, 0 ms, 20 ms and 60 ms conditions except for solo-60 ms: W = 25, Z = −2.9866, p = 0.0169, where the co-contraction was slightly lower). There was then an increase for the two highest applied delay values (60 ms-180 ms: W = 1, Z = −3.8826, p < 0.0001; 60 ms-540 ms: W = 0, Z = −3.9199, p < 0.0001), in which the co-contraction level was not clearly different between the two levels (W = 25, Z = −2.9866, p = 0.0169). While the increase of co-contraction in the noise group was not concurrent with an increase in tracking error, as would be predicted if using the noise compensation strategy, the observed increase in co-contraction for the delay group coincided with an increase in tracking error.

Finally, we examined if the two different perturbations affected the user trajectories, through the cross-correlation delay between the reference and user trajectory (Fig 5A), and the motion smoothness (Fig 5B) as computed by the SPARC measure [23]. Here, the cross-correlation delay measured the difference between the target and participant trajectory, such that if the delayed haptic information of the delay perturbation was not compensated for, it would have resulted in increased cross-correlation delay. Moreover, the smoothness measured if the participant’s motion had greater frequency spectrum complexity as would occur for an uncompensated noise perturbation.

10.1371/journal.pcbi.1012037.g005 Lag and noisiness during the interaction with the robotic partner with delay (left) or noise (right) perturbation: Cross-correlation delay (A) and smoothness (B).

Each dot represents the average value in each block for one participant.

The addition of the robot partner and perturbations affected the cross-correlation delay of both groups (delay: χ2(5) = 36.564, p < 0.0001; noise: χ2(5) = 48.724, p < 0.0001), where interestingly working with the robot partner without perturbation resulted in a reduced cross-correlation delay (both p < 0.0001), suggesting that the haptic connection may have aided a quicker reaction. For the delay group, the cross-correlation delay in high-delay conditions was higher compared to smaller delay conditions (20—540 ms: W = 20, Z = −3.1733, p = 0.0007; 60—540 ms: W = 11, Z = −3.5096, p = 0.0005; 60—180 ms: W = 5.5, Z = −3.6028, p = 0.0003) suggesting that at those delay conditions participants struggled to counteract the delay. In contrast, for the noise group, the cross-correlation delay was higher in the solo and 0-noise condition (p < 0.0001 for all pairwise comparisons with solo; 0–8 mNm: W = 147, Z = 2.6787, p = 0.0437; 0–22 mNm: W = 148, Z = 2.7222, p = 0.0366; 0–67 mNm: W = 163, Z = 3.3752, p = 0.0019) and all other conditions were similar to each other (p > 0.05 for all other pairwise comparisons). This suggests that the presence of noise may have made the participants more reactive.

The participants smoothness was also affected by the perturbation (delay: χ2(5) = 75.771, p < 0.0001; noise: χ2(5) = 93.486, p < 0.0001), where as has been previously observed [19], the addition of the robot partner without perturbation resulted in smoother motion (both p < 0.0001). In both cases, the high perturbation levels were different to that of the no perturbation robot partner (delay: p < 0.05 for 0–180 ms and 0–540 ms; noise: p < 0.05 for 0–67 mNm and 0–184 mNm) suggesting that both perturbations resulted in a more jerky motion.

These results confirm that haptic communication improves interaction performance. While the observed behaviour is then affected by both the delay and noise perturbations, the effect is distinct in the two groups. For the noise group, participants appear to compensate as would be predicted by the co-activation compensation strategy, where their co-contraction increases with the applied perturbation level, and while there is an observed degradation of motion smoothness, there is no tracking performance that is worse than that of the solo condition. In contrast for the delay group, while there is an increase in co-contraction for the two highest applied perturbation levels, this increase does not result in compensation for the performance. Instead, participants appear to use a distinct method to counteract for the applied delay where only at the highest levels of injected delay was there a different lag to the other delay perturbation levels, and this was still not different from their own lag in the solo condition.

 Simulation results

The experimental results suggest that the response of the noise group is consistent with that predicted by the co-activation compensation strategy. However, the delay group behaviour was not consistent with this predicted behaviour, since the co-contraction did not consistently increase with a larger applied delay level. Could the results be explained strictly from the interaction mechanics, without any compensation? To evaluate this, we simulated the experimental scenario (with the same number of trials and blocks) of a participant being connected (by a virtual spring) to a partner with delayed haptic feedback. Here both partners were simulated using the model of haptic communication presented in [4, 5], which is based on four principles: i) The CNS of each participant is able to identify that the haptic feedback that they are receiving is related to the (visual) tracking task; ii) By using a model of tracking control, the CNS can extract from the haptic feedback an estimate of the partner’s tracking error; iii) The participants then each combine their own and partner motion information in a stochastic optimal way, yielding a Bayesian sensor fusion of visual (own) and haptic (partner) information; iv) The viscoelasticity of the haptic connection to the partner is incorporated into the sensor fusion, where more compliant connections are considered as having additional uncertainty. To simulate the effect of the delayed haptic feedback on the haptic communication model of [5], we created 20 simulated participants, where for each measured tracking error (skill level) of an experimental participant there was an equivalent simulated participant. For each simulated participant, we delayed the virtual spring torque for one of the participants with a delay value matching those of the experimental condition.

When both partners were modelled as in [5] such that there was no compensation, the performance (Fig 6C) did not match that observed in the experimental data (Fig 6A) for participants with lower skill. Here, while there was little difference for small applied delay values (and participants with higher skill), the performance for lower skilled participants could become unstable at larger delay levels such that it overshot the participant performance. Therefore, from the differing behaviour of low skill simulated participants, we conclude that the participants had a mechanism to compensate for the delay.

10.1371/journal.pcbi.1012037.g006 Participant specific (and average) tracking performance for the experimental data and in the simulation to analyse the mechanism to compensate for temporal delays.

Experimental results are shown for the delay group (A), as well as for the noise group (B). All simulated results consider only delay compensation, where results are shown for: no compensation (C), compensation as noise (D) and compensation by delay prediction (E). In all subfigures, the grey lines indicate individual participant (real or simulated) performance while the thick solid black line and the dashed black lines denote the median performance and interquartile range of performance across all participants.

Participants did appear able to recognise delay (Fig 3A). However, they may have adopted different strategies for its compensation which could lead to the observed motion characteristics. To investigate this compensation mechanism we extended the model developed in [5] to consider one possible compensation by delay prediction model as well as compensation as noise (see Methods). In the compensation as noise simulation, this consisted of modifying iv) such that both the viscoelasticity of the haptic connection and the delay of the feedback of that connection were considered as additional sources of uncertainty for the haptic information source (Fig 1B). In the compensation by delay prediction simulation, it was instead assumed that as a part of i) and ii) the participant would be able to identify the delay and use their model of the interaction and delay to predict the future value for the haptic feedback (Fig 1C).

The simulation results exhibit differences between the two models. Here, the compensation as noise simulation (Fig 6E) incorrectly predicted the delay response to have a similar trend to what was observed for noise group (Fig 6B), with a roughly constant tracking error behaviour across the applied delay levels. In contrast, the compensation as delay prediction simulation (Fig 6D) predicted an increase in tracking error and in the variance across participants for the higher delay levels.

To understand the differences in the proposed models, we compared the variance between the experimental data and the simulated data for noise, delay and no-adaptation compensation. The F-test did not show a clear difference in variance between the experimental data and the compensation as delay prediction simulation for any level of delay (all p > 0.1). In contrast, for the highest level of delay (540 ms), the results showed a clear difference between the experimental data and both the no-compensation simulation and the compensation by noise simulation (no-compensation: F(19, 19) = 10320643; p < 0.0001; noise: F(19, 19) = 0.1617; p = 0.0002) and a tendency for different variance at the 180 ms level (noise: F(19, 19) = 0.4244; p = 0.0692; no-compensation: F(19, 19) = 0.4044; p = 0.0554). No difference for other levels of delay was found for these strategies (all p > 0.1).

 Discussion

This paper investigated the mechanism of haptic communication during a tracking task by considering the response to delay and noise perturbations. Our results indicate that participants can still exploit haptic communication in the presence of both small delays and moderate levels of noise, where haptic communication resulted in improved smoothness, accuracy and smaller correlation delays. Interestingly, the participants were able to correctly recognise the presence of delays and noise from their smallest values with limited confusion between the correct perturbation and other possible perturbing modalities. They then appear to compensate for each of these two different perturbation types with different strategies.

While our findings showed similar tracking error and smoothness improvements resulting from haptic communication as had been previously observed [1, 4], we further observed a reduction in the cross-correlation delay that had not been previously investigated. This indicates that participants reacted faster to the moving target during human-like robot interaction. This may be explained by two factors: i) since the robot partner is also tracking the target, the interaction force assists the user such that it speeds up their response; and ii) the user uses haptic communication to more quickly update their motion plan. While it is difficult to separate these two factors, it is noted that the observed improvement in cross-correlation delay was still present with all noise levels and with the small applied delay values. Both of these perturbations would have meant that the interaction force sometimes hindered target tracking. The reduced lag may therefore suggest that the information exchange of haptic communication not only improves the quality of the estimation but also further aids participants to react quicker.

In response to increased noise, our experimental findings indicate that while participants had a clear reduction of smoothness, their co-contraction showed an increasing trend with increasing applied noise. This result is consistent with the predictions made by the co-activation compensation strategy [20, 22], therefore indicating that humans use this same strategy during haptic communication. Interestingly, the cross-correlation delay reduced after the application of the noise torque. This finding, that the application of noise can make participants react quicker, merits further investigation as it would not be predicted by existing haptic communication models [4, 5] and may suggest an additional mechanism within haptic communication.

The delay group showed different behaviour to the noise group, where the performance became worse for high delay values, and at these values there was an observed decrease in smoothness and increase in cross-correlation delay and co-contraction. In the context of haptic communication, these results could be obtained through distinct response mechanisms such as: i) participants do not compensate for delay. Here, the lack of a clear performance changes for small delay values is reflective of the delay having a minimal effect on the system dynamics at these values; ii) participants do not explicitly compensate for delay and instead identify the delay as an additional source of uncertainty. For large delay values, performance degrades due to the participants incorrectly modelling the delay as noise; and iii) participants possess a distinct compensation mechanism for delay. Here, the compensation mechanism may be imperfect leading to the observed changes in behaviour.

Our simulations indicate that while the no compensation model can result in similar performance for some participant skill levels, it is not robust, such that for the extreme cases (corresponding to the lower skill level experimental participants) it would predict unstable performance (Fig 6C). The lack of such cases in our experimental results (Fig 6A) suggests the presence of a compensation mechanism for delay, as has been observed in multi-sensory integration [24] and in the adaptation to applied force [16, 25, 26] and visual [17, 27] delays. Here, it is also noted that participants were able to perceive delay even for small values (Fig 3A). While, there is evidence that the mechanisms for perceiving delay and adapting to it are not necessarily the same [16], this has suggested that compensation occurs without perception but not the other way.

The simulation results also suggest that the delay group participants did not increase their co-contraction in a manner consistent with the compensation by noise compensation strategy. Our findings therefore appear to be consistent with a strategy of compensation by delay prediction, where the CNS identifies the delay and distinctively compensates for it. Our simulations consider this compensation to be an explicit time-based compensation as has been observed for the multisensory integration of information across the visual and haptic channels [24] and in other tracking tasks with delay [26, 28]. However, it is worth noting that other studies have shown that force delay may instead be compensated for using a mixture of current and delayed state information [25], or that visual delay may be compensated for using state-based mechanisms such as assuming altered impedance characteristics [27] or scaling of the measurement [17] properties.

Given that previous studies have shown that the perception of delay is associated to a perception of stiffness [13, 14] and that increasing compliance can be modelled by an increase in uncertainty in Bayesian integration [5], the haptic communication model [4] could be updated to consider alternate delay compensation models through an updated haptic channel measurement with additional sources of uncertainty. Our current experimental results are however not able to distinguish between these different possible delay compensation strategies. Moreover, it is worth noting that the two applied delay levels for which noise was perceived (delays of 180 and 540 ms) both also coincided with an increase in co-contraction. It is therefore possible that participants used a mix of the compensation by delay prediction and compensate by noise strategies, where in response to their insufficient compensation participants identify noise and then try to compensate (ineffectively) through co-activation.

In summary, physical connections over the haptic channel can improve user performance and is robust to small to medium sized perturbations in the form of delays and noise. Our findings suggest that this compensation is made possible by the participant being able to uniquely perceive the presence of delay or noise and then compensate specifically to these different perturbations.

 Methods Ethics statement

The experiment was approved by the Research Ethics Committee of Imperial College London. Each participant gave informed formal written consent, and filled in a demographic questionnaire as well as the Edinburgh handedness form [29] before starting the experiment.

 Participants

The experiment was carried out by 40 participants (21 female, 19 male) without known sensorimotor impairment aged 24.02±3.19 years old. All participants performed the task with their dominant hand, where four participants were left-handed. Participants were divided into two groups that each experienced only one type of perturbation: human-robot interaction with i) time delay (data collected in [15]) or ii) with haptic (torque) noise.

 Experimental setup and protocol

The experimental task was designed to replicate the tasks of existing studies of haptic communication [19]. Two participants completed the experiment at the same time, with their dominant arm attached to the Hi5 dual robotic interface [18]. The participants were instructed that within the experiment they might experience a haptic interaction at their wrist. They were then visually separated from one another by a curtain and each participant’s wrist flexion/extension movement interacted with the Hi5 interface, which was controlled at 1000 Hz, while the wrist angle data was recorded at 100 Hz.

Each participant was asked to track a moving target “as accurately as possible” using the wrist flexion/extension of their dominant hand (Fig 2A). The target trajectory was given by q * ( t ) ≡ 18 . 5 ° sin [ 2 . 031 ( t + t 0 ) ] sin [ 1 . 093 ( t + t 0 ) ] , 0 ≤ t ≤ 30 s , where to minimise the learning of the trajectory each trial started from a randomly selected starting time {t0 ∈ [0, 30] s | q*(t0) ≡ 0}. The participant’s wrist flexion/extension was connected to a robotic partner (RP) with angle qr through a virtual viscoelastic band described (in Nm) by τ ( t ) = 1 . 72 [ q r ( t δ ) - q ( t δ ) ] + 0 . 0286 [ q ˙ r ( t δ ) - q ˙ ( t δ ) ] , t δ ≡ t - δ .

The robotic partner (RP) is a reactive controller [4, 19] that mimics human interaction behaviour. This includes accounting for the different skill levels of human participants, where the robot’s skill level (as measured by the RMS tracking error) can be set. Here, the RP replicates human haptic communication behaviour through a sensory augmentation approach in which the information coming from the haptic connection is used to infer the partner’s motion information [4]. This is then combined with their own motion information in a stochastically optimal manner, where less skilled agents have larger uncertainty in their measurements.

For Eq (2) the damping and stiffness constants were chosen to match the conditions of medium stiffness in [4], with which an interaction with an interactive agent was clearly perceived by participants [19]. In the delay group δ ∈ {0, 20, 60, 180, 540} ms was used for the delayed interaction torque (while the robot partner received the torque without delay). For the noise group δ = 0, while the torque was perturbed by Gaussian noise ν: τ ν ( t ) = τ ( t ) + ν , ν ∈ N ( 0 , Σ η ) , where the standard deviation Ση ∈ {7.5, 22.5, 66.7, 184} mNm was used.

Surface electrodes were used to record electromyographical (EMG) activity from the wrist flexor carpi radialis (FCR) and extensor carpi radialis longus (ECRL) muscles. This was calibrated through a process in which participants were asked to flex/extend while their wrist was locked by the device at 0° corresponding to the participant’s most comfortable position. Each participant was asked to produce flexion and extension torques of {1, 2, 3, 4} Nm for 2 seconds, first flexion then extension, with a rest period of 5 seconds between each activation to prevent fatigue. This EMG data was linearly regressed with the measured torque to estimate the relationship between muscular activity and torque. Then the co-contraction was computed as u ( t ) ≡ min { τ f ( t ) , | τ e ( t ) | } , where τf (t) ≥ 0 and τe(t) ≤ 0 are the flexor and extensor torques, computed from the respective EMG signals. The average co-contraction over all participants was computed from each participant’s normalised co-contraction, calculated as u n ≡ u ¯ - u ¯ m i n u ¯ m a x - u ¯ m i n , u ¯ ≡ 1 T ∫ 0 T u ( t ) d t , T = 30 s with u ¯ m i n and u ¯ m a x the minimum and maximum of the means of all trials of the specific participant.

The experiment protocol is described in Fig 2B. In the initial solo block, each participant attempted five trials of the task without a haptic connection to be familiarised with the task and to minimise subsequent learning effects. In the main experiment, participants carried out six blocks, each of ten trials. Each block included seven experimental trials followed by three washout trials of the solo condition. The first experimental condition was without any interaction and in the following five blocks assistance from a robot partner was introduced for both delay and noise experiments. The robot partner’s uncertainty was set after the solo block using a mapping that converted RMS error into measurement uncertainty [4]. Here the robot’s skill was set to be equal to the deviation observed in the participant’s tracking movement during the final solo trial to ensure that the participant and the RP had similar skill level. This was chosen to ensure that there was no clear difference in the participant and RP’s performance during the tracking trials. Each trial took 30 s and was followed by a 5 s break.

The delay and noise within the robotic assistance trials were increased from each block to the next with values {0, 20, 60, 180, 540} ms for delay and {0, 8, 22, 67, 184} mNm for noise. The sequence of the blocks with increasing level of perturbation was identical for each participant within delay and noise experiments. The delay levels were chosen to include small delay values considered in [30] and values greater than the threshold for performance loss found in [31]. The Gaussian noise values were then set to approximate the effect of the delay. Here, the noise torque standard deviation Ση was set so that three standard deviations was equivalent to the likely maximum error torque caused by a given delay, which was approximated by 1.72 ⋅ max{q*(t) − q*(t − δ)}. After each block, the participants had to answer questions about their perception of the interaction (see Supporting Information S1 Text).

 Simulation framework

We evaluated the mechanism for delay via 20 simulated participants that were programmed with the computational model developed in [5] which matched the algorithm used by the RP. In this discrete time model, the control of the wrist is modelled as a linear controller that acts on a double integrator system which describes the wrist angle q. The state space dynamics at time-step i are given by q i + 1 = A q i + B ( u i + τ i ) , q i ≡ [ q i q ˙ i ] , A = [ 1 d t 0 1 ] , B = [ 0 d t / I ] , where dt is the time differential and I = 0.002 kg m2 the wrist’s moment of inertia (as defined for experiments with the Hi5 robot in [5]). The control input ui is determined by the linear feedback control law u = - L p ( q - q * ^ ) + L v ( q ˙ - q ˙ * ^ ) , where q * ^ i denotes the participant’s estimate of the trajectory. Lp and Lv are the proportional and derivative gains determined to minimise a quadratic cost function of error and effort [20]. Moreover, the system is influenced by the haptic interaction torque τi. This torque is set to 0 during solo trials. It instead acts as a spring and damper torque and is set as in Eq (2) for interaction trials connecting the agent to their partner whose dynamics similarly evolves with a form given by Eq (6).

The key feature of the model is that the partners improve their estimate q * ^ through visual feedback and haptic information from the interaction. Here, their own target information is combined with the partner’s target information q ˜ * ^ i, as determined from the interaction force. This integration of the partner’s target information is carried out through a Kalman filter in which the measurement zi is given by z i = [ q i - q i * q i - q ˜ * ^ i ] + [ μ μ ˜ ] , where μ ∈ N ( 0 , Σ z v ) and μ ˜ ∈ N ( 0 , Σ z h ). This assumes that Σ z v characterises the visual noise naturally present in the participant’s tracking, while Σ z h characterises the haptic noise and is composed of the partner’s visual noise Σ z ˜ v and additional noise resulting from the virtual band viscoelasticity Σ z k such that Σ z h = Σ z ˜ v + Σ z k. In this way, the haptic interaction acts both as an input force that could help in target tracking and as a second sensor measurement that can improve the participant’s understanding of their current task state.

Delay compensation strategies

Three delay compensation strategies are considered within the simulation to explore the features of the participants response to delay: i) no compensation; ii) compensation as noise; iii) compensation as delay prediction. In the no compensation strategy participants were simulated to directly use the above interaction model with a delayed estimation of the partner’s error, i.e. z i = [ q i - q i * q i - δ - q ˜ * ^ i ] + [ μ μ ˜ ] .

In the compensation as noise strategy participants were modelled to use the same delayed measurement as in Eq (9). However, they were assumed to consider the haptic signal to be noisier with an additional independent noise Σ z d associated to the given delay level such that Σ z h = Σ z ˜ v + Σ z k + Σ z d .

The magnitude of this additional delay generated noise was determined by simulating the participant’s solo performance with the delay and then mapping the resulting error to a noise value through the error to noise regression determined in [5].

Finally, in the compensation as delay prediction strategy the participants were instead modelled to be able to identify both the presence of the delay as well as its magnitude. They were then assumed subsequently adjust their haptic measurement through forward integration with the known system dynamics Eq (6) and visual target sequence. Here the current error estimate q i - δ - q ˜ * ^ i was updated by iteratively applying Eq (6) (without the unmeasured torque τi) for the number of delayed time steps (given as δ/dt).

 Data analysis

To investigate the delayed force exchange’s effect on participant performance, the Root-Mean-Square Error (RMSE), the smoothness metrics SPARC [23], the cross-correlation delay and the co-contraction were analysed. The cross-correlation delay corresponds to the time interval between the target’s movement and the participant’s resulting motion and was calculated as the time lag at which the cross-correlation between the target and participant’s positions was the highest. To understand how participants perceived the changes in delay, a questionnaire composed of a 5-point Likert-scale item was analysed (see the question list in Supporting information). To check the consistency of the questionnaire responses for Items 2–12, composite reliability and Cronbach’s alpha were used to determine whether responses were consistent between items. This gave values of 0.86 and 0.87, respectively, which confirmed the consistency between the item responses, and indicates that the items were not redundant.

Since we could not directly compare the delay and noise magnitudes, a separate analysis was conducted for each perturbation group, and the group tendencies were then qualitatively compared. Since each metric was found to not be normally distributed, the influence of the perturbation on each metric was explored through Friedman tests. Post-hoc analysis between individual perturbation levels was conducted using a paired Wilcoxon sign-rank test with the Hommel adjustment to control the family-wise error rate. For each objective value (RMSE, SPARC, cross-correlation delay, co-contraction) the analysis was conducted for each participant using the averaged value over all trials in a block. To compare the variance between the simulated data of for the compensation by noise, compensation as delay prediction and no compensation models and the experimental data, an F-test was used for each delay level.

 Supporting information Questionnaire items.

(DOCX)

 References Ganesh G, Takagi A, Osu R, Yoshioka T, Kawato M, Burdet E. Two is better than one: Physical interactions improve motor performance in humans. Scientific Reports. 2014;4:3824. doi: 10.1038/srep03824 24452767 Beckers N, van Asseldonk EH, van der Kooij H. Haptic human–human interaction does not improve individual visuomotor adaptation. Scientific Reports. 2020;10(1):1–11. doi: 10.1038/s41598-020-76706-x Mojtahedi K, Whitsell B, Artemiadis P, Santello M. Communication and inference of intended movement direction during human–human physical interaction. Frontiers in Neurorobotics. 2017;11:21. doi: 10.3389/fnbot.2017.00021 28450834 Takagi A, Ganesh G, Yoshioka T, Kawato M, Burdet E. Physically interacting individuals estimate the partner’s goal to enhance their movements. Nature Human Behaviour. 2017;1(3):0054. doi: 10.1038/s41562-017-0054 Takagi A, Usai F, Ganesh G, Sanguineti V, Burdet E. Haptic communication between humans is tuned by the hard or soft mechanics of interaction. PLoS Computational Biology. 2018;14(3):e1005971. doi: 10.1371/journal.pcbi.1005971 29565966 Ivanova E, Eden J, Carboni G, Krüger J, Burdet E. Interaction with a reactive partner improves learning in contrast to passive guidance. Scientific Reports. 2022;12(1):15821. doi: 10.1038/s41598-022-18617-7 36138031 Batson JP, Kato Y, Shuster K, Patton JL, Reed KB, Tsuji T, et al. Haptic coupling in dyads improves motor learning in a simple force field. In: International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC); 2020. p. 4795–4798. Parise CV, Ernst MO. Correlation detection as a general mechanism for multisensory integration. Nature Communications. 2016;7(1):1–9. doi: 10.1038/ncomms11543 27265526 Cao Y, Summerfield C, Park H, Giordano BL, Kayser C. Causal inference in the multisensory brain. Neuron. 2019;102(5):1076–1087. doi: 10.1016/j.neuron.2019.03.043 31047778 Imaida T, Yokokohji Y, Doi T, Oda M, Yoshikawa T. Ground-space bilateral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7-s time delay condition. IEEE Transactions on Robotics and Automation. 2004;20(3):499–511. doi: 10.1109/TRA.2004.825271 Postolache O, Hemanth DJ, Alexandre R, Gupta D, Geman O, Khanna A. Remote monitoring of physical rehabilitation of stroke patients using IoT and virtual reality. IEEE Journal on Selected Areas in Communications. 2020;39(2):562–573. doi: 10.1109/JSAC.2020.3020600 Hojatmadani M, Rigsby B, Reed KB. Time delay affects thermal discrimination. IEEE Transactions on Haptics. 2022;15(2):451–457. doi: 10.1109/TOH.2022.3156122 35259114 Pressman A, Karniel A, Mussa-Ivaldi FA. Perception of delayed stiffness. In: IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics.; 2006. p. 905–910. Nisky I, Baraduc P, Karniel A. Proximodistal gradient in the perception of delayed stiffness. Journal of Neurophysiology. 2010;103(6):3017–3026. doi: 10.1152/jn.00939.2009 20357066 Ivanova E, Eden J, Zhu S, Carboni G, Yurkewich A, Burdet E. Short time delay does not hinder haptic communication benefits. IEEE Transactions on Haptics. 2021;14(2):322–327. doi: 10.1109/TOH.2021.3079227 33989159 Leib R, Karniel A, Nisky I. The effect of force feedback delay on stiffness perception and grip force modulation during tool-mediated interaction with elastic force fields. Journal of Neurophysiology. 2015;113(9):3076–3089. doi: 10.1152/jn.00229.2014 25717155 Avraham G, Leib R, Pressman A, Simo LS, Karniel A, Shmuelof L, et al. State-based delay representation and its transfer from a game of pong to reaching and tracking. eNeuro. 2017;4(6). doi: 10.1523/ENEURO.0179-17.2017 29379875 Melendez-Calderon A, Bagutti L, Pedrono B, Burdet E. Hi5: A versatile dual-wrist device to study human-human interaction and bimanual control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2011. p. 2578–2583. Ivanova E, Carboni G, Eden J, Krueger J, Burdet E. For motion assistance humans prefer to rely on a robot rather than on an unpredictable human. IEEE Open Journal of Engineering in Medicine and Biology. 2020;. doi: 10.1109/OJEMB.2020.2987885 35402952 Franklin DW, Burdet E, Tee KP, Osu R, Chew CM, Milner TE, et al. CNS learns stable, accurate, and efficient movements using a simple algorithm. Journal of Neuroscience. 2008;28(44):11165–11173. doi: 10.1523/JNEUROSCI.3099-08.2008 18971459 Hasson CJ, Gelina O, Woo G. Neural control adaptation to motor noise manipulation. Frontiers in human neuroscience. 2016;10:59. doi: 10.3389/fnhum.2016.00059 26973487 Börner H, Carboni G, Cheng X, Takagi A, Hirche S, Endo S, et al. Physically interacting humans regulate muscle coactivation to improve visuo-haptic perception. Journal of Neurophysiology. 2023;129(2):494–499. doi: 10.1152/jn.00420.2022 36651649 Balasubramanian S, Melendez-Calderon A, Roby-Brami A, Burdet E. On the analysis of movement smoothness. Journal of Neuroengineering and Rehabilitation. 2015;12(1):112. doi: 10.1186/s12984-015-0090-9 26651329 Kasuga S, Crevecoeur F, Cross KP, Balalaie P, Scott SH. Integration of proprioceptive and visual feedback during online control of reaching. Journal of Neurophysiology. 2022;127(2):354–372. doi: 10.1152/jn.00639.2020 34907796 Avraham G, Mawase F, Karniel A, Shmuelof L, Donchin O, Mussa-Ivaldi FA, et al. Representing delayed force feedback as a combination of current and delayed states. Journal of Neurophysiology. 2017;118(4):2110–2131. doi: 10.1152/jn.00347.2017 28724784 Rohde M, Van Dam LC, Ernst MO. Predictability is necessary for closed-loop visual feedback delay adaptation. Journal of Vision. 2014;14(3):4–4. doi: 10.1167/14.3.4 24599942 Leib R, Karniel A, Mussa-Ivaldi FA. The mechanical representation of temporal delays. Scientific Reports. 2017;7(1):7669. doi: 10.1038/s41598-017-07289-3 28794465 Farshchiansadegh A, Ranganathan R, Casadio M, Mussa-Ivaldi FA. Adaptation to visual feedback delay in a redundant motor task. Journal of Neurophysiology. 2015;113(2):426–433. doi: 10.1152/jn.00249.2014 25339704 Oldfield RC. The assessment and analysis of handedness: the Edinburgh Inventory. Neuropsychologia. 1971;9(1):97–113. doi: 10.1016/0028-3932(71)90067-4 5146491 Friston S, Karlström P, Steed A. The effects of low latency on pointing and steering tasks. IEEE Transactions on Visualization and Computer Graphics. 2015;22(5):1605–1615. doi: 10.1109/TVCG.2015.2446467 Pavlovych A, Stuerzlinger W. Target following performance in the presence of latency, jitter, and signal dropouts. In: Graphics Interface. vol. 2011; 2011. p. 33–40. 10.1371/journal.pcbi.1012037.r001 Decision Letter 0 Haith Adrian M Academic Editor Martin Andrea E. Section Editor 2024 Haith, Martin This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version 0

11 Jul 2024

Dear Dr Eden,

Thank you very much for submitting your manuscript "During haptic communication, the central nervous system compensates distinctly for delay and noise" for consideration at PLOS Computational Biology.

As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers were generally positive about the paper, but did raise some substantial concerns about the modeling for the delay condition. There is a substantial literature on adaptation to delayed feedback which has has argued against the simple delay prediction model put forward here. This literature is largely overlooked, however. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments and in particular addresses the issues raised by the reviewers surrounding the modeling of behavior in the delay condition.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Adrian M Haith

Academic Editor

PLOS Computational Biology

Andrea E. Martin

Section Editor

PLOS Computational Biology

***********************

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: The authors present an interesting study of how human actors might account for noise and delay when interacting haptically with humans or robotic assistants to perform joint action tasks. I find the experiments and analysis to be well done and modelling of human behaviour to be convincing and innovative.

I have one methodological question as to how to differentiate between mechanical effects of the haptic interaction on performance versus a Bayesian interpretation of the interaction in terms of optimal control. And I have some suggestions on how the author's might interpret their results with respect to both haptic perception in the face of delays and in terms of a potential link to the classic binding problem.

Lines 141 and 154: It is indeed interesting that adding the RP without perturbation both increased the smoothness and decreased the correlation delay. Indeed, I have a question about the comparison between solo and with RP but without added noise or delay. This may have been covered in previous publications, but I have not read them recently and in the interest of time (my review is late) I will pose my questions here. At line 276 it is stated " the information coming from the haptic connection is used to infer the partner’s target, which is then combined with their own target in a stochastically optimal manner." In the experiments reported here, what is the RP's "own target"? Was it an accurate version of the actual target? This would mean that the RP is a priori assisting the human subject and the human can infer from instructions or deduce that the haptic information should be helpful and use that information to modulate their own muscle activations to control the wrist. Or is the RP only moving based on the "inferred" target of its human partner, without independent information about the target's trajectory?

In either case, how do the authors account for the stabilizing effect of the damping in the haptic connector to the RP on the smoothness of the hand trajectory? In other words, what would happen if the RP did not assist the human (position gain set to zero and q-dot_r set to zero with the same velocity gain in Eq. 2)? Might this also reduce the tracking error compared to the solo condition, where, from my understanding, no haptic torques are being applied to the participant's wrist? Could the action of the RP in fact be more through a mechanism of impedance matching rather than active assistance and haptic communication per se?

I find it also interesting that adding noise can elicit the perception of delay. Is this conditioned by the fact that the subjects were told that they would interact with a robot or a human? Was it stated or implied that the robot or human would be acting toward the same goal?

Aside from this methodological question, I wonder if the author's could consider other studies of haptic perception in face of delays (e.g. Pressman, A., Karniel, A., & Mussa-Ivaldi, F. A. (2006, February). Perception of delayed stiffness. In The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006. (pp. 905-910). IEEE.) And in view of the sharp change in predicted behavior for larger delays (Fig. 6D), I wonder if the author's have considered the so-called binding problem in face of delays? Might the subjects consider small delays as noise when the delay is small and then shift to a compensation as delay strategy when the delay is large, because, hypothetically, with the larger delay the haptic interaction is perceived as a delayed interaction from a separate entity (the RP) rather than a noise perturbation to a single physical interaction?

Minor remarks

Abstract: Perhaps "Physically connected humans ..." ?

Line 50: Are the experimental finding mentioned here findings from previous studies, or a forward reference to what will be reported here?

Line 78-79: That subjects more reliably detected delay when there was delay vs. when there was not is clear It would nevertheless be interesting to know to what extent subjects perceived delay when there was none, i.e. did the responses to the item "It seemed like I felt haptic interaction with a delay." elicit a response that was different from "strongly disagree"?

Line 75-81: Could you clearly state which items are used in each analysis, and if you relied on only a single item in each case, or if "perceived noise" (item 6) was somehow conditioned by responses to other items, such as "interaction with an agent" or not?

Line 85: How is this result "different to the results in the delay group". The sentence confused me because the delay group also perceived delay even for the smallest delay. I suggest: "Participants in the noise group clearly identified the presence of noise (χ2(5) = 66.411, p < 0.0001) as shown in Fig. 3D. In contrast to the perceived noise in the delay group, noise-group participants perceived the presence of noise from the smallest applied noise level (p < 0.01 for all pairwise comparisons between 0 mNm and 8-184 mNm noise conditions)."

Line 90: "none of the conditions were found to be clear different." From each other? Or from the solo or zero-delay conditions? A post hoc test of all pairwise comparisons would be less powerful then a set of planned comparisons between each of the non-zero delays and the zero-delay condition.

Fig. 3: I quibble with the axis titles "perceived as ..." and wonder if the delay is orthogonally independent from the noise. Unless the subject knew that there would be only noise or only delay, then the answers to the questionnaire tested only for the perceived presence of noise ("perceived noise") or the perceived presence of delay ("perceived delay"). Whether they perceived delay as noise is a matter of interpretation. Did any subjects perceive delay only as increased noise? The argument that delay is distinguishable from noise also depends on the assumption that no noise is actually added when adding delay. Does introducing delay in fact add noise to the haptic signals received by the subject via the coupling? Could a robotic system take the place of the human and, knowing that the RP will introduce only delay, could the robot identify the delay and produce trajectories that are no noisier than the non-delay system?

Line 134: The statement is factually correct, but the "in contrast to" perhaps invites an apples-to-oranges comparison. There is no way, in my mind, to calibrate the potential effect of a given level of noise on the tracking performance to the potential effect of a given level of delay on tracking performance. There is no reason to suppose and equivalence between 184 nMn of noise with 540 ms of delay in terms of effect on the tracking error in the absence of co-contraction, so the fact that the co-contraction in the case of delay did not prevent an increase of tracking error is potentially misleading.

Fig 6: What is the meaning of the different grey lines? Are 6A and 6B simply a different representation of the same results reported in 4A, but with connections between results for individual participants? What is the meaning of the dashed lines?

Why is 6B present, since in the text it is said "To evaluate this, we simulated the experimental scenario (with the same number of trials and blocks) of a participant being connected (by a virtual spring) to a partner with delayed haptic feedback."? Are the authors also simulating the compensation as noise strategy for subjects who received noise perturbations? If so, perhaps an explicit comparison of simulation results in 6E to 6B would also be warranted in the text.

What it the meaning of the grey lines for the simulation results (6C-6F)? Are these individual simulations tailored to each participant? The way that this was done is not clear to me. And if this is true, does 6C-F represent simulations for both the delay group and the noise group? Or just the delay group?

Lines 184-185: Since the RP is reacting to the haptic information from the human, I wonder if it is the same thing to delay the torque produced by the virtual spring vs. delaying the RP's "own target" estimate.

Lines 204-210: Have the authors confused 6D and 6F in the text?

Questionnaire:

What is the difference between items 3 and 7? i.e. what is the difference between "haptic forces" and "haptic feedback"? Were there differences in responses to these two items?

Were the answers to the questions internally consistent? For instance, if a subject answered non-zero for 8 or 9, did they also answer non-zero for 7 and zero for 12?

Reviewer #2: The paper generally addresses an interesting and important problem – haptic collaboration over delayed channels. This problem has many practical implications in today’s world and also may shed light on an interesting and unsolved problem of coping with internal delays (however this implication was not mentioned by the authors). The authors nicely extend their previous studies of collaboration between humans and robotic agents in a task of tracking a target, and in the current study they investigate how this interaction is affected by adding noise or by adding a delay. While the only new experiment is the effect of noise, I do appreciate the analysis of the old and the new study together to make a point and the previous work is correctly acknowledged in the paper. Overall the work is of good quality. However, I have the following major reservations about this work:

(1) The authors completely ignore the existing literature about how delays affect movement control and perception. While the particular task of haptic collaboration over a delayed channel was not extensively studied, the effect of delay was studied in several contexts that are related to the task. Except from the failure to put the study in the correct context in terms of prior literature, these prior studies could yield several other possible models for coping with delay except from modeling it as noise. Delay in force feedback was previously shown to change the perception of the mechanical properties of environment (stiffness, mass). Delay in force field adaptation was proposed to be compensated as a combination of current and delayed movement signals, or as a state-based approximation – position, velocity, and acceleration). Delay in visual feedback was proposed to be compensated as a gain, as a mechanical system, or as an altered inertia). All these could result in possible models that can propose explanations for the inaccurate coping with delay beyond noise. While the results of this paper nicely show that delay is not compensated as noise, suggesting that some type of representation or approximation of delay exists, I wander does it make sense to assume such complete lack of representation as an alternative to accurate prediction of time when evidence in favor of alternative ideas exists that are not noise but also not delay representation?

Here are a few of the references that are relevant. Note that even though quite a few of these papers are from my group, (not all though), by all means i do not request the authors to cite this entire list. But i would like the author to be aware of these studies and decide how to proceed with this knowledge:

Pressman et al., 2007, Perception of delayed stiffness

Nisky et al., 2008, A regression and boundary-crossing based model for the perception of delayed stiffness

Nisky et al., 2010 A regression and boundary-crossing based model for the perception of delayed stiffness

DiLuca et al., 2011 Effects of visual–haptic asynchronies and loading–unloading movements on compliance perception

Leib et al., 2015 The effect of force feedback delay on stiffness perception and grip force modulation during tool-mediated interaction with elastic force fields

DiLuca Rhodes 2016 Optimal Perceived Timing: Integrating Sensory Information with Dynamically Updated Expectations

Leib et al., 2017 The mechanical representation of temporal delays

Avraham et al., 2017 “Representing Delayed Force Feedback as a Combination of Current and Delayed States

Avraham et al., 2017 State-based delay representation and its transfer from a game of pong to reaching and tracking

Farschian et al., 2018 Energy exchanges at contact events guide sensorimotor integration across intermodal delays

Avraham et al., 2019 Effects of Visuomotor Detlays on the Control of Movement and on Perceptual Localization in the Presence and Absence of Visual Targets

Van Polanen et al., 2019 Visual delay affects force scaling and weight perception during object lifting in virtual reality

(2) It is not entirely clear to me how the computational model supports the experimental findings beyond the conclusion that can be reached from the error and co-contraction results? The predictions of the model were consistent generally with the experimental results, but it is difficult to tell if the chosen model indeed explains what happens in the delay case. Indeed the authors are right in their choice of the title stating that the main finding is that delay and noise are not compensated similarly. But this conclusion does not require the computational model. The alternative models I mentioned in the previous point could possibly propose a more compelling explanation.

In addition, I think the following points could further improve the paper, regardless to my previous comments:

1) It would help the interpretations a lot if the authors would explain exactly what are their prediction with respect to each of the computed metrics, and mentioned the specific results in the narrative rather than just the statistical results. For me the result that was most difficult to understand due to the lack of predictions and narrative is the results of the delay cross-correlation – did they expect the delay to change? To which value? What the exact value of the delay teaches us? The authors state in the discussion that: “The reduced lag then suggests that this information not only improves the quality of their estimation but also aids them to adapt quicker then what they normally would” but it is not clear to me why. The other results are a bit more intuitive, but still the way they are presented could be improved – I would like to be able to get an idea about the exact result from the text, and then go to the figure to make sure I agree with the authors based on what I see.

2) Similarly, the exact interpretation of the simulations are not entirely clear. For example, I could not tell based on figure 6 whether a real difference exists between the predictions of the no-compensation and the compensation as delay, which is confusing and makes the message of the paper less clear.

3) The illustration of the different models in Figure 1 is also not clear – why at some point the compensation as delay diverges from the original trajectory. In general, in addition to the general illustration becoming more clear, I would like to see simulated trajectories and experimental trajectories to be able to compare the predictions of the model to the data.

4) Minor: they call the connection between the partners an elastic band but I believe a viscoelastic band is more accurate.

To conclude, I like the direction where the authors are going, but I think that the current work does not address sufficiently the models from the literature, and does not provide enough information about the simulation and its interpretation to merit the paper publication in its current form.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No: 

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Ilana Nisky

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

 10.1371/journal.pcbi.1012037.r002 Author response to Decision Letter 0 Submission Version 1

26 Sep 2024

Submitted filename: Reviewer_Comments_PCOMPBIOL_D_24_00534.pdf

 10.1371/journal.pcbi.1012037.r003 Decision Letter 1 Haith Adrian M Academic Editor Martin Andrea E. Section Editor 2024 Haith, Martin This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Submission Version 1

18 Oct 2024

Dear Dr Eden,

We are pleased to inform you that your manuscript 'During haptic communication, the central nervous system compensates distinctly for delay and noise' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology. 

Best regards,

Adrian M Haith

Academic Editor

PLOS Computational Biology

Andrea E. Martin

Section Editor

PLOS Computational Biology

***********************************************************

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: I am satisfied with the revised manuscript and I appreciate the careful attention that the authors have given to addressing my questions and concerns.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

 10.1371/journal.pcbi.1012037.r004 Acceptance letter Haith Adrian M Academic Editor Martin Andrea E. Section Editor 2024 Haith, Martin This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

29 Oct 2024

PCOMPBIOL-D-24-00534R1

During haptic communication, the central nervous system compensates distinctly for delay and noise

Dear Dr Eden,

I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Zsofia Freund

PLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom ploscompbiol@plos.org | Phone +44 (0) 1223-442824 | ploscompbiol.org | @PLOSCompBiol