<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article
  PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "http://dtd.nlm.nih.gov/publishing/3.0/journalpublishing3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="3.0" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="nlm-ta">PLoS ONE</journal-id>
<journal-id journal-id-type="publisher-id">plos</journal-id>
<journal-id journal-id-type="pmc">plosone</journal-id>
<journal-title-group>
<journal-title>PLOS ONE</journal-title>
</journal-title-group>
<issn pub-type="epub">1932-6203</issn>
<publisher>
<publisher-name>Public Library of Science</publisher-name>
<publisher-loc>San Francisco, CA USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.1371/journal.pone.0135072</article-id>
<article-id pub-id-type="publisher-id">PONE-D-14-56862</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Research Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>The Canary in the Coal Mine Tweets: Social Media Reveals Public Perceptions of Non-Medical Use of Opioids</article-title>
<alt-title alt-title-type="running-head">Qualitative Analysis of Opioid Misuse Using Social Media</alt-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" xlink:type="simple">
<name name-style="western">
<surname>Chan</surname>
<given-names>Brian</given-names>
</name>
<xref rid="aff001" ref-type="aff"/>
</contrib>
<contrib contrib-type="author" xlink:type="simple">
<name name-style="western">
<surname>Lopez</surname>
<given-names>Andrea</given-names>
</name>
<xref rid="aff001" ref-type="aff"/>
</contrib>
<contrib contrib-type="author" corresp="yes" xlink:type="simple">
<name name-style="western">
<surname>Sarkar</surname>
<given-names>Urmimala</given-names>
</name>
<xref rid="cor001" ref-type="corresp">*</xref>
<xref rid="aff001" ref-type="aff"/>
</contrib>
</contrib-group>
<aff id="aff001"><addr-line>Division of General Internal Medicine, School of Medicine, University of California San Francisco, San Francisco, California, United States of America</addr-line></aff>
<contrib-group>
<contrib contrib-type="editor" xlink:type="simple">
<name name-style="western">
<surname>Hildt</surname>
<given-names>Elisabeth</given-names>
</name>
<role>Editor</role>
<xref ref-type="aff" rid="edit1"/>
</contrib>
</contrib-group>
<aff id="edit1"><addr-line>Illinois Institute of Technology, UNITED STATES</addr-line></aff>
<author-notes>
<fn fn-type="conflict" id="coi001">
<p>The authors have declared that no competing interests exist.</p>
</fn>
<fn fn-type="con" id="contrib001">
<p>Conceived and designed the experiments: BC US. Performed the experiments: AL. Analyzed the data: AL BC US. Wrote the paper: AL BC US.</p>
</fn>
<corresp id="cor001">* E-mail: <email xlink:type="simple">urmimala.sarkar@ucsf.edu</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>7</day>
<month>8</month>
<year>2015</year>
</pub-date>
<pub-date pub-type="collection">
<year>2015</year>
</pub-date>
<volume>10</volume>
<issue>8</issue>
<elocation-id>e0135072</elocation-id>
<history>
<date date-type="received">
<day>18</day>
<month>12</month>
<year>2014</year>
</date>
<date date-type="accepted">
<day>17</day>
<month>7</month>
<year>2015</year>
</date>
</history>
<permissions>
<copyright-year>2015</copyright-year>
<copyright-holder>Chan et al</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/" xlink:type="simple">
<license-p>This is an open access article distributed under the terms of the <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/" xlink:type="simple">Creative Commons Attribution License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:href="info:doi/10.1371/journal.pone.0135072" xlink:type="simple"/>
<abstract>
<sec id="sec001">
<title>Objective</title>
<p>Non-medical prescription opioid use is a growing public health concern. Social media is an emerging tool to understand health attitudes, beliefs, and behaviors.</p>
</sec>
<sec id="sec002">
<title>Methods</title>
<p>We retrieved a sample of publicly available Twitter messages in early 2014, using common opioid medication names and slang search terms. We used content analysis to code messages by user, context of message (personal vs general experiences), and key content themes.</p>
</sec>
<sec id="sec003">
<title>Results</title>
<p>We reviewed 540 messages, of which 375 (69%) messages were related to opioid behaviors. Of these, 316 (84%) originated from individual user accounts; 125 messages expressed personal experience with opioids. The majority of personal messages referenced using opioids to obtain a “high”, use for sleep, or other non-intended use (87,70%). General attitudes regarding opioid use included positive sentiment (52, 27%), comments on others peoples opioid use (57, 30%), and messages containing public health information or links (48, 25%).</p>
</sec>
<sec id="sec004">
<title>Conclusions</title>
<p>In a sample of social media messages mentioning opioid medications, the most common theme amongst English users related to various forms of opioid misuse. Social media can provide insights into the types of misuse of opioids that might aid public health efforts to reduce non-medical opioid use.</p>
</sec>
</abstract>
<funding-group>
<funding-statement>This work was supported by The National Cancer Institute (<ext-link ext-link-type="uri" xlink:href="http://www.cancer.gov/" xlink:type="simple">http://www.cancer.gov/</ext-link>) R01CA178875 Ruth L Kirschstein National Research Service Award (<ext-link ext-link-type="uri" xlink:href="http://grants.nih.gov/training/nrsa.htm" xlink:type="simple">http://grants.nih.gov/training/nrsa.htm</ext-link>) T32HP19025. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</funding-statement>
</funding-group>
<counts>
<fig-count count="3"/>
<table-count count="2"/>
<page-count count="10"/>
</counts>
<custom-meta-group>
<custom-meta id="data-availability" xlink:type="simple">
<meta-name>Data Availability</meta-name>
<meta-value>Data were obtained by searching Twitter manually using the search terms provided in the manuscript.</meta-value>
</custom-meta>
</custom-meta-group>
</article-meta>
</front>
<body>
<sec id="sec005" sec-type="intro">
<title>Introduction</title>
<p>Non-medical prescription opioid use is a rising public health concern. Prescriptions for opioid medications have nearly doubled between 1994 and 2007.[<xref rid="pone.0135072.ref001" ref-type="bibr">1</xref>] The burden of illness is high; opioid prescription related deaths (14,800) made up 73.8% of the 20,044 prescriptions drug overdose deaths in 2008.[<xref rid="pone.0135072.ref002" ref-type="bibr">2</xref>] The prevalence of patients who report a non-physician source of opioid medicine is reported to be more than double (69%) those who report a physician source (31%) in a recent analysis of the National Survey on Drug Use and Health.[<xref rid="pone.0135072.ref003" ref-type="bibr">3</xref>] With the concern over the growing epidemic of non-medical use of prescription opioids,[<xref rid="pone.0135072.ref004" ref-type="bibr">4</xref>, <xref rid="pone.0135072.ref005" ref-type="bibr">5</xref>] there is a need to understand the changing public attitudes and beliefs about these medications. The traditional methods of characterizing non-medical use, such as large government-funded telephone surveys, carry limitations including underreporting, and a significant delay between data collection and public data availability.[<xref rid="pone.0135072.ref006" ref-type="bibr">6</xref>, <xref rid="pone.0135072.ref007" ref-type="bibr">7</xref>]</p>
<p>Social media is a means to enhance real-time understanding of attitudes and beliefs regarding opioid use, because it provides an opportunity to share ideas, opinions, and information instantaneously and publicly online. The micro-blogging site Twitter has experienced rapid growth over time with an estimated 18% of US online adults using Twitter in 2013, and about half of those reporting daily use of the application.[<xref rid="pone.0135072.ref008" ref-type="bibr">8</xref>] Twitter permits communication with real-life social networks to be both public and online; therefore, personal information and potentially stigmatizing behaviors that individuals would previously share only with close contacts, such as substance abuse,[<xref rid="pone.0135072.ref009" ref-type="bibr">9</xref>] are often available for research and to inform public health efforts and policy.</p>
<p>We conducted an exploratory, mixed-methods analysis of Twitter messages to characterize the nature of the content relating to opioid medications. In this study, we aimed to describe who uses social media to discuss opioid use or misuse, and what attitudes, themes, and behaviors social media users message about. Our hypothesis was that the anonymity of social media allowed for more candid discussion of opioid misuse and abuse behaviors.</p>
</sec>
<sec id="sec006" sec-type="materials|methods">
<title>Methods</title>
<sec id="sec007">
<title>Study Setting: Twitter</title>
<p>Twitter (<ext-link ext-link-type="uri" xlink:href="http://www.twitter.com" xlink:type="simple">www.twitter.com</ext-link>) is an online social networking site that allows individuals to share information in short messages called “tweets” that are 140 characters or less. Twitter is largely a public forum, and accounts can be individual (from friends and family to celebrities and politicians) or organizational (including national and local non-profit organizations, companies, advocacy groups, and others) to media outlets posting real-time updates. Users form personalized networks by following the “feed” (message stream) of other user accounts, creating a timeline of personalized news and information (<xref rid="pone.0135072.g001" ref-type="fig">Fig 1</xref>). Similarly, users gain influence by having “followers” who receive their tweets. Although the text of each message is limited, the content shared is often rich, especially through links to longer stories, entire websites, pictures, and/or videos.</p>
<fig id="pone.0135072.g001" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0135072.g001</object-id>
<label>Fig 1</label>
<caption>
<title>An overview of Twitter: re-created example using search term “hydros”.</title>
</caption>
<graphic mimetype="image" xlink:href="info:doi/10.1371/journal.pone.0135072.g001" position="float" xlink:type="simple"/>
</fig>
</sec>
<sec id="sec008">
<title>Data Collection</title>
<p>We evaluated the content of a cross-sectional sample of publicly available Twitter messages during a 2 week period in March and early April 2014. We did not query Twitter’s API, but instead manually searched Twitter messages using the following search terms: 1) Duragesic, 2) Fentanyl 3) Hydrocodone, 4) Hydros, 5) Oxy, 6) Oxycodone, 7) Oxycotin, 8) Oxycotton, 9) Vicodin, 10) Vikes, and 11) Oxycontin. We identified search terms based on prior literature on the most commonly prescribed opioids, using generic, trade, and slang terms.[<xref rid="pone.0135072.ref010" ref-type="bibr">10</xref>] We conducted searches during different days of the week and times of day to sample across potential time dependent variations in themes, similar to previous studies using Twitter.[<xref rid="pone.0135072.ref011" ref-type="bibr">11</xref>–<xref rid="pone.0135072.ref013" ref-type="bibr">13</xref>] “We used a consecutive sampling scheme to compile Twitter messages up to a maximum of 50 per search term. We used a 3<sup>rd</sup> party web-available program, tweetseeker.com (<ext-link ext-link-type="uri" xlink:href="http://www.tweetseeker.com" xlink:type="simple">http://www.tweetseeker.com</ext-link>) to export our searches into Excel.</p>
</sec>
<sec id="sec009">
<title>Qualitative Coding</title>
<p>We initially used content analysis[<xref rid="pone.0135072.ref014" ref-type="bibr">14</xref>, <xref rid="pone.0135072.ref015" ref-type="bibr">15</xref>] to code each Twitter message. First, we developed codes deductively using commonly accepted definitions of prescription drug abuse[<xref rid="pone.0135072.ref016" ref-type="bibr">16</xref>] and aberrancy behaviors,[<xref rid="pone.0135072.ref017" ref-type="bibr">17</xref>] an existing opioid misuse tool, the Current Opioid Misuse Measure (COMM)[<xref rid="pone.0135072.ref018" ref-type="bibr">18</xref>], as well as a prior study of Twitter regarding Adderall use amongst college students.[<xref rid="pone.0135072.ref019" ref-type="bibr">19</xref>] We also developed codes based on prior qualitative studies of social media messages, including attitudes regarding breast and cervical cancer screening,[<xref rid="pone.0135072.ref020" ref-type="bibr">20</xref>] physician office experiences,[<xref rid="pone.0135072.ref012" ref-type="bibr">12</xref>] and tobacco product use.[<xref rid="pone.0135072.ref021" ref-type="bibr">21</xref>]</p>
<p>Next, we developed codes inductively, through a grounded theory approach, as additional themes arose not captured by existing thematic coding schemes.[<xref rid="pone.0135072.ref022" ref-type="bibr">22</xref>] Because we were interested in user attitudes and behaviors, we focused on the themes derived from individual user messages. First, one member of the team (BC) examined a 20% sample of all messages to develop a coding framework that included five major themes for personal messages (those messages that reflected personal experience with opioid medication), and four major themes in general messages containing opioid search terms (<xref rid="pone.0135072.g002" ref-type="fig">Fig 2</xref>). Next, the research team (AL, BC, US) examined an additional 30% of all tweets and met to refine the overall coding framework that included author type, message category (personal versus general), and description of reported behavior or perception. For example, we coded the message “This Liquid Oxycodone Got me Leaning (smiling emoticon)” as from an individual author, blogging a personal statement about use, with content of the tweet relating to using medication to obtain a high. We resolved discrepancies by regular meetings and discussion and revised the codebook iteratively. Data were compiled in an Excel spreadsheet to directly compare categorization of tweets across coders. If new categories (including sub-categories) emerged, we changed the coding framework and re-analyzed the messages according to the new structure. Once no new categories or themes emerged, reaching thematic saturation, the coding scheme was finalized and one member of the team coded the remaining messages (AL), followed by a joint review of aggregated messages. Because we included a substantial number of messages in our sample, we report frequencies for our data below.</p>
<fig id="pone.0135072.g002" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0135072.g002</object-id>
<label>Fig 2</label>
<caption>
<title>Conceptual Framework for Categorizing Twitter Messages Containing Personal Experiences and General Perceptions.</title>
</caption>
<graphic mimetype="image" xlink:href="info:doi/10.1371/journal.pone.0135072.g002" position="float" xlink:type="simple"/>
</fig>
</sec>
</sec>
<sec id="sec010" sec-type="results">
<title>Results</title>
<p>Overall, we searched a total of 540 Twitter messages. Of these, we excluded 182 (33%) messages for the following reasons: non-English content (20, 4%), content referring to non-opioid use of search term (138, 26%), and inability to discern the context of the message in a meaningful manner (24, 4%). For example, 48 of 50 messages using the search term “Vikes”, a slang term for Vicodin, contained content relating to the Minnesota Vikings football team. We also excluded messages originating from users names that contained search terms but whose messages did not. We coded the remaining 375 messages.</p>
<p>Of the 375 messages, 333 were from unique user accounts (89%). Individual accounts made up the majority of authored messages (316, 84%), compared to organizations or news outlet sources (59, 16%). The distribution of author type varied depending on the search term (<xref rid="pone.0135072.g003" ref-type="fig">Fig 3</xref>). The majority of messages authored by individuals were general comments on opioid use (191, 60%), and included 65 “retweets,” messages that are seen by a Twitter users “followers” and subsequently re-sent on their own network. There were 125 (40%) messages reflecting personal experience.</p>
<fig id="pone.0135072.g003" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0135072.g003</object-id>
<label>Fig 3</label>
<caption>
<title>Author of Twitter Messages (individuals, organizations, news outlets, other (user name contained search terms, foreign languages, references to non-opioids).</title>
</caption>
<graphic mimetype="image" xlink:href="info:doi/10.1371/journal.pone.0135072.g003" position="float" xlink:type="simple"/>
</fig>
<sec id="sec011">
<title>Personal experiences with opioid use by Twitter users</title>
<p>The majority of Twitter messages of a personal nature contained references to aberrant and opioid misuse behaviors (<xref rid="pone.0135072.t001" ref-type="table">Table 1</xref>). Of the 125 personal messages, 87 (70%) contained themes of opioid prescription misuse, or aberrant behaviors Twitter users were candid in messaging about using opioids to obtain a “high” (26, 21%). An example is a user who messaged “Bought some <bold>oxycontin</bold>, we finna get trippy mane” (sic). Co-use with other substances was explicitly mentioned in 16 (13%) of personal messages, (eg. “I’m eating shrooms, popping <bold>oxycontin</bold>”). A similar number of messages contained themes of seeking to obtain opioid medication (16, 13%).</p>
<p>Other personal themes included sharing one’s pain regimen with other followers without mention of aberrant or misuse behavior (26, 21%). For example, one user messaged “On more antibiotics, ibuprofen, and <bold>vicodin</bold>, with a scheduled wisdom tooth extraction in May. yay!” An equal number of messages contained mention of side effects of opioid medications (16, 13%), with a representative message “…I usually skip the <bold>hydrocodone</bold> because that’s insta-barf medicine for me.” A small percentage of personal messages referenced addiction to opioid medications (5, 4%).</p>
<table-wrap id="pone.0135072.t001" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0135072.t001</object-id>
<label>Table 1</label> <caption><title>Major opioid related content and attitudes contained in Personal Twitter messages, with representative quotes and frequencies.</title> <p>(n = 125).</p></caption>
<alternatives>
<graphic id="pone.0135072.t001g" position="float" mimetype="image" xlink:href="info:doi/10.1371/journal.pone.0135072.t001" xlink:type="simple"/>
<table>
<colgroup span="1">
<col align="left" valign="middle" span="1"/>
<col align="left" valign="middle" span="1"/>
<col align="left" valign="middle" span="1"/>
</colgroup>
<thead>
<tr>
<th align="left" rowspan="1" colspan="1">Theme:</th>
<th align="left" rowspan="1" colspan="1">Representative Message:</th>
<th align="left" rowspan="1" colspan="1">Frequency # (%):</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Aberrant/misuse behaviors</bold></td>
<td align="left" rowspan="1" colspan="1"/>
<td align="left" rowspan="1" colspan="1"/>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Use of opioids for sleep<xref rid="t001fn001" ref-type="table-fn">*</xref></td>
<td align="left" rowspan="1" colspan="1">2 hydros and back to sleep (;</td>
<td align="left" rowspan="1" colspan="1">11 (9%)</td>
</tr>
<tr>
<td rowspan="2" align="left" colspan="1">Co-Use<xref rid="t001fn001" ref-type="table-fn">*</xref><xref rid="t001fn002" ref-type="table-fn"><sup>a</sup></xref></td>
<td align="left" rowspan="1" colspan="1">Vicodin and vodka get me in my drake like a champ</td>
<td rowspan="2" align="left" colspan="1">16 (13%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">2 hydros, 40oz, and Mary</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Seeking<xref rid="t001fn001" ref-type="table-fn">*</xref></td>
<td align="left" rowspan="1" colspan="1">Anyone know where I can get some Hydrocodone's? Hmu</td>
<td align="left" rowspan="1" colspan="1">16 (13%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Selling</td>
<td align="left" rowspan="1" colspan="1">Don’t know if I should sell all these hydrocodone pills hmmm</td>
<td align="left" rowspan="1" colspan="1">2 (2%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Improper Administration<xref rid="t001fn001" ref-type="table-fn">*</xref></td>
<td align="left" rowspan="1" colspan="1">You need to not brag about popping vicodin/oxy if you dont even know how to cold water extract the good shit away from the fillers and APAP</td>
<td align="left" rowspan="1" colspan="1">3 (2%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Using other’s medicine <xref rid="t001fn001" ref-type="table-fn">*</xref></td>
<td align="left" rowspan="1" colspan="1">Thanks. Co-worker slipped me something and, hot damn, it seems to be working. May be Allegra. May be Vicodin. Either way: D</td>
<td align="left" rowspan="1" colspan="1">2 (2%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Poor Response To Medication</td>
<td align="left" rowspan="1" colspan="1">@XXX I'm on Vicodin but it doesn't do shit but make me sleep lol</td>
<td align="left" rowspan="1" colspan="1">11 (9%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Mention of using opiates to obtain “high”</bold><xref rid="t001fn001" ref-type="table-fn">*</xref></td>
<td align="left" rowspan="1" colspan="1">reoccurring hydrocodone dreams that i'm too fucked up to walk forward and i keep falling backwards and spinning and shit</td>
<td align="left" rowspan="1" colspan="1">26 (21%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Side Effects</bold><xref rid="t001fn002" ref-type="table-fn"><sup><bold>a</bold></sup></xref></td>
<td align="left" rowspan="1" colspan="1">I took one of my (prescribed) hydros once and fought the sleep cause i had shit to do…wasnt fun at allDruggies are weird</td>
<td align="left" rowspan="1" colspan="1">16 (13%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Sharing personal regimen</bold></td>
<td align="left" rowspan="1" colspan="1">How I get just slightly enough pain away to not be in bed all day like I'm depressed: 45mgs Oxycodone, 6 Advils, high grade MMJ.</td>
<td align="left" rowspan="1" colspan="1">26 (21%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Addiction</bold></td>
<td align="left" rowspan="1" colspan="1">Ever since I stuck Fentanyl patches on and took tramset (I fink) tablets I been addicted 2 painkillers lol</td>
<td align="left" rowspan="1" colspan="1">5 (4%)</td>
</tr>
</tbody>
</table>
</alternatives>
<table-wrap-foot>
<fn id="t001fn001"><p>* Coding theme based on the Current Opioid Misuse Measure (COMM)</p></fn>
<fn id="t001fn002"><p><sup>a</sup> Coding theme based on prior study of Twitter and Adderall themes</p></fn>
</table-wrap-foot>
</table-wrap>
</sec>
<sec id="sec012">
<title>General Perceptions of opioid use by Twitter users</title>
<p>Many messages from personal accounts (191, 60%) did not refer to personal experiences but to more general perceptions of opioid use (<xref rid="pone.0135072.t002" ref-type="table">Table 2</xref>). Of these messages, we noted 52 (27%) as expressing a positive sentiment toward opioids (eg. “<bold>Hydros</bold> fix everything”), compared to 15 (8%) of messages expressing a negative sentiment toward opioid use (eg. “I don’t understand the culture of young people who think its okay to pop Molly’s, Percocet, <bold>Oxycotin</bold>, Xanax, etc.”) A number of Twitter messages contained commentary on inappropriate use of opioids by others in their social circle (57, 30%). There were 30 (16%) messages that referenced opioid use in entertainment media, music lyrics containing opioid use, or public figures. A number of messages from individuals touched on public health awareness or posted links to media stories on opioid related awareness (eg. “Crazy documentary on the grip prescription drugs have on our society. topdocumentaryfilms.com/<bold>oxycontin</bold>-expr…”)</p>
<table-wrap id="pone.0135072.t002" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0135072.t002</object-id>
<label>Table 2</label> <caption><title>Major opioid related content and attitudes contained in General Twitter messages, with representative quotes and frequencies.</title> <p>(n = 191 includes 65 re-tweets).</p></caption>
<alternatives>
<graphic id="pone.0135072.t002g" position="float" mimetype="image" xlink:href="info:doi/10.1371/journal.pone.0135072.t002" xlink:type="simple"/>
<table>
<colgroup span="1">
<col align="left" valign="middle" span="1"/>
<col align="left" valign="middle" span="1"/>
<col align="left" valign="middle" span="1"/>
</colgroup>
<thead>
<tr>
<th align="left" rowspan="1" colspan="1">Theme:</th>
<th align="left" rowspan="1" colspan="1">Representative Message:</th>
<th align="left" rowspan="1" colspan="1">Frequency # (%):</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Public Health Awareness Statement</bold></td>
<td align="left" rowspan="1" colspan="1">Before you reach for a Hydrocodone, read this: bit.ly/1fiNVuO</td>
<td align="left" rowspan="1" colspan="1">48 (25%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Reference to Entertainment (Music, Rap, TV, Movies)</bold></td>
<td align="left" rowspan="1" colspan="1">Popping Vicodin like my names Dr. House</td>
<td align="left" rowspan="1" colspan="1">30 (16%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Negative Sentiment toward Opioid</bold></td>
<td align="left" rowspan="1" colspan="1">I don’t understand the culture of young people who think it’s okay to pop Molly’s, Percocet, Oxycotin, Xanax, etc.</td>
<td align="left" rowspan="1" colspan="1">15 (8%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Positive Sentiment toward Opioid</bold></td>
<td align="left" rowspan="1" colspan="1">@XXX @XXX hydros fix everything</td>
<td align="left" rowspan="1" colspan="1">52 (27%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Comment on other’s use</bold></td>
<td align="left" rowspan="1" colspan="1">My psyc. Teacher just said the cure for anything is a hydrocodone and a shot of whiskey</td>
<td align="left" rowspan="1" colspan="1">57 (30%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Medical Setting</bold></td>
<td align="left" rowspan="1" colspan="1">RT @XXXmd: @XXXmd I don’t have a document, but in our ED, almost everyone tubed gets ASAP a midazolam and fentanyl drip</td>
<td align="left" rowspan="1" colspan="1">19 (10%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Retweet of other’s message or link</bold></td>
<td align="left" rowspan="1" colspan="1">RT @XXXNews: Addicts abusing Fentanyl don’t realize how their addiction affects their family. fentanylabuse.org 800-303-2482 httpic.twitter.com/QRt1jMWl1E</td>
<td align="left" rowspan="1" colspan="1">65 (34%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"><bold>Others</bold></td>
<td align="left" rowspan="1" colspan="1"/>
<td align="left" rowspan="1" colspan="1"/>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"> - question to others</td>
<td align="left" rowspan="1" colspan="1">What u gata do to be on oxycotton?</td>
<td align="left" rowspan="1" colspan="1">2 (1%)</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1"> - unclear context</td>
<td align="left" rowspan="1" colspan="1">@o_XXX #oxycotton</td>
<td align="left" rowspan="1" colspan="1">5 (3%)</td>
</tr>
</tbody>
</table>
</alternatives>
</table-wrap>
</sec>
<sec id="sec013">
<title>Inclusion of Slang Terms, Misspelled Search Terms</title>
<p>Of 123 messages that included slang or misspelled terms, 55 messages were coded as containing reference to misuse/abuse behavior or attitudes (45%). This is compared to 53 of 252 (21%) of messages that included trade or generic names of opioids.</p>
</sec>
</sec>
<sec id="sec014" sec-type="conclusions">
<title>Discussion</title>
<p>In this qualitative analysis of self-reported attitudes and behaviors about opioids online, we found that Twitter users publicly message both personal experience with opioid medications, and general perceptions regarding opioid medications, including content related to non-medical use of opioid medications. While Twitter messages should not replace traditional survey methods of opioid use, the content contained in social media messages can be a useful, real-time, publicly-available source of information to be used in conjunction to identify emerging trends in use and misuse behavior, as well as a forum to communicate public health awareness on non-medical use of opioids.</p>
<p>The personal experience messages captured a wide variety of opioid related behaviors and are consistent with reported behaviors found in other survey methods about misuse and abuse of opioid medications.[<xref rid="pone.0135072.ref003" ref-type="bibr">3</xref>] Authors messaged about their aberrant use of opioids as a sleep or anti-anxiolytic, or as a way to obtain a high sensation. Similarly, authors shared openly messages about co-use of opioids with other medications, illegal drugs, or alcohol. Although we cannot quantify the magnitude of positive opioid-related sentiment based on our manual sample, the high frequency of messages that display aberrant opioid behaviors in a positive light are deeply concerning, and suggest that the public still lacks sufficient awareness of the mounting harm associated with opioid misuse. Twitter may be useful in the future for tracking public awareness of opioid misuse.</p>
<p>These messages also provide data for public health practitioners in real-time, providing data that is complementary to traditional methods. Characterizing risky behaviors through health surveys or adverse events is subject to limitations in response rates and significant time delays.[<xref rid="pone.0135072.ref007" ref-type="bibr">7</xref>] In-clinic patient reported screening tools are limited by recall bias, social-desirability bias, and concerns about receiving medication.[<xref rid="pone.0135072.ref023" ref-type="bibr">23</xref>, <xref rid="pone.0135072.ref024" ref-type="bibr">24</xref>] The real-time nature of Twitter messaging, as well as the relative anonymity of Twitter, can complement data from current surveys and reporting systems.</p>
<p>Our findings are similar to studies that used Twitter to characterize potential misuse of other prescribed medications, like the use of Adderall as a study aid.[<xref rid="pone.0135072.ref019" ref-type="bibr">19</xref>] Prior studies report Twitter as a means for studying behavior with other health related behaviors such as smoking and drinking alcohol.[<xref rid="pone.0135072.ref021" ref-type="bibr">21</xref>, <xref rid="pone.0135072.ref025" ref-type="bibr">25</xref>] These prior studies have all used automated algorithms to study text data from twitter, while we used traditional qualitative analysis, which permitted some additional insight into each message that automated “big data” analyses cannot provide. As an example, the initial automated search terms in our sample did not result in a precise identification of opioid-related messages, and we restricted our analysis to pertinent messages. There is promise in combining traditional qualitative analysis with automated approaches, such an approach led to greater precision in a study of online doctor ratings.[<xref rid="pone.0135072.ref026" ref-type="bibr">26</xref>]</p>
<p>Our results reveal several lessons in future use of social media data sources for opioid misuse surveillance. First, themes related to personal experience with opioids differ from themes related to general perceptions of opioids, the latter which is influenced by one’s social network, pop culture, news, and entertainment sources. If actual behaviors are to be studied, future search algorithms will need to differentiate personal from general experiences. Second, searching of slang terms (“oxy”) or misspelled opioid search terms (“oxycotin”) yielded informative messages that referenced opioid use, misuse, and abuse. Design of social media surveillance programs need to consider how to include misspellings or slang terms into their algorithms to improve effectiveness.”</p>
<p>In addition to potential use as a surveillance tool, we also found that Twitter can be a source of information for the public. Organizations tended to message about the potential risks and harms of opioid medications, or retweeted new stories about criminal activity involving opioid medications. Prior use of twitter during political movements underscores the immediacy of the medium,[<xref rid="pone.0135072.ref027" ref-type="bibr">27</xref>] and we believe this is an advantage for public health applications as well.</p>
<sec id="sec015">
<title>Limitations</title>
<p>This exploratory analysis has several limitations. By necessity, a manual qualitative analysis has a limited sample size and scope. We cannot speak to the representativeness of the messages we sampled, though we did attempt to include different times of day, days of the week, as well as scientific, trade, and slang terms. We pre-specified the search terms based on a preliminary review of social media related to prescription opioid use. These terms were subject to “noise” from other non-opioid related messages. Future work should examine additional keywords and messages over a longer period of time to target specific behavior or content. We acknowledge that while Twitter use amongst adults is growing, people who message about opioid use are different from the general population, and our findings should be interpreted in that light. However, a recent Twitter based surveillance study have produced similar findings to traditional survey designs in characterizing Adderall abuse, [<xref rid="pone.0135072.ref019" ref-type="bibr">19</xref>] and we believe the advantages in cost and immediacy of data availability confer significant advantages over traditional methods to study high-risk behaviors.</p>
</sec>
</sec>
<sec id="sec016" sec-type="conclusions">
<title>Conclusion</title>
<p>Social media has changed the way individuals communicate about health-related behaviors, including behaviors that are illicit and difficult to capture otherwise. A next step is combining results from a qualitative analysis with larger automated data in order to develop search strategies to monitor opioid use and misuse behaviors that can be compared to gold standard surveillance systems currently in place. In addition, user location information provided on messages may offer additional information to identify incipient public health concerns. Twitter has potential to complement existing public health approaches to characterize opioid medication misuse and to develop effective public health campaigns to address this growing epidemic.</p>
</sec>
</body>
<back>
<ack>
<p>We thank Cassidy Clarity for her assistance with formatting and submission of the manuscript.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="pone.0135072.ref001"><label>1</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Volkow</surname> <given-names>ND</given-names></name>, <name name-style="western"><surname>McLellan</surname> <given-names>TA</given-names></name>, <name name-style="western"><surname>Cotto</surname> <given-names>JH</given-names></name>, <name name-style="western"><surname>Karithanom</surname> <given-names>M</given-names></name>, <name name-style="western"><surname>Weiss</surname> <given-names>SR</given-names></name>. <article-title>Characteristics of opioid prescriptions in 2009</article-title>. <source>JAMA: the journal of the American Medical Association</source>. <year>2011</year>;<volume>305</volume>(<issue>13</issue>):<fpage>1299</fpage>–<lpage>301</lpage>. Epub 2011/04/07. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1001/jama.2011.401" xlink:type="simple">10.1001/jama.2011.401</ext-link></comment> <object-id pub-id-type="pmid">21467282</object-id>; PubMed Central PMCID: PMC3187622.</mixed-citation></ref>
<ref id="pone.0135072.ref002"><label>2</label><mixed-citation publication-type="journal" xlink:type="simple"><collab xlink:type="simple">Centers for Disease Control, Prevention</collab>. <article-title>Vital signs: overdoses of prescription opioid pain relievers---United States, 1999–2008</article-title>. <source>MMWR Morbidity and mortality weekly report</source>. <year>2011</year>;<volume>60</volume>(<issue>43</issue>):<fpage>1487</fpage>–<lpage>92</lpage>. Epub 2011/11/04. <object-id pub-id-type="pmid">22048730</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref003"><label>3</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Becker</surname> <given-names>WC</given-names></name>, <name name-style="western"><surname>Tobin</surname> <given-names>DG</given-names></name>, <name name-style="western"><surname>Fiellin</surname> <given-names>DA</given-names></name>. <article-title>Nonmedical use of opioid analgesics obtained directly from physicians: prevalence and correlates</article-title>. <source>Archives of internal medicine</source>. <year>2011</year>;<volume>171</volume>(<issue>11</issue>):<fpage>1034</fpage>–<lpage>6</lpage>. Epub 2011/06/15. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1001/archinternmed.2011.217" xlink:type="simple">10.1001/archinternmed.2011.217</ext-link></comment> <object-id pub-id-type="pmid">21670373</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref004"><label>4</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Volkow</surname> <given-names>ND</given-names></name>, <name name-style="western"><surname>Frieden</surname> <given-names>TR</given-names></name>, <name name-style="western"><surname>Hyde</surname> <given-names>PS</given-names></name>, <name name-style="western"><surname>Cha</surname> <given-names>SS</given-names></name>. <article-title>Medication-assisted therapies—tackling the opioid-overdose epidemic</article-title>. <source>The New England journal of medicine</source>. <year>2014</year>;<volume>370</volume>(<issue>22</issue>):<fpage>2063</fpage>–<lpage>6</lpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1056/NEJMp1402780" xlink:type="simple">10.1056/NEJMp1402780</ext-link></comment> <object-id pub-id-type="pmid">24758595</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref005"><label>5</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Alexander</surname> <given-names>GC</given-names></name>, <name name-style="western"><surname>Kruszewski</surname> <given-names>SP</given-names></name>, <name name-style="western"><surname>Webster</surname> <given-names>DW</given-names></name>. <article-title>Rethinking opioid prescribing to protect patient safety and public health</article-title>. <source>JAMA: the journal of the American Medical Association</source>. <year>2012</year>;<volume>308</volume>(<issue>18</issue>):<fpage>1865</fpage>–<lpage>6</lpage>. Epub 2012/11/15. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1001/jama.2012.14282" xlink:type="simple">10.1001/jama.2012.14282</ext-link></comment> <object-id pub-id-type="pmid">23150006</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref006"><label>6</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Freifeld</surname> <given-names>CC</given-names></name>, <name name-style="western"><surname>Brownstein</surname> <given-names>JS</given-names></name>, <name name-style="western"><surname>Menone</surname> <given-names>CM</given-names></name>, <name name-style="western"><surname>Bao</surname> <given-names>W</given-names></name>, <name name-style="western"><surname>Filice</surname> <given-names>R</given-names></name>, <name name-style="western"><surname>Kass-Hout</surname> <given-names>T</given-names></name>, <etal>et al</etal>. <article-title>Digital drug safety surveillance: monitoring pharmaceutical products in twitter</article-title>. <source>Drug safety: an international journal of medical toxicology and drug experience</source>. <year>2014</year>;<volume>37</volume>(<issue>5</issue>):<fpage>343</fpage>–<lpage>50</lpage>. Epub 2014/04/30. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/s40264-014-0155-x" xlink:type="simple">10.1007/s40264-014-0155-x</ext-link></comment> <object-id pub-id-type="pmid">24777653</object-id>; PubMed Central PMCID: PMC4013443.</mixed-citation></ref>
<ref id="pone.0135072.ref007"><label>7</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Roehr</surname> <given-names>B</given-names></name>. <article-title>US hospital incident reporting systems do not capture most adverse events</article-title>. <source>Bmj</source>. <year>2012</year>;<volume>344</volume>:<fpage>e386</fpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1136/bmj.e386" xlink:type="simple">10.1136/bmj.e386</ext-link></comment> <object-id pub-id-type="pmid">22246454</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref008"><label>8</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Duggan</surname> <given-names>M SA</given-names></name>. <article-title>Social Media Update 2013</article-title>. <source>Pew Research Internet Project</source>. <year>2013</year>.</mixed-citation></ref>
<ref id="pone.0135072.ref009"><label>9</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Moreno</surname> <given-names>MA</given-names></name>, <name name-style="western"><surname>Egan</surname> <given-names>KG</given-names></name>, <name name-style="western"><surname>Brockman</surname> <given-names>L</given-names></name>. <article-title>Development of a researcher codebook for use in evaluating social networking site profiles</article-title>. <source>The Journal of adolescent health: official publication of the Society for Adolescent Medicine</source>. <year>2011</year>;<volume>49</volume>(<issue>1</issue>):<fpage>29</fpage>–<lpage>35</lpage>. Epub 2011/06/28. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.jadohealth.2011.04.015" xlink:type="simple">10.1016/j.jadohealth.2011.04.015</ext-link></comment> <object-id pub-id-type="pmid">21700153</object-id>; PubMed Central PMCID: PMC3125574.</mixed-citation></ref>
<ref id="pone.0135072.ref010"><label>10</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Manchikanti</surname> <given-names>L</given-names></name>, <name name-style="western"><surname>Helm</surname> <given-names>S</given-names> <suffix>2nd</suffix></name>, <name name-style="western"><surname>Fellows</surname> <given-names>B</given-names></name>, <name name-style="western"><surname>Janata</surname> <given-names>JW</given-names></name>, <name name-style="western"><surname>Pampati</surname> <given-names>V</given-names></name>, <name name-style="western"><surname>Grider</surname> <given-names>JS</given-names></name>, <etal>et al</etal>. <article-title>Opioid epidemic in the United States</article-title>. <source>Pain physician</source>. <year>2012</year>;<volume>15</volume>(<issue>3 Suppl</issue>):<fpage>ES9</fpage>–<lpage>38</lpage>. Epub 2012/07/20. <object-id pub-id-type="pmid">22786464</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref011"><label>11</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Lagu</surname> <given-names>T</given-names></name>, <name name-style="western"><surname>Hannon</surname> <given-names>NS</given-names></name>, <name name-style="western"><surname>Rothberg</surname> <given-names>MB</given-names></name>, <name name-style="western"><surname>Lindenauer</surname> <given-names>PK</given-names></name>. <article-title>Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites</article-title>. <source>Journal of general internal medicine</source>. <year>2010</year>;<volume>25</volume>(<issue>9</issue>):<fpage>942</fpage>–<lpage>6</lpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/s11606-010-1383-0" xlink:type="simple">10.1007/s11606-010-1383-0</ext-link></comment> <object-id pub-id-type="pmid">20464523</object-id>; PubMed Central PMCID: PMC2917672.</mixed-citation></ref>
<ref id="pone.0135072.ref012"><label>12</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Lopez</surname> <given-names>A</given-names></name>, <name name-style="western"><surname>Detz</surname> <given-names>A</given-names></name>, <name name-style="western"><surname>Ratanawongsa</surname> <given-names>N</given-names></name>, <name name-style="western"><surname>Sarkar</surname> <given-names>U</given-names></name>. <article-title>What patients say about their doctors online: a qualitative content analysis</article-title>. <source>Journal of general internal medicine</source>. <year>2012</year>;<volume>27</volume>(<issue>6</issue>):<fpage>685</fpage>–<lpage>92</lpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/s11606-011-1958-4" xlink:type="simple">10.1007/s11606-011-1958-4</ext-link></comment> <object-id pub-id-type="pmid">22215270</object-id>; PubMed Central PMCID: PMC3358396.</mixed-citation></ref>
<ref id="pone.0135072.ref013"><label>13</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Kim</surname> <given-names>AE</given-names></name>, <name name-style="western"><surname>Hansen</surname> <given-names>HM</given-names></name>, <name name-style="western"><surname>Murphy</surname> <given-names>J</given-names></name>, <name name-style="western"><surname>Richards</surname> <given-names>AK</given-names></name>, <name name-style="western"><surname>Duke</surname> <given-names>J</given-names></name>, <name name-style="western"><surname>Allen</surname> <given-names>JA</given-names></name>. <article-title>Methodological considerations in analyzing twitter data</article-title>. <source>Journal of the National Cancer Institute Monographs</source>. <year>2013</year>;<volume>2013</volume>(<issue>47</issue>):<fpage>140</fpage>–<lpage>6</lpage>. Epub 2014/01/08. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/jncimonographs/lgt026" xlink:type="simple">10.1093/jncimonographs/lgt026</ext-link></comment> <object-id pub-id-type="pmid">24395983</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref014"><label>14</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Borzekowski</surname> <given-names>DL</given-names></name>, <name name-style="western"><surname>Schenk</surname> <given-names>S</given-names></name>, <name name-style="western"><surname>Wilson</surname> <given-names>JL</given-names></name>, <name name-style="western"><surname>Peebles</surname> <given-names>R</given-names></name>. <article-title>e-Ana and e-Mia: A content analysis of pro-eating disorder Web sites</article-title>. <source>Am J Public Health</source>. <year>2010</year>;<volume>100</volume>(<issue>8</issue>):<fpage>1526</fpage>–<lpage>34</lpage>. Epub 2010/06/19. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.2105/AJPH.2009.172700" xlink:type="simple">10.2105/AJPH.2009.172700</ext-link></comment> <object-id pub-id-type="pmid">20558807</object-id>; PubMed Central PMCID: PMC2901299.</mixed-citation></ref>
<ref id="pone.0135072.ref015"><label>15</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Jenssen</surname> <given-names>BP</given-names></name>, <name name-style="western"><surname>Klein</surname> <given-names>JD</given-names></name>, <name name-style="western"><surname>Salazar</surname> <given-names>LF</given-names></name>, <name name-style="western"><surname>Daluga</surname> <given-names>NA</given-names></name>, <name name-style="western"><surname>DiClemente</surname> <given-names>RJ</given-names></name>. <article-title>Exposure to tobacco on the internet: content analysis of adolescents' internet use</article-title>. <source>Pediatrics</source>. <year>2009</year>;<volume>124</volume>(<issue>2</issue>):<fpage>e180</fpage>–<lpage>6</lpage>. Epub 2009/07/22. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1542/peds.2008-3838" xlink:type="simple">10.1542/peds.2008-3838</ext-link></comment> <object-id pub-id-type="pmid">19620193</object-id>; PubMed Central PMCID: PMC2818533.</mixed-citation></ref>
<ref id="pone.0135072.ref016"><label>16</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Chabal</surname> <given-names>C</given-names></name>, <name name-style="western"><surname>Erjavec</surname> <given-names>MK</given-names></name>, <name name-style="western"><surname>Jacobson</surname> <given-names>L</given-names></name>, <name name-style="western"><surname>Mariano</surname> <given-names>A</given-names></name>, <name name-style="western"><surname>Chaney</surname> <given-names>E</given-names></name>. <article-title>Prescription opiate abuse in chronic pain patients: clinical criteria, incidence, and predictors</article-title>. <source>The Clinical journal of pain</source>. <year>1997</year>;<volume>13</volume>(<issue>2</issue>):<fpage>150</fpage>–<lpage>5</lpage>. <object-id pub-id-type="pmid">9186022</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref017"><label>17</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Wachholtz</surname> <given-names>A</given-names></name>, <name name-style="western"><surname>Gonzalez</surname> <given-names>G</given-names></name>, <name name-style="western"><surname>Boyer</surname> <given-names>E</given-names></name>, <name name-style="western"><surname>Naqvi</surname> <given-names>ZN</given-names></name>, <name name-style="western"><surname>Rosenbaum</surname> <given-names>C</given-names></name>, <name name-style="western"><surname>Ziedonis</surname> <given-names>D</given-names></name>. <article-title>Intersection of chronic pain treatment and opioid analgesic misuse: causes, treatments, and policy strategies</article-title>. <source>Substance abuse and rehabilitation</source>. <year>2011</year>;<volume>2</volume>:<fpage>145</fpage>–<lpage>62</lpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.2147/SAR.S12944" xlink:type="simple">10.2147/SAR.S12944</ext-link></comment> <object-id pub-id-type="pmid">24474854</object-id>; PubMed Central PMCID: PMC3846312.</mixed-citation></ref>
<ref id="pone.0135072.ref018"><label>18</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Butler</surname> <given-names>SF</given-names></name>, <name name-style="western"><surname>Budman</surname> <given-names>SH</given-names></name>, <name name-style="western"><surname>Fernandez</surname> <given-names>KC</given-names></name>, <name name-style="western"><surname>Houle</surname> <given-names>B</given-names></name>, <name name-style="western"><surname>Benoit</surname> <given-names>C</given-names></name>, <name name-style="western"><surname>Katz</surname> <given-names>N</given-names></name>, <etal>et al</etal>. <article-title>Development and validation of the Current Opioid Misuse Measure</article-title>. <source>Pain</source>. <year>2007</year>;<volume>130</volume>(<issue>1–2</issue>):<fpage>144</fpage>–<lpage>56</lpage>. Epub 2007/05/12. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.pain.2007.01.014" xlink:type="simple">10.1016/j.pain.2007.01.014</ext-link></comment> <object-id pub-id-type="pmid">17493754</object-id>; PubMed Central PMCID: PMC1950245.</mixed-citation></ref>
<ref id="pone.0135072.ref019"><label>19</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Hanson</surname> <given-names>CL</given-names></name>, <name name-style="western"><surname>Burton</surname> <given-names>SH</given-names></name>, <name name-style="western"><surname>Giraud-Carrier</surname> <given-names>C</given-names></name>, <name name-style="western"><surname>West</surname> <given-names>JH</given-names></name>, <name name-style="western"><surname>Barnes</surname> <given-names>MD</given-names></name>, <name name-style="western"><surname>Hansen</surname> <given-names>B</given-names></name>. <article-title>Tweaking and tweeting: exploring Twitter for nonmedical use of a psychostimulant drug (Adderall) among college students</article-title>. <source>Journal of medical Internet research</source>. <year>2013</year>;<volume>15</volume>(<issue>4</issue>):<fpage>e62</fpage>. Epub 2013/04/19. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.2196/jmir.2503" xlink:type="simple">10.2196/jmir.2503</ext-link></comment> <object-id pub-id-type="pmid">23594933</object-id>; PubMed Central PMCID: PMC3636321.</mixed-citation></ref>
<ref id="pone.0135072.ref020"><label>20</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Lyles</surname> <given-names>CR</given-names></name>, <name name-style="western"><surname>Lopez</surname> <given-names>A</given-names></name>, <name name-style="western"><surname>Pasick</surname> <given-names>R</given-names></name>, <name name-style="western"><surname>Sarkar</surname> <given-names>U</given-names></name>. <article-title>"5 mins of uncomfyness is better than dealing with cancer 4 a lifetime": an exploratory qualitative analysis of cervical and breast cancer screening dialogue on Twitter</article-title>. <source>Journal of cancer education: the official journal of the American Association for Cancer Education</source>. <year>2013</year>;<volume>28</volume>(<issue>1</issue>):<fpage>127</fpage>–<lpage>33</lpage>. Epub 2012/11/08. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/s13187-012-0432-2" xlink:type="simple">10.1007/s13187-012-0432-2</ext-link></comment> <object-id pub-id-type="pmid">23132231</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref021"><label>21</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Myslin</surname> <given-names>M</given-names></name>, <name name-style="western"><surname>Zhu</surname> <given-names>SH</given-names></name>, <name name-style="western"><surname>Chapman</surname> <given-names>W</given-names></name>, <name name-style="western"><surname>Conway</surname> <given-names>M</given-names></name>. <article-title>Using twitter to examine smoking behavior and perceptions of emerging tobacco products</article-title>. <source>Journal of medical Internet research</source>. <year>2013</year>;<volume>15</volume>(<issue>8</issue>):<fpage>e174</fpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.2196/jmir.2534" xlink:type="simple">10.2196/jmir.2534</ext-link></comment> <object-id pub-id-type="pmid">23989137</object-id>; PubMed Central PMCID: PMC3758063.</mixed-citation></ref>
<ref id="pone.0135072.ref022"><label>22</label><mixed-citation publication-type="book" xlink:type="simple"><name name-style="western"><surname>Creswell</surname> <given-names>JW</given-names></name>, <name name-style="western"><surname>Creswell</surname> <given-names>JW</given-names></name>. <source>Qualitative inquiry and research design: choosing among five approaches</source>. <edition>3rd ed</edition>. <publisher-loc>Los Angeles</publisher-loc>: <publisher-name>SAGE Publications</publisher-name>; <year>2013</year>. <fpage>xxi</fpage>, <fpage>448</fpage> p. p.</mixed-citation></ref>
<ref id="pone.0135072.ref023"><label>23</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Fishbain</surname> <given-names>DA</given-names></name>, <name name-style="western"><surname>Cutler</surname> <given-names>RB</given-names></name>, <name name-style="western"><surname>Rosomoff</surname> <given-names>HL</given-names></name>, <name name-style="western"><surname>Rosomoff</surname> <given-names>RS</given-names></name>. <article-title>Validity of self-reported drug use in chronic pain patients</article-title>. <source>The Clinical journal of pain</source>. <year>1999</year>;<volume>15</volume>(<issue>3</issue>):<fpage>184</fpage>–<lpage>91</lpage>. <object-id pub-id-type="pmid">10524471</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref024"><label>24</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Becker</surname> <given-names>WC</given-names></name>, <name name-style="western"><surname>Fraenkel</surname> <given-names>L</given-names></name>, <name name-style="western"><surname>Edelman</surname> <given-names>EJ</given-names></name>, <name name-style="western"><surname>Holt</surname> <given-names>SR</given-names></name>, <name name-style="western"><surname>Glover</surname> <given-names>J</given-names></name>, <name name-style="western"><surname>Kerns</surname> <given-names>RD</given-names></name>, <etal>et al</etal>. <article-title>Instruments to assess patient-reported safety, efficacy, or misuse of current opioid therapy for chronic pain: a systematic review</article-title>. <source>Pain</source>. <year>2013</year>;<volume>154</volume>(<issue>6</issue>):<fpage>905</fpage>–<lpage>16</lpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.pain.2013.02.031" xlink:type="simple">10.1016/j.pain.2013.02.031</ext-link></comment> <object-id pub-id-type="pmid">23601625</object-id>; PubMed Central PMCID: PMC3683236.</mixed-citation></ref>
<ref id="pone.0135072.ref025"><label>25</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Cunningham</surname> <given-names>JA</given-names></name>. <article-title>Using Twitter to measure behavior patterns</article-title>. <source>Epidemiology</source>. <year>2012</year>;<volume>23</volume>(<issue>5</issue>):<fpage>764</fpage>–<lpage>5</lpage>. Epub 2012/08/09. <comment>doi: <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1097/EDE.0b013e3182625e5d" xlink:type="simple">10.1097/EDE.0b013e3182625e5d</ext-link></comment> <object-id pub-id-type="pmid">22872115</object-id>.</mixed-citation></ref>
<ref id="pone.0135072.ref026"><label>26</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Paul MW</surname> <given-names>BC</given-names></name>. <name name-style="western"><surname>Dredze</surname> <given-names>M</given-names></name>. <article-title>What Affects Patient (Dis)satisfaction? Analyzing Online Doctor Ratings with a Joint Topic-Sentiment Model</article-title>. <source>Association for the Advancement of Artificial Intelligence</source>. <year>2013</year>.</mixed-citation></ref>
<ref id="pone.0135072.ref027"><label>27</label><mixed-citation publication-type="journal" xlink:type="simple"><name name-style="western"><surname>Tobias</surname> <given-names>E</given-names></name>. <article-title>Using Twitter and other social media platforms to provide situational awareness during an incident</article-title>. <source>Journal of business continuity &amp; emergency planning</source>. <year>2011</year>;<volume>5</volume>(<issue>3</issue>):<fpage>208</fpage>–<lpage>23</lpage>. <object-id pub-id-type="pmid">22130339</object-id>.</mixed-citation></ref>
</ref-list>
</back>
</article>