Published on 02.09.13 in Vol 2, No 2 (2013): Jul-Dec
Development of a Questionnaire and Cross-Sectional Survey of Patient eHealth Readiness and eHealth Inequalities
Background: Many speak of the digital divide, but variation in the opportunity of patients to use the Internet for health (patient eHealth readiness) is not a binary difference, rather a distribution influenced by personal capability, provision of services, support, and cost. Digital divisions in health have been addressed by various initiatives, but there was no comprehensive validated measure to know if they are effective that could be used in randomized controlled trials (RCTs) covering both non-Internet-users and the range of Internet-users.
Objective: The aim of this study was to develop and validate a self-completed questionnaire and scoring system to assess patient eHealth readiness by examining the spread of scores and eHealth inequalities. The intended use of this questionnaire and scores is in RCTs of interventions aiming to improve patient eHealth readiness and reduce eHealth inequalities.
Methods: Based on four factors identified from the literature, a self-completed questionnaire, using a pragmatic combination of factual and attitude questions, was drafted and piloted in three stages. This was followed by a final population-based, cross-sectional household survey of 344 people used to refine the scoring system.
Results: The Patient eHealth Readiness Questionnaire (PERQ) includes questions used to calculate four subscores: patients’ perception of (1) provision, (2) their personal ability and confidence, (3) their interpersonal support, and (4) relative costs in using the Internet for health. These were combined into an overall PERQ score (0-9) which could be used in intervention studies. Reduction in standard deviation of the scores represents reduction in eHealth inequalities.
Conclusions: PERQ appears acceptable for participants in British studies. The scores produced appear valid and will enable assessment of the effectiveness of interventions to improve patient eHealth readiness and reduce eHealth inequalities. Such methods need continued evolution and redevelopment for other environments. Full documentation and data have been published to allow others to develop the tool further.
Med 2.0 2013;2(2):e9
Definitions and Literature
The term eHealth is used in various ways, some (eg, World Health Organization)  using it as an over-arching term incorporating health informatics, telehealth, e-learning, and mHealth, while others think of eHealth as a patient-centered subset of health informatics [ - ]. This paper uses the term “patient eHealth readiness” to refer to the opportunity of patients to use the Internet and apps for health, and eHealth inequalities to mean differences in patient eHealth readiness.
In developing the ideas for this study, literature was first reviewed in November 2010 and updated in May 2013 from Web of Knowledge, PubMed, and Google Scholar using the terms (1) [E-health OR ehealth OR telehealth* OR telemedicine OR (ICT AND health) OR (technology AND health <in topic>) AND (readiness OR preparedness OR (implementation AND measure*) <in title>], (2) E-health literacy, (3) (Digital divide OR digital inclusion OR digital exclusion OR e-health inequalities) AND health, and (4) Inequality AND measure <in title> AND health. Web of Knowledge was used to examine citations of this literature for further relevant studies.
Benefits From Patient eHealth Interventions
There is evidence that direct use of the Internet by patients can benefit patients . For example, systematic reviews show improvements in health-related knowledge, attitudes, intentions and behaviors [ , ], and reduced health service use [ , ]. Studies showing the benefits of patient eHealth interventions, however, are nearly always carried out on populations of Internet users and the effectiveness of any intervention may depend on the skills and opportunities of the population recruited [ ].
Barriers to Patient eHealth Opportunity
There are four domains of barriers to eHealth opportunity that were identified (1) provision of eHealth opportunity, (2) personal abilities of the patient, (3) the support from others they may have to use eHealth, and (4) economic barriers.
Provision of eHealth opportunity varies. For example, while some British general practices  provided information, repeat prescribing, appointment booking, online advice, and patient access to their medical records, other practices had no website [ ]. Internationally, many US practices use preconsultation computer-interviews [ ], but these are rarely used in Britain. In secondary care, most British renal patients have access to their renal medical records online [ ], but few stroke patients have such facility. Even use of globally available websites may show marked regional variation because of varied rates of recommendation to patients. For example, use of an online cognitive behavioral-therapy site for depression varied 30-fold by postcode area [ ].
Physical and psychological attributes of patients contribute to digital divisions in health. Someone may have problems from sight or hearing impairment, arthritis, or lack of mobility in their hands. They may have no prior experience or find it difficult to learn Internet use, have limited literacy or health literacy, or lack confidence either in their Internet use or in making decisions using health information. They may distrust the Internet . Someone’s current health may increase motivation to use the Internet for health [ ], but may restrict Internet use; 81% of those with no recent health problems had used the Internet compared to 65% with recent health problems [ ].
Some factors limiting personal use of eHealth may be diminished if people have support from others. For example, anonymous e-mail support may help people with long-term conditions use the Internet [, ] and volunteers may help older people start using the Internet [ ]. Without such support people may struggle to go online or make the best use of resources.
Finally, economic factors may affect digital divisions in health . Although homes may be capable of Internet connection, families may not be able to afford it. Someone relying on accessing the Internet at their local library may be restricted by transport costs. Some groups, such as those with substance use problems, may be particularly susceptible, and in times of economic recession, barriers to eHealth use may increase. In the United States, broadband use is clearly related to income with 43% of families with incomes between $15,000 and $25,000 compared to 86% of those with incomes between $100,000 and $149,000 having home broadband [ ]. However, with appropriate provision even the poorest can get access; a US study among homeless found that 47% reported computer use in the past month [ ]. Economic factors are relative to the cost of alternative actions in health.
Others have examined barriers to eHealth use and the eHealth readiness of organizations or health services [- ] through measures involving contact with staff or observation of process. The aim of this study was to develop a patient-completed tool giving patients’ perceptions of their opportunity that could be combined with their personal abilities, their support networks, and economic barriers.
Do Digital Divisions in Health Deserve Action?
Should governments or health services address digital divisions in health? Some argue that it is just a matter of time before everyone has Internet access and that digital divisions will disappear. Others remind us that in the diffusion of technology [, ], there are always earlier and later adopters, so there will always be inequalities. Others argue that as technology and eHealth progress, differences in opportunities for patients to use the Internet for health may increase, ultimately leading to worse health inequalities [ ]. Even without the ethical argument for addressing inequality, eHealth inequalities make the adoption of more cost-effective health delivery difficult. If health services must provide eHealth and more traditional services, this diversity of service provision may be expensive. The digital divide has received attention with British government promoted organizations such as Race Online 2012, national regular reporting of digital use [ , ], and other specialized reports [ ]. The current British government is committed to the idea that services should be “digital by default” [ ], which may impact on those without good Internet access or skills.
What Level of eHealth Inequality Is Important?
Like the seven-year difference in life expectancy by social class in England , the size of eHealth inequality needs to be large enough to be of concern. Some differences are binary; if houses in rural areas are not connected to the Internet, then those families cannot use eHealth. Other factors, such as eHealth literacy, will follow a distribution and we need to ask whether the standard deviation of that distribution is unacceptably large. In some cases relatively small differences are worth addressing if that can be done at low cost. A single numerical measure of eHealth inequality would help to judge the effectiveness of interventions.
How Have eHealth Inequalities Been Addressed?
Initially, physical access to eHealth received a good deal of attention. From the late 1980s, there were experiments with public access kiosks  and initiatives to make the Internet available in public libraries. In the United States, 95% of public libraries provided Internet access by the year 2000 [ ]. The third sector, through organizations such as Age UK, have provided both physical access and support using computers for older people [ ]. In the English National Health Service (NHS), NHS Choices had a social and digital inclusion team from 2007 to 2012 [ ], now lost in recent government cuts. There are no quality targets requiring NHS Trusts to provide eHealth services.
Various studies have addressed eHealth inequalities or tried to ameliorate their impact. For example, Kerr et al  explored the effectiveness of a web-based intervention in decreasing inequalities in access to self-management support in patients with coronary heart disease. Jones et al piloted anonymous personal online email support for patients with long-term conditions [ ]. In the United States, an experiment offering older adults computer training in public libraries on finding health information via the Internet was successful [ ]. In England, Fisher et al aim to improve uptake of patient access to their records by supporting general practices [ ].
Digital divisions caused by physical disability have been subject to legislation. Web accessibility laws and regulations have encouraged developers to make websites accessible to those with visual, auditory, motor, neurological, or cognitive impairments. In Britain, the Disability Discrimination Act 1995 , Special Educational Needs and Disability Act 2001 [ ], and the Equality Act 2010 [ ] resulted in organizations reviewing website functionality and causing some organizations [eg, Royal National Institute for the Blind (RNIB)] [ ] to have units aiming to make digital information accessible to those with physical disability. In the United States, section 508 of the Rehabilitation Act of 1973 [ ] required federal agencies to ensure that federal employees with disabilities have equal access to information unless an undue burden would be imposed on the agency.
Why Do We Need to Measure eHealth Inequalities?
Projects and national initiatives such as those described above need to measure eHealth inequalities to know (1) if action is needed, (2) what is the main cause of inequality, and (3) if inequalities are addressed, if the intervention was successful and cost effective. But inequality cannot be directly measured; it has to be measured as a difference in another variable, namely eHealth readiness. To compare eHealth inequality over time, we need a measure of “patient eHealth readiness” that is comprehensive, valid, and reliable. A measure that is also “diagnostic,” allows development of interventions tailored to the needs of populations.
Tools to Assess eHealth Readiness
Others have considered the “readiness” of practitioners, organizations, or communities to adopt telehealth or eHealth [, , , , - , ]. Legare in 2010 [ ] identified six eHealth readiness tools [ , , , , , ], five of which assessed organizational readiness. Legare developed one of these [ ] further, translating it into French [ ] and validated its use with staff. However, no suitable tool that assessed patients’ opportunities to participate in eHealth was identified.
There are two groups of literature that exist at the “patient level” (1) the “digital divide” and (2) eHealth literacy. The digital divide-as the term implies-tends to be measured as a binary division. For example, whether someone has or does not have access to the Internet or has or has not used the Internet in the last three months . The digital divide has usually been assessed and reported by factual measures of Internet use or availability rather than attitudes or psychometric assessment. Work on measures of eHealth literacy [in particular the eHealth Literacy Scale (eHEALS)] [ ] recognized that physical access to the Internet was only part of the story and that personal abilities to use the Internet were important. However, by adopting a more sophisticated examination of eHealth literacy, the basic ideas of digital divide and limitations of access to the Internet were lost.
Simple measures of whether or not someone has Internet access are insufficient as even among Internet users some may be more ready to make progress in using eHealth if they have access to support and are not struggling with the cost of access. In particular, interventions at patient and community levels need tools that can measure their impacts.
The aim of this project was to get the benefits of a scaled (rather than binary) approach (like eHEALS), but to include eHealth provision, support, and economics in the scale. In particular, the study aim was to develop and validate a self-completed questionnaire and scoring system for use in intervention studies hoping to improve eHealth readiness and reduce eHealth inequalities.
The Patient eHealth Readiness Questionnaire (PERQ) and related scores have been developed in two stages. First, four domains (1) provision (from the digital divide literature), (2) personal (from the eHealth literacy literature), (3) support, and (4) economic were used to draft a self-completed questionnaire and take it through three stages of piloting (January-March 2012). Second, a cross-sectional population survey was carried out (April-August 2012) and proposed scoring systems checked and iteratively refined to ensure construct validity.
Although there are good national statistics on home Internet access [, , ] that allow a check on face validity, similar questions need to be included in patient-completed questionnaires to allow comparison before and after interventions. Provision of eHealth services is more difficult to assess as this will depend on the health conditions of interest to respondents and will be country-specific. Nearly all British respondents have a family doctor so asking about General Practice (GP) website provision is applicable to all. Some surveys have only asked about Internet “information” and respondents may not consider using the Internet to contact people. PERQ, therefore, included questions about personal contact.
The most frequently used [- ] measure of personal skills is eHEALS [ ], using eight items to assess eHealth literacy. A Dutch translation of eHEALS was found reliable, but its validity questioned [ ]. Van Deursen and Van Dijk [ ] criticized eHEALS because respondents were not always accurate at estimating their real levels of skill [ ]. Others have noted that self-efficacy may not accurately reflect ability. For example, nursing students’ self-efficacy in numeracy decreased if they had previously been asked to carry out an actual drug calculation [ ], and patients with long-term conditions may be confident in what they do on the Internet, but lack a sense of adventure to try new things [ ]. Van Deursen suggested that incorporation of basic Internet skills is needed to measure all aspects of eHealth literacy [ ]. However, having to “test” whole populations to produce a measure of eHealth literacy is not feasible.
Prior to the Dutch studies [, , ], Hargittai [ ] examined survey measures of Web-oriented digital literacy to serve as proxies for observed skill measures. They studied both observations and survey questions, and recommended measures as survey proxies of observed web-use skills. Their results suggested some composite variables of survey knowledge items were better predictors of people's actual digital literacy based on performance tests than the usual method of asking users' self-perceived abilities. Hargittai’s approach seemed a reasonable compromise towards the gold standard of Van Deursen. The first version of PERQ included the eight eHEALS questions and a single self-efficacy question, [ ] based on [ ], both “grounded” by using questions based on self-assessment (Hargittai’s approach) of the skills identified by Van Deursen and Van Dijk.
Interpersonal and Economic Measures
Although interpersonal support to help people start using the Internet was a major component of the Race Online 2012 campaign , no “measure” of support in using the Internet was identified. Simple questions about who is available to support participants and if there are barriers (eg, of disclosure, or “being a bother”) can be used. Similarly, although economic differences in being able to use eHealth are clearly important, there did not appear to be any standard measures. To ground questions about the perception of cost of Internet access, PERQ included comparative questions about cost of access to health services and the perceived cost of Internet provision.
Whether or not someone uses the Internet for their health depends on whether they are motivated to do so . So if patient eHealth readiness is to be an indicator of digital divisions, it needs to be “standardized” for motivation, similar to the distinction between digital choice and digital exclusion [ ].
PERQ Development and Initial Piloting
Three pilots (PERQ1-3) including repeatability were followed by a baseline survey. (PERQ4):
- The first had 15 people (work colleagues and friends). Questions from eHEALS [ ] were initially included following the four skills questions [ ] and followed by a single self-efficacy question [ ]. eHEALS score and the single self-efficacy rating showed quite good agreement (rho=0.61, P=.02), the single question showed good face validity (see ), and eHEALS was not well understood by one older person. Given the need to shorten the questionnaire, the eHEALS questions were subsequently omitted and the single self-efficacy question, following the four skills questions, retained.
- The second had 20 friends and family of a research assistant; 17 of these were subsequently asked to complete PERQ4 to assess repeatability (reported below).
- The third had a convenience sample of 103 houses likely to have a high proportion of more elderly residents. This was used to test the survey method, response rate, completeness of data, and that non-Internet users would respond. The response rate was 44% and data were reasonably complete. It was found that 5 out of 43 (12%) respondents had not used the Internet.
After each stage, revised questionnaires were circulated among colleagues to check readability (see). The questionnaire was reviewed and approved by the university ethics committee between pilots two and three.
Baseline Population Survey
As one intended use of PERQ was in a geographically based cluster randomized controlled trial (RCT), it was appropriate to test that mode of use. The aim was to use a sample representative of urban, suburban, and semirural postcodes and different levels of affluence to pilot the questionnaire, its distribution, and methods for construction of eHealth readiness and inequality scores. The 2001 census included 14,279 postcodes for the PL postcode area, with a total population of 510,223. There were seven postcodes (total population 3243) with very high populations (being either military camps or university halls of residence) that were excluded. The remaining postcodes were “ordered” by population and a 1% systematic sample (142 postcodes) was taken. Each postcode was “looked up” on the free Zoopla website service giving estimated property values to find current average property values and number of properties in each postcode. To achieve a “practical” sample of just fewer than 1000 properties, all postcodes further than 12 miles from Plymouth University were excluded, leaving 79 postcodes. These 79 postcodes were again ordered by Zoopla average property values, and a further systematic sample of 53/79 postcodes was taken with a total of 975 properties.
The final sample therefore comprised 53 postcodes within 12 miles of Plymouth University, with a total of 975 properties, population of 2126, and an average of 2.18 people per house. Average property prices (January 2012) per postcode ranged from British £78,163 to £459,360. The sample was representative of the range of property prices. Number of properties per postcode (a crude measure of rurality) varied from 1 to 53.
We attempted to deliver questionnaires by hand to all 975 properties in April 2012. There were thirty houses no longer in use, leaving 945 occupied houses as our sample. The research assistant called at each house and if someone answered, she explained the purpose of the survey and if possible handed the resident a questionnaire and covering letter (24 refused to take the questionnaire). If there was no response at the house, the questionnaire and covering letter were posted through the letterbox. In June 2012, reminders were posted through the letterbox of 658 who had not responded. The instructions on the questionnaire, and explained by the research assistant, were for the person with the next birthday in the house to complete the questionnaire.
Analysis and Refinement of Scoring Systems
Survey data were used to assess the ability of PERQ to collect good quality data, and used to develop and iterate a scoring system suitable for use in intervention studies, particularly RCTs. This required that only questions that contributed to the score were included, collected data were complete, consistent, and valid, scales must not have floor or ceiling effects, and must reflect meaningful changes.
Scales that combine a number of “Likert style” attitude questions normally assess reliability using Cronbach alpha. In this study, the construction of the eHealth readiness scale relied on pragmatic combinations of factual questions with some ratings, so assessing scale properties such as repeatability, face, and construct validity, was also pragmatic. Questions were cross checked for consistency and reviewed for their contribution to eHealth readiness scores, face validation against other information sources, or description of sample demographics. Comparisons of scores between subgroups were made using Mann Whitney U tests for groups less than 100 and t tests for groups of 100 or more.
The contribution of each question was checked. Not all questions made direct contributions to scores, some were asked to ground respondents to give them the “right frame of mind” for subsequent questions. Other questions were used as consistency checks.
Scores for each constructed variable were essentially arbitrary, but to have some way of measuring change before and after interventions, an overall score is needed that is at least ordinal, and if possible approximates to a cardinal scale. Similarly, overall scores need to combine component variables in a sensible manner. A pragmatic and iterative approach was taken to examine construct validity of scores by examining the scores of sampled individual respondents with a range of scores. If the order and difference in scores between individuals did not match with an understanding of the barriers to adopting eHealth, the weights of scores were adjusted.
Scoring was also adjusted after examining the repeatability of scores and to cope with occasional missing values. This process of tuning scoring weights continued until all components seemed internally consistent. Various methods of combining the four subscales to produce an overall readiness scale were tried, checking for construct validity by examining differences between Internet users and non-Internet users.
Once the scoring system was finalized, questionnaires and dataset were again reviewed to check that all questions and answers were useful either as contributors to the score, as “grounding” for other questions, or as consistency checks (see).
Modeling of Performance in Measuring Change
Finally, data from the survey were used to model possible changes to participants “states” and to check the ability of the scoring system to measure those changes. Comparisons of scores between subgroups were made using Wilcoxon Signed Rank tests for groups less than 100 and t tests for groups of 100 or more.
This provided a further check that the subscores and weights seemed sensible, and to allow an assessment of methods of analysis and estimation of sample size for possible RCTs.
The anonymized dataset from the cross-sectional survey is available from the author.
Response Rate and Possible Biases
shows by August 2012, 344 (36.4%) of the 945 occupied houses in the sample had returned completed questionnaires. Those 323/945 (34.2%) houses where the research assistant was able to speak to someone were more likely to have returned questionnaires (56% versus 27%) (χ24=90.4; P<.001). The 344 houses providing respondents had higher estimated values than those with no respondent (£176,998 versus £142,019; t925=-6.2; P<.001).
Data Completeness, Consistency, and Contribution
Despite care in design and three stages of piloting, PERQ still had missing and some inconsistent data (see), for example 29/344 (8.4%) people did not complete their age and 6/344 (1.7%) their gender. All questions contributed (see ).
shows the sample was disproportionately female (231/344 ,67.2% women; 107/344, 31.1% men; 6/344, 1.7% gender unknown), and older (mean age 55) than the Plymouth population.
shows the eight sections of the questionnaire (A-H). Non-Internet users answered A, B, C, G, and H and Internet users A, B, and D-H. There were six sets of variables created by scoring or combining responses to questions (1) Need, (2) Internet-Use including range of uses and the four subscales of “eHealth readiness,” (3) Provision including physical provision of Internet and provision of health on Internet, (4) Personal (ie, the individual’s capability to use the Internet for health), (5) Interpersonal Support, and (6) Economic. A “short score” (of half the score) was used in some comparisons and figures.
Scores (0-10) representing “need for health information and support” were constructed from one multi-part factual question (A3), by adding 2 points for each professional contact and health-information seeking behavior in the last three months. Scores had face validity, for example, women had higher Need scores than men. Need was used as a modifier of Provision scores.
Personal use of the Internet in the last three months was similar to national figures (271/342, 79.2% versus 77% from Office for National Statistics, ONS) . As expected, younger people and those from more affluent areas were more likely to use the Internet. Nearly half used it for health (mainly to search for information), but few used discussion forums or social media for health purposes. Most Internet users (262/271, 96.7%) used the Internet at home, at work (94/271, 34.7%), or on mobile (93/271, 34.3%). This section (B) was used for subsequent questionnaire section choice, face validity check, and as a consistency check with other parts of the questionnaire.
For Internet users, provision scores comprised two parts (section D) (1) General Internet Provision (4 points) ascertained by questions about what opportunities there are to access the Internet, and (2) Health Internet Provision (5 points) ascertained by questions on GP website (3.5 points), and Internet condition specific information and support (1.5 points). Provision of online information and support may vary by condition (eg, there are many resources for breast cancer, but fewer for stroke). However, not everyone has a need for health information or support and so may never have had reason to look for their GP website or for health information. So the Need score was used to moderate Provision; more was added to Health Internet Provision if Need was equal to zero (see).
shows the mean Provision score for Internet users was 4.5. There were 18 Internet users who had relatively low General Internet Provision (<1.5/3.5 max) including those who used the Internet only in places other than the home. Individual records were checked for participants with extreme scores and appeared to have face and construct validity (see ).
Component questions showed that nearly half (30/71) of non-Internet users had an Internet connected computer at home. Of 271 Internet users, 249 people had used it at home, but 3 said they had no home Internet connection, of these, 2 had used a mobile device and so it is possible that questions about “home Internet use” need to be clarified. There were 3 other people who had accessed the Internet at home, but did not apparently know if they had an Internet connection, may have not known about the “speed” of their home Internet, so some clarification may be needed for that question. A substantial minority (33/243,13.6%) thought their home Internet connection was not fast enough, a third of these said it was because they would need to pay more, a third because their provider did not offer a faster connection, and a third did not know.
Of 271 Internet users 89/271 (32.8%) had looked at their GP’s website, 51/271 (18.8%) thought their GP had a website, but had not seen it, 7/271 (2.6%) thought their GP did not have a website, and nearly half (122/271, 45.0%) did not know. Of the 89 who had looked at their GP’s website, 64 knew they could order repeat prescriptions online, 6 said their GP did not offer this service, and 14 did not know. Only 1 person knew they could see their medical record online, 20 knew they could not, but 66 did not know. Half (143/271, 52.8%) of Internet users had tried to find information on health topics with all but 9 having found what they wanted, but only 27/271 (10.0%) people had tried to contact an organization or forum or other people online connected with health.
The internal consistency of Provision scores was addressed by comparison of answers to sections B (where people had used the Internet) and D (home Internet provision and use for health).shows that no one had a short score of 5, so there was room for improvement and no ceiling effect.
Personal scores comprised moderated confidence scores for Internet users (section E) and the willingness to try using the Internet for non-Internet users. Internet users rated their skills on four tasks from which skills scores (0-12) were constructed. Users then rated their overall Internet confidence (0-10). This sequence of questions aimed to ground their confidence rating in the reality of their ability and to provide a consistency check on their confidence rating. Skills scores correlated with confidence self-ratings (Spearman’s Correlation=0.60, P<.001) with some outliers; four people rated their skills low, but confidence high and 13 people rated their skills high, but confidence low (see). However, to produce more consistent Personal scores, “moderated” confidence scores of skills*original confidence score/12 were calculated.
Component questions showed that just under half of non-Internet users said they would try using the Internet if they had help, would have a home Internet connection if they had help and it was cheap, and would use the Internet for health at some other place (most frequent choice public library). Questionnaires asked about disabilities. Six non-Internet users and six Internet users said they had disabilities (including arthritis, eye problems, hearing impairment, learning difficulty, and dyslexia) that made using computers difficult, but this information was not used in Personal score calculations on the assumption that respondents would themselves make that adjustment.
The face validity of Personal scores was assessed by exploring associations with frequency, range, and ubiquity of Internet use. As expected there were strong associations between frequency of use and moderated confidence (χ212=81, P<.001). Nevertheless there were outliers, one person who was very confident despite using the Internet less than once a week and four people who used the Internet many times daily, but had low confidence. The latter is more believable as they may use it for limited purposes. Similarly, as would be expected, there were strong associations between range of use and moderated confidence (χ212=61.5, P<.001), but similarly there were some “outliers.” Personal scores had a strong association with “ubiquity,” [ie, the places where people accessed the Internet (χ216=81, P<.001)]. Some might argue that range of health uses should be the outcome measure of any intervention, but this will be dependent on someone’s need for health information. So overall PERQ scores include moderated confidence as Personal score, being a “cleaner concept,” but will additionally report range of health uses.
shows there were some ceiling effects on Personal score. also shows that despite grounding the estimates of confidence by asking about skills, there was still a large minority (51/200, 25.5%) of the sample with maximum scores, being able to do all four Internet tasks and being totally confident in their use of the Internet. This means that these people would not be able to increase their Personal score during the course of a study. This suggests that some “harder” tasks should be included in the skills question, and to focus the questionnaire better on eHealth, this should perhaps include some health-focused questions.
Support scores were largely based on factual questions. There were 22 out of 271 Internet users that did not complete the section on support, half of these (10/22) said (H1) that they had no barriers to Internet use and were confident in using the Internet for health (E3).
Component questions showed that among Internet users, only 58/271 (21.4%) had been given information by health professionals to help them use the Internet. Just under half (117/271, 43.2%) knew where they could find help locally in using the Internet; many of these (78) cited their local library. A quarter (68/243, 28.0%) said there had been times when help would have been useful, and of these, 50 had someone they could ask, of which 47/50 could ask about health. Nearly three-quarters (49/71, 73%) of non-Internet users had someone use the Internet for them. There were 40/71 (65%) that had someone that could help if they wanted to try using the Internet.
That Support scores were less differentiated between Internet users and non-Internet users “made sense” in the way that questions were asked and answered. Exploration of how Support and Personal scores were associated led to a pragmatic combination, using the Personal score to moderate the Support score in the overall Readiness score (see below). There were no ceiling effects on Support.
The Economic subscale was constructed slightly differently to the other three subscales, relying on comparison of perceptions of the cost of using the Internet compared to other health activities such as visiting their GP or local hospital. Internet users and non-Internet users answered the same questions.
Considering the component questions, there were significant differences on the two Internet questions and on the cost of visiting the hospital between Internet users and non-Internet users. For the two Internet questions, this was dominated by the “don’t knows” among non-Internet users; 45% (29/64) of non-Internet users did not know about the cost of home access and 57% (35/61) about the cost of mobile access compared to 4.8% (13/269) and 32.8% (87/265) of Internet users. There was no difference between Internet users and non-Internet users in perceptions of cost in getting to the local library or GP. Most (195/325, 60.0%) did not think it cost much to get to a public library, but a large minority (95/325, 29.2%) did not know. The vast majority (317/337, 94.1%) agreed that visiting their GP cost nothing or very little. Non-Internet users were more likely to think that visiting their nearest hospital cost nothing or little (55/67, 82% versus 185/268, 69.0%; χ24=15.7; P=.003); this may be because more had free bus passes and may be an important reason why the Internet appears relatively more expensive to older non-Internet users.
shows that overall, non-Internet users were likely to have lower Economic scores indicating more barriers to using the Internet (χ28=39; P<.001). There were no ceiling effects so improvements could be measured.
Overall View on Using the Internet for Health
Question H1 sought to identify the most important issue in using or not using the Internet for health. The original intention was to use this question to weight subscale scores in their combination to produce an overall readiness score. This idea was abandoned when it was realized that there was a close relationship between the Support and Personal subscales and an alternative combination method was developed. However, H1 remained a useful consistency check on the subscale scores.shows that most Internet users (185/271, 70%) thought they had no real barriers to using the Internet for health. Among non-Internet users, 61% (40/66) said they had no interest in using the Internet.
Further breakdown of the groups inshow the range of different situations and attitudes. Of the 52 with “no interest in using the Internet,” 12 had used it in the last three months, 11 had home Internet access and had not used it personally, but most had someone else use it for them. Relatively few chose connectivity, economic reasons, or need for support as the main barrier to Internet use for health.
To test the construct validity of subscales, constructed variables were compared to answers to question H1 (). All but two answers, “would use Internet more if could get a better connection” and “would use Internet more if could get someone to help” showed significant differences on the expected variable. Short scores were compared between Internet users and non-Internet users and examples of where non-Internet users had higher scores or the same scores as Internet users were selected and reviewed. These showed construct validity.
Combining the Four Subscales into an Overall eHealth Readiness Score
The initial intention was to create an overall eHealth readiness score by taking the mean of the four subscales, that is (Provision + Personal + Support + Economic)/4. However, exploration of the data led to recognition that Support was much more important for non-Internet users. Those who were already competent Internet users for health needed little support and scored low on Support. This reduced their overall eHealth readiness score and was misleading. Support was therefore added to eHealth readiness in inverse proportion to that person’s Personal score, (ie, people with a higher Personal score had less weight given to their Support score). Through a process of iteration considering whether the impact on overall eHealth readiness made sense, the term 3*Support/(Personal+Support) was added as a “Modified Support” term. This Modified Support score can range from 0 to 3 and the sum of Personal and Modified Support can range from 0 to 10.5. The Economic score also seemed less important in being “eHealth ready” than Personal and Provision scores, so the short score (range 0-5) was used as the contribution to eHealth readiness. So,
eHealth readiness= Provision (0-9) + (Personal + Modified Support) (0-10.5) + Short economic (0-5)
It was then multiplied by 9/24.5 to scale to the range 0-9.shows in this sample scores ranged from 0-7 with mean 4.1 (SD 1.79). Non-Internet users had, as expected, lower scores than Internet users.
|Overall View||Non-Internet user||Internet user||Total||“Nearest” variable||Mean score for those who chose this item versus rest (t test)|
|Home access||No home access|
|(H11) No need for health information.||1||3||34||38||NEED||1.8 versus 3.3|
|(H12) No interest in using the Internet.||11||29||12||52||PERSONAL||1.1 versus 6.2|
|(H13) Would use the Internet more for health if could get a good Internet connection.||0||2||3||5||PROVISION||2.8 versus 3.8|
|(H14) Don’t understand the Internet that much.||9||4||17||30||PERSONAL||2.7 versus 5.7|
|(H15) Would use the Internet more for health if could get someone to help.||0||0||10||10||SUPPORT||2.1 versus 2.1|
|(H16) Would use the Internet more for health if money were no object.||1||2||2||5||ECONOMIC||0.8 versus 2.5|
|(H17) Uses or would use the Internet for health and have no real barriers to that use.||4||0||185||189||PERSONAL||7.2 versus 3.2|
|ECONOMIC||3.0 versus 1.9|
|PROVISION||4.0 versus 1.9|
|SUPPORT||3.1 versus 2.3|
|READINESS||4.4 versus 2.3|
aU=Mann Whitney U
bnsd=no statistically significant difference P>.05
Repeatability and Change of Scores Over Time
Seventeen of the 20 people who completed the second-stage pilot (January 2012) completed PERQ4 in September 2012. Of these, four non-Internet users were excluded as the questions in section C had changed too much between the earlier and later version of PERQ to be comparable. For 13 Internet users their January data was converted to the September version of the questionnaire to allow a comparison and some assessment of “repeatability” and change over time. Each pair of questionnaires was examined for changes to answers and the impact on the scoring system to see if it made sense and if the scoring system was appropriate. This check resulted in some changes to the scoring system. With the final scoring system there was reasonable consistency in scores between January and September 2012 (see) with changes in scores making sense with known changes in personal circumstances for those respondents.
Is PERQ Suitable to Assess Interventions?
shows that PERQ did not have floor or ceiling effects. PERQ produces two scores, eHealth readiness and eHealth inequalities (SD of readiness). The aim of interventions would be to improve overall eHealth readiness (ie, increase the mean score) while keeping variation (SD) the same or reduced. With this sample, the mean eHealth readiness score was 4.24 with standard deviation 1.73, (4.9 for Internet users versus 1.6 for non-Internet users; t4.24,1.73=-25.8; P<.001). Statistically significant changes in mean scores must represent practically (clinically) significant changes. To assess whether this scoring system can measure an attainable and useful improvement in eHealth readiness and what this would mean in terms of individual changes, changes were modelled using the dataset.
shows four feasible changes resulting from interventions or further development in Internet use, modelled using the dataset. The first shows that if 20 non-Internet users get online and access health information there is a substantial increase in score for subgroup and whole sample as well as a reduction in inequality (SD of readiness). While the decrease in SD is not statistically significant (confidence intervals are 1.73-6.38 and 1.50-5.53) [ ] the decrease is at least “heading in the right direction.” The second shows the impact of existing Internet users gaining more routes to access via mobile and learning about patient access to their GP record. Mean readiness is increased, but again, although not statistically significant, it is tending to increase inequality. The third scenario might result from better Internet provision, such as the implementation of faster broadband as is happening in Cornwall. If the level of statistical significance is set at P=.05, then the increase in readiness is significant, but policy makers may consider the actual change of just 20 people getting faster access relatively unimportant. With this sample size it appears relatively easy to achieve a statistically significant change in mean PERQ. The fourth scenario shows the possible impact of GPs in the area starting to engage more in recommending websites to their patients, as has been the case with “information prescription” projects [ - ]. PERQ therefore appears to be sufficiently sensitive to change.
|Assumed changes||Impact on sub group mean readiness score||Impact on whole sample|
|Mean readiness score||Inequality (standard deviation of readiness)|
|20 non-Internet users are supported in getting online. They have not looked at the GP website, but have found other health information online.||Increase 1.8 to 5.0|
|Increase 4.24 to 4.44|
1.73 to 1.61
|20 Internet users who previously used computer at home and at work got mobile access and became aware of GP services including patient access to records.||Increase 4.9 to 6.2 z=4.1; P<.001|
|Increase 4.24 to 4.32|
1.73 to 1.79
|20 Internet users who said their Internet connection was too slow who got a faster connection and many of who used it to contact someone about health.||Increase 4.2 to 4.6|
|Increase 4.24 to 4.27|
|80 Internet users who previously had not had advice on using Internet from HCP, recommended websites by GP.||Increase 4.5 to 4.7|
|Increase 4.24 to 4.29|
Questionnaire and Scoring
A questionnaire (PERQ) and scoring system has been developed comprising four components of patient eHealth readiness (1) provision of Internet and Internet for health, (2) personal capacity to use it, (3) support in using it, and (4) economic barriers to use. The scoring system appears consistent, to have face and construct validity, and to produce a score that can be used to assess interventions that improve eHealth readiness. By examining the standard deviation of scores, eHealth inequalities can be reviewed to ensure that interventions have not worsened inequalities. The questionnaire is being used in two studies locally [, ].
Although national cross-sectional data from the ONS and Oxford Internet Survey (OIS) show the uptake of the Internet, and sometimes include questions on health, there is a need for a tool to measure the impact of interventions in the context of RCTs. Although there are widely used measures of eHealth literacy, these were not sufficiently comprehensive in their scope; in particular they only “worked” for Internet users. Initially the eHEALS questionnaire on eHealth literacy was included within the PERQ questionnaire, but was then dropped as the single self-efficacy question seemed adequate and took less space. There was a need for a measure that covered the full range of individuals from non-Internet users through to frequent Internet users.
Limitations in Scoring System
This pragmatic scoring system has many limitations, but is published with full details and data so that others can refine or continue to develop it. One problem with any measure of eHealth readiness is continually changing technology such as the current shift to smart phones . If measures are to be used for any length of time they need to cope with changing technology. One solution may be to have a framework of generic questions that remain the same over time, but “situate” these by inserting questions related to the “state of the technology.” These questions will change over time as technology changes. The first stage of using a measure of eHealth inequality for a study would be to gain consensus on the current “State of Technology.” This has not been achieved in the development of this questionnaire and scoring system, but remains a future goal.
Second, the scale and scoring rely on self-report. Although one dimension of the proposed scale is provision of eHealth services, and although this could be measured fairly objectively [, ], to have a method consistent with the other dimensions, this is best measured by asking patients (ie, the pragmatic solution of “perception of provision.)” Self-reported measures, such as self-efficacy, as discussed earlier, may not be good predictors of actual ability. PERQ tries to compensate by using “grounding” questions and by pragmatic “constructs” such as “modified confidence.”
Third, having four dimensions makes having one summative number for eHealth readiness difficult. As described, the original plan to make the four components additive either in equal proportion or, using the ideas of Paterson et al [- ] in measures of quality of life, by asking respondents to nominate what is most important. However, when the close connection between the Personal and Support terms was noted, this determined how the four terms should be combined. The final scoring system appears to have face and construct validity, but is nevertheless arbitrary. Others may wish to explore alternatives.
Fourth, the eHealth readiness scale is at best ordinal and not cardinal. This compares to, for example, a difference in mortality that can be expressed as a difference in years of life. While some may argue that 10 years of life at age 20 is “worth more” than 10 years of life at age 70, “years of life” is essentially a cardinal scale. Self-reported questions used to construct an ordinal scale will always have limitations and should be used cautiously.
Fifth, the weights used for individual items were arbitrary. For example, Internet Health Provision included questions about whether patients could access their medical records (weighted 1.5) and could order repeat prescriptions online (weighted 1.0). These weights reflect the judgements of the author in the “difficulty” or “sophistication” of provision. Clearly other weights could be used and the dataset and analysis syntax are provided for others to explore, but this pragmatic approach seems to provide a way of scoring and measuring change.
Sixth, people whose opportunities to use the Internet are less because of limited English will not be identified by this approach, using an English language questionnaire.
Lastly, it is not possible to directly measure eHealth inequalities, but instead to examine the spread of eHealth readiness scores. Those implementing eHealth interventions are interested in improvements in mean score (eHealth readiness) over time, but should also consider the standard deviation of scores (eHealth inequality). In other fields, for example, income inequality, measures such as the Gini coefficient directly measure inequality. Health inequalities are typically measured by differences in mortality, expressed in absolute numbers of life expectancy at birth or some other age between two groups. In this case, we hope that an intervention will improve eHealth readiness, but also reduce the standard deviation. It was important therefore in the construction of this score that a reduction in standard deviation was not artificially induced by a “ceiling effect” on the score. Given the natural progression of the Internet we are unlikely to see eHealth readiness reduce, so “floor effects” are less important. The modelling of possible interventions suggests that the eHealth readiness score is quite sensitive to relatively modest changes in Internet use for health. In determining sample size and setting significance levels, therefore, it is suggested that P<.001 is appropriate. On the other hand, achieving a statistically significant reduction in eHealth inequality may be difficult, but researchers’ and policy makers’ may be able to decide that interventions are at least not making inequality worse.
Clarity is needed about the role of social determinants of eHealth inequalities. Should associations between eHealth use and demographic and social variables be explored, or should the focus be on the immediate “cause” of eHealth inequality? Demographics are clearly important in use of the Internet . Answers to questions on the personal and interpersonal components of a measure may be predicted by social determinants and act as a test of face validity, but should not be part of any eHealth readiness scoring system.
Representativeness of the Sample
That the nonresponders in the baseline survey were more likely to live in lower value houses will have biased this sample towards Internet users . On the other hand, using households as the sampling unit biased the sample towards older people and females (as there are more single, older, female households), so biasing the sample towards non-Internet users. Lower response rates from younger people, particularly from student households, will also bias the sample towards older people and non-Internet users. Overall, the baseline survey overrepresented older people. As the purpose of the sampling was to have a “test bed” for the questionnaire and to develop the scoring system, this may have been an advantage rather than disadvantage. The selection of households and respondents for this survey was pragmatic using easily available open data sources, but was similar in principle to the methods used in the OIS. The OIS used 175 randomly selected “Output Areas” in England, within which 10 addresses were selected at random from the Postal Address File. Interviews aimed to interview the person with the next birthday.
The baseline survey response rate was fairly poor (36%) compared to the 59% achieved by ONS  and to the OIS (49% successful interviews for 4160 houses visited) [ ]. But, with bigger budgets, the data for both were collected by interview (rather than returned self-completed questionnaire) and allowed multiple visits to find a respondent at home. In this survey, the response rate for houses where the research assistant was able to speak to the resident before leaving a questionnaire for self-completion and return was 56%. Greater variation in time of calling/delivery and a budget allowing more persistence should achieve a better response rate.
“Diagnostic” Uses of PERQ
More detailed analysis of PERQ results could indicate the most appropriate interventions for individuals or subgroups. For example, groups that would most benefit from faster access, or support, or for whom economics was the main barrier could have interventions chosen appropriately. It is possible that a “stages of change” approach to classifying individuals might be useful, although the different dimensions (personal, provision, economic, support) need to be taken into account.
The support section of the questionnaire was the least successful. This had proved difficult throughout piloting. In particular, we had sought ways of getting those people who had never needed or sought help to answer the questions by wording the questions about “people in general,” and by stressing that we wanted everyone to answer this section. Nevertheless, 21/271 (7.8%) Internet users failed to answer this section. The second part of the “support section” (F) of PERQ probably did not collect particularly useful information, and given the desire to shorten the questionnaire, could possibly be dropped in further developments.
There was previously no measure of personal eHealth readiness or eHealth inequalities. The concept of a patient eHealth readiness based on provision, personal ability, support, and economic considerations with eHealth inequality as the standard deviation seems to “work” and be acceptable in a British context. The scores produced appear valid and sufficiently sensitive to enable assessment of the effectiveness of interventions to improve eHealth readiness and reduce eHealth inequalities. With suggested modifications PERQ is now being used in two other local studies. It could also be used to help identify interventions addressing eHealth readiness. Such methods need continued evolution; full documentation and data have been published to allow others to develop the tool further. In particular with changes to the “provision section,” PERQ might be adapted for use in nonBritish settings.
The author wishes to thank the following people for their assistance with this paper: the reviewers for their helpful comments on the original submission, Liz Baker who delivered questionnaires, entered the data from the questionnaires, as well as recruiting family and friends for one round of the piloting, Paul Hewson for statistical advice, Jeremy Hobart and Michael Hyland for advice on scale construction, Lesley Goldsmith, Emily Ashurst, Tess Lomax, Jane Grose, Julia Frost, Anita Slade, Phil Abbott-Garner, and Jo Atkey for comments on drafts of the paper, and finally various colleagues, family members, residents of Wesley Court retirement apartments, students of Central Saint Martins, and others for the completion of and comments on early drafts of the questionnaire. The literature review, research for this paper, development and early piloting was unfunded. Sentinel Community Interest Company (GP-Co-operative for Plymouth) funded the costs of data collection for the pilot study of the questionnaire. Analysis and write-up was funded by Plymouth University (in terms of RJ’s time).
Conflicts of Interest
Multimedia Appendix 1
Initial piloting, more details on baseline survey, questionnaire development (versions 1-4), variables, SPSS syntax, dealing with inconsistent and missing data.PDF File (Adobe PDF File), 791KB
- World Health Organisation. World Health Organisation. Trade, foreign policy, diplomacy and health: E-health URL: http://www.who.int/trade/glossary/story021/en/ [accessed 2013-05-24] [WebCite Cache]
- Pagliari C, Sloan D, Gregor P, Sullivan F, Detmer D, Kahan JP, et al. What is eHealth (4): a scoping exercise to map the field. J Med Internet Res 2005;7(1):e9 [FREE Full text] [CrossRef] [Medline]
- Oh H, Rizo C, Enkin M, Jadad A. What is eHealth?: a systematic review of published definitions. World Hosp Health Serv 2005;41(1):32-40. [Medline]
- Eysenbach G. What is e-health? J Med Internet Res 2001;3(2):E20 [FREE Full text] [CrossRef] [Medline]
- Della Mea V. What is e-health (2): the death of telemedicine? J Med Internet Res 2001;3(2):E22 [FREE Full text] [CrossRef] [Medline]
- Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. J Med Internet Res 2005;7(1):e1 [FREE Full text] [CrossRef] [Medline]
- Ahern DK, Kreslake JM, Phalen JM. What is eHealth (6): perspectives on the evolution of eHealth research. J Med Internet Res 2006;8(1):e4 [FREE Full text] [CrossRef] [Medline]
- Jones R, Rogers R, Roberts J, Callaghan L, Lindsey L, Campbell J, et al. What is eHealth (5): a research agenda for eHealth through stakeholder consultation and policy context review. J Med Internet Res 2005;7(5):e54 [FREE Full text] [CrossRef] [Medline]
- Jones R, Goldsmith L. Final Report to NHS Choices. Plymouth: University of Plymouth; 2009. What is the evidence for the benefits and outcomes for digital health services? Final Report to NHS Choices URL: http://www.plymouth.ac.uk/files/extranet/docs/FoH/NHSChoices_2009.doc [accessed 2013-01-31] [WebCite Cache]
- Portnoy DB, Scott-Sheldon LA, Johnson BT, Carey MP. Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988-2007. Prev Med 2008 Jul;47(1):3-16 [FREE Full text] [CrossRef] [Medline]
- Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res 2004 Nov 10;6(4):e40 [FREE Full text] [CrossRef] [Medline]
- Moeinedin F, Moineddin R, Jadad AR, Hamid JS, To T, Beyene J. Application of biomedical informatics to chronic pediatric diseases: a systematic review. BMC Med Inform Decis Mak 2009;9:22 [FREE Full text] [CrossRef] [Medline]
- Sanders DL, Aronsky D. Biomedical informatics applications for asthma care: a systematic review. J Am Med Inform Assoc 2006;13(4):418-427 [FREE Full text] [CrossRef] [Medline]
- Hannan A. Medeconomics. 2013. Self Care - Inform patients via your website URL: http://m.gponline.com/article/1064630/Self-Care---Inform-patients-via-website [accessed 2013-01-31] [WebCite Cache]
- Jones R. Inequalities in general practice website provision. Br J Gen Pract 2011 Nov;61(592):663 [FREE Full text] [Medline]
- Wenner A. University of Plymouth Webinar. 2013. Patient Entered Data Using Instant Medical History (Recorded webinar) University of Plymouth 2008 URL: http://www.plymouth.ac.uk/pages/view.asp?page=29272 [accessed 2013-01-31] [WebCite Cache]
- Renal Association. Webcite. Which Units have Renal Patient View? Accessed from 31 January 2013 URL: http://www.renal.org/whatwedo/JointActivitiesSection/RIXGSection/RenalPatientView/RIXGWhere.aspx [accessed 2013-01-31] [WebCite Cache]
- Jones RB, Goldsmith L, Hewson P, Williams CJ. Recruitment to online therapies for depression: pilot cluster randomized controlled trial. J Med Internet Res 2013;15(3):e45 [FREE Full text] [CrossRef] [Medline]
- Lenhart, Horrigan J, Allen K, Boyce A, Madden M, O'Grady E. Pew Internet and American Life Project. Washington, DC: Pew Charitable Trusts; 2003. The ever-shifting Internet population: a new look at Internet access and the digital divide URL: http://www.pewinternet.org/Reports/2003/The-EverShifting-Internet-Population-A-new-look-at-Internet-access-and-the-digital-divide/01-Summary-of-Findings.aspx [accessed 2013-01-31] [WebCite Cache]
- Rice RE. Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys. Int J Med Inform 2006 Jan;75(1):8-28. [CrossRef] [Medline]
- Renahy E, Parizot I, Chauvin P. Health information seeking on the Internet: a double divide? Results from a representative survey in the Paris metropolitan area, France, 2005-2006. BMC Public Health 2008;8:69 [FREE Full text] [CrossRef] [Medline]
- Sheaves B, Jones RB, Williamson GR, Chauhan R. Phase 1 pilot study of e-mail support for people with long term conditions using the Internet. BMC Med Inform Decis Mak 2011;11:20 [FREE Full text] [CrossRef] [Medline]
- Jones RB, O'Connor A, Brelsford J, Parsons N, Skirton H. Costs and difficulties of recruiting patients to provide e-health support: pilot study in one primary care trust. BMC Med Inform Decis Mak 2012;12:25 [FREE Full text] [CrossRef] [Medline]
- Plymouth SeniorNet. 2013. URL: http://www.routeways.org.uk/index.php?page=silver-dreams [accessed 2013-08-29] [WebCite Cache]
- Lorence D, Park H. Group disparities and health information: a study of online access for the underserved. Health Informatics J 2008 Mar;14(1):29-38. [CrossRef] [Medline]
- National Telecommunications and Information Administration. Published by U.S. Department of Commerce Accessed from 31 January Webcite. 2013. Digital Nation: Expanding Internet Usage (NTIA Research Preview) URL: http://www.ntia.doc.gov/report/2011/digital-nation-expanding-internet-usage-ntia-research-preview [accessed 2013-01-30] [WebCite Cache]
- Eyrich-Garg KM. Sheltered in cyberspace? Computer use among the unsheltered ‘street’ homeless. Computers in Human Behavior 2011 Jan;27(1):296-303 [FREE Full text] [CrossRef]
- Demiris G, Patrick T, Khatri N. Assessing home care agencies' readiness for telehealth. AMIA Annu Symp Proc 2003:825 [FREE Full text] [Medline]
- Oliver DR, Demiris G. An assessment of the readiness of hospice organizations to accept technological innovation. J Telemed Telecare 2004;10(3):170-174. [CrossRef] [Medline]
- Wittenberg-Lyles E, Shaunfield S, Oliver DP, Demiris G, Schneider G. Assessing the readiness of hospice volunteers to utilize technology. Am J Hosp Palliat Care 2012 Sep;29(6):476-482. [CrossRef] [Medline]
- Lehman WE, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat 2002 Jun;22(4):197-209. [Medline]
- Snyder-Halpern R. Development and pilot testing of an Organizational Information Technology/Systems Innovation Readiness Scale (OITIRS). Proc AMIA Symp 2002:702-706 [FREE Full text] [Medline]
- Snyder-Halpern R. Indicators of organizational readiness for clinical information technology/systems innovation: a Delphi study. Int J Med Inform 2001 Oct;63(3):179-204. [Medline]
- Jennett P, Jackson A, Ho K, Healy T, Kazanjian A, Woollard R, et al. The essence of telehealth readiness in rural communities: an organizational perspective. Telemed J E Health 2005 Apr;11(2):137-145. [CrossRef] [Medline]
- Li JH, Ray P. E-Health Readiness Assessment Methodology (EHRAM). In: Dössel O, Schlegel WC, editors. World Congress on Medical Physics and Biomedical Engineering September 7 - 12, 2009 Munich, Germany: Vol. 25/XII General Subjects (IFMBE Proceedings). Berlin: Springer; 2009:151-154.
- Chattopadhyay S, Ray P, Land L, Li J. A Framework for Assessing ICT Preparedness for e-Health Implementation. In: Proceedings of 10th IEEE Intl. Conf. on e-Health Networking, Applications and Service. Singapore: IEEE; 2008:147-152.
- Légaré E, Vincent C, Lehoux P, Anderson D, Kairy D, Gagnon MP, et al. Developing and validating the French-Canadian version of the practitioner and organizational telehealth readiness assessment tools. J Telemed Telecare 2010;16(3):140-146. [CrossRef] [Medline]
- Légaré E, Vincent C, Lehoux P, Anderson D, Kairy D, Gagnon MP, et al. Telehealth readiness assessment tools. J Telemed Telecare 2010;16(3):107-109. [CrossRef] [Medline]
- Wilson EV, Lankton NK. Modeling patients' acceptance of provider-delivered e-health. J Am Med Inform Assoc 2004;11(4):241-248 [FREE Full text] [CrossRef] [Medline]
- Kutlu B, Ozturan M. Determinants of E-Health Readiness of End-Users. International Medical Journal 2012;19:287-291 [FREE Full text] [WebCite Cache]
- Li J, Seale H, Ray P, Wang Q, Yang P, Li S, et al. e-Health preparedness assessment in the context of an influenza pandemic: a qualitative study in China. BMJ Open 2013;3(3) [FREE Full text] [CrossRef] [Medline]
- Zapka J, Simpson K, Hiott L, Langston L, Fakhry S, Ford D. A mixed methods descriptive investigation of readiness to change in rural hospitals participating in a tele-critical care intervention. BMC Health Serv Res 2013;13:33 [FREE Full text] [CrossRef] [Medline]
- Wilkowsk W, Ziefle M, Alagöz F. How user diversity and country of origin impact the readiness to adopt E-health technologies: an intercultural comparison. Work 2012;41 Suppl 1:2072-2080. [CrossRef] [Medline]
- Khoja S, Scott RE, Casebeer AL, Mohsin M, Ishaq AF, Gilani S. e-Health readiness assessment tools for healthcare institutions in developing countries. Telemed J E Health 2007 Aug;13(4):425-431. [CrossRef] [Medline]
- Jennett P, Jackson A, Healy T, Ho K, Kazanjian A, Woollard R, et al. A study of a rural community's readiness for telehealth. J Telemed Telecare 2003;9(5):259-263. [CrossRef] [Medline]
- Jennett P, Bates J, Healy T, Ho K, Kazanjian A, Woollard R, et al. A readiness model for telehealth is it possible to pre-determine how prepared communities are to implement telehealth? Stud Health Technol Inform 2003;97:51-55. [Medline]
- MacFarlane A, Clerkin P, Murray E, Heaney DJ, Wakeling M, Pesola UM, et al. The e-Health Implementation Toolkit: qualitative evaluation across four European countries. Implement Sci 2011;6:122 [FREE Full text] [CrossRef] [Medline]
- Touré M, Poissant L, Swaine BR. Assessment of organizational readiness for e-health in a rehabilitation centre. Disabil Rehabil 2012;34(2):167-173. [CrossRef] [Medline]
- Chipps J, Mars M. Readiness of health-care institutions in KwaZulu-Natal to implement telepsychiatry. J Telemed Telecare 2012 Apr;18(3):133-137. [CrossRef] [Medline]
- Rezai-Rad M, Vaezi R, Nattagh F. E-health readiness assessment framework in iran. Iran J Public Health 2012;41(10):43-51 [FREE Full text] [Medline]
- Sharifi M, Ayat M, Jahanbakhsh M, Tavakoli N, Mokhtari H, Ismail WK. E-health implementation challenges in Iranian medical centers: a qualitative study in Iran. Telemed J E Health 2013 Feb;19(2):122-128. [CrossRef] [Medline]
- Rogers EM. Diffusion of innovations. New York: Free Press; 1995.
- Fenn J. Gartner Webcite. 2013. Understanding Gartner's Hype Cycles URL: http://www.gartner.com/id=1398413 [accessed 2013-01-31] [WebCite Cache]
- Viswanath K, Kreuter MW. Health disparities, communication inequalities, and eHealth. Am J Prev Med 2007 May;32(5 Suppl):S131-S133 [FREE Full text] [CrossRef] [Medline]
- Office for National Statistics. Office for National Statistics. 2012. Households and Individuals, 2012 URL: http://www.ons.gov.uk/ons/rel/rdit2/internet-access---households-and-individuals/2012/stb-internet-access--households-and-individuals--2012.html [accessed 2013-01-31] [WebCite Cache]
- Ofcom. Ofcom. 2010. Digital Participation: 2010 Metrics Bulletin URL: http://stakeholders.ofcom.org.uk/market-data-research/media-literacy/archive/medlitpub/medlitpubrss/2010-metrics-bulletin/ [accessed 2013-01-31] [WebCite Cache]
- Milligan C, Passey D. Current Engagement and future Needs. Ageing and the use of the internet URL: http://www.nominettrust.org.uk/sites/default/files/NT%20SoA%20-%20Ageing%20and%20the%20use%20of%20the%20internet_0.pdf [accessed 2013-01-31] [WebCite Cache]
- Cabinet Office. Cabinet Office. Digital services so good that people prefer to use them (Government Digital Strategy) Accessed from 31 January 2013 Webcite URL: http://publications.cabinetoffice.gov.uk/digital/ [accessed 2013-01-31] [WebCite Cache]
- Department of Health. A review of tacking health inequalities in England over the last 10 years. Tackling health inequalities: 10 years on URL: http://www.bris.ac.uk/poverty/downloads/keyofficialdocuments/Tackling%20HE%2010%20years%20on.pdf [accessed 2013-01-31] [WebCite Cache]
- Jones R. The role of health kiosks in 2009: literature and informant review. Int J Environ Res Public Health 2009 Jun;6(6):1818-1855 [FREE Full text] [CrossRef] [Medline]
- John Carlo B, Paul TJ, Lesley AL, Charles RM. First Monday. 2006. Public access computing and Internet access in public libraries: The role of public libraries in e-government and emergency situations URL: http://www.firstmonday.org/ojs/index.php/fm/article/view/1392/1310 [accessed 2013-08-30] [WebCite Cache]
- Information Daily. 2013. Age Concern launches its Digital Inclusion Network Accessed from on 31 January 2013 Webcite URL: http://www.theinformationdaily.com/2007/05/01/age-concern-launches-its-digital-inclusion-network [accessed 2013-01-31] [WebCite Cache]
- Gita Mendis (Head of Digital Inclusion NHS Choices) (2007-2011). 2013. Webcite URL: http://www.linkedin.com/pub/gita-mendis/6/4aa/438 [accessed 2013-01-31] [WebCite Cache]
- Kerr C, Murray E, Noble L, Morris R, Bottomley C, Stevenson F, et al. The potential of Web-based interventions for heart disease self-management: a mixed methods investigation. J Med Internet Res 2010;12(4):66-80 [FREE Full text] [CrossRef] [Medline]
- Xie B, Bugg JM. Public library computer training for older adults to access high-quality Internet health information. Libr Inf Sci Res 2009 Sep 1;31(3):155-162 [FREE Full text] [CrossRef] [Medline]
- Health Foundation. Health Foundation Inspiring Movement. 2013. Patient record access: turning it on, sharing the learning URL: http://www.health.org.uk/areas-of-work/programmes/closing-the-gap-through-changing-relationships/related-projects/patient-record-access/ [accessed 2013-01-31] [WebCite Cache]
- National Archives. Legislation.gov.uk. 2013. Disability Discrimination Act 1995 URL: http://www.legislation.gov.uk/ukpga/1995/50/contents [accessed 2013-01-31] [WebCite Cache]
- National Archives. Legislation.gov.uk. 2013. Special Educational Needs and Disability Act 2001 URL: http://www.legislation.gov.uk/ukpga/2001/10/contents [accessed 2013-01-31] [WebCite Cache]
- National Archives. Legislation.gov.uk. 2013. Equality Act 2010 URL: http://www.legislation.gov.uk/ukpga/2010/15/contents [accessed 2013-01-31] [WebCite Cache]
- RNIB Digital Accessibility Team. Tiresias. 2013. Making ICT accessible URL: http://www.tiresias.org/index.htm [accessed 2013-01-31] [WebCite Cache]
- United States Government. United States Government. 2013. Resources for understanding and implementing Section 508 Accessed from on 31 January 2013 Webcite URL: http://www.section508.gov/index.cfm [accessed 2013-01-31] [WebCite Cache]
- Khoja S, Scott R, Gilani S. E-health readiness assessment: promoting "hope" in the health-care institutions of Pakistan. World Hosp Health Serv 2008;44(1):36-38. [Medline]
- Office for National Statistics. Office for National Statistics. 2013. Internet Access - Households and Individuals, 2012 URL: http://www.ons.gov.uk/ons/rel/rdit2/internet-access---households-and-individuals/2012/stb-internet-access--households-and-individuals--2012.html [accessed 2013-01-31] [WebCite Cache]
- Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006;8(4):e27 [FREE Full text] [CrossRef] [Medline]
- Fox S. Pew Internet and American Life Project. Washington, DC: Pew Internet and American Life Project; 2005. Health Information Online URL: http://www.pewinternet.org/~/media/Files/Reports/2005/PIP_Healthtopics_May05.pdf.pdf [accessed 2013-01-31] [WebCite Cache]
- Dutton WH, Blank G. Oxford Internet Surveys. Oxford: Oxford University; 2001. Next generation users: the Internet in Britain URL: http://www.oii.ox.ac.uk/downloads/index.cfm?File=publications/oxis2011_report.pdf [accessed 2013-08-30] [WebCite Cache]
- Robinson C, Graham J. Perceived Internet health literacy of HIV-positive people through the provision of a computer and Internet health education intervention. Health Info Libr J 2010 Dec;27(4):295-303. [CrossRef] [Medline]
- Xie B. Experimenting on the impact of learning methods and information presentation channels on older adults' e-health literacy. J. Am. Soc. Inf. Sci 2011 Jun 06;62(9):1797-1807. [CrossRef]
- Knapp C, Madden V, Marcu M, Wang H, Curtis C, Sloyer P, et al. Information seeking behaviors of parents whose children have life-threatening illnesses. Pediatr Blood Cancer 2011 May;56(5):805-811. [CrossRef] [Medline]
- Horvath KJ, Courtenay-Quirk C, Harwood E, Fisher H, Kachur R, McFarlane M, et al. Using the Internet to provide care for persons living with HIV. AIDS Patient Care STDS 2009 Dec;23(12):1033-1041. [CrossRef] [Medline]
- van der Vaart R, van Deursen JAMA, Drossaert HCC, Taal E, van Dijk AMGJ, van de Laar AFJM. Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011;13(4):e86 [FREE Full text] [CrossRef] [Medline]
- Deursen AJAMV, van Dijk JAGM. Measuring Internet Skills. International Journal of Human-Computer Interaction 2010 Sep 17;26(10):891-916 [FREE Full text] [CrossRef]
- Dunning D, Johnson K, Ehrlinger J, Kruger J. Why people fail to recognize their own incompetence. Current Directions in Psychol Sci 2003 Jun;12(3):83-87. [CrossRef]
- McMullan M, Jones R, Lea S. Math anxiety, self-efficacy, and ability in British undergraduate nursing students. Res Nurs Health 2012 Apr;35(2):178-186. [CrossRef] [Medline]
- van Deursen AJ, van Dijk JA. Internet skills performance tests: are people ready for eHealth? J Med Internet Res 2011;13(2):e35 [FREE Full text] [CrossRef] [Medline]
- Hargittai E. Survey Measures of Web-Oriented Digital Literacy. Social Science Computer Review 2005 Aug 01;23(3):371-379. [CrossRef]
- Barnoy S, Volfin-Pruss D, Ehrenfeld M, Kushnir T. Factors affecting nurses' attitudes in Israel toward patients who present them with Internet medical information. Nurs Outlook 2008;56(6):314-321. [CrossRef] [Medline]
- Go ON UK Ltd. Webcite. London; 2013. URL: http://www.go-on.co.uk/ [accessed 2013-01-31] [WebCite Cache]
- del Hoyo-Barbolla E, Kukafka R, Arredondo MT, Ortega M. A new perspective in the promotion of e-health. Stud Health Technol Inform 2006;124:404-412. [Medline]
- Eynon R, Helsper E. Adults learning online: Digital choice and/or digital exclusion? New Media & Society 2010 Dec 06;13(4):534-551 [FREE Full text] [CrossRef]
- Office for National Statistics. Office for National Statistics. 2012. Internet Access Households and Individuals 2011 URL: http://www.ons.gov.uk/ons/dcp171766_256200.pdf [accessed 2013-08-29] [WebCite Cache]
- Rizzo M. Computational Statistics with R. In: Statistical Computing with R (Computer Science and Data Analysis). London: Chapman & Hall/CRC; 2008.
- Beaudoin DE, Longo N, Logan RA, Jones JP, Mitchell JA. Using information prescriptions to refer patients with metabolic conditions to the Genetics Home Reference website. J Med Libr Assoc 2011 Jan;99(1):70-76 [FREE Full text] [CrossRef] [Medline]
- Brewster L, Sen B. 'Quality signposting': the role of online information prescription in providing patient information. Health Info Libr J 2011 Mar;28(1):59-67. [CrossRef] [Medline]
- Burke M, Carey P, Haines L, Lampson AP, Pond F. Implementing the information prescription protocol in a family medicine practice: a case study. J Med Libr Assoc 2010 Jul;98(3):228-234 [FREE Full text] [CrossRef] [Medline]
- Chamberlain D, Heaps D, Robert I. Bibliotherapy and information prescriptions: a summary of the published evidence-base and recommendations from past and ongoing Books on Prescription projects. J Psychiatr Ment Health Nurs 2008 Jan;15(1):24-36. [CrossRef] [Medline]
- Koonce TY, Giuse NB, Storrow AB. A pilot study to evaluate learning style-tailored information prescriptions for hypertensive emergency department patients. J Med Libr Assoc 2011 Oct;99(4):280-289 [FREE Full text] [CrossRef] [Medline]
- Oliver KB, Lehmann HP, Wolff AC, Davidson LW, Donohue PK, Gilmore MM, et al. Evaluating information prescriptions in two clinical environments. J Med Libr Assoc 2011 Jul;99(3):237-246 [FREE Full text] [CrossRef] [Medline]
- Ritterband LM, Ardalan K, Thorndike FP, Magee JC, Saylor DK, Cox DJ, et al. Real world use of an Internet intervention for pediatric encopresis. J Med Internet Res 2008;10(2):e16 [FREE Full text] [CrossRef] [Medline]
- Abbott-Garner P. slideshare. Plymouth University: Plymouth University; 2012. Do superfast broadband and community interventions improve use of e-health and reduce health related car travel? Factorial cluster randomised controlled trial URL: http://www.slideshare.net/OnlineCentresFoundation/phils-phd-presentation [accessed 2013-08-29] [WebCite Cache]
- Boulos MN, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online 2011;10:24 [FREE Full text] [CrossRef] [Medline]
- Paterson C, Thomas K, Manasse A, Cooke H, Peace G. Measure Yourself Concerns and Wellbeing (MYCaW): an individualised questionnaire for evaluating outcome in cancer support care that includes complementary therapies. Complement Ther Med 2007 Mar;15(1):38-45. [CrossRef] [Medline]
- Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ 1996 Apr 20;312(7037):1016-1020 [FREE Full text] [Medline]
- Paterson C. Seeking the patient's perspective: a qualitative assessment of EuroQol, COOP-WONCA charts and MYMOP. Qual Life Res 2004 Jun;13(5):871-881. [Medline]
- Paterson C. Measuring changes in self-concept: a qualitative evaluation of outcome questionnaires in people having acupuncture for their chronic health problems. BMC Complement Altern Med 2006;6:7-7 [FREE Full text] [CrossRef] [Medline]
|GP: General Practice|
|OIS: Oxford Internet Survey|
|ONS: Office for National Statistics|
|PERQ: Patient eHealth Readiness Questionnaire|
|RCT: randomized controlled trial|
Edited by G Eysenbach; submitted 01.02.13; peer-reviewed by H Durrani, M Duplaga; comments to author 12.05.13; revised version received 24.05.13; accepted 16.08.13; published 02.09.13
©Ray Jones. Originally published in Medicine 2.0 (http://www.medicine20.com), 02.09.2013.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Medicine 2.0, is properly cited. The complete bibliographic information, a link to the original publication on http://www.medicine20.com/, as well as this copyright and license information must be included.