a) The Beta-test
A post-test evaluation design was used to evaluate the utility and acceptability of the SCI Nuggets in a sample of family physicians. Nuggets were disseminated weekly to participating physicians, over a period of about 5 months. Within one month following dissemination, physicians were contacted for data collection.
The pilot study engaged a total of 49 primary care physicians; 30 from Canada (18 from Ontario and 12 from Newfoundland) and 19 from Australia. Physicians were selected from members of the Atlantic Practice Based Research Network (APBRN) at Memorial University, the Network for Studies in Primary Care (CSPC) at Queen’s University, and the Rural Clinical School Network at University of New South Wales, Australia.
Participants were recruited to the study using a mailing on letterhead from their respective network. Mailed correspondence was followed up by telephone, to answer questions and provide additional information about participation to prospective participants
Family physicians were eligible for inclusion in the study if they:
- had at least one person with a spinal cord injury in their practice;
- had been in practice at least one year;
- had been the primary care physician for the designated patient for at least one year.
These qualities have been shown in our previous work to affect the quality of care provided to patients with complex conditions (McColl et al. 2005)
As a means of securing physician cooperation in the study, continuing medical education credits were sought and approved in the two Canadian jurisdictions. Study staff submitted applied to the College of Family Physicians of Canada to offer Mainpro M1 CME credits to physicians who agreed to participate in the study, receive the postcards, and review the supplementary web- based materials.
In general, participants were experienced practitioners with average sized caseloads for family doctors (McColl et al., 2009) In Ontario the majority of physicians were practicing in smaller urban centres under 100,000 people and were part of interdisciplinary practices that included a variety of allied health professionals. In Newfoundland the majority of physicians were located in larger urban centres and were practicing in physician-based groups without collocated allied health professionals. In Australia, all were located in small communities.
Numbers of spinal cord injury patients within individual practices was fairly similar within practice sites in Ontario, Newfoundland and Australia — Ontario (mean=1.8); Newfoundland (mean=1.7); Australia (mean = 1.9). When standardized as a number per 1000 in a caseload, patients with spinal cord injuries made up anywhere from 0.57 to 2.5 per 1000 of physician caseloads.
SCI Nuggets were disseminated in hard copy weekly over a period of twenty weeks to participating physicians. All physicians received the hardcopy postcards through regular mail. Most physicians also received electronic notification of the Nuggets, with the exception of a few participants who did not wish to do so.
b) The Evaluation of SCI Actionable Nuggets (1st ed.)
1. Knowledge
A brief, multiple-choice quiz was designed based on the information contained in the 20 Nuggets, and administered to participants at the end of the dissemination period. Participants in the pilot study assessed their own knowledge of issues of patients with SCI as fair to good prior to receiving the Nuggets, or an average of 2.91 out of 5 (SE = 0.14), — at the high end of the “fair” category.
The scores shown in the table below represent a self-rating of knowledge prior to the study, assessed importance of each issue, and post-test scores.
Physicians felt most prepared to manage depression, cardiovascular disease, osteoporosis, and pain. Physicians felt least confident about managing SCI-specific issues, such as syringomyelia, autonomic dysreflexia, wheelchair issues, pressure ulcers, and sexual health. Physicians volunteered the following issues about which they also experienced knowledge gaps:
- when to refer and to whom (6)
- UTI treatment (2)
- social/community supports available (grants/home care/tax benefits etc) (2)
- issues of access and human rights
- medication covered by provincial formulary
- current literature on SCI
- long term effects of SCI
- current treatments for common SCI related issues.
Other challenges participants identified in providing care to their SCI patients were:
- Lack of adjustable exam table (10)
- Lack of ongoing experience with condition due to low prevalence in caseload (4)
- Need to understand interactions with other health issues (4)
- Scheduling challenges because longer appointments are needed (3)
- Lack of availability of allied health professionals to help with care (2)
- Difficulty staying up to date with new advances in SCI research (2)
- Lack of education in undergraduate medicine on needs of SCI patients
- Difficulty finding time in schedule for home visits
- Difficulty addressing psychological needs.
2. Attitudes
Attitudes toward patients with SCI, were assessed by evaluating the accessibility and accommodations offered within the practice. These indicators have been shown in our previous research to be more robust than standardized measures of attitudes toward disability, which are highly subject to social desirability bias, especially among well-educated samples. The average score on accessibility was 72%. Five items are highlighted where practices were least likely to be accessible: staff familiarity with procedures for assisting disabled patients (only 55% scored positively), handrails in halls and stairways (52%), accessibility of information products (46%), adjustable exam tables (26%), lifts or hoists to assist with transfers (9%). The average score for accommodations to patients with spinal cord injuries was 79%, suggesting that physicians were willing to make many of the accommodations suggested. The least likely accommodation was arranging joint appointments with specialists. A remarkable 96% stated that they would make home visits for their patients with SCI.
3. Acceptability & Utility
A de-briefing interview with physicians was designed to elicit feedback on the Nuggets, as well as to receive suggestions and observations for revision of the Nuggets. We asked participants for their impressions of the utility and acceptability of SCI Nuggets as a method of knowledge translation for primary care.
As a general measure of utilization, we asked participants how many of the cards they had actually read. On average, participants declared that they had read 16 of the 20 Nuggets (+ 6.6). Two-thirds (65%) of participants said they reviewed all 20 cards; a further 12% (77%) reviewed at least 15 cards. The primary reason physicians offered for not reviewing cards was time constraints. Also one physician volunteered that he or she did not read cards covering topics that were already familiar.
Based on seven criteria derived from the literature for excellence in knowledge translation, participants were asked to rate the Nuggets from 1 (poor; not helpful) through 5 (excellent). The average score for utility of the Nuggets was 30 out of 35, or 86%. In general, participants rated the cards in the excellent range in all categories, with particularly high scores for professionalism, appearance and content.
c) The Evaluation of SCI Actionable Nuggets (2nd ed.)
To learn more about the evaluation of SCI Actionable Nuggets (2nd ed.) click here to access the article, “Using developmental research to design innovative knowledge translation technology for spinal cord injury in primary care: Actionable Nuggets™ on SkillScribe™”
Actionable Nuggets for Spinal Cord Injury 