UBC Faculty Research and Publications

A Standardized Assessment of the Chinese Renal Nutrition Patient Education Materials at Providence Health… Koo, Winphia; Koh, Jiak Chin 2016

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-Koo_W_et_al_Standardized_assessment_chinese.pdf [ 76.57kB ]
JSON: 52383-1.0320961.json
JSON-LD: 52383-1.0320961-ld.json
RDF/XML (Pretty): 52383-1.0320961-rdf.xml
RDF/JSON: 52383-1.0320961-rdf.json
Turtle: 52383-1.0320961-turtle.txt
N-Triples: 52383-1.0320961-rdf-ntriples.txt
Original Record: 52383-1.0320961-source.json
Full Text

Full Text

A Standardized Assessment of the Chinese Renal Nutrition Patient Education Materials at Providence Health Care  Winphia Koo RD, BSc, Clinical Dietitian, Providence Health Care, Vancouver, BC, Canada (wkoo@providencehealth.bc.ca) Jiak Chin Koh  RD, MSc, Clinical Resource Dietitian, Providence Health Care, Vancouver, BC, Canada (jckoh@providencehealth.bc.ca)  Abstract  Objective: This study evaluated the 15 Chinese renal nutrition handouts developed at Providence Health Care (PHC) by using the Suitability Assessment of Material (SAM) tool. Methods: A quantitative, non-experimental survey design was used to assess overall suitability, content, literacy demand, graphics, layout and typography, learning stimulation/motivation, and cultural appropriateness of the handouts.  The evaluation was done using the SAM tool. Results: Eight of the 15 handouts were found to be superior and seven adequate in overall suitability.  Subscale scores identified three areas of relative weakness – lack of inclusion of summary or review, high reading grade level, and the lack of use of interactive learning stimulation. Conclusions and Implications: This systematic evaluation found that all of the 15 handouts were suitable overall.  A few relative weaknesses were identified and they will be addressed at the next revision.  This study also confirmed that the SAM is a feasible tool to evaluate patient education materials.  Key Words: kidney, diet, handouts, suitability, evaluation  Introduction Health literacy is “the ability to access, understand, evaluate and communicate information as a way to promote, maintain and improve health in a variety of settings across the life course.”1  It can influence the quality of health care experience, which relates to the ability to understand and follow medical advice2 and differential use of health care services.3,4 It is also associated with knowledge4,5 and management6-8 of diseases.  A systematic review showed that limited health literacy can increase health care costs by up to 5%.9 Sixty percent of Canadians are not capable to obtain, understand and act upon health information10,11 in large part due to the growing immigrant population with limited English or French proficiencies.11  Linguistic barriers can adversely impact quality of care, leading to misunderstandings and incomplete education,12 longer hospital stays,13 more adverse events,14 and lower patient satisfaction.12,15  The use of linguistically appropriate education materials in a culturally acceptable setting is found to associate with greater acceptability and effectiveness.16-18   According to Statistics Canada, the Chinese are the largest visible minority group in the Vancouver metropolitan area, and their numbers will continue to rise, reaching 23% by 2031.19  The 2011 Census revealed that 712,000 people in Vancouver speak an immigrant language most often at home, of whom 40% reported speaking Cantonese, Mandarin or other Chinese languages.20  In response to the need, 15 nutrition education materials in Chinese were developed by dietitians at Providence Health Care (PHC) in partnership with British Columbia Provincial Renal Agency (BCPRA).  These handouts feature culturally appropriate food and eating patterns.  Details on these handouts were previously published in the Journal of Renal Nutrition: “the Chinese Nutrition Educational Materials for Renal Patients” Vol 17, No 5 (September), 2007: p357-359, and “the Chinese Nutrition Educational Materials for Renal Patients: A 2010 Update” Vol 21, No 2 (March), 2011:ppe1-e4. Since the above publications, the Chinese renal nutrition handouts have been shared with patients and health care providers within and beyond PHC.  They are electronically accessible to over 60 renal dietitians in British Columbia, and in response to requests received, they have been shared with over 55 health care teams around the world.   Revision of the handouts was planned for quality improvement.  However, a systematic assessment had not been done to evaluate their suitability for the intended audience and to guide the revision.  This study used the Suitability Assessment of Material (SAM) instrument, an assessment tool that has been standardized and validated.21  The purpose of this study was to use the SAM to evaluate the 15 handouts for content, literacy demand, presentation, learning stimulation/motivation, and cultural appropriateness.  The findings would help identify areas for improvement in the revision of these education materials.  Methods  Chinese Renal Nutrition Patient Education Materials   Fifteen handouts were evaluated in this study (Table 1).  Table 1. CHINESE RENAL NUTRITION PATIENT EDUCATION MATERIALS THAT WERE EVALUATED IN THIS STUDY  Title of handout Type of material Number of pages 1. Low Sodium Diet Tables with text 3 2. Low Phosphorus Diet Tables with text 2 3. Potassium (short version) List 5 4. Potassium (long version) List 9 5.  Protein Is Good for Your Body Text-based 2 6. How To Use Your Protein Powder Tables with text 2 7. Fluid Restriction Text-based 1 8. Tips To Control Your Fluid Intake And Thirst Text-based 2 9. Acceptable Blood Values For Dialysis Patients Table with blanks 5 10. Grocery List For Kidney Patients List 5 11. Grocery List For Hemodialysis Dialysis Patients List 5 12. Grocery List For Peritoneal Dialysis Patients List 5 13. Healthy Eating For Diabetes And Your Kidneys Guide 3 14. Meal Planning Guide for Diabetes And Kidney Disease Guide 8 15. Diet Tips To Lower Cholesterol For the Renal Diet Text-based 4   Suitability Assessment of Materials (SAM) Instrument   The Suitability Assessment of Materials (SAM), developed by Doak et al. at the John Hopkins School of Medicine, is an education material evaluation tool that has been validated with individuals from a variety of cultural backgrounds.21  The SAM evaluates education materials on 22 subscales grouped into six categories: content, literacy demand, graphics, layout and typography, learning stimulation and motivation, and cultural appropriateness.  Education material is overall scored as “Superior” (70-100%), “Adequate” (40-69%), or “Not Suitable” (0-39%).  Individual subscale scores highlight specific deficiencies that require improvement.  Mei-Chuan Chang, a doctoral student at the National Taiwan University, translated the SAM, made necessary modifications for the Chinese language, and validated it for use on Chinese materials.22  Specifically, modifications for Chinese texts were done on three of the 22 SAM subscales: reading grade level, layout factors, and typography.  Reading level was determined by the online Readability Assessment System for Chinese reading materials developed at the National Kaohsiung Normal University, Taiwan.23    Participants and Procedures   All PHC dietitians who were able to read Chinese, understood the Chinese culture, had experience in providing nutrition counseling to Chinese speaking patients, and were not directly involved in the development of the handouts being assessed in this study were invited to participate in our study.  Seven dietitians at PHC met the study inclusion criteria, of whom six provided consent to participate.    Upon consenting to participate, the reviewers were invited to a group workshop to discuss the SAM evaluation criteria and to practice using the SAM on a Chinese education material unrelated to the study.  Discrepancies were discussed to ensure consistent interpretation and application of the SAM criteria.  Three subscales “Reading Grade Level,” “Layout Factors” and “Typography” were modified according to the Chinese SAM instrument.  The subscale “Cover Graphic Shows Purpose” did not apply to our handouts and, by instruction, was excluded from the scoring.  The number of pages of our 15 handouts ranged from one to nine.  While the SAM tool suggests only assessing three pages for long materials, the investigators and reviewers decided that all our handouts would be evaluated in their entirety.  Assessment time was estimated at 20 minutes per handout.  At the end of the workshop, each reviewer was randomly given a sealed envelope containing a manageable collection of five handouts.  Therefore, each of the 15 handouts was evaluated by two reviewers, thereby producing an averaged overall SAM score.  The reviewers were given two weeks to independently evaluate the materials.  Score sheets were coded without personal identifiers. Completed score sheets were mailed back to a research assistant who prepared a summary of scores, as well as determined the “Reading Grade Level” subscale for all 15 handouts by using the online Chinese Readability Assessment System as mentioned above.  Results  As shown in Table 2, eight of the 15 handouts received a “Superior” SAM overall rating, seven “Adequate”, and none “Not Suitable.”  Average percentage scores ranged from 57-73%, overall averaging 69%.     Table 2.  OVERALL SUITABLILITY OF CHINESE RENAL NUTRITION PATIENT EDUCATION MATERIALS BASED ON THE SUITABILITY ASSESSMENT OF MATERIALS (SAM) INSTRUMENT  Title of Handout Scores from Two Reviewers Average  Score SAM Rating* Low phosphorus diet 71%, 74% 73% Superior Potassium (short) 67%, 79% 73% Superior Potassium (long) 69%, 78% 73% Superior Tips to control your fluid intake and thirst 71%, 76% 73% Superior Low sodium diet 64%, 80% 72% Superior How to use your protein powder 67%, 75% 71% Superior Acceptable blood values for dialysis patients 65%, 76% 71% Superior Healthy eating for diabetes and your kidneys 69%, 71% 70% Superior Protein is good for your body 67%, 71% 69% Adequate Grocery list for kidney patients 60%, 78% 69% Adequate Grocery list for peritoneal dialysis 54%, 75% 65% Adequate Fluid restriction 62%, 68% 65% Adequate Meal planning for diabetes and kidney disease  62%, 66% 64% Adequate Grocery list for hemodialysis patients 57%, 69% 63% Adequate Diet tips to lower cholesterol for the renal diet 57%, 57% 57% Adequate *Overall suitability score using the SAM: Superior 70–100%; Adequate 40–69%; Not Suitable 0–39%.                    Table 3 provides the aggregate results of the entire handout collection in each subscale.  Particularly noteworthy are the three of the 22 subscales which were often scored as “Not Suitable,” namely inclusion of summary or review, reading grade level and the use of interactive learning stimulation.  Table 3. RESULTS FOR SAM SUBSCALE SCORES  SAM Subscales Superior Score of 2 n (%) Adequate Score of 1 n (%) Not Suitable Score of 0 n (%) Not applicable  n (%) 1. Content a. Purpose is evident b. Content about behaviours c. Scope is limited d. Summary or review included  30 18 26 1    (100%) (60%) (86.7%) (3.3%)   12 4  2     (40%) (13.3%) (6.7%)     27      (90%)       2. Literacy Demand a. Reading level grade (Chinese online software)  b. Writing style, active voice c. Vocabulary uses common words d. Context is given first e. Learning aids via “road signs”    21 11 17 22    (70%) (36.7%) (56.7%) (73.3%)  8    6   19 11 6   (26.7%)  (20%) (63.3%) (36.7%) (20%)  22   2    2   1    (73.3%)  (6.7%)  (6.7%) (3.3%)    1    1     (3.3%)   (3.3%) 3. Graphics a. Cover graphic shows purpose b. Type of graphics c. Relevance of illustrations d. List, tables, etc explained e. Captions used for graphics   24 18 17 23   (80%) (60%) (56.7%) (76.7%)   6   12 13 6     (20%) (40%) (43.3%) (20%)      1        (3.3%)  N/A by design    4. Layout and Typography a. Layout factors (Chinese SAM) b. Typography (Chinese SAM) c. Subheads “chunking” used   22  13  (73.3%)  (43.3%)  8   30 7    (26.7%) (100%) (23.3%)    3      (10%)    7     (23.3%) 5. Learning stimulation/Motivation a. Interaction learning stimulation used b. Behaviours are modeled and specific c. Motivation – self-efficacy  3    16  16  (10%)  (53.3%)  (53.3%)  3    14  13  (10%)  (46.7%)  (43.3%)  24     1    (80%)    (3.3%)   6. Cultural Appropriateness a. Match in logic, language, experience b. Cultural image and examples   22  1    (73.3%)  (3.3%)  8    29  (26.7%)  (96.7%)          Discussion  Evidence supports that culturally tailored, language specific, written education materials contribute to awareness of and access to health promoting practices.24  Accordingly much time and effort have been spent on developing patient education materials.  However, good intentions alone will not always suffice, especially when dealing with linguistically, culturally and educationally diverse populations.  It is therefore important to objectively evaluate patient education materials.  The PHC/BCPRA Chinese renal nutrition patient education materials have achieved widespread international recognition and use.  Yet they previously had not undergone objective evaluation of their quality.  Ideally, a focus group of the intended audience would be used to evaluate the materials.  However, focus groups can be subjective, and, in our case, would have been overly costly due to the extensive collection of our handouts.  A more practical method was to employ an objective, tested, and validated tool.  Through the instrument of SAM, this study has allowed us to evaluate the renal nutrition handouts in a timely and fiscally sound manner.  The overall suitability scores for our materials were rather tight, mostly between 60-70%, demonstrating consistency across the handouts.  By contrast, a study evaluating 29 prostate cancer materials showed a much wider variation in quality, ranging from 39 to 80% in the overall SAM scores.25   Subscale scores highlighted three areas of relative weakness in our handouts.  In content, a summary is needed.  Secondly, most of our materials were rated as having reading levels above the 9th grade.  Designing patient education materials with appropriate reading levels is challenging: 90% of patient education materials for prostate cancer were scored as not suitable for reading level,25 while 88.9% of materials for stroke patients and caregivers had a level above the 8th grade,26 beyond the skills of 52.6% of their cohort.26  These findings support the SAM rating criteria for reading levels between 6th to 8th grades being “adequate” and at 5th or lower grades as “superior.”  Thirdly, the handouts was deficient in learning stimulation, with formats that feature “question-and-answer” or “presentation of problems and questions for reader responses” being desirable.  All of the above will be considered in the upcoming revision of the handouts. There are several issues to be considered in the application of the SAM instrument both specifically to our study and in general.  There is subjectivity in the interpretation of criteria and scores.  We attempted to minimize this by having a workshop for the reviewers to establish consistency in interpreting the criteria.  In addition we had two reviewers independently rate each of our materials and their scores were averaged.  Although we used the SAM in a formal study setting, we wish to note that it is also intended for use in an informal setting by even a single care provider to assess education materials.21  Many patient education materials available today may not undergo any formal evaluation. In this study, we have shown that our internationally recognized handouts, while indeed good, had room for improvement.  Our results showed that objective assessment of patient education materials is both necessary and achievable.  While such may not always need to be undertaken in the academic manner of this study, it is our hope that our successful experience will lead to more widespread assessment of patient education materials, and thereby ultimately have a positive impact on patient care.   Implication for Research and Practice   This study demonstrated that the SAM tool is systematic, objective, easy to use and cost effective in the evaluation of the renal nutrition handouts in Chinese.    References 1. Rootman I, Gordon-El-Bihbety D. A vision for a health literate Canada: report of the expert panel on health literacy; 2008. http://www.cpha.ca/uploads/portals/h-l/report_e.pdf   Accessed May 29, 2015. 2. Canadian Council on Learning. Health literacy in Canada: a healthy understanding; 2008. http://www.ccl-cca.ca/pdfs/HealthLiteracy/HealthLiteracyReportFeb2008E.pdf  Accessed May 29, 2015. 3. Berkman ND, DeWalt DA, Pignone MP, Sheridan SL, Lohr KN, Lux L. Literacy and health outcomes; 2004. Evidence report/technology assessment no. 87. http://archive.ahrq.gov/downloads/pub/evidence/pdf/literacy/literacy.pdf  Accessed May 29, 2015. 4. Berkman ND, Sheridan SL, Donahue KE, Harlpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Annals of Internal Medicine. 2011;155:97-107. 5. Williams MV, Baker DW, Parker RM, Nurss JR. Relationship of functional health literacy to patients’ knowledge of their chronic disease: a study of patients with hypertension and diabetes. Archives of Internal Medicine.1998;158(2):166-72. 6. Peterson PN, Shetterly SM, Clarke CL, Bekelman DB, Chan PS, Allen LA, Matlock DD, Magid DJ, Masoudi FA. Health literacy and outcomes among patients with heart failure. Journal of American Medical Association. 2011;305(16):1695-701. 7. Davis TC, Wolf MS, Bass PF, Thompson JA, Tilson, HH, Neuberger M, Parker RM. Literacy and misunderstanding prescription drug labels. Annals of Internal Medicine. 2006;145(12):887-94. 8. Kripalani S, Henderson LE, Chiu EY, Robertson R, Kolm P, Jacobson TA. Predictors of medication self-management skill in a low-literacy population. Journal of General Internal Medicine. 2006;21(8):852-6.  9. Eichler K, Wieser S, Brugger U. The costs of limited health literacy: a systematic review. International Journal of Public Health. 2009;54(5):313-24.  10. Canadian Council on Learning. Health literacy in Canada: initial results from the International Adult Literacy and Skills Survey; 2007. http://www.ccl-cca.ca/pdfs/HealthLiteracy/HealthLiteracyinCanada.pdf  Accessed May 29, 2015.  11. Simich L. Health literacy and immigrant populations; 2009. http://canada.metropolis.net/pdfs/health_literacy_policy_brief_jun15_e.pdf  Accessed May 29, 2015. 12. Woloshin S, Bickell NA, Schwartz LM, Gany F, Welch HG. Language barriers in medicine in the United States. Journal of the American Medical Association. 1995; 273(9):724-8. 13. John-Baptiste A, Naglie G, Tomlinson G, Alibhai SMH, Etchells E, Cheung A, Kapral M, Gold WL, Abrams H, Bacchus M, & Krahn M. The effect of English language proficiency on length of stay and in-hospital mortality. Journal of General Internal Medicine. 2004;19:221-8. 14. Divi C, Koss RG, Schmaltz SP, Loeb JM. Language proficiency and adverse events in US hospitals: a pilot study. International Journal for Quality in Health Care. 2007;19(2):60-7. 15. Weech-Maldonado R, Elliott MN, Morales LS, Spritzer K, Marshall GN. Health plan effects on patient assessments of Medicaid managed care among racial/ethnic minorities. Journal of General Internal Medicine. 2004;19:136-45. 16. Cooper LA, Hill MN, Powe NR. Designing and evaluating interventions to eliminate racial and ethnic disparities in health care. Journal of General Internal Medicine. 2002;17:477-86. 17. Hawthorne K. Asian diabetics attending a British hospital clinic: a pilot study to evaluate their care. British Journal of General Practice.1990;40:243-7. 18. Song H, Han H, Lee J, Kim J, Kim KB, Nguyen T, Kim MT. Translating current dietary guidelines into a culturally tailored nutrition education program for Korean American immigrants with Type 2 Diabetes. The Diabetes Educator. 2010;36(5):752-61. 19. Statistics Canada. 2006 Community Profiles. 2006 Census (Catalogue no. 92-591-XWE). Vancouver, BC (Code 933); 2007. http://www12.statcan.ca/census-recensement/2006/dp-pd/prof/92-591/index.cfm?Lang=E  Accessed May 29, 2015. 20. Statistics Canada. 2011 Census of population: linguistic characteristics of Canadians. (Catalogue No. 98-314-X2011001); 2012. http://www12.statcan.gc.ca/census-recensement/2011/as-sa/98-314-x/98-314-x2011001-eng.pdf  Accessed May 29, 2015. 21. Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills: 2nd ed. Philadelphia, PA: Lippincott; 1996. 22. Chang, MC (2011, July 26). E-mail conversation. meichuan2@gmail.com. 23. Industrial Technology Education Department, National Kaohsiung Normal University of Taiwan.  (n.d.). Online readability assessment system for Chinese materials. Retrieved from 24. Ahmad F, Cameron JI, Stewart DE. A tailored intervention to promote breast cancer screening among South Asian immigrant women. Journal of Social Science & Medicine. 2005;60:575-86. 25. Weintraub D, Maliski SL, Fink A, Choe S, Litwin MS. Suitability of prostate cancer education materials: applying a standardized assessment tool to currently available materials. Patient Education and Counseling. 2004;55:275-80. 26. Hoffmann T, McKenna K. Analysis of stroke patients’ and carers’ reading ability and the content and design of written materials: recommendations for improving written stroke information. Patient Education and Counseling. 2006;60:286-93.   Acknowledgements:  The authors would like to thank Aggie Black, PHC Research Challenge, PHC Renal Program, PHC Clinical Nutrition, our research assistant Angela Yee Kay Kwok, our research mentor Ann Vanderbijl, BCPRA, and all dietitians, volunteers and patients who helped with the development and revision of the Chinese renal education materials. 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items