BRIEF COMMUNICATION

Year : 2010  |  Volume : 23  |  Issue : 2  |  Page : 386

Academic-Community Partnership: An Orientation for Employees of a Community Mental Health Agency to Its Research Program

JM Gonzalez¹, DI Velligan², L Lopez³, J Mintz², GR Bond⁴, AL Miller^2
1 University of Texas Health Science Center, Psychiatry Department, San Antonio, Texas, USA
² University of Texas Health Science Center, USA
³ Center for Health Care Services, San Antonio, USA
⁴ Dartmouth Medical School Department of Psychiatry, Lebanon, USA

Correspondence Address:
J M Gonzalez
University of Texas Health Science Center
USA

Source of Support: None, Conflict of Interest: None

Check

PMID: 20853242

Context: Community agency employees' interest and involvement in academic-community research partnerships are keys to successful collaborations. One main barrier to success can be employees' lack of knowledge about research. We present data on an "orientation to research" program for community agency employees in a large U.S. city designed to improve knowledge about research in general and that specific to the agency.
Methods: We developed an agency intranet website, a scavenger hunt to facilitate learning through the intranet research website, and a ten-item quantitative knowledge assessment tool. Academic and agency partners were actively involved in the design of the program and its evaluation.
Findings: More educated and long-term employees had higher pre-test scores but not post-test scores. Significant improvement in post-test scores was observed for employees after completion of the program. Informal feedback about course content and the academic-community partnership was positive.
Conclusions: This report examines the feasibility of a structured knowledge program targeted at community agency employees at all levels within an agency. We believe that this approach is generalizable to other settings to the extent that there are shared interests, resources, and investment of the academic partner and agency.

Keywords: Academic-community partnership; research knowledge; program evaluation

Introduction



Models for conducting community-based health research emphasize the importance of shared goals, mutual acceptance of research, and respect between partnering organizations^1-4. One strategy for improving inter-agency communication is for university researchers to systematically educate staff at community agencies about research, addressing topics like ethical conduct of research (including institutional review boards), project development, grant writing, research methods and statistical analyses^3,5-7. However, in employing educational strategies, the importance of including agency staff at all levels of the organization is often overlooked. In a prior survey of attitudes and views of academic-community partners, we found that community mental health center staff, such as case managers and clerical staff, had minimal knowledge about research yet were interested in having more information about their agency’s ongoing research efforts⁸. They reported that they had been approached by clinic patients interested in research projects and they were curious about the type of research being conducted at their agency. Providing information about the organization’s research involvement to all community agency staff could substantially improve the success of research, as they play pivotal roles in promoting research and carrying out its protocols. We found no published reports of structured educational programs aimed at increasing general knowledge about research for community agency staff at all levels of the organization, including the staff most involved in day to day aspects of research but who are often not involved in projects’ initial development, ongoing inter-agency research collaborations and the application of project findings.



In this paper we report on the process of developing and implementing a research orientation program for community agency employees that included a broad overview of research in general and of their agency’s specific involvement in research. Academic faculty at a local university and agency staff of the local county mental health provider collaborated to develop the “orientation to research” program for new and ongoing (i.e. long term) employees. We present data on initial program outcomes measured by assessing participants’ knowledge prior to and following participation in the program.



Methods



San Antonio, Texas USA is a metropolitan community with 1.3 million residents. The University of Texas Health Science Center in San Antonio has over 3,000 students and is the only medical school in the South Central region of Texas. The local mental health agency, the Center for Health Care Services, provides care to up to 11,000 children, adolescents and adults each year in outpatient settings, treating a wide range of psychiatric disorders with an emphasis on serious mental illness and substance abuse. Researchers at the University of Texas Health Science Center and staff at the Center for Health Care Services co-developed the program.







Figure 1:  Research Home Page of the Center for Health Care Services Website



In a prior survey, agency employees reported there was no central location to access information about research ongoing within their agency and indicated that the internet would be the preferred location for such information⁸. The aims of the research orientation program for staff were 1) to increase knowledge about the agency’s research involvement, 2) to make information about the agency’s research more accessible, and 3) to improve attitudes toward research. The program consists of the following components: an agency intranet research website (see Figure 1), a “scavenger hunt” to help staff learn about the intranet research website, and a ten-item quantitative knowledge assessment tool (see Tables 1 and 2). Table 1 provides an overview of the content of the intranet site, which is updated regularly and serves as a resource for information on current projects as well as provides educational information about research more generally. To familiarize participants with the website, the scavenger hunt facilitates and encourages accessing the various research sites and related information (see Table 2).



Table 1:  Intranet Site Content Areas: A site used by agency employees to learn about research and research practices within the agency.









Table 2:  Pre- (and Post-)Assessment and Scavenger Hunt for New and Ongoing Employees







Researchers and agency staff developed a ten-item test for pre- and post-training knowledge assessment. Some test items were included to assess knowledge of general research topics that would facilitate greater understanding and communication about research studies (e.g., defining “randomization” and discussing the concept of “voluntary withdrawal” from research protocols). Agency specific questions were also included, such as who to contact within the mental health agency with concerns or questions about research (see Table 2).



The agency piloted the orientation program (i.e. in-person introduction of the website, the pre-and post-knowledge assessments and the scavenger hunt) with six employees at various levels within the agency, recording length of time to complete the entire program and eliciting feedback. Based on the pilot experience, several modifications were made to the website, assessment and scavenger hunt. For example, agency employees noted that the official study titles were replete with scientific jargon and thus study aims were not easy to understand. We made changes to simplify the terminology. The general format of the program was generally well-received and understood by employees. The research orientation was instituted as an annual employee requirement. In-person group orientation led by a program trainer is the standard method for orienting new employees at the agency, while self study is standard for ongoing employees. The new employee orientations were conducted in-person with 4 to12 employees lasting an hour to an hour and a half. The program trainer provided a 5-minute rationale for the program and a 20-minute introduction to the intranet, pre- and post-assessments, and the scavenger hunt. The program trainers were agency employees with bachelor’s degrees in a behavioral science who were participating in the university research projects. Ongoing employees completed the annual requirement on their own.



Participants for whom findings are reported here were new and ongoing employees scheduled for orientations between April 2008 and January 2009. Employees required to participate were those involved in any of the clinic operations (i.e. with direct patient contact). Participants completed pre- and post-training tests and the scavenger hunt and handed their completed exams to human resources personnel. A grade of 80% on the post-test was considered passing. Employees who did not score 80% reviewed their missed questions with the agency research coordinator to ensure comprehension of missed items.



We present pre- and post-test scores for employees by the level of education required for their positions (up to high school; bachelors; graduate degree) and separately for new and ongoing employees. Comparisons of employees in the several educational groups on baseline scores were made using chi-square analyses and we carried out paired t-tests and analyses of variance for pre-post changes in group scores. The study was approved as non-research by the University of Texas Health Science Center Institutional Review Board and the Center for Health Care Services Research Review Committee.



Results



Pre- and post-training tests were completed by 307 employees (120 new, 187 ongoing employees). Total numbers of new and ongoing employees during this time period are not known thus participation rates cannot be formally calculated; nevertheless, we estimate a participation rate of 75-80%. Of participants, 135 had positions requiring a high school education, 96 had positions requiring a bachelor’s degree and 76 had positions requiring graduate degrees. Ongoing employees were more educated (32% of existing staff had post-college education versus 13% of new employees, X²  = 26.7, df = 2, p <0.0001) (see Table 3). Prior to training, 33% of the total sample scored a perfect 10/10. Of those, 28% (n=28) then scored either a 9 or 8 on post-test. No particular item was noted to be incorrect on the post-test. In an analysis of variance of pre-test scores, with employee type and education level entered as design factors, ongoing employees scored significantly higher than new employees (9.08 ± .08 versus 8.69 ± .11, F = 7.0, df =1,303, p = 0.005). Employees in positions with higher education levels had significantly higher pre-training scores than those with lower education levels (8.55 ± .09, 8.87 ± .11 and 9.24 ± .15, p <0.001).



Table 3:  Frequency Count of Employee Status by Education Level.







In the total sample, the overall mean pre-training score was 8.85 ± 1.12 and mean post-training score was 9.23 ± .81, showing a significant improvement (paired t = 6.64, df = 307, p <0.001). A 2 (employment status) x 3 (education level) factorial analysis of variance of pre-post change indicated that new employees improved more in their scores than ongoing employees (F = 8.2, df =1, 301, p <0.005) and persons in positions requiring no more than a high school education improved to a greater degree than persons in positions requiring more education (F = 6.6, df =2,301, p <0.002). Based on post-training scores, only 2% (n=7) of the sample did not meet the 80% correct criterion to pass.



Some employees provided informal feedback after the orientation. Among comments were: “I didn’t realize the Center was that involved in research”; “How can I get involved in the research projects you have?” and “Those sound like some very interesting studies”.  Numerous staff reported that research involvement gave the agency more prestige. Other staff noted they had seen researchers in their clinics but never knew what their purpose in the clinic was. Several noted the training helped them to understand the researchers’ role and indicated that they had plans to introduce themselves at the next opportunity.



Discussion



In an orientation to research program, community agency employees showed significant improvement in research knowledge based on scores of a brief assessment. This is the first such evaluation we are aware of for a research orientation program implemented for all employees of a service organization. We aimed to improve research knowledge, while as a secondary outcome we hoped to improve attitudes toward research, which is a common barrier to successful community-based research. Although there are numerous strategies for educating involved agency personnel on grant writing and related topics^3,5-7, other community staff within their agencies are affected by, and affect, research within the agency. Informal feedback suggests the knowledge provided through our program was appreciated by employees, even by those whose jobs keep them on the periphery of the research⁸. This program is straightforward and can feasibly be individualized and implemented at other community agencies.



On average, baseline knowledge about research was good, as 89% of employees obtained a passing score on pre-assessment. Scores were particularly good at baseline for ongoing employees and those in positions requiring graduate degrees. Most of the other two-thirds of employees significantly improved their knowledge of research after program participation. The scavenger hunt task was completed but not scored by agency staff; however, informal feedback indicated the hunt was a primary tool for learning about the website. We did not receive informal feedback from ongoing employees completing the orientation on their own, thus these informal conclusions are limited to feedback by employees who received the training in-person. Other than in pilot testing, we did not seek formal feedback which would have aided in understanding how employee subgroups viewed the module. We recommend orientations for new employees be conducted in-person when feasible to allow orientation staff to field questions, to increase interactions among employees about research, and to highlight the agency’s interest and investment in research for employees. As another limitation, we note that the finding that persons in jobs requiring lower education levels improved more from pre- to post-program testing than persons in jobs requiring more education may be due to a ceiling effect in the assessment instruments. A final limitation is that some employees did not complete the research module (approx. 20-25%) even though agency rules required them to do so, and we do not know if their pre-post scores or reactions to the program would have differed from our study employees.



This orientation was feasible and generalizable, to the extent that the academic and agency partners share interests, resources and investment in research. In our case, the agency took the lead role for website development, conducted the in-person orientations, scored the assessments, and updated website information on current research projects. Development and evaluation of programs to improve the quality of the academic-community partnership in other settings should consider the education level of employees and whether they are new to the agency or already familiar with its practices. In future studies, we plan to examine program impact on attitudes toward research and employee involvement in research.



Readers interested in further information about the program development, content and strategies can contact Linda Lopez: [email protected].



Acknowledgement



This manuscript was supported by NIMH Grant R24MH072830. We express great appreciation to employees of the Center for Health Care Services.



References



1. Christopher S, Watts V, McCormick AK, Young S. Building and maintaining trust in a community-based participatory research partnership. American Journal of Public Health. 2008; 98: 1398-1406.



2. Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annual Review of Public Health. 1998; 19: 173-202.



3. Israel BA, Schulz AJ, Parker EA, Becker AB, Community-Campus Partnerships for Health. Community-based participatory research: policy recommendations for promoting a partnership approach in health research. Education for Health. 2001; 14: 182-197.



4. Wells KB, Staunton A, Norris KC, Bluthenthal R, Chung B, Gelberg L, Jones L, Kataoka S, Koegel P, Miranda J, Mangione CM, Patel K, Rodriguez M, Shapiro M, Wong M. Building an academic-community partnered network for clinical services research: the Community Health Improvement Collaborative (CHIC). Ethnicity and Disease. 2006; 16: S3-17.



5. Cashman SB, Adeky S, Allen AJ, Israel BA, Montaño J, Rafelito A, Rhodes SD, Swanston S, Wallerstein N, Eng E. The power of the promise: Working with communities to analyze data, interpret findings and get to outcomes. American Journal of Public Health. 2008; 98: 1407-1417.



6. Dobransky-Fasiska D, Brown C, Pincus HA, Nowalk MP, Wieland M, Parker LS, Cruz M, McMurray ML, Mulsant B, Reynolds CF 3rd, RNDC-Community Partners. Developing a community-academic partnership to improve recognition and treatment of depression in underserved African American and white elders. American Journal of Geriatric Psychiatry. 2009; 17: 953-964.



7. Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. Journal of the American Medical Association. 2007; 297: 407-410.



8. Gonzalez JM, Miller AL, Lopez L, Milam C. Voices from the community: Challenges and opportunities. Proceedings of the National Council for Community Behavioral Healthcare 38th Conference. May1-3, Boston, Massachusetts; 2008.