The Multisite Implementation Evaluation of Tribal Home Visiting (MUSE), one of many knowledge-building projects funded by the Office of Planning, Research and Evaluation within the Administration for Children and Families, seeks to understand how home visiting services are planned and delivered in tribal communities. One of the study’s key tenets is a focus on stakeholder engagement throughout its 5-year timeline. Community input in the early stages has already led to significant changes in the study design.
Stakeholder engagement is especially important when working with tribal communities due to their complicated histories with research and evaluation, says MUSE Project Coordinator Tess Abrahamson-Richards. Even recently, tribes have shared stories of information being misrepresented outside of their communities or data being used without appropriate consent.
“MUSE aims to honor indigenous and traditional knowledge while gathering information to strengthen tribal communities through home visiting,” Abrahamson-Richards said. “This vision of respect was built into the study.”
The Road to Engagement
MUSE’s focus on substantial, iterative, and ongoing engagement aligns with principles established by its federal partners. In 2013, the Children’s Bureau released a roadmap for collaborative and effective evaluation created by its tribal evaluation workgroup.
The roadmap identifies 15 priorities for relationship, knowledge, and skill building, including—
- Evaluation practice that involves the community in determining priorities
- Dialogue with the community to anchor evaluation within cultural and ethical practices
- Mentoring for evaluation design and implementation
- Protocols for tribal approval of evaluation research
In its first year, MUSE researchers organized six webinars and held two in-person meetings with grantees of the Tribal Maternal, Infant, and Early Childhood Home Visiting Program. They also delivered a virtual orientation for evaluation consultants and formed a technical workgroup that includes grantees, consultants, and federal partners.
During one of the meetings, MUSE researchers held a “quality café,” which included small-group discussions among program managers, directors, local evaluators, and home visitors aimed at identifying what makes a successful home visiting program. MUSE researchers assigned topics to 10 meeting room tables, then asked participants to rotate from table to table. Each group elected a facilitator to guide the conversation, while a note taker remained at each table to help participants translate notes from previous groups.
The approach engaged participants with various levels of power and expertise, according to Abrahamson-Richards. Their input helped inform study questions and instruments, including the decision to use the terms “successful and effective” rather than “quality” in protocols, surveys, and questionnaires.
“People didn’t like the word quality because they felt there was too much judgment behind it,” she said. “Changing the wording also moved us away from jargon to language that is more accessible to a broader audience.”
Many Steps, Many Benefits
Effective stakeholder engagement requires significant time and preparation, including multiple opportunities for change based on community feedback. Research approval bodies representing each site will have the chance to approve research measures before the evaluation takes place. These extra steps, Abrahamson-Richards adds, will lead to increased confidence in the study design and increased likelihood that the findings will inform tribal home visiting practice.
An open mind and willingness to act on feedback can also benefit home visiting research done outside of tribal contexts. For example, an organization serving homeless youth or families may not be comfortable running a randomized control trial in which certain participants receive a different level of services. Many other communities have their own long histories of being marginalized and feeling distrustful of outsiders.
“Striving for a true partnership with the people you’re working with is incredibly valuable and universal to researchers and evaluators,” Abrahamson-Richards said. “There is always value in asking questions at the outset and learning about the context in which your study will operate.”
Learn more about MUSE in our Innovation Roundup Brief on “Home Visiting Research and Evaluation Supported by the Maternal, Infant, and Early Childhood Home Visiting Program.”