Better Outcomes for Substance Abuse Addiction: Spotlight on Leonard Jason and the Oxford Houses

Alcohol and drug addiction is a complex and pervasive chronic condition that causes substantial harm, not only to individuals suffering from addiction, but also to those around them. Recognizing the importance of understanding this complex condition, a team of Chicago-based community psychologists, led by Dr. Leonard Jason of DePaul University, has focused its efforts on studying recovery outcomes for those in recovery from addiction. 

Oxford House: Self-Help for Sobriety without Relapse

At a young age, Dr. Jason discovered the atrocities of the Holocaust, and this knowledge motivated him to work for social change. He understood that most community changes are not brought about by authority figures such as politicians, but by ordinary members of the community. After learning about the Oxford House model, he recognized it as a prime example of individuals successfully using the support of their communities to overcome their difficulties. For over 20 years, Dr. Jason and his team have been researching the role that Oxford Houses play in substance abuse recovery.

Oxford Houses (OH) are self-governing recovery homes that are rented by six to ten people who are in recovery from alcohol or drug addiction. Oxford Houses are run democratically, require residents to pay an equal share of rent and expenses, and have a zero tolerance policy toward alcohol and drug use. What makes OH’s distinct from similar institutions is that there are no residency time limits, something that current addiction research deems very important, as there is no universal timeline for a successful recovery process. 

For years, the results of Dr. Jason’s research have provided food for thought to many who are interested in addiction and recovery within the scientific community. Through a number of multi-million dollar grants from the National Institute of Health, Dr. Jason and his team have empirically validated the positive effects of Oxford House residency when compared to usual care facilities, such as higher abstinence rates (69% vs. 35%), higher monthly income ($989 vs. $440), and lower incarceration rates (3% vs. 9%). These findings led the Substance Abuse and Mental Health Administration (SAMSHA) to recognize the OH model as a successful recovery model and also resulted in the reinstatement of Illinois’s revolving fund to help open new houses, among other benefits. Outside of academic publications, descriptions of Dr. Jason’s past and current research are systematically distributed at the Oxford House conferences.

Ed Stevens, a graduate student in DePaul’s doctoral Community Psychology program, believes that the team’s research answers three very important questions about the OH model:   1) Does it work? 2) Can it be replicated? and 3) Can its elements be used for other systems? Ed calls OH a “grassroots model” and a “classic form of empowerment” in which people are trying to solve their own problems and overcome their own obstacles. Olya Belyaev-Glantsman, a doctoral student in DePaul’s Community Psychology program who has worked with Dr. Jason since 2001, states, “It feels great knowing that thousands of lives are positively affected by Dr. Jason’s work.”  Ed Stevens adds, “I love the topic. I love people we are involved with; it’s an honor to work with them.” 

The feedback from Oxford House representatives on their experience working with the DePaul University research team is also overwhelmingly positive. For example, when asked about his involvement with the OH research team, Rory, a state coordinator and an Oxford House resident for seven years, said that it makes him feel good to be a part of this team, and that he is glad the team sought his help. “After being part of a problem for so long, I feel like I am now part of the solution,” he continued. Rory has been working with DePaul’s team for three years, helping with recruiting, tracking, and placing individuals into Oxford Houses. He also holds another important role – through his own experience, as well as his active involvement with Oxford House, Inc., he helps the team to better understand the addiction and the recovery process. Rory laughs when he reflects on his first experience at the team’s weekly meeting: “[Coming in with no research skills,] at first I was lost,” but over the years he has become comfortable interpreting data and is often the one the team turns to when it needs help explaining the results of its studies.

The topic of recovery is extremely important and through this truly collaborative partnership, the Oxford House Inc. and the DePaul’s research team are constantly sharing resources and learning from each other on the road to better outcomes for substance abuse addiction. The years of systematic research by Dr. Jason’s team helped raise awareness of this successful and cost effective recovery approach.

Olya Belyav-Glantsman
 DePaul University 

______________________________

For more information about Oxford House Inc., see http://www.oxfordhouse.org/userfiles/file/

For more information on Dr. Jason’s work, see http://condor.depaul.edu/ljason/

To see Dr. Jason talking about OH, see http://www.youtube.com/watch?v=15lIOw1VVCw

This is the second in a series of bulletins highlighting the use of community psychology in practice, written by the members of the Outreach Group of the Community Psychology Practice Council. For more information about the series, contact Bill Berkowitz at Bill_Berkowitz@uml.edu.  

Up to $1200 for your Community Project! Applications are Open for the 2012 SCRA Mini-Grants

Free money for my community? Why, sure!

Welcome back to the SCRA blog! We're happy to be back after a busy summer, and have great news - the second round of SCRA Community Mini-Grants are now available!

The SCRA Community Mini-Grant  

Who Can Apply:  Current SCRA members, along with their community/organizational partners

What:  Funding (up to $1,200) for small, time-sensitive, community-based projects; up to ten awards available annually.  Grant criteria include timeliness, potential for success/impact, community involvement, and alignment with SCRA’s vision, mission and values.  Grantees must be willing to share the process and outcome of their funded actions with others. 

Where:  Communities across the globe.

When:  Completed pre-application forms are accepted on a rolling basis.

Why:  Because SCRA wants to help communities in “real-time” and recognizes that a small amount of funding can often make a meaningful difference. 

How:  Find the application online at  www.scra27.org/communityminigrants.  Pre-Application forms will be reviewed within a week to ten days of submission, at which point some applicants will be invited to fill out the longer, 3-page application form for funding.  All pre-applications and applications will be subject to a blind-review process.

The Community Mini-Grant is generously funded by SCRA and administered by the Practice Council.   

Questions? Clarifications? Email: SCRACommunityGrants@gmail.com

***At least one person associated with each application must be a current SCRA member.  Additionally, the Mini-Grant funds cannot be used to support projects that are already being financially supported by SCRA (e.g., through the Policy Grant or various student research grants).***

Community Psychologists: Boosters for the Local Economy

Every so often, it’s helpful to take a step back and realize how many intriguing community ideas are out there. Creative, feasible, and potentially powerful ideas. Tested and replicable ideas, ripe for the taking. And simple ideas – sometimes so simple that you might wonder, “Why didn’t I think of that?”

But with the Web, you needn’t think so hard; you mostly need to know where to find good ideas and put them to use. In other words, there are many available ways to do business in the community; both figuratively – or in the case of this blog post, literally.

For example: In this presidential year, where jobs and the economy are dominating issues, we might consider ideas community psychologists could apply to boost the local economy. I can mention three that recently crossed my path:

* Do you have a favorite local business? If so, you can get together with others to support it, in an organized way, by being part of a cash mob. Just mobilize your friends and neighbors, agree on a date and time, and show up at the store with cash in hand. Then simply spend money, hopefully on items of value to you. Cash mobs, a flash mob spinoff, have become something of a national phenomenon. More at www.cashmobs.com.

* No cash to spend? Then invent your own. Many of us have become familiar with homegrown currencies, such as Ithaca Hours, or BerkShares, in Massachusetts (www.ithacahours.com, and www.berkshares.org). Another very simple ideas was Downtown Dollars, in Ardmore, Pennsylvania, where merchants sold Downtown Dollars at a 50% discount to Ardmore area shoppers, who could then spend those dollars at local stores. See http://money.cnn.com/galleries/2012/pf/1201/gallery.community-currencies/7.html.

* Got a business idea? Then pitch it. In Grand Rapids, Michigan, five contestants had five minutes each to pitch their best business idea to a local jury, with prizes of up to $5000. One idea that caught my eye was a creating a “signature sandwich” for Grand Rapids, to put it on the culinary map. (See http://www.npr.org/2011/03/18/134930914/In-Michigan-Entrepreneurs-Pitch-Ideas-for-5-000.) Could you do something like that where you live?

These ideas just skim the surface, without yet touching on individual creative ideas some of your acquaintances have probably had. What an asset it would be to have a useful compendium of these and other community ideas – but that’s for another posting.

Can community psychologists be “job creators”? Not directly, as a rule.

We don’t hear much about our colleagues starting businesses – although they, and we, could. True, most of us may not aspire to become Captains of Industry, or even CEO’s; but we can surely support local businesses, encourage new ones, and apply our knowledge and skills to create a local economic climate where community psychology values will take hold and flourish. 

Bill Berkowitz

University of Massachusetts Lowell

Profiles of Applied Arts-Based Programs

Hello again! For this posting, we have decided to provide mini profiles of some organizations that we think provide an interesting approach to arts-based work and community psychology.

dance4life

“We are a growing, worldwide movement, led by nearly half a million young people. Through education, we are challenging stigma and taboo, and young people are learning how they can protect themselves and those around them.”

            Working in 27 different countries with national non-profit organizations, dance4life has created a school based program for youth to explore issues of sexual health, sexuality, and HIV/AIDS. This movement, as it is referred to, facilitates sexual health education with youth in school systems while using a 4-step model. These steps include: INSPIRE, EDUCATE, ACTION, CELEBRATE. Step one: A dance4life team travels to different schools to use music and dance to inspire young people to get involved in the fight against HIV/AIDS while providing opportunities for open communication about HIV and related topics. Step two: The dance4life team facilitates workshops and discussions about leadership, decision making, reproductive rights, and healthy sexuality. Step three: Youth involved in the dance4life movement begin to take action. This may take various forms such as using the skills they developed in the workshop, sharing their knowledge with those in their communities, or making change in their communities. Step four: Every two years a celebration on World AIDS Day is planned for those youth who took action (agents4change).   

http://www.dance4life.com/en/organisation/dance4life!/

The Possibility Project

“A society where teenagers are valued, respected and play a leading role in creating a better world.”

            The Possibility Project merges performance art with community action to “empower teenagers to create safe, peaceful, and productive lives and communities.” Throughout an entire year youth throughout New York City meet to engage in “issue-oriented discussions,” exploring diversity, activism, and leadership. Throughout the program youth are also trained on play writing and other skills related to performance art. Together, the youth write a musical that addresses social issues that they face in their lived realities while also creating a community action project. The Possibility Project has four core values that influence their approach to community based work: cross-cultural understanding, youth leadership, integrity, and excellence.

http://the-possibility-project.org/

The Center for Urban Pedagogy

“We believe that increasing understanding of how these systems work is the first step to better and more diverse community participation.”

            The Center for Urban Pedagogy (CUP) aims to improve civic engagement through the use of art. Projects pursued by CUP cover a range of social issues such as juvenile justice, urban planning, immigration, and many others. These collaborative projects involve artists, community-based workers, and the staff at CUP. The goal of the projects is to increase diversity in community participation and civic engagement by breaking big system concepts down into a language and state where community members are better equipped to become involved, fight for change in their communities, and shape policies that impact their lives.

http://welcometocup.org/

Crossroads Institute for Arts, Learning and Community

”To spread knowledge and build capacity regarding the use of the creative arts for learning and social change”

            The Crossroads Institute is a team of researchers, artists, educators and youth leaders that works with children, youth and adults across the U.S. to develop and disseminate community-based arts practices. They have developed creative publications and seminars that are available to other artists, educators and communities interested in promoting social change. They also provide free access to their online library which includes best practices from community-based arts programs. This organization has taken a collaborative approach by partnering with local and national community-based organizations and universities.

http://xroadsinstitute.org/

ArtsCorps Detroit

“In partnership with community agencies, non-profit organizations and neighborhood groups, ArtsCorps Detroit identifies creative arts-related projects”

            ArtsCorps Detroit is a community-based program at Wayne State University. This program offers service learning opportunities for students and community members to help revitalize the greater Detroit community through the arts. For example, LOTS of Art! is a project which involves working closely with neighborhood groups to find creative ways, such as painted  murals or dance platforms, to address abandoned spaces in the city. This program also has a research component that addresses the effectiveness of arts in promoting personal growth and organizational/community change.

http://artscorpsdetroit.wayne.edu/about.php

The Kresege Foundation

“We seek to foster the power of arts and culture to recharge and rebuild communities of all sizes throughout the United States”

The Kresege Foundation is a great organization for community psychologists who are interested in funding arts-based projects focused on social change. One of their program areas is Arts & Culture. Within this program, there is an Arts and Community Building focus area.  Overall, this foundation is interested in funding organization projects dedicated to integrating the arts and community building activities. In addition, they commission and publish research on efforts to integrate cultural organizations into community building efforts; and evaluating activities for their community arts mini-grant initiatives for grassroots arts and cultural projects that address pressing social issues.

http://www.kresge.org/programs/arts-culture

Arts-Based Approaches: A “Snapshot” of our Work!

Prior to entering into Wichita State’s Community Psychology program, I, Kyrah, was heavily involved in grassroots-level work in my home community. I was involved with organizing a youth group as a means to promote positive youth development by ‘stirring up their gifts’. The youth in our community are at particular risk for gang involvement, teen pregnancy and overall idleness. So, this service-oriented, youth-led group provided a way for youth to develop their talents and to become active participants in community affairs. Many of our activities involved using the arts as a tool for individual and community level change. For example, we’ve been fighting for years with the city to have a community center built. We rallied at council meetings, and many youth expressed their concerns by reading poems (about their personal experiences and their community) to council members in an effort to convey how important this decision would be for them. In addition, while our community is affected by crime and blight, it was important for the youth to showcase themselves as community strengths. So, whenever we organized community rallies there was always an element in the program that involved youth showcasing their talents (e.g. singing, interpretive dance, poetry) with a particular message (e.g., neighbor relationships, anti-violence). In terms of practice, this work was focused on helping youth understand that they could use their talents/gifts in the arts to make changes in their community.

 Of course at this time I had no idea that this was called ‘community psychology’ or an arts-based approach. In college, I learned that the two could be merged and could be counted as research! Since then, I’ve tried to think about ways that arts-based approaches can be integrated into projects that I work on. For instance, our research team is a part of a university-community partnership which has implemented a project to support predominately African American middle school-aged children who are at risk for disproportionate health outcomes, contact with the juvenile justice system and academic failure. We hold a summer college enrichment camp to expose our youth to the six program areas and to build on their understanding of academic success and college. Last summer, I suggested that we include a poetry workshop into the program. So the team worked with the students to develop ‘I Am’ poems, which is a poem form used to describe one’s characteristics, beliefs, desires and ambitions. The youth then performed their poems for their senior citizens at a local senior center. We were able to use poetry as an educational tool to practice writing (e.g., nouns, verbs, imagery, etc.) as well as to promote self-reflection and creative thinking.  This is important because many of our students struggle in reading and writing. So if we can make it fun, we may be able to plant a seed for further academic development. We plan to do another poetry workshop this summer. This time I hope to be able to measure some of their attitudes about the experience to understand in what ways it helped them.

My experience with art-based approaches has been mostly in the form of community psychology practice. In the future, I hope to be able to incorporate these approaches into my own research. While I am very interested in how arts-based approaches can promote positive youth development, my primary research focus is women’s health- particularly maternal and child health (i.e., infant mortality). I would like to look into arts-based approaches that help understand the prenatal experiences of women of color. I would be very interested in hearing about any work that anyone is doing in this area. I am quite passionate about understanding the ‘stories’ of underrepresented populations in addition to highlighting the strengths that they possess through arts-based methods.

I, Katherine, have just begun my journey into arts-based methods. Currently, I am working on a Photovoice project with survivors of sexual violence within a college campus setting, and I have begun developing an international project with an organization that uses music and dance to educate and empower youth around issues of sexuality and sexual health. This second project will also extend beyond music and dance, and will incorporate video journaling, performance ethnography, and the Photovoice methodology (Wang & Burris, 1997).

For this posting I will primarily focus on the Photovoice project with survivors of sexual violence as the other project is still in the planning stages.

I have learned an incredible amount from the research participants in this project; far more than I ever expected to learn from a Master’s thesis in fact. For those of you who may not be familiar with Photovoice I will give a very brief overview of the process. The process begins as most all research projects do: with a theme and research questions. Framing questions are then developed from the research questions, and are stated as simple, brief questions similar to the research questions. For the current project two framing questions were originally developed: What is most helpful to someone after they have experienced assault? What should the campus community offer survivors like you? Participants responded to these questions by taking a photograph and writing a short narrative about each photo. The group held multiple meetings during which the photos and narratives were discussed more in-depth.

Currently the project is in the participatory data analysis stage, during which the participants decide what themes are most important across their photos and narratives. There have already been very rich recommendations made by the participants for how the campus community response may be improved to better reflect the diversity of survivors on campus. The next and final part of the project involves compiling the photos and narratives into a sustainable community outreach tool. For this project we are planning to explore the use of digital stories to use throughout the campus community to share what was learned and to make recommendations on how to better reach out to the diversity of survivors on campus.

Many ethical issues were explored throughout the development and beginning stages of the project. However, while there were many barriers to developing a Photovoice project with survivors of sexual violence, I am very happy to have barreled through them and pursued this opportunity. To begin, the process of confidentiality has been interesting. Participants have expressed pride in their voice and participation, and tentatively plan to attend the events at which the digital story is shown. Secondly, the framing questions were developed in order to try and prevent the participants from having to relive their violent experiences. However, participants have used the framing questions as an opportunity to re-evaluate their experiences to some extent. Additionally, a great deal of time was spent deciding whether or not the project should recruit male survivors. Some perspectives suggested this could be problematic considering how it may have influenced the project participants (who are all women); however, when I posed this question to the participants it seemed as though they would have enjoyed learning more about the male survivor perspective. This, of course, was said to be dependent on where the survivor was at in their own healing process.

As can be seen from the brief overview of our work, we have used different forms of arts-based methods, but believe in the unique influence these approaches have on the research process. That influence, is voice and story. These methods have created beautiful opportunities for participants to share their stories in a communicative way that may capture their voice better than an interview protocol or a focus group transcription. Furthermore, participants have an opportunity to use an active voice through poetry readings, taking pictures, and writing narratives. Such active opportunities are usually left out of traditional research dissemination processes.

**This post was written by Katherine Cloutier from Michigan State University, and Kyrah Brown from Wichita State University.

References

Wang, C., & Burris, M. A. (1997). Photovoice: Concept, methodology, and use for participatory needs assessment. Health Education & Behavior, 24(3), 369-387. doi: 10.1177/109019819702400309

Arts-Based Methods and their Value to Community Psychology

 Hello again! As you may remember from our last posting we (Katherine and Kyrah) are doing a posting series on arts-based research (ABR) approaches to community-based participatory research (CBPR). In this post we highlight some arts-based approaches to CBPR, as well as, the potential for these methods.

        ABR approaches incorporate music, dance, photography, visual art, and theatre into the research process through data collection, analysis, or dissemination (Leavy, 2009). Incorporating such methods into a CBPR framework creates a unique research agenda. Such an agenda reflects feminist values (using art to understand a person’s perspective or viewpoint), promotes a non-hierarchical structure to the research process (co-creating art to be used as data that will directly influence social structures or policies), challenges the status quo of conventional research paradigms, and allows for a unique approach to understanding community context through culturally relevant channels of communication (Haraway, 2001; Harding, 1987; Leavy, 2009; Rappaport, 2005; Wallerstein & Duran, 2003). As is the case with a great deal of community psychology work, there is also an explicit focus on social justice (Rappaport, 2005).

       Photovoice is one arts-based participatory research method. This approach was introduced by Wang and Burris (1994) to understand women’s health issues in rural China. Photovoice projects are almost entirely participant driven from developing the research questions, analyzing data, to disseminating what was learned. Data collection for Photovoice projects begins by presenting participants with simple, concise questions, called framing questions (these reflect the project theme). Participants then go through multiple rounds of taking photographs and writing  narratives for each framing question. These are discussed during in-person meetings. Participants also help to qualitatively analyze the data prior to putting together a public outreach tool (Wang, 1999). These tend to take the shape of photography exhibits or digital stories. Either option presents the photos, narratives, and themes that emerged throughout the project. This culmination is particularly emancipatory as the audience (which is decided on by the participants) tends to be community leaders who can learn from the participants’ voices and implement policies and regulations according to the perspectives of those whose viewpoints are often unheard.

       Ethnodrama, ethnotheater, and performance ethnography are other approaches to arts-based participatory research (Leavy, 2009). These approaches are sometimes viewed as distinct, but in general refer to the process of analyzing, translating, and disseminating research through dramatic performance (Leavy, 2009). Research may be previously collected through interviews, focus groups, field notes, or other conventional methods and writing a script based on research findings may also allow the research to make composite sketches about specific themes found throughout the data (Leavy, 2009). Performance ethnography also creates a unique opportunity for the researcher to recognize/assert their role. As suggested by many feminist scholars, deciding the researchers’ role in the performance and how their role interacts with the script or other performers establishes reflexivity as an integral part of the research process (Leavy, 2009).

        Overall, arts-based scholars have explored methods related to narrative analysis, poetry, music, performance, dance/movement, and visual arts (Leavy, 2009). These approaches may be bridged together (for example using music and performance as one), or supplemented by conventional research methods.

       We believe that there is a special place for ABR in community psychology.  We want to note that arts-based research is not new; and that it is a set of practices that have been used for years to capture the experience of individuals and communities.  It has been gaining popularity as a form of qualitative research among psychologists. Because arts-based methods are action oriented and participatory in nature, many psychologists have found it useful for engaging individuals and groups in the process of addressing the needs and strengths of the community. It creates an opportunity for researchers and practitioners to use diverse, creative strategies for conducting and evaluating research. It is no doubt that community psychologists understand the importance of social context, and arts-based practices can be a powerful tool used to identify and analyze the contexts in which people live.  It is a tool that can be crafted by community and used as a sustainable catalyst for change.

         While there are challenges to using arts-based approaches (e.g., community buy-in; time-consuming), the benefits of such work can be far-reaching. It is common for researchers to use arts-based methods (e.g., photovoice) to supplement the overarching research agenda. Consider a project that involves addressing gang violence among adolescents. There are important methods (i.e., surveys, neighborhood data) used to assess the context and the needs of a community. However, imagine that teens are actively involved in photographing and writing narratives about their community. Imagine the youth presenting their findings to key stakeholders and suggesting what to do about gang violence. The point is that arts-based methods can be a viable tool for citizen participation, using and building on community strengths, empowerment, and sustainable change.  In essence, community psychology research and practice can benefit from arts-based approaches.

**This post was written by Katherine Cloutier from Michigan State University, and Kyrah Brown from Wichita State University.

References

Haraway, D. (2001). Situated knowledges: The science question in feminism and the privilege of partial perspective. In M. Lederman & I. Bartsch (Eds.), The gender and science reader (pp. 169-188). London: Routledge.

Harding, S. (1987). Introduction: Is there a feminist method? In S. Harding (Ed.), Feminism and methodology: Social science issues (pp. 1-14). Bloomington: Indiana University Press.

Leavy, P. (2009). Method meets art: Arts-based research practice. New York: The Guilford Press.

Rappaport, J. (2005). Community psychology is (thank God) more than science. American Journal of Community Psychology, 35(3/4), 231-238. doi: 10.1007/s10464-005-3402-6

Wallerstein, N., & Duran, B. (2003). The conceptual, historical, and practice roots of community based participatory research and related participatory traditions. In M. Minkler & N. Wallerstein (Eds.), Community-based participatory research for health (pp. 27-52). San Francisco: Jossey-Bass.

Wang, C. C. (1999). Photovoice: A participatory action research strategy applied to women's health. Journal of Women's Health, 8(2), 185-192.

Wang, C., & Burris, M. A. (1994). Empowerment through photo novella: Portraits of participation. Health Education & Behavior, 21(2), 171-186. doi: 10.1177/109019819402100204

Ethics and Community Psychology

We all are enthusiastic about the current push to identify the competencies we bring to our work in communities and to learn how to best train students to acquire the competencies. Did you know that one of the ethical principles of psychologists is to be competent? Standard 2 of the ethical principles is “competence” – that a psychologist will only practice within the person’s competence boundaries, is obligated to acquire training to become competent and maintain competence, etc. See http://www.apa.org/ethics/code/index.aspx .

One of the proposed competencies is evaluation. The importance of this competency was brought home to me recently when I reviewed an evaluation of a civic organization that had been conducted by a company that claims competence in conducting evaluations. In brief, like so many nonprofits these days struggling to survive, the Board of Directors of the civic organization was considering eliminating a ten year old program (call it “S”) because it was not financially self sustaining. Prior large financial donations from corporations and foundations had faded away so the parent nonprofit organization was subsidizing program S to keep it going. As overall resources tightened, the Board decided to rethink its continuance of its subsidy of program S and therefore to question the value of S’s brand. In this weak economy, I presume that many organizations are similarly scrutinizing their programs, divisions, etc. to excise the weaker units.

The Board of Directors contracted with a local company (a full service management company, in existence 15 years, that has contracts ranging from the federal government down to small community based organizations) to conduct an evaluation of program S. On paper, the company appeared competent. It defined evaluation as: “.. a process that critically examines a program. It involves collecting and analyzing information about a program’s activities, characteristics, and outcomes. Its purpose is to make judgments about a program, to improve its effectiveness, and/or to inform programming decisions. Evaluation is essentially the systematic investigation of the merit, worth, or significance of any object, activity, or program.” So far, so good. The materials go on to assert that a great evaluation should employ “rigorous methodology” and should be “inclusive,” “complete,” take in “diverse viewpoints,” etc

And yet …. I noted that the company’s content-filled website does not list the number of employees nor does it reveal a single name or expertise or background of its employees.

The sum total of the “data” for the completed “evaluation” was from one 90-minute focus group involving seven participants in the program (out of a pool of over 200). The final report was presented as a power point (only) and was wholly nonanalytic. Much time went into the company learning about program S and into recording and transcribing the focus group proceedings. They claimed to have used qualitative analysis software and “developed codes” (the codes being “strengths, challenges, suggestions.”) And despite all the accoutrements of a “rigorous methodology,” the body of the evaluation merely consisted of somewhat random quotes from the focus group participants, dealing with trivial or person specific issues OR that were trite. That suggests to me that the questions posed were not sufficiently incisive and the personnel conducting the focus group were not sufficiently skilled to guide the discussion so as to probe more deeply. In any case, this evaluation can be characterized by the Gertrude Stein quote – “there is no there there.”

Further, several of the negative quotes were so specific that the nonprofit staff could easily identify the person making the comment. (The staff had recruited the focus group participants.) For example, one person is quoted as saying that the staff had never taken him/her up on his/her volunteer offer to do x. I learned that the focus group participants were not informed that anything they said could be quoted, verbatim, although they were not attributed by name.

The company’s final recommendations were out of touch with the organization’s reality, e.g., one recommendation was to hire more staff to organize volunteers (whereas the organization is operating in financial crisis mode now and in the foreseeable future). The “next step” was to use the focus group results to “rebrand” program S with enhancements, even though the “focus group results” were inadequate to inform any substantive or feasible change. Needless to say, the organization (which had invested scarce resources in this effort) was unimpressed. The evaluation did not assist the Board of Directors in exercising its responsibility. Another program evaluation thrown in the trash.

We can (and must) do better in terms of the competence we bring to our work.

Gloria Levin