The first step is to enlist a partner or two for this challenging research. Choose people who are naturally curious, a bit geeky and likely to enjoy what they’ll learn from it.
Identify whether any part of your program model uses what a respected institution (like a federal government agency) has deemed an “Evidence-Based Practice” or a “Promising Practice.”
A place to start is to review the following document listing 12 online clearinghouses of EBPs and PPs. (This list was found on the Upstream Investments of Sonoma County website. The last column in the document refers to Upstream Portfolio’s three tiers of evidence.)
Next step is to explore the clearinghouse websites to find the most relevant EBPs and/or Promising Practices, those describing your own program model. You need to narrow the focus of your research to one or two models or practices whose evaluation literature you’ll be able to review.
You want to locate prior literature reviews describing numerous existing evaluation studies, as well as one-off evaluations, of your program model. When you find those that are sufficiently recent and relevant to your research, you need to locate and review the actual evaluation reports. Make sure the reports specify exactly who the programs studied served (is this the same target group as your program’s?), how long the program studied lasted (similar to your model?), the evaluation’s findings (are the changes observed the same ones your program seeks to make?) and so forth. Don’t waste time running down irrelevant research articles.
Published evaluation reports are often available for free online. A Google Scholar title, author or keyword search may turn up your desired report in a proprietary, subscription-only database of scholarly articles, which makes access difficult. But the same search may also turn up a free copy of the same report on another website. Scan those search findings carefully and know it may take some time to click on the top 10 hits to find a free full copy of what you’re seeking.
f you’re not able to locate a free copy of what you want, ask people in and outside your organization to find someone with access to these databases who is willing to share it with you or to send you the articles you need. You can also see if the local public or university library can access such databases for you. It’s expensive to buy individual articles but it is an option. Another option is to read the abstracts found in the subscription databases; these are succinct descriptions of evaluation articles. It is usually easy to determine from the abstract whether particular articles will be worth the effort to locate in full.
Once you’ve collected at least a half dozen literature reviews and/or evaluation articles that describe your program model or some aspect of it, you and your partner should write summaries of the evidence each one presents in your own words. Add your thoughts about how that study adds weight to your proposition that your program has some evidence of effectiveness–if not what the clearinghouse reviewers would consider “strong” or “sufficient.”
Now, with your partner, develop an outline for your Lit Review. Decide on a few topics to cover in a five- to eight-page paper; more topics or articles would make the piece unfocused and unpersuasive. To create your outline, review all the research summaries to determine a logical sequence for your topics.
Drop in and tweak your brief summaries of relevant articles according to the outline’s order. Include the findings of more than one article in a single paragraph if they are similar. Write introductory and concluding sentences to each paragraph about the relevance of this research to your program. Then write short transition paragraphs to smooth the flow of ideas.
Write a conclusion driving home your points about the evidence that your program is effective, then write an introduction laying out briefly what you will present.
After the Introduction, write a Research Methods section describing the limitations of the research cited based on the populations studies, data collection methods, and other factors. Recommend a future research focus based on the gaps cited.
Use the American Psychological Association format for all research citations. At the end of a sentence, place the author(s) and publication date of the article in parentheses before the period, e.g., “(Arbreton & McClanahan, 2002).” Also, compile all your References into an alpha list at the end of your document, using this format that includes its online location: “Amendola, M. & Oliver, R. (2010). Aggression Replacement Training Stands the Test of Time, Reclaiming Children and Youth, 19(2), 47-50. Retrieved from http://eric.ed.gov/?id=EJ895352.”
Share the draft Lit Review with as many people you can; choose people very familiar as well as less familiar with your program. Ideally, also invite those familiar with evaluation methods to review it. Ask them to read it for clarity, flow and plausibility.