Methods & study design

Two decisions to make before you collect

Once you know which measures you want, two practical questions remain: how will you collect the data, and what design will let you say something credible about change. These guides walk through both, in plain terms.

Best read after you have drafted your measures in the Evaluation Tool Builder. When you are ready to write your design down, use the study design builder.

Guide 1

Choosing how to collect your data

The right tool is the one your participants and staff can actually use, that keeps the data safe, and that you can get back out again for analysis. Most social prescribing programs serve older adults, so accessibility and a low technology barrier matter as much as features. Match the platform to three things: the sensitivity of what you are collecting, the capacity you have to set it up, and whether you need to follow the same people over time.

MethodBest forCostGetting data outWatch-outs
Paper + manual entryOlder adults, in-person intake, places with poor connectivityLow, but staff time to enterSomeone types responses into a spreadsheetTranscription errors and effort; store and transport securely
Google FormsQuick, simple surveys with little setupFree with a Google accountExports straight to Google Sheets or ExcelNot built for identifiable health data; check where data is stored and your organization's policy
Microsoft FormsOrganizations already using Microsoft 365Included in many M365 plansExports to Excel inside your tenantKeeps data in your organization's environment, which often fits privacy rules better than consumer tools
SurveyMonkeyPolished surveys, skip logic, wider reachFree tier is limited; paid plans for export and volumeExport on paid plansConsumer platform; check data residency before collecting anything sensitive
QualtricsComplex branching, panels, research studiesEnterprise or academic licenceStrong export and analysisPowerful but a steeper learning curve; usually accessed through a university
REDCapLongitudinal studies, participant IDs, sensitive dataFree to institutions in the REDCap consortiumBuilt-in exports and an audit trailNeeds an institutional host (a university or health authority) and some setup skill; the strongest fit for privacy and follow-up

How to decide

Start from the data, not the tool. If you collect names, contact details, or health information, Canadian privacy law applies, and the rules differ by province (for example PHIPA in Ontario, the Health Information Act in Alberta, and PIPA and FIPPA in British Columbia). Prefer a tool that keeps data in Canada or under a compliant agreement, and check your own organization's policy before you choose. If you need to follow the same people from intake to a six-month follow-up, you need stable participant identifiers and a tool that supports scheduled, repeated entry; this is where REDCap earns its setup cost. If you are running a short, anonymous experience survey, a free form tool is usually enough.

Capacity is the other half of the decision. A tool no one has time to configure is worse than paper used well. Pick the simplest option that meets your privacy and follow-up needs, write a one-page data-handling note so everyone treats responses the same way, and pilot it with a handful of participants before you collect at scale.

A practical default. For a small program: paper or Microsoft/Google Forms at intake, entered into one well-labelled spreadsheet, with a clear consent line and secure storage. For a research-capable program following people over time: REDCap hosted through a university or health-authority partner.

Guide 2

Choosing a study design

A study design is the logic that lets you move from a number to a claim. The question it answers is simple to state and hard to satisfy: if something changed, how confident are you that your program is the reason. You do not need a randomized trial to learn something useful, but you should match the strength of your claim to the strength of your design, and say plainly what your design cannot rule out.

Pre-post testing, the workhorse

Most programs use a single-group pre-post design: measure each participant at intake, measure them again later, and compare. It is feasible, cheap, and gives every participant their own baseline. Its weakness is that it has no comparison, so a change could come from things other than the program. People often improve on their own over time (maturation), those recruited at their worst tend to drift back toward their average (regression to the mean), and the season or a policy change can move everyone at once (secular trends). Pre-post measures change well; on its own it is weak for attribution.

You can strengthen a pre-post design without a control group. Use the same measure and the same recall window at each point. Leave enough time between measurements that real change can occur; for an older, higher-needs population, six months usually detects change better than three. Report loss to follow-up honestly, because the people who drop out are rarely a random sample. Triangulate the numbers with a few participant stories, which can show whether the mechanism you expected is the one doing the work.

Single-group pre-post

Measure the same people before and after. The default for most programs.

Strengths
  • Feasible and low-cost
  • Each person is their own baseline
  • Shows whether participants changed
Limits
  • No comparison, so other causes are not ruled out
  • Vulnerable to maturation and regression to the mean
  • Loss to follow-up can bias results

Pre-post with a comparison group

Compare change in participants with change in a similar group who did not take part.

Strengths
  • Much stronger basis for attribution
  • Helps rule out trends that affect everyone
Limits
  • A fair comparison group is hard to find in community settings
  • Groups can differ in ways you cannot see
  • Raises practical and ethical questions about who is left out

Interrupted time series

Track a routinely collected number across many points before and after a change.

Strengths
  • Uses data you already collect, such as referral counts or service use
  • Separates a real shift from ordinary fluctuation
Limits
  • Needs many time points before and after
  • Other events at the same moment can confound the picture

Post-only or retrospective change

No baseline; ask afterward, or ask participants to rate how much they changed.

Strengths
  • Possible when you have no baseline data
  • Quick and low-burden
Limits
  • Relies on memory and is open to bias
  • Cannot show change as credibly as a real baseline

Qualitative and mixed methods

Stories, interviews, and significant-change accounts, on their own or alongside numbers.

Strengths
  • Explains how and why change happened
  • Captures outcomes that no scale measures
  • Persuasive with funders and policymakers
Limits
  • Self-selected stories may not represent everyone
  • Needs light structure and consent to synthesize well

Putting it together

Most social prescribing evaluations land on a single-group pre-post design for participant outcomes, an interrupted time series for routinely counted measures like referrals, and a small set of stories for meaning. That combination is realistic for a community program and still gives a funder something concrete. When you are ready to commit your design to paper, the study design builder turns a few answers into a short design summary you can attach to a plan or a report.

More guides will be added here over time, including consent and data handling, sampling and response rates, and analysis basics. Tell us what would help through the feedback tab.