Contents
Considerations for prevention impact and its measurement, for the Victorian context
Primary prevention work can have more immediate impacts on individuals than previously supposed
The evidence distilled in this review challenges a long-held assumption that because primary prevention works across the whole population rather than specifically with high-risk individuals, it lowers risk gradually and in a diffuse manner. While it is generally recognised that this will benefit the whole community over the long-term, it is often assumed that primary prevention is limited in its impact on individuals in the shorter-term (reference 26). A growing number of evaluations have shown that well-crafted primary prevention interventions can have measurable impact on rates of perpetration and victimisation of violence against women within intervention timeframes (reference 1). That is, primary prevention interventions for violence against women can indeed have relatively short-term impact for individuals, including on those who may already be perpetrating or experiencing violence.
The review identified only a limited number of early intervention activity impact evaluations. Of those that did exist, and which showed impact in high income countries, most aimed to reduce women’s victimisation, such as through empowerment-based self-defence or targeting alcohol consumption. The review surfaced one robustly-evaluated activity targeting men with substance abuse issues, but it did not have an impact on violence prevention outcomes (reference 7). Another assessed an intervention with young people who may have experienced violence, offering supported housing to out-of-home care leavers, which had an impact on partner violence victimisation (reference 8). There were no other examples of interventions implemented in high-income contexts with individuals or groups considered at increased risk of experiencing or perpetrating violence, or addressing the reinforcing factors identified in Change the Story (reference 4).
Several of the interventions identified in the review that were evaluated as impactful had been implemented in high-income countries, and in settings or with target populations similar to existing Victorian prevention programs. Among these, the research confirmed that interventions in settings such as education, sports, and maternal and child health, and with population groups including young people and new parents (where good practice prevention work already exists in Victoria) can indeed reduce victimisation and or perpetration of violence against women for participants in intervention timeframes. There is therefore good reason to believe that such Victorian programs, if implemented with the quality and intensity necessary, would similarly have an impact. While many excellent process and outcome evaluations have been undertaken on Victorian prevention work, the review did not surface any robust impact evaluations in the Victorian context.
Primary prevention work can prevent new violence and/or reduce existing violence
Recent evidence has addressed the question of whether primary prevention interventions showing short-term impact are preventing ‘new’ violence, or reducing existing patterns of violence perpetration or victimisation. A meta-analysis of six randomised controlled trials of partner violence prevention interventions showed that it is possible for such interventions to do either, or to do both.
This supports the premise articulated in Change the Story that ‘primary prevention complements and enhances early intervention, response and recovery activity by helping reduce recurrent perpetration of violence (reference 4).’ It does not mean, of course, that separate response and early intervention activities are not needed. Far from it, as all three are crucial elements of a comprehensive and integrated system. However, it does demonstrate that primary prevention can contribute to both early intervention and response by stopping or reducing recurring violence, while shifting the social norms and structures that maximise the possibility of maintaining changed behaviours and rebuilding lives after violence.
There are caveats to this: none of these six interventions were implemented in contexts similar to Victoria’s in terms of existing gender norms, systemic support for prevention, or past 12-month prevalence levels (in each of the six cases, past 12-month prevalence of physical intimate partner violence was in double digits, versus under 3% in Victoria (footnote 1)). While analyses do not yet exist for interventions in contexts with similar past 12-month prevalence to Victoria’s, this factor may play a role in whether existing or new violence is being prevented over a short timeframe, and to what extent. Reductions of the same magnitude on either new or recurring violence would be unlikely when compared to high 12-month prevalence contexts, and the nature of the violence being prevented (i.e. whether new or recurring) may also differ.
This review of studies does not tell us the specific impact an intervention might have on higher risk groups who are not already experiencing or perpetrating violence, as these are subsumed under the category of those not experiencing/perpetrating violence at baseline in those studies. As such, we cannot say whether these primary prevention interventions would be more effective at preventing the onset of violence for these groups than an early intervention one would.
Impact evaluations require careful design, based on a clear theory of change
The results described above demonstrate the challenge of understanding exactly how an intervention is working and the specific behaviours it is changing. More research is needed to fully understand the factors that influence the type of impact a primary prevention intervention might have, and on which sub-sets of the population. The authors of the meta-analysis described in the previous section recommended that interventions and their evaluations ‘need to specify a clear theory of change and pathways of impact for outcomes of interest (reference 9).’ They further cautioned that care should to be taken in defining expected outcome measures for impact evaluations in this field: binary measures usually employed to assess impact (‘any’ versus ’no’ experience/perpetration of violence in a certain time period) ‘masked some of the more subtle intervention effects (reference 9).’ This is important because:
Conclusions on whether a program is perceived “to work” are highly influenced by the [intimate partner violence] outcomes that the investigators choose to report, and how they are measured and coded. Lack of attention to outcome choice and measurement could lead to prematurely abandoning strategies useful for violence reduction or missing essential insights into how programs may or may not affect intimate partner violence (reference 9).
Indeed, a consistent finding across the research included in this review was the need for caution and attention in undertaking impact evaluation in a complex field such as prevention of violence against women. It is well-established that prevention interventions need to pass through a number of stages of developmental and outcome evaluation, and subsequent refinement, before they can be expected to impact on rates of perpetration and/or victimisation, especially at the population level (reference 63-66). It must be noted that there are a number of factors that need to be in place for an impact evaluation to be worth the time, expertise and investment required to do it properly (reference 67).
Finally, for impact evaluation to be meaningful, it needs to be scoped and resourced at a level sufficient to ‘power’ a statistically significant analysis. The What Works review found that ‘studies, particularly cluster randomised controlled trials were often underpowered to show meaningful effects (reference 1).’ They noted several randomised controlled trials that appeared to show large violence against women impact, but statistical significance could not be claimed due to the relatively small population size or number of clusters in the trial (reference 1). The scale of the intervention itself, and of the population it aims to impact, is an important consideration here too: it is more difficult to assess impact at a statistically significant level if the intervention population size is small.
Considerations for maximising impact in the Victorian context
Specific foundational conditions and quality criteria are essential for impact
Whether prevention activity does achieve impact, and to what extent, is a matter of design quality, delivery intensity and appropriate contextualisation of the work. This review found that international meta-analyses of existing evidence have distilled several quality design principles that appear to be foundational conditions for impactful prevention practice. These align with and reinforce those in existing frameworks used in Victoria, most notably Change the Story (reference 4). They include, for example, rigorous planning and development of context-specific theories of change, use of an explicit gender analysis that addresses the multiple drivers of violence, the inclusion of tailored and targeted activities for specific population groups, and so forth.
International analyses tend to give more attention to the quality principles of direct engagement activities with individuals and groups. Such types of intervention are more common in international programming (especially aid and development contexts) than setting-wide approaches or those targeting environments or organisations. However, these could provide useful up-to-date guidance to the development of place-based prevention activity in Victoria seeking to engage communities and groups (reference 10) (footnote 2).
Impact has a quasi-linear relationship with implementation ‘intensity’
An important finding of this review is that prevention interventions need to be delivered at a certain level of intensity to reach impact – a relationship described as ‘more is more’. This is not a new concept for prevention work in Victoria: it was noted, for instance, in reference to good practice for schools-based programs as early as 2009 (reference 68). But evaluative evidence gained over recent years has provided additional and more specific detail on what the concept of ‘adequate intensity’ entails (though this is still emerging and evolving). The concept is especially relevant, again, for work directly engaging individuals and groups. ‘Intensity’ appears particularly important in relation to duration of the activity, in the ‘dosage’ of messaging, knowledge transfer and/or skills-building activity (e.g. number of hours in workshops or other direct participation activities) and, finally, in the number of skilled prevention practitioners compared to the size of the community/population targeted (reference 10). While exact figures vary (i.e. for ‘adequate’ dosage, duration, or number of skilled practitioners), depending on the setting and context, some ‘threshold’ estimates are emerging for specific types of intervention or setting.
This has implications as much for existing practice as it does for the development of a place-based saturation model. It means that attempting to ‘stretch’ programming to reach more people, but with fewer opportunities for direct engagement, for example, or with larger practitioner-to-population ratios, can undermine the utility of the program overall. Some program reviews found that below-adequate intensity can even engender backlash (reference 16). In the absence of impact evaluations of Victorian programs, ensuring their implementation at a level of intensity that meets or exceeds emerging minimums (footnote 3) would be essential for any program aiming to have impact on individual perpetration or victimisation of violence against women. For the development of a place-based saturation model, a comprehensive planning phase will be essential to determine the resources needed to establish and maintain adequate intensity for the specific activities envisaged.
Additionally, the existence of an enabling infrastructure to ensure adequate expertise, coordination and quality assurance, along with a high-level of support to victim-survivors are foundational to impactful prevention practice. This is discussed further below and similarly needs to be a core consideration in the design of any place-based saturation model.
Multi-component interventions are more impactful
The review confirmed existing understandings in prevention programming that discrete projects or interventions are not enough to achieve the broader systemic and normative change needed for sustainable reductions in violence against women at the population level (reference 4). The very few interventions worldwide that have measured and demonstrated population-level impact all employed multiple techniques, and worked across several levels of the social ecology. One violence against women intervention meta-analysis found a magnification effect from interventions employing multiple components across the spectrum from prevention to response, and at different levels of the social ecology, compared to stand-alone interventions (reference 24). This supports the conclusion from multiple evidence reviews that effective and available response systems, and an enabling legal, policy and institutional environment, can themselves magnify the impact of prevention activity.
As significant effort has been underway in Victoria for many years to improve responses to, and prevention of, violence against women, mutual reinforcement from cross-spectrum activity may already be operating, and/or could be magnified by improved coordination. However, due to the lack of impact evaluations of the Victorian work, the mutually-reinforcing effect of legislative, policy and programmatic activity is largely unknown. For that reason, implementation and evaluation of a multi-component approach through a place-based saturation model provides an opportunity to test such a hypothesis and provide evidence to inform future coordination efforts.
Initial implications for a place-based saturation model in Victoria
There are emerging principles to apply to multi-component prevention work
Evidence pertaining to promising practice in planning and coordination of multi-component prevention work is nascent and requires further investment and attention (reference 27). A direct relationship between ‘more components’ and ‘more impact’ cannot be assumed, and research is still evolving on the conditions, processes and criteria that best enable or support a ‘mutually-reinforcing effect’ in prevention of violence against women. However, several guiding principles or features have been identified in the literature that can support mutual reinforcement.
The first is that, in prevention of violence against women interventions, the combination of interventions should be based on the principle of addressing multiple drivers and reinforcing factors. The authors of the What Works review suggest that the reason multi-component interventions have greater impact is likely because ‘more components’ means greater reach and range of influence to address multiple drivers. Reaching multiple drivers, they contend (based on their review of evaluations), is critically important to maximising impact.
This has implications for program design in that the aim of addressing multiple drivers and reinforcing factors (in a quality way) should be what guides the choice of components employed – techniques, settings and levels – in a multi-component prevention program. Therefore, a deep understanding would be necessary of the nature of current (past 12-month) violence in the specific context where the intervention is being planned, including its specific drivers and reinforcing factors and their relative weight. This is a departure from approaches that seek mutual reinforcement by simply multiplying techniques, settings or levels of action alone.
Further principles for planning and coordinating multi-component work include sequencing activity in such a way as to support the phases of behaviour change, and synchronising implementation of different activities across settings and levels. The development of a place-based model in Victoria should draw on lessons from the multi-component community mobilisation models of SASA! and SHARE in this regard. These interventions employed phased approaches, aligned with theories of behaviour change, and involved coordinated and progressive staging of multi-component activity over time, at each level of the social ecology.
Lessons from impactful place-based interventions should be attentive to contextual differences
The review found that very few interventions globally have had a population-level impact. Among them, a handful of policy and legislative reforms were correlated with reductions in victimisation across implementation jurisdictions. The other interventions that showed population-level impact did so at the community-level, within a geographically bound place. These interventions were developed, implemented and evaluated in social, political and economic contexts different to Victoria’s, which has several implications.
Firstly, these interventions were largely implemented in relative isolation from an enabling policy environment, high-level institutional support from relevant sectors, or a coordination infrastructure to support scaling. Programmers considered this limited their ability to expand such impactful work beyond local communities to a jurisdictional level (footnote 4). Based on prevention theory, it may be assumed that this lack of broader support has, if anything, dampened the (nevertheless significant) impact of these interventions at the place/community population level. On this count alone, the potential to achieve at least the same level of impact may be expected in Victoria, if not more given Victoria’s stronger enabling environment.
However, there is another major factor operating counter to that assumption. Starting rates of past 12-month prevalence (footnote 5) were significantly higher in each of the three implementation contexts than in Victoria. This means it is unlikely that similar interventions would have the same level of impact in the same timeframes (and would need to be measured over a larger population to attain the same statistical significance). It also points to the need for more specific measures in Victoria, capturing not just acts of violence but its drivers, using qualitative as well as quantitative measures.
Footnotes
Past 12-month prevalence rate of intimate partner violence for women in Victoria was 1.5% in 2021–22, down from 2.3% in 2016.
Quality principles for activities with groups, for instance, include participatory learning methods ‘that emphasise empowerment, critical reflection, communication and conflict resolution skills-building’.
These can be sourced for some types of intervention, from the evidence reviews cited here. For others, it may be necessary to seek detail as to dosage, duration and ratio of practitioners-to-participants from the descriptions and evaluations of similarly-conceptualised interventions evaluated as impactful.
Interview with key informant.
The studies evaluated against past 12-month prevalence of physical, sexual and emotional violence, using standardized indicators established by the World Health Organisation.