Key concepts for practice

Ethics

Your evaluation work should be guided by principles of ethical practice.

Ethical practice is underpinned by three key ideas: respect for persons, justice and beneficence (putting the welfare of participants at the centre)(xx).

On a practical level, these principles mean that evaluation practice should:

• reduce risks and limit any harm to participants
• be inclusive where possible
• ensure participants know about all the risks and benefits of the research, so they can give informed consent.

Key modern themes in ethical evaluation include the rights and treatment of communities that have experienced injustice, exclusion or disadvantage. You should therefore be aware of principles around inclusion, intersectionality, Aboriginal self-determination, and Indigenous data sovereignty.

Ethical practice

Please ensure your project follows good ethical practice. In some cases, your project might need formal approval from an official research and ethics committee. The National Health and Medical Research Council National Statement on Ethical Conduct in Human Research is a foundational document that provides high level detail on ethical protocols in research.

We strongly advise you consult the Australian Evaluation Society’s Code of Ethics for useful information on ethics in evaluation practice. When working with Aboriginal and Torres Strait Islander communities, you should also draw on the Australian Evaluation Society’s First Nations Cultural Safety Framework [PDF].

Ethical considerations can arise at any stage of a project, however, they are most relevant at the data collection, storage, analysis, and reporting stages. Assessing risks and benefits for participants is essential. A risk can be understood as any potential for harm, discomfort or inconvenience to a participant or community. All monitoring, evaluation and learning (MEL) practice should consider, weigh up and manage ethical risks.

Suggested resources:

• The National Health and Medical Research Council National Statement on Ethical Conduct in Human Research
• Also refer to the Australian Code for the Responsible Conduct of Research.

Informed consent

It is ethical to seek informed consent from participants in your evaluation. This means ensuring that a potential participant has all the information about the purpose and process of the research.

Some participants may choose not to participate. For example, a participant:

• may not want their feedback to be audio recorded
• may not consider the purpose of the research to be important.

The process of seeking and receiving informed consent can take different forms. Researchers and evaluators usually produce a plain language statement containing all the information a participant needs to make an informed decision. This statement might be handed out in hard copy, included at the beginning of an electronic survey, or read out during an interview. To provide consent, participants might sign a hard copy consent form, check a box on an electronic survey, provide affirmation from a secure email address, or provide verbal consent that you must record in writing.

If you are working with children and young people, in most situations you will require written consent from them and their parents or carers.

Suggested resources:

Check out information from the Australian Evaluation Society on ethics and informed consent.

Intersectionality

Intersectionality is a term coined by Professor Kimberle Crenshaw (1989)(xxi) (Columbia Law School) to describe how experiences of oppression can intersect and be compounded for individuals based on multiple identity characteristics, such as gender, ethnicity, ability, sexual orientation, sex, religion, age or residential location.

The primary prevention of violence against women is concerned with how gender inequality leads to discrimination or violence. However, other factors such as colonisation, racism, ableism, ageism, homophobia and transphobia can also play a role in violence against women.

Taking an intersectional approach is a way of thinking and a practice skill. As Women’s Health Grampians (2020) advises, principles of an intersectional approach for practitioners include to ‘learn, listen, open, reflect, and apply’(xxii). As an evaluator, you can draw on these principles to consider how your evaluation approach and activities take account of intersecting stakeholder characteristics.

Practitioners should aspire to (and plan for) evaluations that are safe, responsive, and inclusive. At a minimum, evaluative practice should make no further contribution to the cumulative and compounding forms of discrimination already experienced by some participants. Consider, for example, how your work might promote a sense of:

• safety (e.g., does an interview location feel physically and emotionally safe/non-intimidating)
• accessibility (for example, are interviews scheduled at convenient times? Is interview language relevant and accessible? Is a location or the media used accessible for participants with a disability?)
• inclusion (e.g., are participants made to feel welcome and like they belong? How might promotional language have encouraged their inclusion? Do surveys include questions on demography or identity characteristics so you can explore intersectional experience? In surveys, exploring answers against these characteristics is called data disaggregation)
• visibility (e.g., have you used open ended questions that seek or enable the sharing of experiences or stories outside norms or stereotypes?).

Intersectionality should be an ongoing consideration in your evaluative work, rather than a ‘point in time’ task.

Suggested resources:

The resources and templates page has more on intersectionality. See also:

• Our Watch’s Counting on Change which explains taking an intersectional approach to progress measurement in prevention
• Women’s Health Grampians (2020) Intersectionality Guide: A tool for CORE members [PDF].

Aboriginal self-determination

Historical research and evaluation has often minimised or excluded Aboriginal perspectives and de-valued Aboriginal ways of knowing. In many cases, this has done explicit harm to Aboriginal peoples and undermined or prevented Aboriginal self-determination.

For these reasons, self-determination is a critical consideration in policy, practice, and evaluation relating to family violence prevention and response. In Victoria, the Dhelk Dja Partnership Forum connects community and government to collaboratively address family violence issues.

Dhelk Dja’s (2021) Monitoring, Evaluation, and Accountability Plan explains that, ‘Self-determination requires government to value and respect Aboriginal knowledge, systems and expertise and to transfer authority, decision-making control and resources to Aboriginal people’(xxiii).

Respecting self-determination is an important principle for all evaluators. This principle should inform how you design, plan, monitor and evaluate your primary prevention projects, and how you share learnings. Evaluations and research should be conducted in an ethical and culturally safe manner.

In the context of primary prevention work, Aboriginal communities consider that colonisation is a cause of gender based and family violence(xxiv), in addition to the gendered drivers of violence(xxv). Aboriginal lived experience and perspectives on change must therefore inform all aspects of a project, from design through to data collection. In particular, ‘Aboriginal measures of progress and success [should] align with Aboriginal holistic understandings of health and wellbeing for the whole community’(xxvi).

Suggested resources:

• Australian Evaluation Society (AEU) has published a First Nations Cultural Safety Framework [PDF] that provides excellent advice for evaluators committed to working in a culturally safe way
• Australia Institute of Aboriginal and Torres Strait Islander Studies (2020) Code of Ethics and Guidelines for Aboriginal and Torres Strait Islander research [PDF].

Indigenous data sovereignty

In the context of research and evaluation, data sovereignty refers to the question of who owns or has governance over data that is collected. Indigenous data sovereignty is the right of Aboriginal peoples to exercise ownership over their data(xxvii). This extends to the design, collection, analysis, interpretation, and dissemination of data (and information).

Concerns about Indigenous data sovereignty arise because traditional Western methods of research consider that ownership of data sits with a researcher and the organisation they represent. For Aboriginal people, evaluation and research can be experienced as theft or appropriation of their information.

It is important to consider issues around data sovereignty, especially when working with any community that may have experienced historical data exploitation. Incorporating some forward thinking about data sovereignty into your MEL practice can help to build trust and show that you are acting ethically.

Suggested resources:

• Check out the range of resources made available by the Maiam Nayri Wingara Aboriginal and Torres Strait Islander Data Sovereignty Collective
• The Framework for embedding First Nations Data Sovereignty principles into design and evaluation [PDF] by Kowa is practical and accessible
• See also the statement on Indigenous Data Sovereignty Principles at ANU’s Mayi Kuwayu project.

Primary prevention policy and practice in Victoria

There are many different policies and strategies governing primary prevention activity in Victoria. Understanding this environment is helpful when considering your own project and MEL practice.

Ending Family Violence is Victoria’s overarching strategy for ending and responding to family violence across Victoria. It is accompanied by the Family Violence Outcomes Framework, which includes four domains and associated indicators for mapping progress towards ending violence across Victoria. These are:

Domain 1 – Prevention
Domain 2 – Victim Survivors
Domain 3 – Perpetrators
Domain 4 – Systems

Focusing on measuring outcomes allows us to see and understand what works (or doesn’t) in creating long-term change.

Free from Violence is Victoria’s overarching strategy for preventing family violence and all forms of violence against women. It is a key part of the Victorian government’s family violence reforms. The strategy is being implemented in three stages over a 10-year period, reflecting that societal change is a long-term process.

The Free from Violence Outcomes Framework [PDF]) contains the prevention domain and four prevention outcomes from the overarching Family Violence Outcomes Framework. The Free from Violence Outcomes Framework is currently in the process of review to explore the development of short-term and intermediate outcomes, indicators and measures.

The Free from Violence Strategy’s Monitoring and Evaluation Strategic Framework [PDF] is a principles-based document that governs our evaluation of Free from Violence. It provides advice on evaluation approaches and key priority areas for evaluation.

You can find Victorian and national primary prevention strategies, policies and frameworks on our resources and templates page.

Endnotes