Chapter 2: Study methods

.
Contents

The Man Box 2024 is a mixed methods study that collected data using a national survey of Australian men and focus group discussions. The Men’s Project led the research with support from Respect Victoria’s research team. Respect Victoria provided subject matter expertise on primary prevention and technical advice to inform all stages of research design, implementation and analysis. The Men’s Project engaged Wallis Social Research and CloudResearch to collect, clean and analyse the survey data, with guidance from staff at The Men’s Project. Wallis Social Research recruited focus group participants and facilitated the focus group discussions. The Jesuit Social Services Human Research Ethics Committee approved all components of the study. An advisory group convened by The Men’s Project helped guide analysis and identify risks throughout 2023.

The Men’s Project authored the research report The Man Box 2024 (13), which primarily focuses on findings from analysis of survey data. Professor Michael Flood, part of the research team who designed the Australian Man Box tool, provided expert commentary on these findings, which is included as a chapter in the report. Respect Victoria worked in close collaboration with staff from The Men’s Project throughout preparation of the publication.  

Respect Victoria conducted the qualitative data analysis and interpretation presented in this report.  

2.1. Qualitative focus group methods

Study design

The Man Box 2024 study included a qualitative exploratory study using focus group discussions to elicit conversations with men about how they experience masculine norms and what factors influence their attitudes and behaviours related to these norms.

The qualitative phase of data collection was designed to understand the perspectives of men who do not personally endorse some or all of the Man Box rules, even if they feel social pressure to conform to those rules. This meant excluding men from the study who, by virtue of their higher level of endorsement of Man Box rules, are more likely to condone violence against women or have perpetrated violence against women (13). This choice was made for two reasons.  

First, it allows a focus on men who are more likely to be open to discussions about how their attitudes and behaviours in relation to the Man Box rules – and by extension, the gendered drivers – align or diverge and why. This helps to address an ongoing gap in primary prevention research around how to engage more men in primary prevention. It is critical to understanding how to design approaches to primary prevention that resonate with the majority of Australian men.  

The findings presented in this report are therefore intended as complementary to efforts to build more and better understanding about how to shift the attitudes and behaviours of men who are more likely to condone and/or perpetrate violence against women and other forms of gender-based violence. The Australian National Research Agenda (44) highlights work with men who use or are at higher risk of using violence as a priority to support the aims of the National Plan to End Violence against Women and Children 2022–2032 (2). The analysis presented in this report supports this work through considering how other men can do more to shift gendered social norms such that discriminatory or violence-supporting attitudes and behaviours are no longer socially rewarded or accepted.  

Second, Respect Victoria and The Men’s Project research teams identified that including focus group participants who highly endorse the Man Box rules, and who are therefore more likely to have perpetrated violence against women (13), raised ethical considerations for the study. These included considerations of safety and comfort of other participants, and additional demands on focus group moderators to manage the potential risk of participants who might seek to dominate discussion or collude with other participants by encouraging sexist or violence-supporting views. Managing these ethical considerations would have logistical implications requiring time and resourcing to address; doing so within the parameters of the project would have limited the total number of focus group discussions conducted for the study. Ultimately, the combined research team chose a study design that allowed for seven discussions, including with fathers, and with gay, bi+ and trans men.

Eligibility and recruitment

Eligible participants were those between the ages of 18 and 45 who identified as a cisgender or transgender man, and resided in Australia. In August and September 2023, Wallis Social Research recruited focus group participants based on a screening phone call to determine key characteristics for group segmentation of age, parental status, sexual orientation, education, occupation and the degree to which they endorsed the Man Box rules.  

The Men’s Project and Respect Victoria then categorised participants into focus groups (see Table 1) to generate a range of perspectives about navigating masculine norms based on common experiences including fatherhood; gay, bi+ and trans identity; or educational and occupational pathways. Men were also categorised by their level of endorsement (low and moderate) of four Man Box rules (footnote 1). This reduced the risk of conflict over differences in belief systems within each group, and the risk of discomfort to participants when encountering potentially extreme views about violence and gender.

Table 1: Focus group demographics, by group

GroupMan Box endorsementAge cohortGender and sexualityEmployment/study statusFatherhood status
1Low18–30 Cisgender and heterosexual Working in male-dominated trade Did not say 
2Low18–30 Cisgender and heterosexual Studying at university or employed (degree) Did not say
3Moderate18–30 Cisgender and heterosexual Working in male-dominated trade Did not say
4Moderate18–30 Cisgender and heterosexual Studying at university or employed (degree) Did not say
5Low18–30 Trans, gender diverse, gay or bi+ Varied Did not say
6Low31–45Cisgender and heterosexual Varied Has a child/children 
7Moderate31–45 Cisgender and heterosexual Varied Has a child/children 

(Source: Adapted from The Man Box 2024, p 149)

Data collection and management  

In August and September 2023, Wallis Social Research conducted seven focus group discussions with a total of 33 men from across Australia in groups of four to five participants. Participants came from varied metropolitan, regional and rural locations, and included men from culturally diverse backgrounds. Discussions lasted approximately 90 minutes and were based on a semi-structured discussion guide with questions addressing:

  • perceptions of masculine norms and influences
  • expressions of emotional vulnerability
  • expectations for intimate partnerships

intended bystander actions in cases of homophobic bullying and street harassment of women.  

Participants received e-vouchers valued at $100 from The Men’s Project in recognition of their time and participation. Discussions were conducted via Microsoft Teams and were audio-recorded, transcribed and securely stored. Transcripts were uploaded into NVivo 14 for analysis.

Table 2: Targeted demographic characteristics of focus group participants

Demographic characteristicCohorts recruited for
Fatherhood status
  • Has a child/children
  • Does not have children
Education level
  • Year 12 or below
  • TAFE, certificate level
  • Bachelor degree or above
Employment status
  • Unemployed  
  • Student  
  • Employed  
  • Home and caring duties 
First Nations status
  • Aboriginal and/or Torres Strait Islander  
  • Not Aboriginal or Torres Strait Islander  
Language spoken at home
  • A language other than English
  • English  
Disability
  • Has a disability, health condition or injury that has lasted, or is likely to last, six months or more (includes stress-related, mental health and intellectual as well as physical conditions)  
  • No disability  
Geographic location
  • Capital city  
  • Regional town  
  • Rural/remote 

(Source: Adapted from The Man Box 2024, p 150)

Analysis  

Our research team developed a codebook (45) based on themes from the discussion guide of relevance in the places where men live, work and socialise (46). This draws on Change the story’s definition of settings relevant to prevention as ‘environments in which people live, work, learn, socialise and play’ (1 p 136). We piloted the codebook by coding three transcripts, each of which was coded independently by two researchers. We then refined the codebook based on discussion of the coding process. After initial coding of all seven transcripts in NVivo 14, we re-organised the codebook under the following themes found to be prominent in the data:

  • fatherhood
  • gender equality in households
  • online spaces
  • partnerships
  • peer interactions
  • public safety (focusing on sexual harassment and homophobic harassment)
  • workplaces. 

This phase of analysis focused on identifying participants’ experiences of competing pressures related to gendered norms, and the different ways they enacted masculine norms in different contexts, as well as barriers and levers influencing how they enact those norms within each theme. This lens for analysis draws on the view that masculinities are multiple, dynamic and situational, as articulated in the Men in focus – Evidence review (10 p 30–31).  

We exported all quotes under each code into Excel to organise and refine analysis within each theme, iteratively combining and expanding themes through a reflexive writing process (47). We engaged in periodic coding workshops to discuss emerging findings and iteratively refine analytic approaches.  

All quotes presented in this report include a reference to the population segment the group comprises and the group’s level of endorsement of the Man Box rules (see Box 6 for more detail). 

Box 6: A note on focus group quotes

This report presents quotes from the focus group participants to illustrate the discussion. With the exception of men in the gay, bi+ and trans focus group, all focus group participants identified as cisgender heterosexual men. The following notation is used to indicate which focus group the quote is drawn from.

University-educated

Men who were studying at university or employed with a university degree.

Male-dominated trade

Men who were employed in a trade that has been historically male-dominated, such as carpentry or mechanics.

Fathers

Men who had children. Men employed in any industry were included in these groups.

Gay, bi+ and trans

Men who identified as gay, bisexual or a sexuality other than heterosexuality, as well as men who identified as trans or gender diverse.

Low Man Box endorsement

Men who indicated low levels of agreement with a short-form Man Box measure used to assess overall endorsement of Man Box rules.

Moderate Man Box endorsement

Men who indicated moderate levels of agreement with the short-form Man Box measure.

See Table 1 for more information about the focus groups.

↑ Return to the top of this page ↑ 

2.2 Quantitative survey methods

Study design

The Man Box 2024 study included an online survey designed to measure men’s perception of social pressure to conform to 19 stereotypical masculine norms that require men to be stoic, aggressive, emotionally invulnerable, physically attractive, in control of their partner, the breadwinner, unengaged in domestic labour, heterosexual and frequently seeking out sex (see the Man Box rules in Box 5). The survey also sought to measure how much they personally agree with these norms, as well as a range of attitudes, behaviours and experiences that impact the safety and wellbeing of themselves and others.

Participants

The survey sample was comprised of 3,519 cisgender or transgender men who resided in Australia, with 2,523 participants aged 18 to 30 and 996 participants aged 31 to 45. The participants were broadly representative of the population of Australian men.

For further information on the quantitative research methods, see The Man Box 2024 (13 p 20–27, 135–148).

Analysis presented in this report

Alongside findings from Respect Victoria’s analysis of the focus group data presented in Chapters 3 and 4, we include boxes with key findings drawn from survey analysis in The Man Box 2024 (13). These findings have been selected to:  

  1. complement themes that emerged in focus group discussions
  2. highlight the differences in men’s perception of social pressure to adhere to masculine norms and their personal belief in these norms
  3. examine how these reported social pressures and personal attitudes have changed over time through comparing 2018 and 2024 Man Box study findings.

2.3 Limitations

There are several limitations to this research that should be considered when reading this report.

The sampling approach for the qualitative study was designed to engage men with a diverse range of experiences and backgrounds. However, the recruitment approach and design of the discussion guide created limited opportunities for participants in most of the groups to discuss how social pressure to perform masculinity can vary based on one’s cultural or racial background, trans identity, sexuality, faith, disability and other factors. This restricted the depth of our application of an intersectional lens throughout the analysis (28). As such, the data does not allow for specific recommendations for work with men with different lived experiences of discrimination, or in-depth analysis of how intersecting forms of oppression shape masculinities, and what this means for preventing violence against women and other forms of gender-based violence. We integrate discussion of intersectionality in this report to the extent possible based on the nature of the focus group data. In particular, we apply this lens to the data from the gay, bi+ and trans men’s focus group, which explored experiences of the intersection between gender, sexuality and masculinities.  

This study did not include participants under the age of 18. Different study design and ethical considerations are required in considering research with children and young people, and these were out of scope for this project. As such, this data reflects only the attitudes and beliefs of men considered as legal adults in Australia. Our analysis includes their observations and memories of how masculine norms have shaped their attitudes and behaviours across their life course rather than representing contemporary experiences of boys and adolescents.

As described in Section 2.1, men who demonstrated high endorsement of the Man Box rules during participant screening were excluded from the focus groups. As a result, this report does not include findings about how men who are more likely to perpetrate intimate partner violence discuss their relationship to the Man Box rules. The quantitative phase of the study, however, did include men of all levels of endorsement of the Man Box rules, generating insights about this group. For information on the impacts of men’s level of Man Box endorsement, see The Man Box 2024 (13).  

The discussion guide had more questions than could be asked in the allotted time of 90 minutes per discussion, so not all groups were asked all questions. The semi-structured discussion guide also focused on a smaller subset of themes than the full quantitative survey. Future qualitative research could explicitly examine, for example, the relationship between men’s endorsement of traditional masculine norms and their attitudes towards sexual violence, use of violent pornography and their perpetration of sexual violence.

Footnotes

Chapter 2 footnotes
  1. Man Box endorsement was based on responses to four items from the Man Box scale:

    1. In my opinion, men should figure out their personal problems on their own without asking others for help;

    2. In my opinion, a man who doesn’t fight back when others push him around is weak;

    3. In my opinion, a man should act strong even if they feel scared or nervous inside;

    4. In my opinion, it is not good for a boy to be taught how to cook, sew, clean the house or take care of younger children.

A participant was categorised as ‘low endorsement’ if they disagreed with all four statements, ‘moderate endorsement’ if they agreed with one of the statements, or ‘high endorsement’ if they agreed with two or more of the statements. These screening questions were selected based on advice from the recruitment team that recruitment would be less likely to alienate men if the selected questions did not elicit strong emotion and did not introduce the theme of violence.