Do Australian police engage in racial profiling? A method for identifying racial profiling in the absence of police data

ABSTRACT

Expanding on a technique developed by Epp, C. R., Maynard-Moody, S., & Haider-Markel, D. (2014). Pulled over: How police stops define race and citizenship, this study develops a methodology for identifying racial profiling from the descriptions of police stops obtained from a survey of members of the public. The analysis subjects 981 accounts of police stops in Victoria described in a survey conducted in 2018–2019 to a ‘threshold’ analysis followed by a multinomial regression controlling for age, gender, LGBITQ status and disability. The results provide strong evidence that race is associated with decision-making by police in Victoria about who to subject to high discretion stops and unjustified post-stop conduct and a consistent pattern of unjustified law enforcement attention on and treatment of racialised compared with white people. This is particularly apparent for Aboriginal, African, Pasifika and Middle-Eastern/Muslim appearing people. The unjustified nature of this policing provides evidence that it is racial appearance, not crime, that is triggering police attention and subsequent behaviour.

KEYWORDS:

• convenience sample
• methodology
• police
• racial profiling
• threshold analysis
• Victoria

Introduction

The experience of being racially profiled leaves individuals and the communities they come from criminalised, humiliated and marked as suspicious, second-class citizens (Cunneen, 2001; Epp et al., 2014). This has serious impacts on health, inclusion and belonging (Selvarajah et al., 2022; Weber, 2020). Racial profiling prevention should therefore be a primary focus of law enforcement agencies. While in 2015, Victoria Police introduced a ban on racial profiling (Victoria Police, 2015), it appears to have no mechanism to monitor the prevalence of racial profiling or enforce the ban. Understanding the prevalence of racial profiling in Australia is a critical first step towards its elimination. The policing of Aboriginal people in Australia has been the subject of a royal commission, numerous inquests and reports (see for e.g. Blagg et al., 2005; Cunneen, 2001) and, to a lesser extent, so too has the policing of Middle-Eastern/Muslim (See for e.g. Collins et al., 2000; McElhone, 2019; Sentas, 2014), Asian (White et al., 1999) and African (Smith & Reside, 2010; Weber, 2020) communities. Furthermore there exists substantial research into the impact of procedural justice by police on racialised¹ communities (Murphy & Cherney, 2011; Murphy & Mazerolle, 2018; Sargeant et al., 2023). However, in Australia, there has been relatively little attention paid to the question of whether police are more likely to, without justification, stop and investigate racialised people compared with white people (Cunneen, 2001, p. 31; but see Weatherburn & Thomas, 2022). Building on the work of Epp et al. (2014), this is the first quantitative study in Australia to investigate whether police in Victoria disproportionately subject racialised communities to unfair or unnecessary treatment or scrutiny. Its publication comes at a time when the contribution of police to the trauma of First Nations people is under intense scrutiny through the Yoorrook Justice Commission and marks the ten-year anniversary of the settlement of the Haile-Michael v Konstantinidis racial profiling claim.

There are a number of strategies utilised across international scholarship to identify racial profiling. All involve a mechanism for comparing the police treatment of Indigenous and racial or ethnic minorities to white people. The dominant strategy for identifying racial profiling is called benchmarking. It involves comparing the rates at which different racialised populations are stopped, searched, arrested, charged or cautioned compared with their relative size in the population of interest (Equality and Human Rights Commission, 2013; Lamberth, 2004; Wortley & Jung, 2020). This strategy requires access to police data on the frequency of various activities directed against different racial groups, and a population benchmark, such as census data or surveys of the populations who use particular roads or areas (Equality and Human Rights Commission, 2013; Lamberth, 2004; Wortley & Jung, 2020). While it is possible to draw conclusions about reasonableness of the police action from carefully constructed benchmarking studies, (see for example Kadane & Lamberth, 2009) in general, benchmarking studies do not assess the reasonableness of the police activity, just its racial disproportionality. While this provides us with important information about systemic racism (Kendi, 2019, p. 17), it does not, necessarily, pin-point the police role in generating it.

The other dominant strategy for assessing racial profiling is to undertake a ‘hit rate’ analysis. A hit rate analysis is a technique that measures the relative ‘find’ or ‘arrest’ rates between different racial groups following one particular police tactic: a search, (Baumgartner et al., 2018; Gelman et al., 2007; HMICFRS, 2017; Persico & Todd, 2008). A lower ‘hit’ rate for a particular racial group creates an inference that police are applying a lower standard of objective reasonableness when they decide to search members of that racial group. This provides evidence of racial profiling (Tanovich, 2004). This method of detecting racial profiling does not analyse whether or not the police had objective evidence of reasonable suspicion before they searched a person, as required by law in most jurisdictions. Instead, it uses a ‘hit’ (either the finding of contraband or an arrest) to infer whether the search was reasonable.

A third test for racial profiling, ‘the threshold test’, derived from Epp et al. (2014), has a particular advantage in comparison to the others. This is because it directly assesses the reasonableness of police activities, rather than drawing inferences about them. This method examines available information about the reasons for and the circumstances of police decisions to stop and subsequently engage in post-stop conduct and makes a legally-based assessment about whether the police had objectively reasonable grounds for their action. The rates at which different racialised groups experience objectively reasonable policing can then be compared. The threshold test can be used to examine a variety of police interactions with the public including, decisions to stop, search, question, detain, issue directions or provide medical treatment. For example, in comparing vehicle stop experiences, information that the police have observed a person going through a red light would justify their decision to stop the vehicle, but would not, on its own, justify a search of the vehicle or its passengers. The test requires information about the circumstances leading up to and during the interaction combined with a legal assessment of whether police activities have met the relevant reasonableness standard. The information needed to conduct this assessment could be derived from police notes, CCTV or on body worn cameras. Importantly, it can also be obtained through conducting a survey of members of the public who have been stopped by police (Epp et al., 2014). The threshold test is, consequently, a methodology for detecting racial profiling in circumstances where the police do not collect or make public, data on their activities, or where that data is unreliable.

Methodologies for understanding police practice drawing from the accounts of members of the public are a necessary response to the closed and secretive institutions of policing. This study was conducted at a time when its authors’ attempts to obtain police data through a freedom of information (FOI) request was resisted through a protracted litigation in VCAT. Studies based on public accounts also make a vital contribution to the literature through drawing on the experience of the policed to build an understanding of police practice (Epp et al., 2014; Futterman et al., 2016; Rios, 2006; Smith, 2020). These methodologies are particularly important in Australia where police do not routinely make available data on their activities.

Policed people’s accounts of police stops may be inaccurate because members of the public do not have access to the information police had before the stop and their account may not be a timely record of the interaction. They have, however, some significant advantages. A survey of individuals can be constructed using text and multiple-choice answers that enables a member of the public to give a very detailed account of the stop and what was said to them. Such accounts are rarely completed by police officers after stops and even more rarely contain reasons for why the individual was selected. Furthermore, members of the public completing a survey do so in circumstances where they derive no benefit to themselves in describing the conduct, in contrast to police accounts derived from official records which may have been ‘manipulated by officers to protect themselves against possible review by supervisors’ (Chan, 1997, p. 79) or the courts (Davies, 2009, p. 111,112). While body worn camera evidence, if reliably recorded, overcomes some of these disadvantages, it remains inaccessible under FOI (see s33 Freedom of Information Act (Vic) 1982) to independent researchers.

Data collection and validity considerations

In Victoria it is very difficult to access data on police stop experiences in a randomised manner. The electoral roll does not include non-citizens, those who have not enrolled to vote or those who choose to keep their details confidential. Furthermore, in Australia the electoral roll is not electronically available to social science researchers. Thus, to take a random sample would require hours at the electoral office writing down names and addresses and many thousands of dollars in postage and follow-up postage to potential participants (see for e.g. Oliveira & Murphy, 2015, p. 264). An alternative strategy would be to pay a company with access to this data, to create a random sample. Even so, due to call refusal and other factors, the samples these options create contain biases.

As an alternative, this study obtains a convenience sample of the population in Victoria through advertising a survey through paid ads on the Facebook pages of the Flemington & Kensington Community Legal Centre, (‘FKCLC’), and the Victorian Aboriginal Legal Service, (‘VALS’) as well as through staff contacts at the Centre for Multi-Cultural Youth, Federation of Community Legal Centres, Djirra (Aboriginal Family Violence Service), and other legal services throughout Victoria. The first author also distributed 1000 postcards to public libraries and community centres including the Asylum Seeker Resource Centre and ran a paid ad on 3CR community radio.² These recruitment sites were chosen to increase the chance that Aboriginal and other racialised people would encounter the survey. The 77-question online survey on the Qualtrics platform was specifically constructed through text box and multiple choice questions to draw out from members of the public, the circumstances, and reasons for police interventions and post-stop conduct (Hopkins, 2022, pp. 247–294).

Over a period of seven months (22 October 2018–28 May 2019), 981 people responded to the survey about their last experience of being stopped by a member of Victoria Police either as a pedestrian or in a vehicle. This produced a sample of 691 individuals stopped while driving of whom 21 per cent were racialised and 260 individuals stopped as pedestrians or bicycle riders, of whom 31 per cent were racialised. In August 2015, Victoria Police introduced a ban on racial profiling. Of the vehicle stops described by participants, 94 per cent occurred after August 2015. Of the pedestrian stops, 72 per cent occurred after August 2015. The data from this survey then, largely reflects the experiences of people after Victoria Police’s anti-racial profiling policies were introduced.

The participation of the 981 self-selected individuals who responded to the survey is non-representative in a number of ways including as a result of being accessed through the networks of particular institutions rather than randomly across the whole Victorian population. As a consequence, they may be more likely to have had a bad stop experience with police or to have an interest in policing, so sparking their decision to follow the social media of these institutions and subsequently respond to the survey (Haynes & Robinson, 2019; Smith, 2012). This means that it is impossible, from this study, to make claims about the rates of different types of police stops or behaviours against population benchmarks. However, this is not the aim of this study.

The strength of this research design is that it involves making comparisons within the self-selected group of people who answered the survey, all of whom are subject to the same set of selection biases. This study is based on the primary assumption that there is no relationship between a person’s racial appearance AND the type of stop they describe WITH their decision to respond to the survey. This means it is possible to compare the odds of white people’s experiences with those of racialised people.

This assumption is detailed in the Appendix, but briefly: suppose we assume that racialised people are more likely to take the survey than white people, because the survey was advertised on the Victorian Aboriginal Legal Service and Flemington & Kensington Community Legal Centre Facebook sites. Additionally, suppose we assume that people stopped for high discretionary reasons will be more motivated to take the survey because they may have higher levels of frustration with the police and thus stronger opinions about the police than others (Smith, 2012). Then there are two possibilities:

1. These two reasons for differential survey taking rate are separate (independent), and do not influence one another, so that white people and racialised people are equally likely to be frustrated with the police when stopped for discretionary reasons, and their likelihood of taking the survey is affected in the same way. In this case the odds ratio is unaffected (unbiased) by different survey taking probabilities.

2. These two reasons for differential survey taking rate are not independent, and do influence one another. For example maybe white people are more likely to be frustrated at the police for being stopped for discretionary reasons than racialised people. If the independence assumption is violated then the odds ratio is biased by different survey taking rates. In this case we cannot make inferences about the study population from our sample.³

Option (a) is the primary assumption on which the validity of this research is based. The assumption that the decision to take the survey is not influenced by an interaction between racial appearance and stop experience, if correct, allows us to draw inferences from data that is not representative of either racial appearance or stop experience in the general population. We make no inferences about the racial appearance or the stop experiences of the general population. Indeed we expect there to be a higher number of Aboriginal and racialised people answering the survey than in the general population, due to where the survey was publicised.⁴ Further, we expect people will report more negative than positive interactions. We do however make inferences about the interaction between racial appearance and stop experience. These inferences can be drawn from a non-representative data set, in light of the primary assumption.

Below is an example of a frustrated white man, who was subject to a high discretion stop and responded to the survey.

Creating reasonableness or justification variables

A key variable of interest in a threshold analysis is whether or not the police intervention was ‘reasonable’ or ‘justified’. The literature reveals that police stop individuals for a range of reasons that are more consistent with their internal working practices than the reasonable suspicion of criminality (Sentas & Grewcock, 2018). The law on the other hand, defines legal thresholds for police intervention (George v Rockett [1990] HCA 26). These legal thresholds take into account behavioural factors where those factors describe what could amount to a reasonable suspicion or belief that an offence has occurred. In this study, a stop or post stop conduct is ‘justified’ if the description of the incident contains an objectively reasonable basis for police to exercise a power to intercept, to question, to demand, to use force or to search.⁷ Police actions are ‘unjustified’ if there appears in the description to be no objectively reasonable basis for the police to act. For example, if the police tell an individual, ‘you are driving erratically’, this meets a legally justified traffic safety threshold and would therefore be characterised in this study as a low discretion stop. This study would characterise as ‘justified’ police then performing a drug and alcohol test. However, unless further evidence was acquired, it would be ‘unjustified’ using this studies definition, for police to then search the vehicle, though they may claim do so by ‘consent’. Behaviour is not used as a control in this study because it is used as a factor to determine whether the description could amount to a reasonable suspicion of a legally proscribed offence. The central point in the analysis is whether the police reason provided or a description of the behaviour of person meets a legal threshold for police intervention.

This study classified each of the 981 stop accounts into various categories depending on the degree to which the police reason for the stop and subsequent police treatment of the individual, was objectively reasonable or justified. It took into account the facts described from the standpoint of the policed person (Smith, 2020, p. 10, 11), the relevant legislative and case law framework guiding police powers in Victoria, and the Victoria Police Manual.⁸ For vehicle stops, the cases were classified into three categories: low, medium and high discretion. ‘Low discretion’ vehicle stops are those where there is objectively reasonable evidence of a traffic safety offence before the stop to justify police action.⁹ These stops include, for example, where the driver has driven through a red light or engaged in speeding. ‘Medium’ discretion vehicle stops are those where there is objectively reasonable evidence of a minor traffic infringement observed prior to the stop such as failure to indicate before changing lane. ‘High’ discretion vehicle stops are those where there is no evidence of an offence before a stop is initiated by police. These include stops where police gave no reason for the stop or explained them as a ‘random intercept’, or random breathalyser stop. When the police in Victoria engage in ‘random’ breath/drug tests and licence and registration checks¹⁰ these are not truly random stops in the statistical sense (such as rolling a die) or based on a criteria that minimises bias. If police stopped every fifth vehicle for example, this criteria would minimise racial bias. Police in Victoria do not however roll a die or follow a criteria that minimises the potential for bias. They stop people based on internal rules that include ‘picking up a feeling’ about who to stop (Cultural and Indigenous Research Centre Australia, 2013, p. 40).

Pedestrian stops were separated into three categories depending on whether police had observed an offence before conducting the stop (objectively justified), were stopping the person because they were a witness to an offence (similarly objectively justified), or were stopping the person before an offence had been identified (objectively unjustified).

For both vehicle and pedestrian stops, the study categorised police post stop conduct into whether it was objectively justified, or not. For example, the study categorised a question about where a driver was going, after they had been stopped for speeding, as ‘justified’ because it enables police to gain an understanding of the offence and falls within the purpose of the authorised stop reason. However, the study categorised asking a driver where they were going following a random intercept or asking a cyclist what was in their pocket, following a stop for not wearing a helmet, as ‘not justified’ because the question indicated a stop purpose beyond the purported stop authorisation. This classification scheme takes into account the growing case law in Australia and internationally on ‘pretext’ stops (R v Arthur [2018] SADC 116; R v Buddee [2016] NSWDC 422; R v Cook [2017] NSWLC 24; R v Large [2019]; NSWDC 627; R. v. Gayle, (2015) ONCJ 575) and ‘investigative stops’ (R v Le [2017] NSWDC 38; R v Mann (2004) SCC 52) and requires a forensic analysis of the particular power that authorises police to engage in a stop and whether their subsequent conduct can be construed as falling within the purpose of that power. Diagram 1 provides a visual aid to understand the above categories (Figure 1).

Figure 1. Threshold analysis.

Of the 691 vehicle stops described in the survey, 246 (40.2%) were low discretion; 99 (16.2%) were medium discretion; and 267 (43.6%) were high discretion (without objective justification). 179 (33.3% of) vehicle stops were proceeded by unjustified post-stop conduct stops, and 359 (66.7%) with justified conduct.

Below is an example of a high discretion vehicle stop proceeded by unjustified post-stop conduct. This and two further examples from the police stop survey illustrated in this article are selected to contextualise the meaning of the terms this study uses.

Of the 260 pedestrian stops, in 132 cases (56.9%) police made no allegations before the stopping the person. In 85 cases (36.6%) police alleged an offence before stopping a person. In 15 pedestrian stop cases (6.5%) police alleged that the person was witness to an offence. In 140 pedestrian stop cases (62.5%) police post stop conduct was not justified while in 84 (37.5%) post-stop treatment was justified. In the following example, a pedestrian was stopped without any allegations he had committed an offence being made by police. He was then subject to unjustified post-stop conduct.

In the following case a group of women were moved on from a train waiting room by police for no apparent reason. In the studies classification scheme, this is a ‘no offence observed’ pedestrian stop followed by unjustified post-stop conduct.

Because the survey was a convenience rather than representative sample of the population, these figures do not describe the relative frequencies of these types of stops by police across the general population. However, due to the analytical design of this study, this caveat does not apply to our findings on racial profiling.

Creating racial appearance variables

The second critical variable in a threshold analysis is ‘racial appearance’. This study uses racial appearance rather than racial background as the key variable to explore the role of race in police decision-making because it seeks to capture what police observe when they first encounter a person. Racial appearance was the key variable used in Epp et al., (2014, p. 171) in Kansas City, USA. Similarly, Baumgartner et al., (2018, p. 53) used the officer’s observation of the driver’s racial appearance (Black, White, Asian, Native, Hispanic, Other) as their key variable. In contrast, Wortley and Owusu-Bempah (2011) in Toronto used racial identity. In that study participants were asked whether they identified as Black, White or Asian. This present study does not use the Wortley/Owusu-Bempah approach for three reasons: firstly, and particularly for First Nations people, racial/ethnic identity and racial appearance are not always the same (Eatock v Bolt [2011] FCA, 1103, paras. 167–190). Secondly, the three categories used in Toronto do not capture the key racial/ethnic groups present in Victoria. Thirdly, this study is concerned with an observer’s perception of race or ethnicity rather than the racial/ethnic identification of participants.

To determine racial appearance, the survey developed for this study asked people to describe both their racial background and their guess as to how a person, observing them for the first time, would describe their racial appearance. For example, white Muslims who indicate they appear Muslim because they observe Muslim dress or grooming norms are coded as appearing ‘Middle Eastern/Muslim’. In general, this study uses people’s own description of their racial appearance; however, it makes an exception to this rule for white-appearing Aboriginal people who are known to police or were stopped in a group of Aboriginal people. These coding decisions assume that police in these circumstances will, more likely than not, assume the person is an Indigenous person.¹²

The following tables draw on the vehicle stop data from the survey to describe how racial appearance is coded into the two separate variables used in this analysis. The first step in creating the racial appearance variables was to code the survey responses into seven racial appearance categories as listed in Table 1. The table lists the racial appearance category with its frequency. Because each individual could only describe one vehicle stop, each count is a unique individual in the vehicle stop data. They could also describe one pedestrian stop. This will appear as a count in the pedestrian stop data set.

Table 1. Recoded racial appearance (vehicle stops).¹⁷

Racial appearance	Count	Per cent
1. White	544	78.7
2. Aboriginal (this includes known to police and in a group)	29	4.2
3. African	37	5.4
4. Middle Eastern /Muslim	30	4.3
5. Pasifika^18	12	1.7
6. Asian	14	2.0
7. Other (Hispanic/Indian/Ambiguous)	24	3.5
Total	691	100%

These seven groups were then coded into a binary category by classifying people as either of white or racialised appearance (Table 2).

Table 2. Dichotomised racial appearance.¹⁹

Dichotomised racial appearance	Count	Per cent
1. Racialised	147	21.3
2. White	544	78.7
Total	691	100%

This study also created a trichotomous racial appearance variable. The first category of this variable is called ‘targeted’ and consists of four racial appearance groups: African, Middle Eastern/Muslim, Pasifika and Aboriginal. This study defines these groups as targeted because each of these groups have raised concerns in Victoria over the last decade about being the disproportionate targets of police attention (Hopkins et al., 2017, p. 44). The next category is the ‘other racialised group’ (Asian, Hispanic, Indian, Ambiguous). The third category is ‘white’. This variable allows a direct comparison between the experiences of the ‘targeted’ racialised group and white people (Table 3).

Table 3. Trichotomised (targeted) racialised appearance.²⁰

	Frequency	Percentage
1. Targeted (African, Middle Eastern/Muslim, Pasifika, Aboriginal)	110	15.9
2. Other racialised (Asian, Hispanic, Indian, Ambiguous)	38	5.5
3. White	543	78.6
Total	691	100%

Research hypotheses

Having described how the key variables of interest are defined, we now describe the research hypotheses. Racial profiling involves the disproportionate use of unjustified police power on racialised minorities (Epp et al., 2014, p. 5). Consequently, the analytical task is to determine whether the forms of policing classified as unjustified are disproportionately experienced by racialised and targeted racialised people in particular. If so, the study has demonstrated racial profiling.

To undertake this analysis we developed two sets of research hypotheses. You will note these hypotheses do not analyse medium discretion stops for vehicle stops. Nor do they analyse stops where police alleged the person was a witness to an offence for pedestrian stops. While they remain categories within the stop reason variables and are not excluded from the model, the regression analysis compares the extremes: high discretion with low discretion for vehicle stops and no offence observed with offence observed for pedestrian stops.

Vehicle stops

• 1.(a) Police have greater odds of subjecting racialised drivers to high discretion compared with a low discretion stops, than white drivers.

• (b) Police have greater odds of subjecting targeted racialised drivers to high discretion compared with low discretion stops, than white drivers.

• (c) Police have greater odds of subjecting racialised drivers to unjustified compared with justified post-stop conduct, than white drivers.

• (d) Police have greater odds of subjecting targeted racialised drivers to unjustified compared with justified post-stop conduct, than white drivers.

Pedestrian/cyclists stops

• 2.(a) Police have greater odds of stopping racialised pedestrians and cyclists before they have identified an offence compared with after they have identified an offence, compared with white pedestrians and cyclists.

• (b) Police have greater odds of stopping targeted racialised pedestrians and cyclists to a stop before they have identified an offence compared with after they have identified an offence, compared with white pedestrians and cyclists.

• (c) Police have greater odds of subjecting racialised pedestrians and cyclists to unjustified compared with justified post-stop conduct compared with white pedestrians and cyclists.

• (d) Police have greater odds of subjecting targeted racialised pedestrians and cyclists to unjustified compared with justified post-stop conduct compared with white pedestrians and cyclists.

Explanation of odds ratios

The key measure of interest in this analysis is the ratio of the odds of experiencing one type of police conduct over another for racialised respondents compared with white respondents. An odds ratio is a measure of relative outcome. It:

• (ii) Calculates the odds of one group (racialised people) having a particular outcome (high discretion stop) compared to another outcome (low discretion stop); and

• (iii) Calculates the odds of the comparison group (white people) having a particular outcome compared to another outcome; and

• (iv) Compares the result of (i) and (ii) by dividing one by the other.

If the odds were the same for both groups, then the odds ratio would have the value 1. Odds higher than 1 indicate the group faces higher odds of experiencing the outcome than the comparison group. Odds below 1 indicate the group faces lower odds. In this study if a group has a higher odds of experiencing a high discretion compared with a low discretion stop than white people, this means that they disproportionately experience policing that is less justified than white people. Below is a visual representation of the percentage proportions of different vehicle stops types experienced by racialised people and white people in the survey. This data is raw and uncontrolled. A visual inspection reveals that racialised people appear to be disproportionately more likely to experience a high discretion stop than a low discretion stop compared to white people. To determine if there was statistically valid evidence of this difference, the study subjected the data to multinomial regression controlling for age, gender, disability and LGBTQ status (Figure 2).

Figure 2. Proportions of vehicle intercept types experienced by racialised appearing people compared to white appearing people (uncontrolled data).

Statistical methods

Because some of the dependent variables are categorical and trichotomous, this study uses multinomial regression to perform the statistical analysis in this research. Multinomial regression produces a p-value as well as an odds ratio (‘OR’) reported with a 95% confidence interval. To maintain consistency across the results, this study uses multinomial regression for all statistical tests. The multinomial regression becomes binomial when the variables are dichotomous.

Confidence intervals

Statistical models estimate parameters (for example the odds ratio) of the population (all police stops) from a sample (survey respondents), and they do so while also modelling the uncertainty associated with estimating a population parameter from a sample. This uncertainty can be quantified with confidence intervals. Informally, a confidence interval is a range of plausible values for the population parameter (e.g., odds ratio). More formally, confidence intervals are constructed such that if we collected many samples from the same population, and for each sample we calculate a say 95 per cent confidence interval for a particular parameter, then at least 95 per cent of the time, that interval will contain the true population parameter. We present confidence intervals for odds ratios, and these should be interpreted as explained above, with confidence intervals containing an odds ratio of 1 indicating no difference between the groups is a plausible population value.

P-values

A p-value is a number between 0 and 1 that summarises the incompatibility between a particular set of data and the null hypothesis if the underlying assumptions in our proposed model for the data hold true (Wasserstein & Lazar, 2016). Our null hypothesis is that the police treat all groups the same way. If the null hypothesis is true, and police treat all groups the same way, then for example a p-value of <0.05 would occur less than 5 in 100 times we sampled from the population. The smaller the p-value, the greater the statistical incompatibility of the data with the null hypothesis. This incompatibility can be interpreted as casting doubt on or providing evidence against the null hypothesis (Wasserstein & Lazar, 2016). The smaller the p-value, the stronger the evidence is against the null hypothesis.

Vehicle stops

Missing data

About 30 per cent of survey participants recording details about a vehicle stop stopped filling in the survey before the end of the 77 questions. While the survey contains a complete data set for racial appearance and age (the survey design required a response to these variables) other independent variables such as gender, disability and sexuality were answered at the end of the survey and are missing in 33–38 per cent of cases.¹³ Dependent variables (whether the policing was reasonable or not) are missing in 15–20 per cent of cases. While variables are mostly missing monotonically (i.e., once a variable is missing, no further variables are completed) this is not always the case.

Missing data is a common issue in social science research (Rose & Fraser, 2008). According to Jakobsen et al., the rate of missingness in this study’s data set (up to 38 per cent) is amenable to multiple imputation (Jakobsen et al., 2017). This study addresses the issue of missing data using multiple imputation as a sensitivity analysis. To conduct this analysis we assume that the data is ‘Missing at Random’, that is that the missingness is related to the observed predictors (Kang, 2013) such as stop type, age, gender, racial appearance, LGBTIQ and disability status. The results are reported from an analysis of the 419 vehicle stop cases that have complete data for all variables including controls.

Controls

All regressions are performed controlling for age, gender, LGBTIQ status and disability. This study controls for these factors because the literature suggests that police decision-making is influenced by each of these factors (Mogul et al., 2011; Ritchie, 2017). Police may be more ‘suspicious’ of an individual due to their youth, their mental health disposition or their trans or gender non-conforming presentation. Controlling for these variables allows us to hold them constant, so that the inference (odds ratios, confidence intervals and p-values) about racial appearance can be interpreted as comparing people of the same age, gender, disability and LGBTQI status.

Results—vehicle stops

Summary and interpretation of results from Table 4

Stop reason

We have strong evidence of racial disproportionality for stop reason for both racialised (p = 0.001) and targeted racialised (p < 0.001) people relative to white people. The odds of a police officer stopping a racialised person for a high discretion stop rather than a low discretion stop is about 2.8 times higher than the odds for a white person (OR 2.77 [1.53, 5.02], p < 0.001), controlling for age, gender, LGBTIQ and disability.

Table 4. Multinomial regression results for hypotheses 1(a–d) using vehicle stop complete cases.

	Reference group	p-value	Adjusted p-value^21	Odds ratio	Confidence interval
1a. Odds of high discretion stop for racialised compared with white people	low discretion stops	0.001	0.001	2.772	1.530–5.021
1b. Odds of high discretion stops for targeted racialised compared with white people	low discretion stops	<0.001	<0.001	3.615	1.793–7.289
1c. Odds of unjustified post-stop conduct for racialised people compared with white people	justified post-stop conduct	<0.001	<0.001	4.949	2.904–8.435
1d. Odds of unjustified post-stop conduct for targeted racialised people compared with white people	justified post-stop conduct	<0.001	<0.001	7.402	3.948–13.88

Note: The odds ratios compare high discretion with low discretion stops and unjustified with justified stops.

The results for 1(b) show that the odds of a police officer stopping a targeted racialised person for a high discretion stop rather than a low discretion stop is about 3.6 times higher than a white person (OR 3.62 [1.79,7.30]), controlling for age, gender, LBTIQ and disability.

Post-stop police conduct

The results for hypothesis 1c show that the odds of a police officer subjecting a racialised person to unjustified post-stop police conduct rather than justified conduct is about 4.9 times higher than the odds for a white person (OR 4.95 [2.90, 8.44], p < 0.001), controlling for age, gender, LBTIQ and disability.

The result for hypothesis 1d shows that the odds of a police officer subjecting a targeted racialised person to unjustified post-stop police conduct rather than justified conduct is about 7.4 times higher than the odds for a white person (OR 7.40 [3.95, 13.88] p < 0.001), controlling for age, gender, LBTIQ and disability.¹⁴

The results from the imputed data used for the sensitivity test were qualitatively consistent with these results (Hopkins, 2022, app. D). The effect sizes were slightly smaller for the imputed data than in the complete cases. (For example the odds ratio for hypothesis 1(d) was 7.271 rather than 7.402. The p-value remained p < 0.001.)

Statistical methods

Pedestrian and bicycle stops

Missing data

Most of our commentary on statistical issues made in relation to vehicle stops applies to pedestrian stops; however, pedestrian stop records had lower levels of missing data.

Of the people who answered questions about a pedestrian stop, 84 per cent provided complete answers about gender, disability and LGBTIQ status. Between 86 and 89 per cent answered a sufficient number of questions so that a stop type variable and post-stop conduct variable could be created in their case. To account for the missing data, the results are presented from the 203 complete pedestrian and bicycle cases (cases for whom all variables in the analysis were present) with results from the analysis of imputed data provided as a sensitivity test.

Results—pedestrian and bicycle stops

Summary and interpretation of Table 5

The results of these tests do not provide evidence to support hypotheses 2a and 2c over the null hypothesis. The results do, however, provide evidence in support of hypotheses 2b and 2d. The results provide evidence (p = 0.039), controlling for other variables, that the police stop targeted racialised pedestrians before an offence is detected at greater odds (OR 2.85 [1.25, 6.48]), than they stop white pedestrians before an offence is detected (hypothesis 2b). Furthermore, the results provide evidence (p = 0.024), that the police subject targeted racialised pedestrians to unjustified post-stop conduct with greater odds than white pedestrians (OR 3.25 [1.40, 7.55]) (hypothesis 2d). The imputed data used as a sensitivity test produced results that were consistent with the compete case analysis, although the effect sizes in the imputed data were slightly larger (Hopkins, 2022, App D).

Table 5. Odds ratio of stop reason and post-stop conduct for racialised and white people – pedestrian and bicycle stops (controlling for age, gender, LGBTI and disability).

	Reference group	p-value	Adjusted p-value	Odds ratio	Confidence interval
2a. Odds of being stopped before an offence is detected for racialised people compared with white people	After an offence is detected	0.103	0.156	1.754	0.892–3.447
2b. Odds of being stopped before an offence is detected for targeted racialised people compared with white people	After an offence is detected	0.013	0.039	2.845	1.250–6.476
2c. Odds of being subject to unjustified post-stop conduct for racialised compared with white people	Justified post-stop conduct	0.078	0.156	1.842	0.934–3.633
2d. Odds of being subject to unjustified post-stop conduct for targeted racialised people compared with white people	Justified post-stop conduct	0.006	0.024	3.252	1.400–7.554

Discussion

Vehicle stops

This is the first study in Australia to quantitatively evidence the existence of racial profiling in police vehicle stops. The results provide strong evidence (p < 0.001) that police subject racialised people to high discretion stops compared with low discretion stops at about 2.8 [CI 1.5–5.0] times the odds of white people. For Aboriginal, African, Pasifika and Middle Eastern/Muslim-appearing people the odds are about 3.6 [CI 1.8–7.3] times those of white people. These results control for age, sex, disability and LGBTQ status. This is evidence that Victoria Police subject racialised drivers to disproportionate levels of unjustified vehicle stops. This means that racialised drivers in Victoria are more likely to trigger scrutiny from police in circumstances where no crime has been observed than white drivers. This indicates that on a consistent basis, whether officially or through informal practice, race is used by police across Victoria to stimulate further enquiries. These results are consistent with qualitative studies (Blagg et al., 2005; Smith & Reside, 2010; Weber, 2020). The unique nature of this study’s design is its quantitative study of the differences in experiences between white and racialised people in their interactions with police. It is this kind of comparative analysis that Cunneen (2001, p. 34) notes has been largely missing in Australian scholarship to date.

The results for unjustified post-stop conduct following a vehicle stop were even starker. Police subject racialised people to about 4.9 [CI 2.9–8.4] times the odds of unjustified post-stop conduct compared to white people, while for Aboriginal, African, Pasifika and Middle Eastern appearing people the rate was about 7.4 [CI 3.9–13.9] the odds of white people. These results are consistent with international findings (Epp et al., 2014; Higgins et al., 2008). Furthermore, the results obtained were consistent across the raw and imputed data. The missing data is therefore likely to have had little impact on the robust nature of the results obtained in this study. While the odds for targeted racialised drivers being stopped for a high discretion stop are 3.6 [CI 1.8–7.3] times those of a white driver, their odds are 7.4 [CI 3.9–13.9] times higher than white drivers of being subjected to unreasonable post-stop investigation.¹⁵ This means that white people who are unnecessarily stopped are, more frequently, waved on, while for African, Aboriginal, Middle Eastern-Muslim and Pasifika individuals the process of interrogation is only beginning. Starting with an unjustified stop, and followed by intrusive and unnecessary questioning, searching, and other demands, the extended arsenal of the criminal justice system is disproportionately triggered by particular racialised groups.

Pedestrian and bicycle stops

The results reveal that police are more likely to stop African, Aboriginal, Pasifika and Middle Eastern/Muslim-appearing people on the street before they identify an offence than white people: (OR 2.85 [1.25, 6.48]). Police were also more likely to subject African, Aboriginal, Pasifika and Middle Eastern/Muslim-appearing people to unjustified post-stop conduct than white people: (OR 3.25 [1.40, 7.55]). These results provide clear evidence of institutionalised racial profiling against specific racialised groups in pedestrian and bicycle stops. These all provide evidence of a racially disproportionate pattern of unjustified police conduct.

This study only found evidence for racialised differences in police treatment of pedestrians and bicyclists in stop reasons and post-stop conduct when comparing the treatment of African, Indigenous, Middle Eastern/Muslim and Pasifika people with white people, not all racialised people. One explanation could be that the overall policing of pedestrians and cyclists may be far more discretionary than drivers. Data from interviews with survey participants confirmed the high levels of frustration and disagreement people have about most pedestrian and cyclists stops, even those stops where the police purport to have a reason.¹⁶ Furthermore, 66 per cent of pedestrian stops involved unjustified post-stop conduct, in contrast to 33 per cent of vehicle stops. This could mean that the divisions between ‘no offence’ and ‘offence observed’ and justified and unjustified pedestrian stops are not as clearly differentiated as those divisions chosen for vehicle stops.

Validity

This study is based on the experiences and perspectives contained in the accounts of police stops described by members of the public rather than police records. This approach is justified for three reasons. First there is an absence of reliable police data on stop and question practices in Victoria. Second, civilian accounts offer a sharp counterpoint to institutionalised and frequently incomplete administrative police data sets used by other studies, providing a necessary window into the experience of being policed. Third, using a legal analysis of the reasonableness of police conduct described in these accounts, the methodology set out in this study is capable of responding to the question at the core of racial profiling identification: are particular racial groups disproportionately subjected to unreasonable policing?

This study assumes that a person’s decision to answer a survey about police stops may be influenced by their racial appearance or their stop experience, but that these two variables do not interact to influence that decision. While this primary assumption is ultimately no more than an assumption—evidence exists that while minority groups may have a greater awareness of police injustice overall (Buckler & Unnever, 2008) this might not be how they perceive individual encounters with the police. There is evidence that people with a greater sense of entitlement (power) have a greater sense of injustice when they experience individual treatment that ‘violates their expectations’ (Sawaoka et al., 2015). Furthermore, they are more likely to do something about it (Sawaoka et al., 2015). Similarly, Luria et al. (2016) find that people with higher status are more likely to make complaints. In a society like Australia, where whiteness is privileged, we might therefore expect that white people would be more likely to have a sense of injustice following an experience of a high discretion stop or unjustified post-stop conduct and therefore be more likely respond to a survey in these circumstances. If this were the case, the decision to respond to the survey, while not the same as making an official complaint, may well be biased towards accounts of white people reporting unjustified stops and post-stop conduct. The results may therefore under report the odds of racialised people experiencing these types of stops. In other words, it is possible that any conclusions drawn from this study may underestimate the extent of racial profiling.

Evidence from a random sample does not require making this assumption providing more assurance. However, in circumstances where there are no means to obtain a randomised sample of the public, the methodology adopted in this study, along with what may be a conservative but untestable assumption, is appropriate.

A potential limitation of this study may be that it does not control for pre-existing beliefs about racial bias by police. However its findings are not based on people’s views about whether their individual encounter with police was racially biased. The survey asked people to describe using text box and tick box answers their most recent experience of being stopped by police to determine whether the encounter and police conduct was triggered by some objective explanation (such as an allegation that a driver had swerved or had gone through a red light). Some readers might assume that participants are likely to describe themselves in a more favourable light than the police, however because the study compares groups of participants all of whom may be expected to share any such tendency, the comparative findings remain valid.

The findings of this study are consistent with a benchmarking analysis conducted on police data obtained under court subpoena from two inner suburbs of Melbourne, Victoria from 2006 to 2009 (Hopkins, 2021; see also Hopkins & Popovic, 2023). While using very different methodologies and different data sets, the broad consistency of the benchmarking findings with the present study lends support to the validity of the present methodology and the reproducibility of its findings. Based on the available evidence, we conclude that where it is not possible to conduct a random sample, a threshold analysis of a convenience sample from members of the public, using the assumption described in this article, if and as applicable in the circumstances of the study, is a valid method for identifying racial profiling in the absence of reliable and relevant police data.

Conclusion

This study evidences the operation of a dual system of policing in Victoria—a textbook version more likely triggered when police interact with a white person and a speculative, intrusive one more likely triggered when police interact with particular racialised groups. The messages police practices send are clear—if you are a member of a particular racial group you are suspicious, untrustworthy and out of place. Racial profiling has been unlawful since the 1975 introduction of the Federal Racial Discrimination Act (‘RDA’). These findings, based predominately on stops after Victoria Police’s August 2015 ban on racial profiling, suggests that this ban is ineffective and that breaches of the RDA continue. This paper, like others in the international literature (Baumgartner et al., 2018; Epp et al., 2014; Epp & Erhardt, 2021), has identified that particular police practices, including routine intercepts, pretext stops and consent searches are more likely to be targeted at particular racialised groups. These studies suggest that racial profiling could be greatly reduced if police agencies across Australia eliminated these practices. There is a pathway for each of these steps. Pretext stops should be banned following the reasoning in R v Buddee [2016] NSWDC 422, consent searches can be legislatively banned (see for example the UK, PACE Code A, (2015) [1.50]) and routine vehicle intercepts and street checks should be restricted to where police possess reasonable grounds to believe an offence has occurred (see Michael McDonald & Taylor, 2019). Furthermore, adherence to these limits on police powers, and their impact on racialised people, must be transparently and comprehensively monitored through an independent agency, complaints must be independently investigated, and breaches must be subject to an escalating disciplinary response that includes dismissal (Hopkins et al., 2017). Increasing police accountability is a critical task and yet governments continually fall short in this area. Alternative suggestions such as, replacing traffic police with traffic cameras (Barnes, 2019), and police patrols with health and community workers may produce a more reliable reduction in racial profiling.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 The term ‘racialised’ refers to people who, in white dominant societies are socially constructed as having a ‘race’. In contrast ‘white’ people are seen as ‘unraced’. For example, descriptions of white people in Australia do not often describe their race.

2 This research was approved by the UNSW Human Research Ethics Committee (HC180500) and was conducted with the support of Flemington & Kensington Community Legal Centre and the Victorian Aboriginal Legal Service. The first author attached the survey to a website that took potential participants through a consent and information form.

3 See Appendix—Gordana Popovic, Unbiasedness of odds ratios under differential survey taking rates.

4 This was the case: 4% of the survey respondents were Aboriginal, while 1% of the Victoria population is Aboriginal (ABS Census 2021).

5 Case 127 in the vehicle stop data file.

6 Peelian principles prioritise policing by consent.

7 The exception to this is batch drug and alcohol testing of vehicles where the police select groups rather than individual vehicles.

8 This is a manual, updated quarterly, available by subscription from the Victoria Police.

9 Included in this category are cases where the driver had been stopped for a preliminary breath test under s54 of the Road Safety Act 1986 (Vic) in circumstances where batches of drivers are pulled over without the opportunity for selectivity on the part of the police.

10 These are stops under s53, 55A and 59 of the Road Safety Act 1986 (Vic).

11 Case 809 in the pedestrian stop data file

12 These assumptions are subject to a sensitivity analysis contained in Appendix D of (Hopkins, 2022).

13 Qualtrics training recommended demographic questions be left to the end of the questionnaire. The questionnaire is available at (Hopkins, 2022, app. A).

14 As a sensitivity test, at the end of the study I controlled for stop discretion; the p-value continued to be <0.001 for the effect of racial appearance on unjustified post-stop conduct.

15 A sensitivity test showed that the association between race and post-stop conduct was very strong, even after controlling for stop reason.

16 For example allegations of jaywalking, drunk in a public place, stealing from a clothing bin.

17 This table comes from Table 6.1 in (Hopkins, 2022).

18 In this article we use the term ‘Pasifika’ to include people who appear to be of Pacific Islander (ie from Samoa, Tonga etc.) or Maori appearance.

19 Table 6.2 (Hopkins, 2022).

20 Table 6.3 (Hopkins, 2022).

21 These p-values have been adjusted using the Holm-Bonferroni multi-test correction.