ACCC vs Coles - why is a case about data, so weak on data?

I've been watching the ACCC vs Coles case with interest and dismay. As a statistician (who has written expert witness reports), here are my observations and where I think the ACCC has failed to make a case.

As a quick note, I've contacted two different lawyers at JWS to try to assist them; neither has bothered to reply. I also contacted one of the barristers. If the people being paid to try and win the case aren't going to do their best, then I'll put this on the public record.

For case that is fought on numbers, the inability of the ACCC to provide one, simple, compelling summary of the data is woeful. I'd ask what everyone has been doing for the past couple of years?

It's hard to know where the fault lies here. Is it the solicitors, the barristers or the expert(s) who are incapable of realising that the case they're presenting is, at best, weak, despite what seems clear and obvious?

I wonder if part of the hubris is simply that the ACCC believes so strongly that they never bothered to do the hard work of presenting overwhelming evidence, instead relying on catching people out on email. Maybe they never had anyone point out how poor the case was, and when I did, it was easier to ignore this. My experience is that when it comes to data, good barristers know what they don't know. The best ones I've worked with will say 'I think there's something wrong here, but I don't understand what the problem is'.

The theory states that experts are to respond to the questions in the letter, but any half-decent expert should be able to identify the most useful questions to answer, given the time and cost of the court. The legal system is adversarial, but the goal is still to find the truth.

I'm mostly critical of the expert witness engaged by the ACCC (the Wright Report), an accounting firm without any training or expertise in data, and it shows with what's been made public so far. The work done by the accounting firm looks fine, but it only summarises which promotion was run at what time and at what price. That's not "an analysis".

To quote my email to JWS "My experience is that for data to make a difference in a case, it needs to be absolutely compelling."

The case presented by the ACCC lacks data, which, in my opinion, is compelling.

What is, in my words, 'absolutely compelling'? Examples of work I've produced:

• The probability of these two events occurring by chance is one in a million.
• The number of things you said had occurred is more than all the things that could occur.
• The way the result was determined could not change unless this very extreme position were taken (that is, unrealistic).

At least one of these cases you'd know from about a decade ago, as it was in the media for weeks, as high-profile as the ACCC vs Coles.

To reach the point of the one-line summary requires hard work, often 100+ paragraphs of laying out an argument in such a way that there is no alternative position. Importantly, these conclusions hold weight because there is no reasonable alternative position, as there is a probability distribution, a p-value, or a confidence interval. None of which is evident anywhere in the ACCC vs Coles.

The argument of the Applicant

The ACCC's argument seems reasonable. Coles had a product at Price 1 for a long time. Coles then increased the price to Price 2 for a short time and then settled on Price 3 (which was greater than Price 1 but less than Price 2). No one disputes these facts.

The behaviour is described as "…the making of false or misleading representations by the Respondent (Coles) in connection with its “Down Down” promotional program."

What did the ACCC do? Did they produce a single compelling summary of the data? No. There's a table in Annexure A of the Opening Submission, drawn from the (First) Statement of Agreed Facts, that is, unsurprisingly, without a heading or anything obvious to look at. I'm labouring the point, but to include a table without a heading is appalling.

It's not a great start. I'd assume that the ACCC is putting forward their strongest arguments in the Opening Submission. The ACCC has two variables on any given day. They know the price of an item, and can therefore determine the duration of a price (which doesn't change very often). Technically, there's a third variable, the promotion (white ticket, down down, etc.).

That's all the ACCC have, that's all they present, and from this, with the use of a magic wand, the ACCC wants the court to believe that you infer "…the making of false or misleading representations by the Respondent (Coles) in connection with its “Down Down” promotional program."

There is nothing in the data as presented that makes this behaviour seem "…false or misleading representations…" I can't fathom how the ACCC can't see that, based on the data they've presented. The reason I hold this view is that you need to compare what you believe to be true with an alternative position and see how well the argument holds up. If we were to consider a reasonable and legal business that had kept prices stable for a long period of time, then increased prices (and saw a noticable drop in sales), and choose to run a fair and reasonable discount and inform consumers via a sign, the data would be exactly the same as the data the ACCC is presenting about Coles.

There is no way to differentiate between "false or misleading representations" and a business increasing prices, then running a fair and reasonable discount (because sales drop). If you can't distinguish between these two positions with the data, why even present the data? Why even have the case?

One approach to thinking about data is to consider the extremes and see how your argument holds up. Here, it's obvious no one thought about this. The first extreme is that the price increases for an hour (or a minute, or a second) and then is dropped the next hour (or minute, or second).

If a consumer saw this, most people would agree that this behaviour is congruent with the idea that the discount is false or misleading. What about the other extreme, where prices increase not for weeks, but for months or years? Any discount from this sustained price increase will be 'real'. The more important issue is that consumers have overpaid simply to satisfy the ACCC. I am sure Coles would be happy to make even more profit, which is what you're asking.

By considering the extreme views and realising they contradict each other, you're left with a weak argument that needs serious work to overcome the inherent shortcomings of not being able to infer directionality. Directionality is a term I use, it doesn't have a technical definition. Think of directionality like vectors. As a counterexample, where directionality was congruent was the third example of my own work. When the weightings were changed in either direction, the outcome remained the same. This is a clear, simple outcome that anyone can understand. This translates into the response being 'your claim that if the weightings were different, you would have been successful is not true.' There is no short, concise statement about the data from the ACCC.

The Applicant's expert witness report

It's very easy to criticise the Wright report, not because of the work done, but because of the work that wasn't done. I don't know if this is a result of poor instruction from JWS, the barrister(s), or the expert not considering the required information to argue the case.

The PDF is 132 pages, which seems excessive for what we're told, or not told. Maybe I'm being unreasonable as an outsider. When you spend more pages talking about your past work than solving the problem it's clear this is performative. The expert is provided with the data on each of the 12 items in the sample. I will note that no one explains how this sample of 12 items was selected, or whether a statistician was involved. Why 12? Solicitors and barristers should not be involved in this process.

Back to the report, section 3 "Analysis of Pricing Data" does not contain any information I would consider "analysis". The author has stated in 3.3.5:

" I have verified the graphs against the source sales data contained in Annexure 1 of the Second SOAF to ensure the graph visualisation and the summary tables accurately reflect the source dataset, prepared using the methodology described above."

That's all there is. This report takes data that has been provided and graphs the data. No wonder the ACCC doesn't have a simple, compelling case with the data. The expert they've engaged has provided information already available.

Reading through the Instructions from JWS they do not refer to anything that might be useful to their case. All they ask is for the expert to produce graphs of the data supplied. Are the instructions lacking because JWS doesn't know what to ask for, or because they've received nothing useful?

The Respondent's expert witness report

The ACCC provided a report that told you almost nothing and allowed Coles to write a report that would support their case. Of all the mistakes made by JWS, this is an unforgivable error, as by serving an expert report with no evidence, this allows Coles to respond and provide additional information that it is unlikely they would have been able to refer to without engaging an expert on the preferred point. Maybe Coles could have found a way to put this information in, but it was too easy.

The instructions to the expert ask (in paragraph 17):

"My instructions are to prepare a report that:

a. States whether I agree or disagree with the graphs at Annexure E of the Wright Report; and

b. Graphically depicts, for the 12 agreed sample products in the Proceeding (Sample Products), on each day in the period between 1 January 2021 and 4 May 2023

(Relevant Period), the following information:

i. the Price of the Sample Product on a per unit basis;

ii. the Status of the Sample Product;

iii. the Cost Price of the Sample Product on a per unit basis;

iv. the total Supplier Funding for the Sample Product on a per unit basis; and

v. the Cost

These instructions allow the Respondent's expert to put forward the case that the price increases are related to an increase in the cost price. Unfortunately, the work done by Edwards in Appendix C is redacted. Based on the references in the ACCC's submission, the ACCC accepts that prices did increase. This information further weakens the ACCC's case, as they can no longer refute that prices increased (from the supplier).

As an external expert, I don't see either report making a simple, compelling argument (noting I can't read part of the Edwards report). The Edwards report is more useful than the Wright report. Given that the ACCC brought the case against Coles, there is no justification to provide so little analytical evidence in such an important case.

What could have been done?

I don't have access to all the information, I don't have a quick and simple answer.

There are three options:

1. Do some form of (additional) statistical analysis
2. Find additional data
3. A combination of 1 & 2

As a starting point, you could treat the data as time series data, which, of course, the experts didn't. If you look at any of the graphs presented, the x-axis is time, which implies this is time-series data. Time series data is a sequence of data points, measurements, or observations recorded in chronological order, typically at successive, equally spaced time intervals. The price each day sure seems a lot like time series data…

I'd be surprised if analysing the data as time series data achieves much, but at least this would be a start. Time series is an entire domain of statistical analysis, and there might well have been something to look at.

The ACCC probably needed additional data to support their case, given that all they have is the price and duration. As I've noted, this is probably insufficient given the lack of directionality. Coles were able to present additional data to their advantage by showing that supplier prices increased, which was a justification for the price to consumers increasing.

The ACCC could well have conducted a survey to prove its point or found additional data. They could have at least tried.

In terms of statistical analysis, there are three approaches. Probability distributions, test of differences or a relationship.

A test of differences is the easiest option to understand and apply. The ACCC would have needed to present an alternative group of pricing of products that could be either be agreed upon to be misleading, or not misleading and then conduct an analysis to show how the sample is either similar or different, depending on the point they want to make.