Twelve invisible sub-criteria under ‘methodology’: the Council of State annuls the award of the Walloon Tonus audit assignments in Charleroi and Frameries to Ernst & Young and Deloitte
The Walloon Region assessed the bids for the audit follow-up of the management plans of Charleroi and Frameries on the ‘methodology’ criterion (40 points) using twelve sub-criteria — including ‘specific knowledge of the municipality’ and ‘participation in budget work (CRAC and DGPL)’ — that appeared nowhere in the specifications or the contract notice, and the Council of State therefore annulled, with reference to the Court of Justice’s Lianakis judgment, the award to Ernst & Young and Deloitte on TCLM’s action.
What happened?
The Walloon Region put out a services contract for the audit follow-up of municipal management plans under the Plan Tonus Axe 2. The contract covered 52 municipalities, was estimated at 1,000,000 euros for an estimated duration of one year and was divided into lots. The Official Journal of the European Union published the notice on 6 October 2007. Article 115 of the specifications listed four award criteria in decreasing order of importance: the proposed methodology (40 points), price (30 points), the quality and precision of the proposed report template (20 points) and the proposed additional assignments (10 points). The specifications said nothing more about the methodology. After the bids were opened on 19 November 2007, it turned out that the audit firm TCLM had bid for, among others, lot 9 (Charleroi), where Ernst & Young also bid, and lot 22 (Frameries), where Deloitte was the competitor. On 4 December 2007 the Walloon Government’s Minister for Internal Affairs and the Civil Service awarded lot 9 to Ernst & Young for 25,650 euros excluding VAT (31,036.50 euros including VAT) and lot 22 to Deloitte for 10,560 euros excluding VAT (12,777.60 euros including VAT). In Charleroi the minister found Ernst & Young’s methodology more interesting because it proposed additional analyses, both for the municipality and for the consolidated entities. In Frameries he considered both bidders’ methodology and report template equivalent, preferred TCLM on additional assignments, but Deloitte won on price. TCLM was notified of the decisions on 16 January 2008 and brought an action for annulment on 13 March 2008. Deloitte and Ernst & Young intervened. The decisive element was point 2.3 of the bid evaluation report. It stated that, for lots with several bids, the methodology had been assessed using three headings — municipality, consolidated entities and methodology — each with four parameters, such as the 2002 methodology note, proposals for additional analyses, specific knowledge of the municipality or the entities, analysis of accounting documents, interviews with local decision-makers and participation in the budget work of the CRAC and the DGPL. The Council counted twelve sub-criteria in it. The Region argued that these were not sub-criteria but objective elements of the bids underpinning the assessment of a known criterion; an authority knows the subject matter less well than bidders and cannot lay everything down in advance. Ernst & Young added that the parameters had not been weighted but placed on an equal footing, and that TCLM had not shown what harm it suffered. The Council disagreed. In an open procedure, all the elements the authority takes into account to identify the most economically advantageous tender, and their relative importance, must be known when bidders prepare their tenders (ATI EAC, C-331/04; Lianakis, C-532/06). According to the Court of Justice, the principles of equal treatment and transparency preclude the authority from setting weighting coefficients and sub-criteria afterwards. It did not necessarily follow from the specifications that the ‘methodology’, stated as a criterion without further detail, had to be assessed on elements such as ‘specific knowledge of the municipality’, ‘specific knowledge of the entities’ or ‘participation in budget work (CRAC and DGPL)’. That such sub-criteria and their weighting could have influenced the preparation of the bids was, according to the Council, not open to dispute. The Region had awarded or withheld points depending on whether bids met those twelve sub-criteria, as the table in annex 4 of the report showed, thereby breaching the publicity obligation of article 16 of the Law of 24 December 1993. The plea was well founded in its third limb. The Council annulled both award decisions. The costs of 425 euros were shared: 175 euros for the Walloon Region and 125 euros for each intervening party.
Why does this matter?
This is an early and clean Belgian application of Lianakis. The judgment shows where the line lies between ‘explaining’ a broad award criterion and building in one’s own assessment grid after the fact. A criterion such as ‘methodology’ without any further description gives the authority wide latitude, but that latitude may not be filled with a structured list of twelve assessment points that appears only in the evaluation report and drives the allocation of points. In particular, the authority’s argument — ‘these are not sub-criteria but elements of the bids themselves’ — failed, as did the argument that the parameters were not separately weighted. The Council looked at what actually happened: points were awarded or withheld depending on whether bids met the parameters. The nature of some sub-criteria is also striking. ‘Specific knowledge of the municipality’ comes close to assessing the bidder rather than the bid, a tension with the distinction between selection and award that TCLM expressly raised. The Council did not decide the case on that point, but anyone wishing to take such elements into account must in any event disclose them in advance. Finally, the judgment is a reminder that intervening beneficiaries bear part of the costs when the award falls.
The lesson
For contracting authorities: if you use a qualitative criterion such as ‘methodology’, decide before publication which elements you will assess it on, and put those elements — and where relevant their weighting — in the specifications. An evaluation grid drawn up only after the bids are opened and on which the scores rest is an invitation to annulment. If you want to remain flexible, describe the criterion so that every bidder knows what you are looking at. Be particularly careful with elements such as local knowledge or experience, which belong rather to selection. For bidders: when challenging an award, request the full evaluation report, with annexes and scoring tables. That is often where the assessment grid that never appeared in the specifications is found. Compare each parameter with the text of the specifications and the notice. And if an award criterion is so vague that you do not know what you will be assessed on, ask a question during the tender period: the answer becomes binding for everyone.
Ask yourself
Do your specifications state, for each qualitative award criterion, what you will concretely assess it on, or does the evaluation committee have to work that out only after opening? Does your evaluation report award points on the basis of parameters found nowhere in the specifications or the notice? Would bidders have drafted their bids differently had they known your assessment grid in advance? Do you use elements at the award stage, such as ‘knowledge of the local situation’, that actually assess the bidder rather than the bid? And as a bidder: have you requested the full evaluation report with annexes and set the parameters used against the specifications?
About this database
The Council of State (Raad van State / Conseil d'État) is Belgium's supreme administrative court. In disputes over public procurement — from contract awards to tenderer exclusions — the Council of State is the final arbiter. The rulings in this database are summarised by TenderWolf in plain language, with practical lessons for tenderers and contracting authorities. View all rulings →