In brief Subjectivity is not the problem when evaluating creative concepts. Unreasoned judgements, differing standards and subsequent changes are. A robust evaluation is based on independent individual scoring, grade anchors described in advance, a defined calculation rule and documentation that says more than “the chemistry wasn’t right”.
Score individually first, then discuss
Each jury member first evaluates on their own and records a short justification for each criterion. Only then is there a moderated discussion. Deviations are not a mistake as long as they can be traced back to the announced criterion. Simply averaging five tastes does not create a standard.
Grades with anchors
School-style grades are more honest than claiming that a number between 0 and 100 is objective in itself. For them to hold up, each grade needs an anchor described in advance:
| Grade | Degree of fulfilment | Anchor |
|---|---|---|
| 1 | 100 % | Very strong, coherently derived, distinctive and convincingly feasible fulfilment |
| 2 | 75 % | Good and sound fulfilment with minor gaps |
| 3 | 50 % | Generally usable, but with recognisable gaps in derivation, relevance or implementation |
| 4 | 25 % | Weak or only partly fitting; essential requirements remain open |
| 5 | 0 % | Essential parts of the task are missed, or no realistic implementation is recognisable |
Five coarse levels create jumps. Between a good 2 and a weak 3 lies a quarter of the possible points. Clear anchors and short justifications absorb this. A finer scale does not automatically solve the problem; it merely equips gut feeling with more decimal places.
From grade to score: a worked example
With a weighting of 70 per cent quality and 30 per cent price, a maximum of 700 quality points and 300 price points are available.
- Price: The lowest offer receives 300 points. All others receive: lowest price divided by own price, times 300.
- Quality: The quality points are also anchored to the best offer and scaled up to 700. If the best offer only reaches 80 per cent of the possible points, it would receive only 560 quality points without this normalisation, while the lowest offer receives the full 300 price points.
700 quality points then mean “best offer in the competition”, not necessarily complete fulfilment. The conversion and minimum quality requirements must therefore be fixed in advance.
Making the presentation fair
- Elements of the presentation that are relevant to the evaluation are announced in advance.
- The jury receives the presentation documents only at the respective presentation.
- Start time, technology, documents, number of participants and time for questions are the same for everyone.
- Only jury members who have seen all presentations in full evaluate the presentation.
- A 15 to 30-minute break between agencies, so that the jury can regroup and the agencies do not meet in the stairwell.
Justifying what was evaluated
For each criterion, the documentation should show which feature was evaluated, which standard applied, which strengths and weaknesses were found, how this affected the grade and how the offer compared with the others. The legal basis is in particular Sections 49, 140 and 142 BVergG 2018.
“Modern, likeable and creative” may be an honest impression. As a justification, it is not enough. A more robust version would be, for example:
“The approach derives the central idea coherently from the access problem of the prioritised target group and applies it specifically to the planned touchpoints. For the regional roll-out, however, a sound production approach and a description of the required resources are missing. Therefore a high, but not the highest, rating in the feasibility criterion.”