Peer review

Also known as: Critical friend review

In a peer review, local policies, programmes and practices are evaluated by colleagues from other cities – peers who act as ‘critical friends.’ In contrast with the study visit, visitors here have the primary aim of critiquing and improving the practices of the host.

Peers face similar issues and work in similar structures. Each peer is familiar with the relevant issues being assessed and brings their own perspective and expertise to the exercise.

Those making the evaluation are called ‘peers’ because they come from backgrounds similar to colleagues whose work is reviewed. Assessment is structured, following a common standard. In Eurocities’ EU-funded CASCADE project, this standard was called a ‘benchmark for integrated learning,’ and focused on key challenges in the implementation of local energy policies.

A strength of the process is that – as peers – participants can readily understand the goals of their counterparts in the host city, the pressures on them, and the complexity of their environment. Though such reviews often take place in the host city, they need not necessarily do so.

Method

Self-assessment report: The host city drafts a report evaluating its own practices against a predefined common standard, while also providing insights into the local context.
Peer review of the report: Other participant cities carefully scrutinise the host city’s self-assessment report.
Peer learning visit: Typically spanning three days, this visit involves participants interviewing 15-20 local stakeholders, encompassing staff from local administration, decision-makers, service providers, NGOs, etc., to gain a deeper understanding of the host city’s practices.
Feedback session: Following the visit, participants present their initial findings and suggestions to the host city.
Detailed feedback report: A comprehensive feedback report is provided to the host city approximately a month after the visit.

The peer review method is not only evaluative but also facilitative. It fosters a learning and exchange environment where participants share insights and garner new knowledge from the cities they review, enabling reflection and enhancement of their own practices.

Video of Eurocities’ EU-funded MIXCITIES peer review methodology.

Examples of application

Eurocities’ EU-funded CASCADE Project

- The standard against which the host city assessed itself was termed as a ‘benchmark for integrated learning.’
- The focus was primarily on local energy policies’ implementation challenges.

Eurocities Social Affairs Forum

- Here, peer reviews are termed ‘critical friend reviews’.
- The process involves a series of interviews with the host city during site visits.
- Each set of interviews is followed by an immediate, initial analysis by the reviewers.
- Feedback is structured using the ‘strengths, weaknesses, opportunities, and threats’ (SWOT) analysis, and Plus/Delta tools.
- Immediate feedback, emphasising strong elements and areas for improvement, is provided to the host city.

Eurocities’ EU-funded FastTrack Project:

- Unlike CASCADE and Social Affairs Forum, FastTrack’s peer review does not necessitate a visit to the host city.
- Participants submit their action plans, which are reviewed in group settings over one and a half days.
- Each city pitches their plan and subsequently receives feedback from peers.
- Here, 24 cities developed a deployment plan on a single mobility innovation. Before submitting these plans, each city underwent a peer review.
- Every city had five minutes to present their plan and query a group of cities.
- Peer review discussions usually lasted around 30 minutes, culminating in a targeted review for all cities within a span of three hours. Ideally, these discussions should be followed by written feedback.

Top tips

Thorough preparation: The host should be well-prepared, with comprehensive self-assessment reports and logistics arranged for peer visits.
Openness to feedback: Host cities should be receptive to critiques and willing to implement suggested improvements.
Active participation: Peers should actively engage in discussions, interviews, and feedback sessions to ensure comprehensive understanding and effective evaluations.
Effective communication: Clear and concise communication is vital for understanding the context and giving/receiving constructive feedback.
Structured approach: Use structured analysis tools like SWOT and Plus/Delta to organise feedback systematically.

Pros & cons

Pros:

Objective evaluation: Having peers from similar backgrounds allows for more balanced and relatable assessments.
Mutual learning: Both the host and the visiting peers can learn from each other’s experiences and practices.
Enhanced implementation: The feedback received can lead to improvements in policy and programme implementation.
Network building: Engages cities in collaborative learning, fostering relationships and mutual support.

Cons:

Time-consuming: The process can be extensive, requiring significant time investment from all parties.
Varied expectations: Differences in expectations and assessment criteria can lead to disagreements.
Logistical challenges: Organising visits and coordinating between different entities can be complex.

Outcomes and impact

Outcomes of the peer review process typically include enhanced understanding of local policies, refined implementation strategies, and identification of strengths and areas for improvement. The feedback provided serves as a catalyst for positive change and innovation, improving the effectiveness and efficiency of programmes and policies.

Impact

Pitfalls

Lack of follow-through: Without proper implementation of feedback, the benefits of the review process may not be realized.
Inadequate preparation: Poorly prepared self-assessment reports and insufficient preparation can hinder the effectiveness of the review.
Resource intensive: The process may require substantial resources, including time and effort, which can be a limitation for some cities.