4.5 ESG Standard 2.4 Peer-review experts - Institutional quality audit

Standard

External quality assurance should be carried out by groups of external experts that include (a) student member(s).
 
ENQA Review 2022 - Suggestion for further improvement

FINEEC is encouraged to consider advance preparation of student and external stakeholder reviewers by providing e.g. short videos on certain specific aspects (the system, the role etc) before the review case specific training. These could be available for all categories of experts.

Institutional quality audit compliance 


TABLE 1. Profile of experts in the third-cycle audits conducted in 2018-2024

TABLE 1. Profile of experts in the third-cycle audits conducted in 2018-2024
Profile of expertsTotal number of experts in third cycle audits (N=146)Total number female experts in third cycle audits (n=74)Total number male experts in third cycle audits (n=72)
National HE experts1064858
International HE experts402614
Labour market representatives36927
National student representatives271314
International student representatives1082


Independent and impartial peer reviewers conduct FINEEC’s audits. The HEI can choose either a national or an international audit team, and the roles and number of international members are determined on a case-by-case basis. The audit team typically consists of four members: two representatives from the higher education sector, one student representative, and one representative from the labour market outside higher education. For large HEIs (more than 20,000 students), a fifth member is appointed. A FINEEC project manager participates in the process as an audit expert and supports the team’s work. The Higher Education Evaluation Committee appoints the audit team and its chair based on a proposal prepared by the project manager. Prior to the appointment, the HEI can comment on the proposed team composition, particularly regarding potential conflicts of interest. All audit team members must complete FINEEC’s training, and disqualification provisions under the Finnish Administrative Procedure Act are applied to ensure impartiality (see below).

The recruitment of experts is a central part of the audit process. FINEEC identifies potential experts through annual internal planning, open calls, and recommendations from HEIs, former auditors, student unions, and European quality assurance agencies. Experts may also be directly contacted, particularly representatives of labour market. FINEEC cooperates closely with national student unions (SYL and SAMOK), the European Students’ Union, HEIs, and quality assurance agencies to identify qualified student and international experts. When composing teams, FINEEC considers the audit criteria, principles outlined in the audit manual, and the profile of the HEI. In addition, FINEEC also ensures broad institutional, geographic, and sectoral representation in the audit teams. An important principle is that experts from across the Finnish higher education system are involved. The audit team must collectively have expertise in the following areas:

  • the Finnish higher education system
  • quality systems
  • strategic management
  • planning, implementation, evaluation and development of HEIs’ core functions
  • student and labour market perspectives
  • experience in evaluations or audits.

In addition, the chair must have prior experience in evaluating HEIs and possess a broad and in-depth understanding of the higher education system. All audit team members are equal in their role as evaluators. The team selects a vice-chair from among its members. A person is disqualified from serving on the audit team if they are involved in the matter or if their impartiality is otherwise compromised. The provisions on disqualification in the Administrative Procedure Act (434/2003, Chapter 5, Sections 27–29) apply. Good administrative practice requires that disqualified individuals do not participate in any way in the processing or evaluation of the matter. Disqualification may arise, for instance, if a person is employed by the HEI under audit or has previously served in one of its decision-making bodies. Auditors must inform FINEEC of any factors that may affect their eligibility. 

The audit team must comply with the following operating principles and ethical guidelines in its work:

Fairness and impartiality: 

  • Auditors must act fairly and impartially.
  • Auditors must be aware of their position of authority and the responsibility that comes with it.

Reliability and transparency

  • The evaluation must be based on FINEEC’s published criteria and the information collected during the audit.

Confidentiality and data protection

  • All information obtained during the process, apart from what is published in the final report, is confidential.
  • Materials containing personal data are treated as confidential.

Professionalism

  • Auditors and FINEEC must act professionally and responsibly.

Respect and interactivity

  • The audit is conducted in good cooperation and interaction with the HEI.
  • All persons involved in the audit are treated respectfully.

In addition, FINEEC has recently introduced principles for the responsible use of AI in audits that apply also to peer experts. The principles are available on FINEEC’s website.

During the third audit cycle (2018–2024), a total of 36 audits of Finnish HEIs and two cross-border audits of non-Finnish universities (the University of Graz, Austria, and the University of Ljubljana, Slovenia) were conducted. Of the Finnish institutions, 22 were universities of applied sciences and 14 were universities. Seventeen audits were carried out by international audit teams. Most teams consisted of four members, while two teams had five members. Each team included representatives from the higher education sector, students, and the labour market. The Finnish experts represented various Finnish HEIs and regions. As a rule, representatives from the higher education sector came from the same sector as the HEI being audited.

In total, 146 experts (74 women and 72 men) participated in the third audit cycle. These included 106 national higher education experts (48 women and 58 men) and 40 international higher education experts (26 women and 14 men), as well as 36 labour market representatives (9 women and 27 men). Student perspectives were ensured through the participation of 27 national student representatives (13 women and 14 men) and 10 international student representatives (8 women and 2 men). International experts came from 16 countries, most commonly Sweden (9), Germany (8), Norway (4), Austria (2), and the Netherlands (2), with additional members recruited from other European countries. The relatively high number from Sweden reflects that FINEEC also conducts audits in Swedish. 

FINEEC actively promotes diversity and gender balance in audit teams, although some variation remains across expert categories. Overall, a 50/50 gender balance was achieved in audit teams composed of four peer experts. Beyond gender balance, the composition of audit teams demonstrates diversity in terms of national and international representation, stakeholder background, and experience profiles. The inclusion of experts from 16 countries has strengthened the international perspective of the audits, although the geographical distribution remains somewhat concentrated in Northern and Central Europe. Broadening the geographical scope further could enhance the diversity of perspectives. The uneven gender distribution of the labour market representatives also requires attention. Similarly, ensuring a balanced representation of student experts, both nationally and internationally, remains important for capturing a wide range of learner perspectives. Expanding the international expert pool and further strengthening balance across expert groups have been identified as improvement areas for the fourth audit cycle. 

FINEEC has continued the use of open calls for Finnish HEI experts introduced in the third cycle. Most recently, an open call was launched for students from UASs, with support from SAMOK, to recruit new student experts for upcoming audits. This approach aimed to test a new recruitment method, increase openness, and improve the effectiveness of the recruitment process from the perspectives of both FINEEC and SAMOK. A corresponding call was not issued to university students, as most universities choose international audits, which are scheduled for later in the cycle. The recruitment of labour market representatives also continues to present challenges, and FINEEC plans to test more systematic approaches to recruitment of labour market experts.

Peer experts are provided training and support by the FINEEC project managers throughout the audit process. This constitutes a key strength of FINEEC’s audit process and is also evidenced by the positive feedback provided by peer experts on project managers’ work in the third audit cycle (see Annex 1). Shared audit training materials, a systematic audit process, and the practice of experienced project managers working alongside new project managers ensure that the training of peer experts is carried out consistently across audit projects. As a follow-up action to audit feedback from some student experts, FINEEC produced concrete material Tips for auditors addressing the key issues from the auditor’s perspective. The set of tips include:

  • What do you need to know about audits?
  • FINEEC audit framework
  • Reading the audit material
  • Evidence-based and criteria-based evaluation
  • Interviews
  • Reporting
  • Key concepts for first timers
  • Students’ tips for preparing for the audit.

FINEEC staff also met with former student members of audit teams to learn from their experiences and to enquire on their views on how students could be better prepared for the audits. The tips for students were put together by the students as part of this process. In addition, FINEEC has explored, together with members of audit teams, possibilities for further strengthening advance preparation, including the potential use of short videos for the fourth audit cycle. These will be considered as part of the continued development of auditor training in the fourth cycle.

Areas improved

  • Shared training material for audit project managers. 
  • Training material Tips for auditors. 
  • Open call for UAS students (mostly national audits in the first couple of years, university audits mainly international).
  • Open call for HEIs to nominate experts.

Areas to be improved

  • Recruitment of labour market representatives continues to be challenging. FINEEC plans to test more systematic ways, such as open calls, to recruit labour market representatives. 
  • Expanding the international expert pool and further strengthening the balance across expert groups in the fourth audit cycle. Open call for international experts is planned.