4.6 ESG Standard 2.5 Criteria for outcomes - Institutional quality audit
Standard
Any outcomes or judgements made as the result of external quality assurance should be based on explicit and published criteria that are applied consistently, irrespective of whether the process leads to a formal decision.
Institutional quality audit compliance
TABLE 2. Audit decisions 2018-2024 (third audit cycle)
| HEIs | Pass | Fail |
| Finnish universities | 14 | 0 |
| Finnish UASs | 22 | 0 |
| Non-Finnish universities | 2 | 0 |
The audit process and report are guided by the audit criteria. The assessment and judgments are based on criteria that have been published in the audit manual. Audit teams receive training on the criteria and are supported throughout the audit process by a FINEEC project managers. FINEEC project managers’ role is to participate in audit teams’ discussions and offer guidance in matters pertaining to the audit criteria. Project managers also have primary responsibility for editing the audit report, including ensuring that all audit criteria are adequately addressed, findings presented in a transparent manner and that judgements are firmly based on the evidence collected during the audit. In addition, the audit process includes an internal peer-reading in which an expert from the unit who has not been involved in the audit project reads and comments on the report. Furthermore, all audit reports are reviewed by FINEEC’s Director and Head of Unit, providing an additional level of quality assurance.
Fourth-cycle audits include two evaluation areas that are both assessed on a four-point assessment scale of Excellent, Good, Satisfactory or Insufficient. Passing the audit requires HEIs to attain at least a Satisfactory rating in both evaluation areas.
Based on the unit staff’s experience and discussions with stakeholders, the fourth-cycle planning team decided to expand the assessment scale by introducing a new Satisfactory rating. The previously used three-point scale consisted of Excellent, Good, and Insufficient. In practice, the Good rating proved too broad, encompassing HEIs ranging from those approaching Excellent to a few cases bordering on Insufficient. In addition, the gap between the Good and Insufficient levels was deemed too wide, with the description of Insufficient being particularly critical. A four-point assessment scale was also used in the first and second audit cycles.
The introduction of a Satisfactory rating allows for more nuanced differentiation in the assessment of HEI performance and includes also more follow up. Furthermore, the Insufficient rating has been revised to encompass not only the quality system but also the actual quality of activities. This change ensures that HEIs with serious deficiencies in the quality of their activities can receive an Insufficient rating even if their quality management processes function at a satisfactory level.
Chapter four of the audit manual outlines criteria that must be met for an HEI to receive an assessment Good in either evaluation area.
To achieve a rating of Excellent, the HEI meets all the criteria for the level of Good and, in addition, has carried out strategic, long-term and effective development in the evaluation area. The development of the HEI’s operations benefits both the institution itself and its internal and external stakeholders. This is underpinned by an innovative and widely inclusive organisational culture in which diverse forms of information are systematically used in the management and development of operations. The HEI serves as an example to other HEIs at the national or international level.
A Satisfactory rating means the HEI’s operations partially meet the criteria for the level of good. However, in some sub-areas there are clear development needs requiring action to reach the level of good. In the case of an Insufficient rating, there are major shortcomings in the quality of the HEI’s operations or its practices for ensuring and developing the quality of its operations are inadequate.
HEIs that receive a Satisfactory rating in one or both evaluation areas are required report to the Higher Education Evaluation Committee on development measures undertaken one year after the audit decision. The reporting concerns are specified by the committee in conjunction with the audit decision. If the HEI fails to submit its report, the Higher Education Evaluation Committee has the option to re-examine the outcome of the audit.
The audit decision-making process is structured to ensure consistent and transparent outcomes. First, audit teams evaluate whether HEIs provide sufficient evidence to pass the audit and, after reaching a consensus, present their conclusions in an audit report. The report is edited by FINEEC project manager to confirm that the audit’s conclusions have been reached based on the provided evidence. The Head of Unit is the presenting official, who makes a proposal to the Higher Education Evaluation Committee. In making the proposal, the presenting official may differ from the assessment of the audit team. The Higher Education Evaluation Committee is responsible for the final decision on whether HEIs pass or fail. The chair or vice-chair of the audit team is also present at the Evaluation Committee meeting to answer any questions committee members may have. It is the Evaluation Committee’s responsibility to ensure the audit has been conducted impartially and that the audit criteria are applied consistently. In making its decision, the Evaluation Committee may decide against the audit team and the presenting official. The Evaluation Committee follows the provisions of the Administrative Procedure Act to ensure that participating members do not have any conflicts of interest.
There are two possible audit decisions: pass (Satisfactory, Good, Excellent) and fail (Insufficient).
- If the HEI passes the audit, it receives a quality label and is included in the register of audited HEIs maintained on FINEEC’s website. The quality label is valid for six years from the decision of the Higher Education Evaluation Committee. If the audit team assesses evaluation area I or II as satisfactory, the HEI must report to the Higher Education Evaluation Committee on the development measures undertaken one year after the audit decision. The Higher Education Evaluation Committee will specify in its decision which areas the reporting concerns. If the HEI fails to submit its report, the Higher Education Evaluation Committee has the option to re-examine the outcome of the audit.
- If the HEI does not pass the audit, a re-audit is required within 2–3 years of the decision. HEIs commit to the re-audit process in the audit agreement with FINEEC. In its decision, the Higher Education Evaluation Committee specifies the sub-areas of the evaluation areas (I–II) to be re-evaluated. If the HEI passes the re-audit, it will receive a FINEEC Quality Label valid for six years from the date of the positive re-audit decision.
HEIs that pass their audits have their results entered into the audit register on FINEEC’s website, receive an audit certificate as well as a FINEEC Quality Label, which is valid for six years. In cases where sufficient evidence is not provided and the HEI does not meet the criteria, a re-audit is required.
Since the completion of the third audit cycle, steps have been taken to strengthen the use of audit criteria in identifying excellence in HEIs. During the third cycle, a Quality Label for Excellence was introduced to recognise HEIs that had distinguished themselves through exceptionally high-quality enhancement work. The label was awarded based on a separate set of criteria and an assessment carried out by a four-member excellence panel appointed from the Higher Education Evaluation Committee. In the planning of the fourth audit cycle, a decision was made to integrate the recognition of excellence directly into the audit criteria and process. As a result, the Quality Label for Excellence was replaced by an Excellence Award, which HEIs receive automatically when they achieve an Excellent rating in one or both evaluation areas.
Areas improved
- The assessment scale has been expanded for the fourth-cycle audits. The level Good was overly broad in the third cycle. A new level Satisfactory has been added to the scale. The description for the level Excellent has been expanded. In addition, the level Insufficient now includes both major shortcomings in actual quality and quality management while it previously only included quality management.
- The Quality Label for Excellence has been replaced with the Excellence Award and the process for awarding excellence has been integrated in the audit process.