1 Introduction and summary of recommendations
1.1 Introduction
To live is to learn. Both the infant taking in their first impressions, and the 90-year-old signing into online banking are learning, whether they want to or not. In our culture, the education system is our most formal learning arena, and it can – at its best – be one of the most important.
Primary and secondary education and training currently comprises almost 900,000 pupils and apprentices.1 In addition, we have close to 30,000 students enrolled in vocational colleges and more than 300,000 students taking higher education. In all such education programmes, the goal is for pupils and students to have the best opportunities in society. Norway also has a long tradition of placing education in high regard: Everyone has the same right to quality education, regardless of, e.g., social or ethnic background, gender or where in the country they reside. Schools, training establishments, vocational colleges, universities and university colleges are tasked with forming and educating pupils, apprentices and students so that they develop relevant competence for the present and future. As stated in the objects clause of the Education Act, education shall “open doors to the world and the future”. In order to succeed with the high ambitions for Norwegian education, we must always seek to improve the facilitation of learning. And here the question is how learning analytics may be helpful.
But what is learning analytics? Learning analytics is the analysis of learning for learning. Teachers and instructors have always been concerned with how they can assess the learning process of their pupils and students and in turn use this assessment to adjust and change their instruction. Since teachers have always been making such adjustments, it may feel a bit foreign to discuss this topic using a new term – learning analytics. The term learning analytics is a rapidly emerging phenomenon in education and society. The latter word analytics has a broader meaning than the conventional understanding of analysis. Analytics involves using digital technology to sort, analyse and interpret data to identify new knowledge and gain new insight. Thus, what is novel in learning analytics is that both data and analysis are digital. What is also new that those who collect and process the learning data of pupils and students have other items on the agenda than the teachers and instructors. How we should approach this development in education is one of the great challenges of our time.
There is no longer any question of whether schools, vocational colleges and higher educational institutions should adopt digital technology. They already do – every single day. Nor is there any question of whether we should collect digital data on pupils and students, as this is also done every single day. The big questions we need to ask are therefore not about digitalisation per se, but about what kinds of roles technology can play in learning analytics and which aspects of the tasks of teachers, instructors, school administrators and programme administrators we want technology to support, change or challenge.
We know that there is currently considerable ambiguity surrounding many aspects of digitalisation in general, and particularly with respect to pupil and student data. On the one hand, there is a clear potential in the opportunity to analyse what pupils and students are doing digitally. In a situation where the digital learning platform is a more frequent meeting place than the physical classroom, instructors are curious about how learning analytics can promote student learning. Teachers explore what information they need in order to gain a better understanding of the pupil’s learning process, and what best informs them how to develop their instruction and better adapt it to the pupils. In a digital school, parents expect that they will be able to gain better insight into their children’s academic development and thereby also support their children’s learning to a greater extent. School administrators and study programme administrators are monitoring data to see what can be collected and analysed, and what can provide a basis for better learning and progress and completion.
On the other hand, there are also unresolved questions and concerns. In course evaluations, students question whether educational institutions actually have a legal basis for allowing instructors to have access to data at the time they are signed into the learning platform. In the staff room, teachers discuss what the purchased teaching aid actually measures when the results display “42”. Pupils say they need to know when the computers are tracking what they are working on, and they want the chance to work without being monitored, as well as the opportunity for trial and error without all their mistakes being stored and made visible to others. Parents are concerned about what data commercial actors can receive about their children during the school day. Politicians and school administrators fear that the actual interest of the suppliers is not to earn revenue from licenses, but to harvest valuable and abundant data on natural persons.
In other words, learning analytics offers opportunities, but also pose considerable challenges. The desired digitalisation of schools and society is primarily what has initiated these processes, and the result to date has been the relatively free rein of market-driven technology development, thus allowing the principle of the right of the strongest to prevail. Society and education have therefore been virtually unprepared for the forms and paths that the functionalities of learning analytics have taken. The situation to date is a muddled picture of all aspects of learning analytics. In the public debate, digitalisation is often referred to as an ecosystem and thus compared to processes in nature that interact in balance. However, one may ask whether the rapid pace of digitalisation has more in common with an impact event than an ecosystem in balance.
In this context, we must emphasise that this perspective on digitalisation in education is not a romantic utopia of wanting to return to nature and get away from technology – on the contrary. In his essay Lyckad skövling i ny natur2 [Successful Destruction in New Nature], entomologist and ecologist Fredrik Sjöberg describes the diversity of insects found in an old shooting range – destroyed nature – and reflects on how biodiversity in many urban environments surpasses that found in nature. This is also how we can look back on the 40-year long digitalisation process. Our ambition is for this report to be a first step towards a balanced ecosystem for the aspect of digitalisation concerning learning analytics, where learning and data protection are the keywords for cultivating a system that benefits human beings.
In the Norwegian education system, we have many digital tools with functionalities for learning analytics, but little systematic knowledge about the extent to which learning analyses actually takes place. In other words: We do not know whether the analysis of learning actually has consequences for continued learning. It is only when this cycle leads to a change in the direction of improving learning that it can be considered learning analyses. When the cycle of learning is not completed, learning becomes lost in the shuffle. In keeping with the ecology metaphor: this report will discuss the need to establish conditions for the learning habitat in a digital age.
Using pupil and student data in an attempt to enhance learning will always be associated with risk and uncertainty. When we digitise the data used to enhance learning, this brings about new and different types of risk. New forms of artificial intelligence are entering the learning analytics cycle. New types of data will emerge, yet the quality of each functionality included in the learning resources is nevertheless determined by human judgement.
An important point for the Expert Group is that we cannot view learning analytics solely as a technological phenomenon. There is not, nor can there ever be, an obvious, static path from the collected data to fulfilling the ambition of enhanced learning. This report identifies a number of areas where new challenges arise when learning analytics are performed. In many ways, the discussions in the report emphasise that the human aspects of using technology are always the most decisive. At the same time, there is a considerable risk associated with failing to consider the opportunities that learning analytics offer in terms of promoting learning for current and future pupils and students. The Expert Group has therefore placed considerable focus on the possibilities of learning analytics in its investigation work.
Is there a need for a new concept – learning analytics – in Norwegian education? The Expert Group has asked itself this question in its work on the report. The question is timely as many concepts have been introduced since the turn of the millennium with great fanfare by various actors seeking to change the way education is implemented, but which have quickly been disposed of at the education sector’s waste depot. Time and further academic debate will eventually show what role the concept of learning analytics can play.
However, an important finding for the Expert Group is that few people in the education sector have a clear understanding of what learning analytics entails. It is not just about the concept itself being unclear, but also about understanding how the collection, analysis and representation of data is part of – or can be included – in promoting learning. If so, this is an eye-opener, given that it is such perspectives that the education sector should be best equipped to address. An uncomfortable, inaccessible concept like learning analytics is perhaps what we need to shed light on how complex and demanding the process towards supporting learning is.
1.2 Summary of the Expert Group’s recommendations
The Expert Group has given weight to developing recommendations and proposals that contribute to a learning analytics that is secure and sound, with a clear pedagogical purpose. One of the main objectives of the recommendations is to take clear steps to strengthen trust in the safeguarding of privacy throughout the educational pathway. This would reduce the risk involved in using new technology in education. At the same time, there is also a risk of missing out on new opportunities by not addressing the question of how technology can be used to enhance learning. The Expert Group will therefore facilitate the exploration and development of good pedagogical practices where learning analytics is included, within secure frameworks. Such practices must be based on discussions among education professionals about the pedagogical purpose of learning analytics should be, and how learning analytics can affect learning processes, teaching situations and roles in education. The Expert Group’s recommendations aim to set the direction for such discussions and how they can contribute to further development of practice.
The Expert Group presents four main recommendations to support good and justifiable learning analytics. The recommendations address different levels of the education sector but should nevertheless be viewed in context. This is first and foremost because navigating an educational pathway should entail a certain degree of coherence and predictability, but also because the recommendations partly interact and complement one another.
The Expert Group recommends clarifying the legal basis for learning analytics in primary and secondary education and training, higher education and tertiary vocational education. The purpose of this recommendation is to clarify when the processing of personal data in learning analytics is lawful and to ensure better predictability.
The Expert Group recommends developing a data protection code of conduct in primary and secondary education and training. The purpose of this recommendation is to strengthen pupils’ and students’ data protection and facilitate good data protection practices, increased awareness and enhanced competence regarding data protection.
The Expert Group recommends establishing frameworks for good learning analytics in primary and secondary education and training. The purpose of this recommendation is to strengthen the free choice of pupils and teachers and to provide a better basis for pedagogical decisions regarding learning analytics to enhance learning.
The Expert Group recommends developing broad guidelines for good and justifiable learning analytics in higher education and tertiary vocational education. The purpose of this recommendation is to facilitate good data protection practices and justifiable learning analytics that promote student learning and increase the quality of education.
1.2.1 Legal basis for the processing of personal data in learning analytics
Primary and secondary education and training
The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in primary and secondary education and training. The proposal is based on the general scheme in the proposal for a new Education Act (Prop. 57 L (2022–2023)). The provision will be added to the Education Act and in the corresponding provision of the Independent Schools Act:
The Expert Group proposes including a new paragraph in section 25-1 of the Education Act on the processing of personal data in learning analytics and the tasks in the Act where such processing will be necessary. Proposed new paragraph:
“Municipalities, county authorities and training establishments may process personal data about pupils and apprentices by means of machine analysis and alignment where this is ethically and pedagogically sound and necessary to perform tasks and duties in the Act and regulations pursuant to the Act. Examples of such tasks and duties may be to adapt the instruction, the work on quality development in section 17-12 and formative assessment in section 3-10 of the Regulations pursuant to the Education Act. The degree of personal identification shall not be greater than necessary for the purpose in question.”
Higher education
The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in higher education. The provisions shall be inserted in the Universities and University Colleges Act and the Regulations pursuant to the Act:
The Expert Group proposes inserting a new paragraph in section 4-15 of the Universities and University Colleges Act on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:
“The educational institution may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties include quality assurance work and the responsibility to ensure that instruction is provided in accordance with recognised ethical and pedagogical principles, cf. section 1-5.”
The Expert Group proposes specifying the provisions on quality assurance work in section 4-1 of the Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:
“The institutions may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”
Tertiary vocational education
The Expert Group recommends clarifying the legal basis for processing personal data in learning analytics in tertiary vocational education. The provisions shall be inserted in the Vocational Education Act and the Regulations pursuant to the Act:
The Expert Group proposes inserting a new paragraph in section 4 of the Vocational Education Regulations on the processing of personal data in learning analytics and for which tasks such processing may be necessary. Proposed new paragraph:
“The vocational colleges may process personal data about students by means of machine analysis and alignment where this is ethically and pedagogically justifiable and necessary to fulfil tasks and obligations pursuant to the Act. Examples of such tasks and duties may be quality assurance work and having learning and instruction methods that are suitable for the students to achieve the learning outcomes, cf. section 2-1 of the Vocational Education Academic Supervision Regulations.”
The Expert Group proposes specifying the provisions on quality assurance work in section 4-1, third paragraph of the Vocational Education Academic Supervision Regulations so that these provisions explicitly apply to the processing of personal data in learning analytics. Proposed new paragraph:
“The vocational colleges may process personal data by means of machine analysis and alignment where necessary for its systematic quality assurance work. The degree of personal identification shall not be greater than necessary for the purpose in question.”
1.2.2 Data protection code of conduct in primary and secondary education and training (School Code of Conduct)
The Expert Group recommends that, in cooperation with the sector, a code of conduct should be drawn up to safeguard data protection in schools. At a minimum, the School Code of Conduct should include the following:
the development and administration of specific data protection requirements in resources that have functionality for learning analytics
the preparation and administration of guidance materials for school owners, school administrators, teachers, pupils, parents, developers and suppliers
the preparation and administration of national data protection impact assessments for resources that have functionality for learning analytics
the facilitation of competence development on and exchange of experiences from data protection work in schools
The Expert Group recommends that, as part of the School Code of Conduct, concrete, verifiable data protection requirements should be drawn up for resources that have functionality for learning analytics. The requirements in the School Code of Conduct must be identical for both licensed and free resources. At a minimum, the requirements should be aimed at reducing the risks associated with the following four data protection principles:
fairness
transparency
data minimisation
accuracy
The Expert Group recommends that a national actor, as part of the School Code of Conduct, prepare and administer overall risk analyses, data protection impact assessments (DPIAs) and data processor agreements for resources that have functionality for learning analytics. The Expert Group emphasises that the responsibility for processing lies with the school owners. As the data protection situation in schools is precarious, we recommend as a first step to make arrangements for school owners to share their analyses and assessments with one another.
The Expert Group recommends that, as part of the School Code of Conduct, arrangements be made for developing competence on and exchanging experiences related to data protection efforts. It would be advantageous if an already existing relevant network can carry out this task.
The Expert Group recommends that the administration model for the School Code of Conduct include a steering group with representatives from key actors and user groups.
The Expert Group recommends that the School Code of Conduct be based on relevant measures and guidelines that are already firmly rooted in the school sector, but it emphasises that the code of conduct must take a comprehensive approach to data protection in schools.
The Expert Group recommends linking the School Code of Conduct with a national service catalogue for digital learning resources. This link must be in line with the procurement legislation.
The Expert Group recommends that the continued work on the School Code of Conduct:
be developed with a realistic level of ambition and include thorough investigations and evaluations along the way
be aligned with existing learning technology standardisation efforts and privacy by design
includes all processing of personal data in schools, including processing that does not have learning analytics as a purpose
involves pupils and parents, where relevant
1.2.3 Frameworks for good learning analytics in primary and secondary education and training
The Expert Group recommends that national authorities facilitate usage-based pricing models for digital teaching aids, and that a study be initiated on how trials involving usage-based pricing models can be scaled up.
The Expert Group recommends that the national service catalogue for digital learning resources supports good learning analytics in schools.
The Expert Group recommends that centrally defined quality criteria be developed for resources that have functionality for learning analytics. It is teachers, school administrators, school owners and developers who will be using these quality criteria. The criteria can be based on existing guidelines for quality assessment of teaching aids.
The Expert Group recommends that suppliers and developers cooperate on the use and further development of the quality criteria so that they provide guidance for product development.
The Expert Group recommends that suppliers be required to make available user-oriented information that justifies and explains how the resources work. Suppliers must also be able to document that the technical specifications in the resources correspond to the user-oriented information.
The Expert Group recommends a grant scheme for purchasing and developing digital teaching aids that have functionality for learning analytics. The grant scheme should stimulate innovative learning analytics functionality and artificial intelligence (AI), and must set requirements for data protection and responsible use of AI. Resources must also be required to comply with centrally defined quality criteria.
The Expert Group recommends that funding be announced for innovation, research and development pertaining to digital learning resources that have functionality for learning analytics and adaptivity, as well as funding for research on the use of such resources in authentic learning situations.
The Expert Group recommends measures aimed at student teachers, teachers, school administrators and school owners, so that they can develop competence in learning analytics. Competence in learning analytics and knowledge of artificial intelligence should be included in both basic education and supplementary and continuing education programmes.
The Expert Group recommends that school owners ensure that pupils receive adapted and comprehensible information, so that they can consider issues relating to learning analytics. Furthermore, it is recommended that school owners regularly evaluate whether pupils feel that the school is safeguarding their right to participation.
1.2.4 Guidelines for good and justifiable learning analytics in higher education and tertiary vocational education
The Expert Group recommends the development of broad national guidelines for good and justifiable learning analytics in cooperation with the relevant sectors. The national guidelines must be adaptable to local conditions. At a minimum, the guidelines should include the following action points:
data protection
participation
openness
free choice
procurements
The Expert Group recommends that a government agency develop and administer the broad guidelines for good and justifiable learning analytics in close cooperation with sectoral actors such as Universities Norway and the National Council for Tertiary Vocational Education. The Expert Group emphasises that the responsibility for good and justifiable learning analytics lies with the institutions.
The Expert Group recommends that the broad guidelines be revised regularly in light of rapid technological developments and at least every five years.
The Expert Group recommends that the guidelines include common solutions, local resources and resources that are freely available online.
The Expert Group recommends that a government agency develop a support system to aid educational institutions in preparing risk analyses (DPIAs) and data processor agreements. The government agency shall also assist educational institutions in connection with procurement processes and system development projects.
The Expert Group recommends that the guidelines explain what constitutes good learning analytics that promote student learning.
The Expert Group recommends that competence in learning analytics be included in training programmes for basic pedagogical competence in higher education and tertiary vocational education. In addition, the Expert Group recommends that learning analytics be included in various courses aimed at instructors, administrators and support staff who assist instructors and who participate in quality assurance work.
The Expert Group recommends that teacher training ensures that newly qualified teachers have the requisite competence in learning analytics and knowledge of artificial intelligence. The institutions must consider how they can ensure such competence in instruction and in learning outcome descriptions.
The Expert Group recommends that funding be announced for innovation, research and development pertaining to digital learning resources that have functionality for learning analytics and adaptivity, as well as funding for research on the use of such resources in authentic learning situations.
The Expert Group recommends that the institutions ensure that students receive adapted and comprehensible information so that they can consider issues relating to learning analytics. Furthermore, it is recommended that the institutions regularly evaluate whether students feel that the school is safeguarding their right to participation.