Academics at Charles Sturt University (CSU) have questioned whether the data collection systems used by schools and universities to improve education outcomes are actually beneficial.

Lecturer in the CSU School of Teacher Education in Bathurst Associate Professor Alan Bain and Dr Nick Drengenberg, Academic Secretary in the CSU Office of Governance and Corporate Affairs, combined years of empirical research with examination of numerous studies conducted over more than two decades. They say the value of the use of 'learning analytics' is dubious, misleading, and has not demonstrably improved teaching or learning.

"While the objective of using data to understand and better predict what will lead to improved teaching methods and learning outcomes is worthy, there's no evidence to demonstrate that learning analytics contributes to any meaningful and consistently positive outcomes," Professor Bain said.

"This is despite vast amounts being spent to capture and analyse teacher and student behaviours."

Professor Bain explained that analytics systems in other areas traditionally monitor behaviours of particular groups, such as shoppers, or stock brokers, or teachers and students, in order to predict their likely behaviour. The systems gather large amounts of data from a range of sources such as from computer logons, downloads, and social media usage, and in education largely via learning management software.

"The problem is that this captured data is has little to do with actual teaching or learning behaviour," Professor Bain said.

The academics' arguments are published their new book, Transforming the Measurement of Learning and Teaching in Higher Education (Routledge, 2016).

"Our findings about the big ideas in the book are not from a single study," Professor Bain said. "They are the product of a portfolio of studies over 20 years, some at Charles Sturt University (including by doctoral students) and others in the US in secondary and higher education."

The studies are described in another book co-authored with CSU colleague Dr Lucie Zundans-Fraser, Rising to the Challenge of Transforming Higher Education – Designing Universities for Learning and Teaching (Springer, 2016).

"The findings both informed and were informed by the Smart Learning project including the development of the Smart Tools software which generates analytics data focused on what learners and teachers do when they participate in learning experiences," Professor Bain said.

Professor Bain led the project for two years at CSU and Dr Drengenberg was involved as a member of the leadership team. Their findings have significant implications for the quality and productivity of what universities and academics do.

Dr Drengenberg said, "Our motivation for writing this book was observing the constant contest, within and outside education, over the measurement of its key work. Why is it almost impossible to reach agreement over the quality of educational work, and the measures of that quality? Why do the measures that are used so often originate from outside education, rather than inside, and from the management of educational work rather than from the practice of it? We have used learning analytics, as the latest attempt to measure learning and teaching, to explore these questions. I think readers will see many of these debates in a very different light, using the ideas we have developed in the book.

"We make a number of recommendations in the book for the tertiary education sector, and we have contributed to the University's analytics framework. Some of the ideas discussed in the book have been used as part of the Smart Learning initiative, and as part of its work in learning analytics."