I am a firm believer in using data and analytics to get the most out of teaching. There are two different data sets that I’m interested in.

1. The data we can get from teaching systems we use, systems like Moodle, Panopto, Padlet etc. We are asking staff to put more and more of their teaching online especially during the pandemic, getting meaningful data out of these systems is vital to make sure that students are accessing and consuming the content. Staff are spending time and effort creating resources, and it’s really useful to see if these are being accessed, and in the case of videos how long they are watching them. Finding out that students are not watching all of a video may initially be a bit demoralising, it’s important to ask why they are switching off and changing the resource to make sure that they are watching in its entirety.
2. The other important data comes through from student surveys, We need to make sure that the information that comes from student surveys is properly looked at and addressed, it really helps getting the feedback from surveys as minor changes to the student experience can have major effects, positively or negatively.

When using data there are a number of things that you must be mindful of, In the past, I have written reports in SQL and you need to be very definite with the questioning, how you process the report can give very different results and it’s important that you are looking for meaningful data.

Also when talking about survey questions, it’s important that the questions are clear and do not lead to confusion and students not being able to answer the questions properly, the length of some of these surveys can be an issue, if the students become a bit bored, they can just start not reading the questions and putting any answer down, just to finish the survey. We have found that adding a predicted time for the completion of the survey helps the participants to know how long it’s going to take.

The questions need to be accessible and meaningful to all participants. We must also be careful that we are not adding any bias to the questions, along with making sure that the responses don’t show any bias. In the research done by Dirk Templaar et al (2020) they looked at two types of bias in self-reporting surveys: response styles (i.e., a tendency to use the rating scale in a certain systematic way that is unrelated to the content of the items) and overconfidence (i.e., the differences in predicted performance based on surveys’ responses and a prior knowledge test). They found that response style bias accounted for a modest to substantial bias, whereas the overconfidence bias was limited. They found that the bias present in surveys adds predictive power in the explanation of performance and other questionnaire data. I see that as we see what we want to see!

To bring this all together, Data is a really powerful tool that helps us to improve what we are doing, it can help us gauge the student’s journey and their satisfaction with the course, along with ways to improve the student’s experience. However, we need to be really careful to make sure that the data we use is accurate and correct and that we aren’t just seeing what we want to see.

References

Tempelaar, D., Rienties, B. and Nguyen, Q. (2020). Subjective data, objective data and the role of bias in predictive modeling: Lessons from a dispositional learning analytics application. PLOS ONE, [online] 15(6), p.e0233977. Available at: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0233977.

Wallace, T.L., Kelcey, B. and Ruzek, E. (2016). What Can Student Perception Surveys Tell Us About Teaching? Empirically Testing the Underlying Structure of the Tripod Student Perception Survey. American Educational Research Journal, 53(6), pp.1834–1868.