Category: What data points are needed to demonstrate learning?

Overview Reflections What data points are needed to demonstrate learning?

Block 2: Summary

In the last block, we focused on teaching with data. My goal was to consider the data collection and visualisation through the perspective of an educator because my professional life is devoted to platform selling and creating dashboards for platform users. I also wanted to understand a bit more about the perspective of educators on what data is important and why.

In this block, the themes that emerged for me include:

“Some data was better than no data – sometimes” (Brown, 2020)
It’s important to know who the big players are, and dig deeper into why they may want to play in the education space (van Dijck et al, 2018)
The data points collected are often behavioral and can be used for adaptive learning, but they may not always be directly correlated to learning (van Dijck et al, 2018)
The data going in affects the outcome of the algorithms. Do educators have the skills and knowledge to determine how biased the algorithm may be and how to adapt their dashboards to it Brown, 2020)?
The trend towards more datafication and commodification of education is changing the role of the teacher and perhaps how they are evaluated (Williamson et al, 2020).

In my own visualisations, I tracked sleep, emotions, and distractions. These, I believed, were things the teacher on the other side of a Zoom screen may want to be aware of as they could impact student engagement. The data points may give insight into the students well being, but the data points themselves may infringe upon the students data privacy as highlighted by van Dijck et al (2018) because they would require a minute by minute tracking of the students.

Working in technology, I have an assumed trust of certain players and distrust of others. In reflection, I asked myself:

If I was an educator, would I need these data points to influence my lesson plan, would I see them as superfluous, or would they change the way that I teach and ‘know’ my students?

The key take away here was that while I am very conscious of how I am tracked online through the use of cookies and which apps I use, I hadn’t taken the same level of data privacy into account from an education perspective. While some data may be better than none (Brown, 2020), does knowing if a student slept well, or is anxious, radically change the lesson plan, or the way I would teach? Moreover, would I have the skill set needed to critically understand the dashboard and adapt accordingly, or would the data unknowingly limit my teaching methods (Brown, 2020)?

The behavioral data points may assist from a gamification standpoint and lead to personalised, or adaptive learning, as highlighted by the education examples by van Dijck et al (2018); however, we have to think critically about what behavioral data points actually correlate with engagement and ultimately learning. In understanding how the majority of platforms are configured, I can attest that how we track data is limited. For example, engagement is likely being tracked from a simple mouse click. You can track that a video’s play button was clicked on and at what time the video was stopped, but unless you are video taping the user’s face as they watch the video, you don’t know what happened after they hit ‘play’.

Taking this a step further, why would we want to track every action? Well, from the technology perspective, you need to find a way to keep selling software. More data equals product enhancements and new technologies, which equals more revenue and happy shareholders. The data collected can also be sold for profit. At first glance, it is easy to trust the ‘Big Five tech companies’ (van Dijck et al, 2018). The marketing places a positive spin on the data collection, product enhancements and new solutions. It’s only when the data becomes creepy (i.e. speaking about something only for it to show up as an ad on Facebook two minutes later), or there’s a breach, that most become aware they’re being tracked and have a problem with it. Do we want children to be tracked every minute from the moment they enter the school system? Personally, I would hate it. Having old pictures and memories show up on Facebook is already more than enough. The readings and visualisations this week have made me reconsider if I would want this tracked by anyone other than myself.

Lastly, we should consider how the data collected and dashboards impact the role of the teacher and how they are evaluated (Williamson et al, 2020). As highlighted by Williamson et all (2020), the data points can become “proxy measures of the performance of staff, courses, schools, and institutions as a whole”. But is education a place where we should focus on customer satisfaction? Few K-12 students would be able to distinguish their anger at a bad grade due to possibly their own lack of preparation from their customer satisfaction of how the teacher taught the material and their skill set as an educator.

Sources:

Brown, M. 2020. Seeing students at scale: how faculty in large lecture courses act upon learning analytics dashboard data. Teaching in Higher Education. 25(4), pp. 384-400

van Dijck, J., Poell, T., & de Waal, M. 2018. Chapter 6: Education, In The Platform Society, Oxford University Press

Williamson, B. Bayne, S. Shay, S. 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in Higher Education. 25(4), pp. 351-365.

What data points are needed to demonstrate learning?

Week of Questions

This week I tracked the number of questions that I was writing down on a notepad throughout the week. I categorised them into three categories – work, personal, and school related.

When selecting the type of data that I wanted to focus on this week, I wanted to be intentional about not tracking something related to time. Instead of looking at the time interval when I had a question, I wanted to see the higher level theme of where my train of thought flowed on a daily basis as well as throughout the week.

The visualisation reads from top to bottom with a few questions likely missing from Sunday as I’m writing this around 3-4PM in the afternoon. There is no mention of time, rather the symbols are representative of my question list for the day. To use Monday as an example, I started the day with writing down a personal question, then opened my laptop to begin working. As I worked, I wrote down a few work related questions. As the day passed, you’ll notice that most of the later part of the day were personal questions again.

As a first reflection, this was harder to track than originally thought. There were several moments this week when I stopped myself throughout the day to ask if I had remembered to write every question down. I am sure there are a few that I missed – questions that were in the back of my mind, but I simply forgot to write down, or multiple things were happening at once, distracting me from the data collection task.

This reflection reminded me of an article that I read back in November about how the pandemic is likely affecting memory. At the time, I felt like I was losing it because I was forgetting things left and right. In reading about memory, I stumbled on the article and bookmarked it to remember the tips given. Hammond (2020) reported that the pandemic is likely affecting our memory because our day-to-day has become so monotonous that we have fewer things to anchor our memory, less social interaction and overall lack of variety. These data points have in the past shown a correlation with worsening memory, so it’s not usual that we are questioning our memories more than usual the longer the pandemic continues.

Secondly, the amount of personal questions stood out to me as I was reflecting on the day, but they actually doesn’t come as a surprise. This week was not a usual work week in that I was participating in an online conference, which has several presentations that were focused on us as individuals, e.g. a motivational speaker. This week as result had a big focus on asking personal questions, e.g. ‘what do I want to…’, ‘can I …’, ‘should I…,’ etc.

In a previous post, I mentioned that I try my best to separate my work and personal life. This week, that separation became obsolete as the presentations I participated in had a personal focus, even though it would be classified as a work activity.

In the past three weeks, the most important reflection for me is a renewed appreciation for how ‘personal‘ the data actually is when looking at data points from the perspective of, or on an individual level.
It’s also highlighted the importance of identifying the question you want to answer prior to embarking on the journey of data collection.

As someone who doesn’t work as an educator, each data collection has provided an opportunity for self reflection and learning about my own habits and behavior. This week focusing on questions was one that dug deeper than tracking something ‘surface level’. By this, I mean, for example, the distinction between talking about the weather and asking a question to truly understand how someone is feeling. It’s made me reflect on whether or not a student should have access to the list of data points collected, as well as the questions (i.e. what learning) are being asked.

Additional Reflections:

I’d like to do more of a deep dive into the questions, but with the public nature of this post, I have chosen to keep it high level and not share my sub-categories, or the questions themselves as that would be sharing what I see to be personal data.
One way to view the ‘missing questions’ is human error… and I have to admit, I find tracking data using technology much easier than relying on myself to do it properly.
Looking back at the questions, it’s in reality more like a to-do list some days rather than a list of questions that require me to figure something out, get an answer, or reflect.

Hammond, C. (2020, November 16). Lockdown has affected your memory – here’s why. BBC. Retrieved from https://www.bbc.com/future/article/20201113-covid-19-affecting-memory

What data points are needed to demonstrate learning?

Tweetorial Week

This week, we participated in a Tweetorial as a part of the course. These weeks always come with a bit of dread because I’m not a fan of Twitter. I find the platform chaotic and rude. Rarely, have I opened Twitter and after 20 minutes of scrolling metaphorically walked away enlightened, happy, or relaxed.

I also find the notifications quite distracting throughout the day because in comparison to others like Instagram that often just tells ‘here’s some new accounts to follow’, Twitter notifications have a different sense of urgency and thought weight associated to them.

In this context, I decided to track my actual and emotional engagement with the platform this week. The question I was asking myself was even if I didn’t receive a notification, was I still thinking about Twitter.

Notifications, Reading Tweets, Engaging (i.e. liking, retweeting, or responding), and thinking about Twitter

The image above represents my week in 7 lines with Monday being the first and Sunday the last. The four horizontal lines divide the day up into 6 hour chunks, the bottom being from 00:01 – 06:00.

Blue represents a received notification
Purple represents reading content in Twitter
Turquoise represents engaging (i.e. liking, retweeting, or responding)
Yellow represents any time I found myself thinking about Twitter or the activity

Note: I concluded the activity around the time I started writing this blog post and turned off all notifications from Twitter again.

There was little activity on Monday as I read about the Tweetorial late in the evening when checking the course website. The week was quieter as well sine the activity itself had finished, but I had forgotten to turn off all Twitter notifications, so they were still popping up on my phone.

The middle of the week has the most activity as I opened the app every evening to engage with the question posed. What I found was that it took quite some time to read through the tweets from classmates to see what was happening in the conversation prior to engaging with the question myself.

The notifications themselves came in at any time dependent on other student activity and any other ‘older’ notifications that I had set up before creating a new Twitter account for the course. Enabling the notifications opened a can of worms as I had no memory of what I’d previously set up and as a consequence became very distracted as the flood of notifications was constant.

To answer my question – I was thinking about Twitter often throughout the week. This occurred as I saw notifications pop up on my phone, but also when ‘nothing’ was happening. I would think back to a particular response, or if I had anything else to add to the conversation.

In a way, the thinking became a useful way to passively engage with the course as the week progressed. This reflection in particular made me reflect further about the responses posted on what data is collected in learning platforms, and if it actually correlates to learning.

For example, if a student is thinking about a video watched and has a conversation about it outside of the platform, how would the teacher (or data collector know) if this was not recorded in the platform? They would only have access to the fact that the student watched the video, how many times they watched it, and if they answered any questions, commented, etc about. Nothing outside of the platform would be tracked, yet that doesn’t mean the student is not learning while engaging in the outside conversation.