I used the second Sample Data file that provides data about a group of students that are doing a blended English course that consists of VLE activities + online lessons that are equally important (corporate training).
Questions I had to answer:
1) What data will be useful for an English teacher and could help them plan necessary interventions or inform their teaching practices?
*attendance and names of the students (attending classes is essential in such courses as it is the time for practicing the new language) – chart 1
*VLE data and names of the students (I included only the total number of posts on the platform (posts +replies) as it suggests students’ ‘production’ activities as opposed to time spent on the platform) – chart 2
*tests results and names of the students (no benchmarks for the teacher provided, leaving the results open to the teacher’s interpretation) – chart 3
*student’s performance = average test results ((test1+test2+test3+test4)/4) – chart 4
*based on the indicators of all the students, high/low/average performance/attendance/VLE engagement were defined. Chart4 can serve as ‘data-informed portraits of individual students’, as Brown put it (p. 396).
2) What data be can left out?
*VLE logs, forum views, pdf views- these data say very little about the engagement with the course. It’s more about learning habits or the students’ context.
Benefits for teachers:
1) Quick visual summary of test results and attendance, it saves the teacher’s time
2) Such data enable teachers to see students that need more support
3) Students’ profiles (chart 4) can be useful when teachers have many students as a quick summary of their activity and their performance on the course
4) Such charts may provide more visibility to other stakeholders, like parents or the students’ managers, which is often expected from teachers
5) The numbers presented in the well-known forms will be better trusted than the teachers’ notes or feedback in free form
Constraints:
1) These data can only serve as indicators of the situation that still need to be linked to the context to make sense
2) The issues of privacy and surveillance: who does these data belong to? Are the teachers aware of what data are collected and how and who might use them?
3) Defining what is low/average/high performance and engagement based on the indicators of a particular group is contentious. It reminds me of the assessment results adjustment in the UK when the measurements of the previous cohort impacted the future of the next generation.
4) Chart 4 looks very primitive. It shows that the person achieved top results in attendance and performance, but in reality they didn’t. It’s either because ‘the designer of the dashboard’ has limited tech expertise or the tools I could use had their constraints.
Conclusion:
One of the conclusions drawn by Brown in his research (2020) was that ‘some data was better than no data’ (p.392). I believe that this is also true in relation to the usefulness of my DIY dashboards. Overall, they have little potential to revolutionize one’s pedagogical strategies. Still, teachers might find them handy for lesson planning, drawing some actionable insights and, maybe, reflecting on one’s own efficiency. Having students’ profiles at hand can also be of help when reporting on students’ achievements or preparing for individual consultations with learners.
As many researchers argue, ‘instructors appear responsive to data about teaching when they can identify useful connections to their daily work and when the data is framed as legitimate by their professional or disciplinary beliefs’(from Brown, p.385). So before introducing any kind of dashboards, it is essential to ensure that the educators understand how the algorithms work, and how these data can inform their day-to-day practices.
“it is essential to ensure that the educators understand how the algorithms work” — this is why some argue that teachers should be trained in “data literacy”. Do you think it would be enough to “train” educators to understand how algorithms work in order to better inform their practice, or, as Raffaghelli and Stewart (2020) argue, do they need some more “critical data literacy” about the powerful role of data and algorithms in reshaping practices, and how, if necessary, to resist them?
You are right, in today’s datafied society developing ‘digital literacy’ in the instrumental sense of this notion is not enough. ‘What is required is an extended critical big data literacy that includes citizens’ awareness, understanding and critical reflection of big data practices and their risks and implications, as well as the ability to implement this knowledge for a more empowered internet usage’ (Sander, p.14).
It is particularly relevant to teachers whose professional activities are increasingly datafied and assessed by algorithms. Besides, educators often serve as role models to their students, which could help them (learners) develop some helpful critical perspectives too.