1) Does this activity enhance learning?
2) Besides the number of comments or words, what else can be quantified in a written message?
3) Tracking comments as measuring learning, is it a good idea? Can technology perform this well?
From the perspective of contemporary learning theories, like constructivism and connectivism, writing feedback comments can be part-and-parcel of learning, since it suggests participation and connecting to ‘more knowledgeable others’, interpretation and meaning-making in the process of reflection, and, in the ideal world, brings to creating a community of learners. I also find contributing to my peers’ blogs beneficial for my learning, since it enables me to extend my understanding of the subject, revise some ideas as well as gain new insights and inspiration.
In digital educational formats, like MOOCs, automated tracking of feedback comments is one of few ways to measure students’ engagement with the learning content, which has its obvious limitations in terms of evaluating quality and relevance of messages. In my self-reporting, I tried to address this through depicting meaning and implying that 100% of my comments are relevant. Is the technology able to do this today? Maybe not at this stage. However, if it is, will these data enhance learning? Perhaps, the comments analysis is not that helpful for students, but can be insightful for teachers in the form of signals that need investigation, like abuse, sudden disappearance of all comments, ignoring particular learners, too many identical comments (maybe a bot) etc.
In conclusion, tracking comments may be more informative than tracking the time spent on the platform in terms of measuring learners’ engagement, but it is still a ‘trade-off between the reliability of the data that can be collected and the richness of what can be measured’ (Eynon, p.408). and should never be used as the only criterion for assessment or feedback.
Hi Iryna,
Love your visualisation and the choice of factor to record. It is true, we do focus on feedback comments, and as you say, they may not tell us as much as we would like (lots of comments doesn’t means lots of engagement in the way we mean) but they may say something useful, we just have to be a little bit more realistic.
Thank you, Tracey! It would be curious to see data showing if there’s any correlation between # of comments and final grades. I’m sure there are interdependencies, not absolute, of course.
This is a really inventive dataviz and a good set of comments. I think you’re right to speculate that analysis of written posts in learning environments will advance in coming years. There’s a lot of excitement about natural language processing in the learning analytics field, and related forms of semantic textual analysis to excavate meaning and comprehension from typed text. Often this goes along with network analysis, and attempts to analyse knowledge construction among connected groups of students. So the other but related intriguing thing you’ve identified is the role of specific theories of learning in defining what might be measured. Yes, if we think learning is a broadly dialogic process, then measuring it must require analysis of dialogue (broadly conceived). But most analytics aren’t there yet, so perhaps we are left only with measurements related to much more individualistic theories of learning? But this, it seems, may change.
‘so perhaps we are left only with measurements related to much more individualistic theories of learning?’
Good point. I believe that measuring learning that takes place in a community is even more complicated (more variables, more sophisticated correlations) than gauging progress of an individual. Algorithms are infamous for ‘counting what’s easier to count’, so it’s still a long way towards reliable LA here. Also, it seems that domineering learning theories are not always the main drivers.
Hey Iryna, this is a really cool dataviz and I like the elements of visualization. You linking it to the different theories and connecting to MOOCs is also of great value. Like you said most times the data to predict performance has been generated from engagement with the LMS. it will be interesting to compare such data against comments and feedback from an assessment over a period to ascertain the better indicator. Nice work
Thanks for your feedback, fejartu!