Block 2: Week 7 Teaching Dashboard

Task

Generate charts/graphs for a dashboard from one of the data set provided, with a focus on what a teacher might find useful.

  • Data: Sample data 2.
  • Software: Excel.

Senario

A 12-week postgraduate class, with the possibility for on-campus and online attendance. Resources, such as the Course Handbook, are shared through the VLE (including a link to Resource Lists, accessible through the Library).

The course is summatively assessed by four separate tests, each with a pass mark of 40%, of which one Fail (below 40%) may be condoned. The overall course mark is an average of the four individual marks, and also has a pass mark of 40%. It is not credit-bearing, but those who pass are offered a certificate of completion. Attendance at on-campus activities is not compulsory.

Dashboard components

Firstly, there is no contextual data in the dataset; we have no personal details nor anything about their previous academic history. Some overall summary statistics might be of interest to the teacher.

example student dashboard
Overall class data

Such charts could assist in seeing if the level of the assessment appeared to be appropriate for the cohort and if it provided differentiation. Whether the assessment actually was successful in determining if the students reached the aims of the course, would require closer examination of the assessments themselves.

The teacher might be happy that all students passed. However, this does not give any indication of why students performed as they did, or if the assessment is not just an indication of factors that are not about academic ability.

Any teacher would be interested in outliers, especially those who indicate that they are not thriving for any number of reasons; of course, interest would only be of real good if realised early in the course (or better still, as part of the course/curriculum design). Student S0021 can be seen to have the lowest overall test scores, attendance (on campus and online), and online engagements, according to the data provided; S0004 the highest.

sammple student dashboard
Does Test Score correlate with Class Attendance?

There does appear to be a correlation between in-class attendance and test score but that does not mean there is an causation: the reasons that a student does not attend could be the same or related to the reasons that they find it hard to study. Of course, this all assumes that the student with the lowest score did badly; given their previous academic ability, say, this may constitute a considerable achievement. The student with the highest score may be automatically considered to have done well, but their mark may not be a true reflection of their ability.

Class attendance, on-campus or online, is easily measured but the reasons behind it and its actual effect on the student are not easily known. Graphs like this however can be easily misread.

sample student dashboard
Do those who attend in-class tend not to attend online?

The teacher may be interested to know if students who attend on-campus tend not to attend online, and vice versa. The data does not appear to suggest this. However, this is based on number of log-ins to the VLE which is a poor proxy for online engagement, though counting attendance at on-campus sessions may not be much better.

Again, our student S0002, may have taken this course because they do not have to attend on-campus, and in that one visit to the VLE may have downloaded their Handbook and complete reading list, and subscribed to the discussion boards (so updates come to them via email), meaning that they do not have to log in again.

The one occasion where lack of online activity may be worth investigating is at the start of a course, where lack of activity may mean the student is experiencing initial technical difficulties or has not received log-in details, and due to this, may also find it difficult to access help.

Does forum activity correlate with test score?

The teacher may be interested in finding if involvement in the discussion online is in any way coorelated to test scores; not surprising if the teacher has put a considerable amount of effort into this themselves. The data suggests not, but it is only about whether students read, posted or replied, not the quality of the experience they had. Again, graphs such as this can be very suggestive if they draw attention to the activity of the highest and lowest scoring students, and if this is used to explain past outcomes or predict future ones.

Conclusions

Having access to a data set, invites curiosity, especially if one has a connection to it. Insufficient knowledge of the amount and kind of data needed, does not stop one from creating what can be a persuasive visual display of ‘knowledge’.

sample student dashboard
Sample Dashboard

2 Replies to “Block 2: Week 7 Teaching Dashboard”

  1. Yes, you’ve amply discussed the dangers of confusing correlation with causation. This raises two questions for me: 1) If you can see the correlation, how would you (as an educator) explore causation? Would this require some form of additional analysis, and what kind of analysis, or simply act as a cue for in-person dialogue? 2) A lot of the initial hype about big data in general was that correlation itself was enough–you didn’t need accounts of causation if you could see, with big enough data sets, what was actually happening. Of course, the accuracy of such correlationist analysis has since been questioned. You can put multiple groups of analysts to work on the same problem with the same datasets, and they still might draw different findings because of entirely defensible yet subjective analytic decisions. The famous case is the correlation of red cards in soccer matches with skin colour–so-called “soccer referee bias”–and an experiment involving 29 research groups, who came to multiple different conclusions from the exact same data. So the question is whether correlation analysis can ever be enough in educational analytics, given its documented contingency rather than claimed accuracy. And if the analytics run automatically based on an in-built model, what’s to say a different model wouldn’t produce very different results?

    1. 1)
      That’s an interesting question. I’d start by reminding myself about Spurious Correlations https://www.tylervigen.com/spurious-correlations, a website full of correlations where causation should never be explored. Then, I would have to ask myself if I thought that could be any causation (for the activities for which I had data say, between attendance and test scores). I would be keen not to get persuaded that because I have data I have knowledge, and if I don’t know enough I just need more data! If I designed this course, I have only given the students an opportunity to learn: it doesn’t mean that if a student has 100% attendance and takes part in all the activities they will pass, or that some student avoids all my activities and passes anyway. Because it’s always more than the course, and I would try to do my best for students but in the end, their outcomes are due to so much more than their academic ability (and that’s way beyond what a learning system can measure); perhaps this course is not particularly accessible and all I am seeing in the data is differentiation by learning difference?
      If I wanted to know anything about academic progress, I would be better off having some formative feedback points; this way not only would I have some clue about how each one was progressing I might be able to support them too (it’s a bit late if I wait until all the data is collected). An online quiz won’t be enough because their progress is going to be due to so many personal factors (and as I said in my original post, we can’t assume that student S0021 is doing badly because they have the lowest scores or S0004 well because they scored highest.)
      2)
      Correlation=causation when N= very large: I haven’t heard this but it seems very convenient for those with the big data sets, and I think they would have to come up with a very robust defence for such a statement.
      I hadn’t heard of the ‘soccer referee bias’ example but it doesn’t surprise me at all; the more complex the question, the more assumptions are brought in and the large role they play. I suppose the one good point about this is that there isn’t an automated system built with an uninterrogatable algorithm that will give an instant verdict on a referee’s decision.

Leave a Reply

Your email address will not be published. Required fields are marked *