Final Reflection

Leave a Reply

The visualisations during this course helped me to understand the importance of the data that is collected and that the way in which it is visualised is of similar if not more importance. Visualisations or dashboards are how most people will view and interact with the collected data which makes these a powerful tool in learning, teaching and governing. The data and dashboards normally come as a package from the vendor of the platform and as such the decisions about what data is collected and how it is visualised have already been made.

Through the different blocks a realisation I made was that the data that is desirable depends on where you stand within the hierarchy of education. The hierarchy in my view is represented in the following:

Figure 1 – Education Hierarchy

While there are organisations that provide consultancy and services to the individual levels of the hierarchy it is those in the hierarchy that ultimately have decision-making power. The process by which the data that is desirable is decided upon are by the policies that the government implements. At each layer, the policies are refined which gives guidance to the lower tiers around what data is required to meet those policies. The systems that are implemented to gather the data are normally decided by each level, as long as the data that is returned is in the correct form to satisfy the policy. The data gathered is more specific moving down the hierarchy but as it is passed back up through the hierarchy the data becomes more generalised meaning that the context is removed.

Within the learning with data block the focus was on learning management systems (LMS) that are used to provide an online service to students similar to that of a classroom and from this service they provide information back to teachers, for example around student engagement and indicators that a student might be struggling with material (Bulger 2016). One issue with these systems is the lack of context in the collected data, context is important due to the life altering consequences a successful or otherwise student career may have on a student’s life. If context can be given to the data such as through wearables for example (Knox et al 2020), then it is possible that services may be able to be provided to the student that could turn an unsuccessful student experience into a successful one. Another issue is the design of the systems for an individual style of learning or personalised learning experience, the education system that is mostly still currently in place is focused on the class or group environment. This class environment has been refined for a long time and to shift to this new individualised method without much research and study is worrying as students may be getting an inferior education as a result.

The next block covered how the data was used in teaching and how best to represent that for teachers. This section showed the limitation that is inherent with such data which is best described by the following quote by Williamson et al (2020), “Data and metrics set limits on what can be known and what can be knowable. They define what is rendered visible or left invisible, thereby impacting on how certain practices, objects, behaviours and so on gain value, while others are not measured or valued.” I believe the reason for such weight being put behind this data is the companies offering these platforms as well as think tanks who recommend this new type of learning push the data as being a huge benefit and market it as a way of being proactive, so no student is left behind. The issue with this is the lack of research into this new form of pedagogy (van Dijck 2018) and the expertise required to understand the data as data science is an entirely separate field. In my view a possible solution to these issues would be some form of regulation of the education sector which would put clear rules in place around what is required to be collected and what can give that all important context that is missing. This role of regulation would be required at the Department of Education level so that is has clear standing in the hierarchy.

The final block discussed governing with the data for example setting policy or ensuring that certain policies are being applied which in turn will provide the data to prove the policy is working. In the past Departments of Education around the world were the main group who had influence on how education would be taught and would set the policies (Williamson 2017). As this new form of learning through platforms has taken hold governments do not have the expertise to create such systems especially in the Global South and so must work with external companies normally in the Global North to achieve scale and be able to interpret the data that is generated (Prinsloo 2020). This setup creates a lot of black boxes that no one person understands how it works and how the output from these systems came to be. It is in this section where the context is lost from the data and it is also within governing that this data is important to have so better decisions can be made.

At the end of this process the main takeaways for me were, context is key, data gathering has limitations, and education is more of a social activity than an individual experience. Teachers based in a classroom environment have the ability to be able to monitor and investigate challenges students may face in real-time and the teacher’s relationship with their students would allow for context to be known surrounding issues. As discussed above in block two when data is gathered unless it is a complete representation of the scenario there are always missing pieces to the information as we do not currently have a method of gathering up data from every aspect of a person’s life. Finally, the way in which online platforms are offering the ability to learn is more of an individualistic approach which removes a lot of the expertise learned about how to teach people. The group and social aspect of education has been around for hundreds of years and regardless of what new system is recommended that may offer benefits it should be heavily investigated and researched before the old methods are thrown away.

References

Bulger, M. 2016. Personalized Learning: The Conversations We’re Not Having. Data & Society working paper. Available: https://datasociety.net/pubs/ecl/PersonalizedLearning_primer_2016.pdf

Knox, J, Williamson, B & Bayne, S 2019, ‘Machine behaviourism: Future visions of “learnification” and “datafication” across humans and digital technologies‘, Learning, Media and Technology, 45(1), pp. 1-15.

Williamson, B. Bayne, S. Shay, S. 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in Higher Education. 25(4), pp. 351-365.

van Dijck, J., Poell, T., & de Waal, M. 2018. Chapter 6: Education, In The Platform Society, Oxford University Press

Williamson, B. 2017. Digital Education Governance: political analytics, performativity and accountability. Chapter 4 in Big Data in Education: The digital future of learning, policy and practice. Sage.

Prinsloo, P. 2020. Data frontiers and frontiers of power in (higher) education: a view of/from the Global South. Teaching in Higher Education, 25(4) pp.366-383

Week 11 – University & Personal Life Visualisation

1 Reply
Figure 1 – Visualisation

Following on the governance block I have focused here on showing general information on my university life and my personal life. From the readings for this block, it seems as though the view decision makers have is that of averaged, abstract data that can show little about who the person is but give broad indications of what is occurring.

Description of the Visualisation

Everything to the left in the visualisation is related to my university studies and everything to the right is covering a general area of my personal life.

On the University side I have split it into three areas which are:

  1. Reading – course specific reading
  2. University Admin – checking tasks, gathering all the readings, reading feedback
  3. Reflections, Tasks – Visualisations, writing reflections

On the personal side I have three areas also:

  1. Food – Eating, preparing, cleaning up food
  2. Exercise – Walking, yoga, etc
  3. Relaxation – TV, playing video games, reading books

Legend

Figure 2 – Legend

Reasoning

Through the readings on governance and over the course of this module I have become aware that decision makers are looking for vague or general information on students, teachers, schools, districts, etc. The closer to the student a viewer of the data is the more detailed information they seem to require, meaning teachers want to have as much information as they can to hand to make a decision on a given student, an administrator wants to see how a class is achieving and district supervisor wants to see how a school is performing.

With this visualisation while it is showing my data in one-hour segments, it would be possible to utilise the same visualisation to show data for an entire class or school. What I have seen as important over the course of this module is to reduce training on how this data can be interpreted and one way to do that is have the same visualisation, so people understand how to read it and then have the visualisation be able to represent a student, a class, or a school. While this has its drawbacks in so far as removing a lot of granularity to the data it ensures that training can be given, and the pitfalls can be called out by the training.

The idea also with this visualisation is that it gives the decision maker some method of being able to express certain areas where they want to see improvement in such as ‘we want people to spend more time reading, how do we achieve that?’ Or ‘we want to ensure students are accessing more exercise as this improves student retention, how can we encourage that?’

Block 3 – Reflection

1 Reply

This block and the associated readings helped me understand more closely what the data is that decision makers have used previously and currently in governing education. This data in some form has played a key role in education for a long time, in the past ‘government education departments were key centres of calculation that were able to collect and aggregate data on school’ (Williamson, 2017) but in recent years large multinational technology companies have become the key centres of calculation and a form of the data is then handed off to the relevant government departments.

I question the methods in which data are currently used in governing and within that the impact the data gathers and data processors have on the resulting data.

When I look at the methods of data collection and representation that have been described in the literature, such as databases, inspections and reports (Williamson, 2017), the term that comes to mind is abstraction. As you add more and more entities or steps in the processing of data that data is more and more abstract from its original form. Abstraction in the form I am most familiar with is programming, within programming these levels of abstraction are for the most part ‘black boxes’, it depends on the situation, but you might not be able to peer inside and see what a program is doing. The output from these black boxes can have little context as you cannot peer inside the box and understand how the output came to be. The same stands for the output of education data, behind the data is a student and possibly their future career but as the levels of abstraction are applied to the data the detail of that student is removed and decisions are made solely on the numbers and not on any extenuating circumstances that might be at play. Case in point the inspectors within Ofsted that could not show that a school was improving just by the data they were asked to collect, because of the data being limiting and not being able to show the context a school could have improved but the data be unable to show that (Ozga & Williamson, 2016).

In ‘Data frontiers and frontiers of power in (higher) education’, Prinsloo (2020) shows that organisations in the Global North are using their considerable influence to push countries in the Global South to adopt technology so that they do not miss out on the Fourth Industrial Revolution. Governments and citizens of the Global South do not want to miss out on being apart of the next revolution and so are keen to work with organisations in the Global North that can help them catch up on the Global North. One of the issues with this approach is that most of the prominent organisations are from the United States and their platforms are designed for the US education system, by countries within the Global South adopting these they also adopt the pedagogical approaches that these systems have. The dashboards that are produced as part of these are also from the view of US education policy meaning that not only is the technology being developed outside of these countries, but they also have no say in the education policy that come with these systems.

As countries are facing increased pressure to adopt EdTech, one such method that these countries have of pushing back on these EdTech platform providers is to put regulation in place that would ensure that they adapt to the country’s rules. It is very difficult for one university or one school to push a technology company to adapt the platform for their requirements, but a government can exert such influence that the platforms make the necessary changes. Governments place regulation around a large number of areas of the economy but one such area lacking regulation is education and I believe such a step would bring much needed benefit to a sector that currently must agree to terms with companies on a case-by-case basis.

References

Prinsloo, Paul, 2020. Data frontiers and frontiers of power in (higher) education: a view of/from the Global South. Teaching in higher education, 25(4), pp.366–383.

Williamson, Ben, 2017. Digital Education Governance: political analytics, performativity and accountability. In: Big Data in Education, London: SAGE Publications.

Ozga, Jenny & Williamson, Ben, 2016. Trust in numbers? Digital Education Governance and the inspection process. European educational research journal EERJ, 15(1), pp.69–81.

Week 10 – Category Visualisation

1 Reply
Figure 1 – Visualisation

As part of governance, it seems as though the view that is desirable is that of averages and broad categories. As data becomes processed at each level the detail is removed from it.

Description of the Visualisation

Along the top of the visualisation, you have a horizontal line that is a linear timeline from the beginning of data capture to the end of capture. Every element on this line is also part of the large circles below.

The circles are split up into the following:

  • Left – University
  • Middle – Personal
  • Right – Work

Legend

Figure 2 – Legend

Reasoning

People who are governing and utilising data for that governance are looking for data that is more general. As the data is processed from its raw form the nuance or specifics of the data are generally removed. That is why I have gone with the above visualisation as it shows the administrator or school leaders areas of the students’ day and based on that they can see where the students focus is.

This iteration of the data could show that possibly students should be encouraged or nudged to do more university reading or spending more time outside. Then the governor’s can search out data that has been processed in another manner to show what the students are reading during that time or possibly if there is a reason for limited outside time such as lack of amenities.

There is always the possibility with any data that possibly it was the way in which it was processed that has removed some of its value for scenarios.

Week 9 – Plant Visualisation

1 Reply
Figure 1 – Plant Visualisation

In this block of governance, I wanted to visualise the data in a way that could be used for viewing a student’s day and possibly seeing patterns. Once the patterns are seen then it could be possible to build policy from that. I kept track of my day and the main elements I did during each day.

Description of the Visualisation

This is five different plants (one for each day) with branches / leaves coming off them and under each branch / leaf is the activity during that hour period. These cover the hours of 10:00 to 18:00 as these are the hours that a school or university can track what a student is doing.

Under each branch I placed a symbol for what I did during that hour, this makes it easier to know roughly what I did. Perhaps there could be another type of visualisation to show a time breakdown if required.

Legend

Figure 2 – Legend

Reasoning

From the tweetorial and readings I have done this week one of the items that stood out was that of high-level information being utilised. By high level I mean how much class time does a student have, how long do they spend on the education platform, how much time do they spend on campus, etc. All these items are gathered from much more granular detail such as card swipe entries into different areas of the building, but decision makers are looking for totals or averages.

For these reasons I went with broad areas but broke them down by hour, but it is most likely that these would be further processed to just an average for a class or course for some decision to be made on if the necessary targets are being achieved.

Block 2 – Reflection

1 Reply

The last three weeks of visualisations I have tried to put emphasis on how best to show the data in a valuable way for teachers. My feelings through this section have been of frustration for several reasons:

  • What does the visualisation show?
  • What value does the data gathered hold?
  • Can any correlations be drawn from this data to research backed assumptions?
  • A piece of text might be better at explaining the outcome of the data collection than a dashboard

Williamson (2020) explained the frustration I felt for the first two reasons:

Data and metrics set limits on what can be known and what can be knowable. They define what is rendered visible or left invisible, thereby impacting on how certain practices, objects, behaviours and so on gain value, while others are not measured or valued.

As can be seen by this statement I faced a tough choice when choosing what data to capture and to display I thought “I’m going to miss huge swathes of data that could provide context for the data I’m collecting.” I may be able to show how many times I accessed my phone or laptop, but I cannot show that between 10am and 6pm on Wednesday the reason I did not access my laptop or phone was because the electricity was out because the visualisation doesn’t allow for that. If someone has limited access to broadband for example how can that data be gathered and shown.

In ‘The Platform Society’ (van Dijck 2018) it is stated that “critics draw attention to the fact that none of the presumed benefits cited by platforms have been proven empirically.” If the benefits have yet to be proven by some form of study, the platforms and proponents of online education can make any claim they wish. This is how unregulated areas of the economy function such as the vitamin industry making claims such as a vitamin can stop a virus without any evidence or how certain treatments in the cosmetic sector can make claims like ‘this will reverse ageing.’ If we place education as one of the pillars of society, we should possibly look at some level of regulation to stop wild claims being made.

And finally, after doing six visualisations I struggle to accept that dashboards are the best way to get across information. Within every visualisation I am making personal choices about what colours and styles to utilise and this is very similar to how dashboards and the underlying software is designed. This should not be the case it should be targeted information with as mentioned above some evidence behind what it is showing. With these issues and limitations of dashboards I would believe for the moment they should not be used. I would look at some form of text recap as much information and specifically individual student details cannot be expressed in a dashboard. There is a lot of work around machine learning and text so it would still be possible to provide a version similar to a dashboard but more granular.

References

Williamson, Ben, Bayne, Sian & Shay, Suellen, 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in higher education, 25(4), pp.351–365.

van Dijck, José Poell, Thomas & de Waal, Martijn, 2018. The Platform Society, Chapter 6 Oxford: Oxford University Press USA – OSO.

Week 8 – Time Spent and Apps Opened Visualisation

1 Reply
Figure 1 – Time / Apps Visualisation

I kept track of how much time I spent on my phone over the week and tried to keep rough track of what apps I used during that period. I limited the apps on the visualisation to those that I accessed twenty times or greater. Within keeping track of how much time I spent on the phone I was able to see how many times I unlocked the phone.

Description of the Visualisation

There are three lines on the visualisation representing the following in order:

  1. Time spent on the phone (clock at the end of the line)
  2. Number of times phone was unlocked (tally counter at the end of the line)
  3. How many times an app was opened (window at the end of the line)

Each line gives some basic information but when combined together the viewer would be able to draw some correalations as to what the user was doing during a given day.

Legend

Figure 2 – Legend

Reasoning

The idea behind this week was to track my phone usage similar to how it was described in ‘The Platform Society’ (van Dijck et al 2018) that AltSchool would monitor their students, “each pupil has an iPad or Chromebook, and every activity is automatically recorded and analyzed” (p. 7). Within that though was to show how maybe that the information that is gathered might be a bit thin on substance meaning that there might be little to gain from tracking it.

The visualisation itself has a clock, tally counter and windows these were used because they are something people have come in contact with and would be familar with.

In different sections of the visualisation the same colours needed to be reused due to running out of colours but I think this also highlights the same situation in dashboards. Within dashboards you can have several different items being shown on the screen and all depending on the colour pallette available might get confused at what is being shown.

Build a Dashboard – Reflection

2 Replies
Figure 1 – Dashboard Example

My overall experience with this task was one of confusion and a simple question, what do I show that is relevant? The answer to that was always very close to this is not showing anything or if it is what real value is it showing other than numbers on a screen. It showed me in the same way the literature has been explaining that gleaming any meaningful information or understanding from dashboards does require more of a background to the pupils instead of just the numbers and a way to be able to critically analyse that data.

We were provided with two sets of data to work with and create some form of a dashboard, I decided to work with Sample Data 2 as Sample Data 1 was missing a lot of background information. Sample Data 1 had only numbers and it was difficult to understand if each number was associated with a particular section and if that was the case why some lines did not have the course ID. (Figure 2)

Figure 2 – CSV Data for Sample Data 1

As can be seen from Figure 2 it seems as though this information is in regard to “course 14” but with no definitive information on that it would be wrong to draw that conclusion.

As can be seen from Figure 1 I decided on straightforward graphs as I was unsure about how to display certain correlations I thought would be good. I struggled with how best to explain in a graph / dashboard that if a user logs in less to the platform and performs poorly show the teacher that information. At the same time, I was conflicted about trying to do that as that has been shown to not be accurate, how much someone interacts with a platform does not mean that they will do better or worse in an exam.

In my exploration of this I noticed that I was drawn more to the exam results than any other item on the CSV file. This idea seems to be one area that is the same throughout society we put so much emphasis on exam results that they are all that matter at the expense of everything else. If there was more granular information about a particular subject or task it is quite possible that this would be more valuable as it would be able to show knowledge increase over time and not just one point in time as exams do.

While these files had a certain set of data it made me think about what data should be collected to make the dashboards more valuable and it became rather complicated. As you start to measure you start to think about measuring everything but as you do that you encroach on people’s freedoms and is an education dashboard a valid reason to do this? Probably not.

Week 7 – Line Visualisation

1 Reply
Figure 1 – Line Visualisation

Over the last week I tracked what I did throughout the day. I only marked down a new entry when I moved to a new activity, I did not track how long I did the activities for.

Description of the Visualisation

I took inspiration from Dear Data, Week 7 Complaints, when doing this visualisation.

Each line on the left hand side of the image represent tasks done prior to 13:00 and each line on the right hand side represents tasks after 13:00. The measurements are for 5 days and each day has a different inclination, day 1 starting from the top to day 5 at the bottom.

Legend

Figure 2 – Legend

Reasoning

I wanted to show what it might be like for a teacher looking at data on a dashboard. There are a lot of lines but if there is limited training how do you know which lines are important or something that should be actioned. Even with a legend or some explanation about the graph how can a teacher understand the reasoning or the underlying logic for why something is being shown to them.

Due to limited colours I decided to use the same colours for different tasks depending on what side of 13:00 the task was. I think this brought out a good point, in that teachers are bombarded with graphs and dashboards with different lines and colours but again no training or context to what they mean. One graph could show a yellow line being something important but in another graph a yellow line could be simply an average marker.

Week 6 – Picnic Visualisation

2 Replies
Figure 1 – Picnic Visualisation

This week I focused on gathering the information I have gathered over the previous three weeks which were:

  • that of distractions (meals, tea, snacks, etc)
  • time outside
  • focus on course materials (reading, twitter, responding to posts, etc)

The reason for this was to show in one visualisation how busy any given week is.

Description of Visualisation

The visualisation is showing a picnic on a bed of grass, there is a blanket in the middle with different plates of food on it and the sun is out and shining.

The idea here is that each of the blades of grass represents a different distraction faced throughout the week. The individual rays from the sun are showing how often I have been outside in the week. On the blanket are different plates which represent different activities surrounding the module/course.

Legend

Figure 2 – Picnic Legend

Reasoning

The idea behind this was to show a teacher that in students lives there are many surrounding items that occur. The blanket which is specifically for items on the course is what the teacher can see but it is important for the teacher to be able to understand that surrounding it from all directions are items that can affect a student.

Teachers are normally very good at understanding that there are other items going on in peoples lives. As we move more towards AI teachers and algorithms making choices on students the above items do not seem to be factored in. I understand that there is discussion about wearables, but that doesn’t factor in personal issues, wearables are more focused on tracking and monitoring of the physical person not the person in a more human / personal sense.