Tag Archives: reflection

Final Reflection

Leave a Reply

The visualisations during this course helped me to understand the importance of the data that is collected and that the way in which it is visualised is of similar if not more importance. Visualisations or dashboards are how most people will view and interact with the collected data which makes these a powerful tool in learning, teaching and governing. The data and dashboards normally come as a package from the vendor of the platform and as such the decisions about what data is collected and how it is visualised have already been made.

Through the different blocks a realisation I made was that the data that is desirable depends on where you stand within the hierarchy of education. The hierarchy in my view is represented in the following:

Figure 1 – Education Hierarchy

While there are organisations that provide consultancy and services to the individual levels of the hierarchy it is those in the hierarchy that ultimately have decision-making power. The process by which the data that is desirable is decided upon are by the policies that the government implements. At each layer, the policies are refined which gives guidance to the lower tiers around what data is required to meet those policies. The systems that are implemented to gather the data are normally decided by each level, as long as the data that is returned is in the correct form to satisfy the policy. The data gathered is more specific moving down the hierarchy but as it is passed back up through the hierarchy the data becomes more generalised meaning that the context is removed.

Within the learning with data block the focus was on learning management systems (LMS) that are used to provide an online service to students similar to that of a classroom and from this service they provide information back to teachers, for example around student engagement and indicators that a student might be struggling with material (Bulger 2016). One issue with these systems is the lack of context in the collected data, context is important due to the life altering consequences a successful or otherwise student career may have on a student’s life. If context can be given to the data such as through wearables for example (Knox et al 2020), then it is possible that services may be able to be provided to the student that could turn an unsuccessful student experience into a successful one. Another issue is the design of the systems for an individual style of learning or personalised learning experience, the education system that is mostly still currently in place is focused on the class or group environment. This class environment has been refined for a long time and to shift to this new individualised method without much research and study is worrying as students may be getting an inferior education as a result.

The next block covered how the data was used in teaching and how best to represent that for teachers. This section showed the limitation that is inherent with such data which is best described by the following quote by Williamson et al (2020), “Data and metrics set limits on what can be known and what can be knowable. They define what is rendered visible or left invisible, thereby impacting on how certain practices, objects, behaviours and so on gain value, while others are not measured or valued.” I believe the reason for such weight being put behind this data is the companies offering these platforms as well as think tanks who recommend this new type of learning push the data as being a huge benefit and market it as a way of being proactive, so no student is left behind. The issue with this is the lack of research into this new form of pedagogy (van Dijck 2018) and the expertise required to understand the data as data science is an entirely separate field. In my view a possible solution to these issues would be some form of regulation of the education sector which would put clear rules in place around what is required to be collected and what can give that all important context that is missing. This role of regulation would be required at the Department of Education level so that is has clear standing in the hierarchy.

The final block discussed governing with the data for example setting policy or ensuring that certain policies are being applied which in turn will provide the data to prove the policy is working. In the past Departments of Education around the world were the main group who had influence on how education would be taught and would set the policies (Williamson 2017). As this new form of learning through platforms has taken hold governments do not have the expertise to create such systems especially in the Global South and so must work with external companies normally in the Global North to achieve scale and be able to interpret the data that is generated (Prinsloo 2020). This setup creates a lot of black boxes that no one person understands how it works and how the output from these systems came to be. It is in this section where the context is lost from the data and it is also within governing that this data is important to have so better decisions can be made.

At the end of this process the main takeaways for me were, context is key, data gathering has limitations, and education is more of a social activity than an individual experience. Teachers based in a classroom environment have the ability to be able to monitor and investigate challenges students may face in real-time and the teacher’s relationship with their students would allow for context to be known surrounding issues. As discussed above in block two when data is gathered unless it is a complete representation of the scenario there are always missing pieces to the information as we do not currently have a method of gathering up data from every aspect of a person’s life. Finally, the way in which online platforms are offering the ability to learn is more of an individualistic approach which removes a lot of the expertise learned about how to teach people. The group and social aspect of education has been around for hundreds of years and regardless of what new system is recommended that may offer benefits it should be heavily investigated and researched before the old methods are thrown away.

References

Bulger, M. 2016. Personalized Learning: The Conversations We’re Not Having. Data & Society working paper. Available: https://datasociety.net/pubs/ecl/PersonalizedLearning_primer_2016.pdf

Knox, J, Williamson, B & Bayne, S 2019, ‘Machine behaviourism: Future visions of “learnification” and “datafication” across humans and digital technologies‘, Learning, Media and Technology, 45(1), pp. 1-15.

Williamson, B. Bayne, S. Shay, S. 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in Higher Education. 25(4), pp. 351-365.

van Dijck, J., Poell, T., & de Waal, M. 2018. Chapter 6: Education, In The Platform Society, Oxford University Press

Williamson, B. 2017. Digital Education Governance: political analytics, performativity and accountability. Chapter 4 in Big Data in Education: The digital future of learning, policy and practice. Sage.

Prinsloo, P. 2020. Data frontiers and frontiers of power in (higher) education: a view of/from the Global South. Teaching in Higher Education, 25(4) pp.366-383

Block 3 – Reflection

1 Reply

This block and the associated readings helped me understand more closely what the data is that decision makers have used previously and currently in governing education. This data in some form has played a key role in education for a long time, in the past ‘government education departments were key centres of calculation that were able to collect and aggregate data on school’ (Williamson, 2017) but in recent years large multinational technology companies have become the key centres of calculation and a form of the data is then handed off to the relevant government departments.

I question the methods in which data are currently used in governing and within that the impact the data gathers and data processors have on the resulting data.

When I look at the methods of data collection and representation that have been described in the literature, such as databases, inspections and reports (Williamson, 2017), the term that comes to mind is abstraction. As you add more and more entities or steps in the processing of data that data is more and more abstract from its original form. Abstraction in the form I am most familiar with is programming, within programming these levels of abstraction are for the most part ‘black boxes’, it depends on the situation, but you might not be able to peer inside and see what a program is doing. The output from these black boxes can have little context as you cannot peer inside the box and understand how the output came to be. The same stands for the output of education data, behind the data is a student and possibly their future career but as the levels of abstraction are applied to the data the detail of that student is removed and decisions are made solely on the numbers and not on any extenuating circumstances that might be at play. Case in point the inspectors within Ofsted that could not show that a school was improving just by the data they were asked to collect, because of the data being limiting and not being able to show the context a school could have improved but the data be unable to show that (Ozga & Williamson, 2016).

In ‘Data frontiers and frontiers of power in (higher) education’, Prinsloo (2020) shows that organisations in the Global North are using their considerable influence to push countries in the Global South to adopt technology so that they do not miss out on the Fourth Industrial Revolution. Governments and citizens of the Global South do not want to miss out on being apart of the next revolution and so are keen to work with organisations in the Global North that can help them catch up on the Global North. One of the issues with this approach is that most of the prominent organisations are from the United States and their platforms are designed for the US education system, by countries within the Global South adopting these they also adopt the pedagogical approaches that these systems have. The dashboards that are produced as part of these are also from the view of US education policy meaning that not only is the technology being developed outside of these countries, but they also have no say in the education policy that come with these systems.

As countries are facing increased pressure to adopt EdTech, one such method that these countries have of pushing back on these EdTech platform providers is to put regulation in place that would ensure that they adapt to the country’s rules. It is very difficult for one university or one school to push a technology company to adapt the platform for their requirements, but a government can exert such influence that the platforms make the necessary changes. Governments place regulation around a large number of areas of the economy but one such area lacking regulation is education and I believe such a step would bring much needed benefit to a sector that currently must agree to terms with companies on a case-by-case basis.

References

Prinsloo, Paul, 2020. Data frontiers and frontiers of power in (higher) education: a view of/from the Global South. Teaching in higher education, 25(4), pp.366–383.

Williamson, Ben, 2017. Digital Education Governance: political analytics, performativity and accountability. In: Big Data in Education, London: SAGE Publications.

Ozga, Jenny & Williamson, Ben, 2016. Trust in numbers? Digital Education Governance and the inspection process. European educational research journal EERJ, 15(1), pp.69–81.

Block 2 – Reflection

1 Reply

The last three weeks of visualisations I have tried to put emphasis on how best to show the data in a valuable way for teachers. My feelings through this section have been of frustration for several reasons:

  • What does the visualisation show?
  • What value does the data gathered hold?
  • Can any correlations be drawn from this data to research backed assumptions?
  • A piece of text might be better at explaining the outcome of the data collection than a dashboard

Williamson (2020) explained the frustration I felt for the first two reasons:

Data and metrics set limits on what can be known and what can be knowable. They define what is rendered visible or left invisible, thereby impacting on how certain practices, objects, behaviours and so on gain value, while others are not measured or valued.

As can be seen by this statement I faced a tough choice when choosing what data to capture and to display I thought “I’m going to miss huge swathes of data that could provide context for the data I’m collecting.” I may be able to show how many times I accessed my phone or laptop, but I cannot show that between 10am and 6pm on Wednesday the reason I did not access my laptop or phone was because the electricity was out because the visualisation doesn’t allow for that. If someone has limited access to broadband for example how can that data be gathered and shown.

In ‘The Platform Society’ (van Dijck 2018) it is stated that “critics draw attention to the fact that none of the presumed benefits cited by platforms have been proven empirically.” If the benefits have yet to be proven by some form of study, the platforms and proponents of online education can make any claim they wish. This is how unregulated areas of the economy function such as the vitamin industry making claims such as a vitamin can stop a virus without any evidence or how certain treatments in the cosmetic sector can make claims like ‘this will reverse ageing.’ If we place education as one of the pillars of society, we should possibly look at some level of regulation to stop wild claims being made.

And finally, after doing six visualisations I struggle to accept that dashboards are the best way to get across information. Within every visualisation I am making personal choices about what colours and styles to utilise and this is very similar to how dashboards and the underlying software is designed. This should not be the case it should be targeted information with as mentioned above some evidence behind what it is showing. With these issues and limitations of dashboards I would believe for the moment they should not be used. I would look at some form of text recap as much information and specifically individual student details cannot be expressed in a dashboard. There is a lot of work around machine learning and text so it would still be possible to provide a version similar to a dashboard but more granular.

References

Williamson, Ben, Bayne, Sian & Shay, Suellen, 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in higher education, 25(4), pp.351–365.

van Dijck, José Poell, Thomas & de Waal, Martijn, 2018. The Platform Society, Chapter 6 Oxford: Oxford University Press USA – OSO.

Build a Dashboard – Reflection

2 Replies
Figure 1 – Dashboard Example

My overall experience with this task was one of confusion and a simple question, what do I show that is relevant? The answer to that was always very close to this is not showing anything or if it is what real value is it showing other than numbers on a screen. It showed me in the same way the literature has been explaining that gleaming any meaningful information or understanding from dashboards does require more of a background to the pupils instead of just the numbers and a way to be able to critically analyse that data.

We were provided with two sets of data to work with and create some form of a dashboard, I decided to work with Sample Data 2 as Sample Data 1 was missing a lot of background information. Sample Data 1 had only numbers and it was difficult to understand if each number was associated with a particular section and if that was the case why some lines did not have the course ID. (Figure 2)

Figure 2 – CSV Data for Sample Data 1

As can be seen from Figure 2 it seems as though this information is in regard to “course 14” but with no definitive information on that it would be wrong to draw that conclusion.

As can be seen from Figure 1 I decided on straightforward graphs as I was unsure about how to display certain correlations I thought would be good. I struggled with how best to explain in a graph / dashboard that if a user logs in less to the platform and performs poorly show the teacher that information. At the same time, I was conflicted about trying to do that as that has been shown to not be accurate, how much someone interacts with a platform does not mean that they will do better or worse in an exam.

In my exploration of this I noticed that I was drawn more to the exam results than any other item on the CSV file. This idea seems to be one area that is the same throughout society we put so much emphasis on exam results that they are all that matter at the expense of everything else. If there was more granular information about a particular subject or task it is quite possible that this would be more valuable as it would be able to show knowledge increase over time and not just one point in time as exams do.

While these files had a certain set of data it made me think about what data should be collected to make the dashboards more valuable and it became rather complicated. As you start to measure you start to think about measuring everything but as you do that you encroach on people’s freedoms and is an education dashboard a valid reason to do this? Probably not.

Block 1 – Reflection

1 Reply

This block gave me a view of data capture and the overwhelming feeling I took away from it was:

The data that is collected is not what should be collected

The learning management systems (LMS) that are being utilised today gather information on the interactions and time users (the systems are tracking everyone) spend with the system. The value of this information improving peoples learning is questionable since it is a limited set of variables to do with interaction with the system. As discussed by Bulger (2016) most if not all current education systems are responsive, meaning that the system needs a ‘cause’ to then ‘react’. A teacher in most cases would be far more adaptive and be able to adapt to students prior to an issue arising.

Over the course of the three visualisations, I have tracked personal details such as how much interaction I had with course materials, how much sleep and exercise I got and finally how often I would have food or drink or snacks. These items show the level of interaction, how effective that interaction might be due to cognitive capacity and the distraction of having a cup of tea instead of focusing on course material.

The above items in my view are more important to understand, as the social life of the person shows more about what that person might be able to achieve. For example, if I do not sleep well that has a greater effect on my learning than what a system can tell from me rewinding a video several times. One of the ways to allow the LMS to see such a situation would be through wearables, those wearables as discussed by Knox et al (2020) could also provide feedback to students that would optimise when to learn and rest.

Personalisation within education has the goal of a single teacher to a single student (Friesen 2020). One of the main questions coming out of personalisation is if that teacher were a machine that could pass the Turing test is that not equal to a human teacher? In Benjamin Bloom’s paper “The 2 sigma problem” (1984) it shows that having the one to one, master – student relationship is highly beneficial. The problem though at this moment is that the master in machine learning does not exist, the technology has that goal of reaching it one day but currently it is not the case. We currently have a system as discussed above that has a limited set of variables to work from and no context around those variables e.g. sleep. A human working in a one-on-one relationship with a student can determine many different variables and through dialogue can understand the mood of the student and on a day to day basis manage their studies based on this.

With personalisation there is always a risk that a system might create a feedback loop for students. If the system recommends a certain subject or the administrator wants more focus on certain subjects they could ensure that the system nudges people in a certain direction (Knox et al 2020). Once a student is on that path perhaps the system continually pushes that new direction and the student unbeknownst to them is being nudged down a path they would not have wanted if they had freedom to explore.