The data imaginary, as termed by Beer (2018), refers to the marketed promises and potential of datafication and data analytics while raising fears of “missing out” or “acting too slowly”. The data imaginary is used to expand the boundaries of or intensify what is acceptable for datafying, while simultaneously reducing resistance and increasing adoption. These new, uncharted data territories are thought of as data frontiers (Beer, 2019; Prinsloo, 2020) and the expansion into these territories lead to the idea of data colonialism (Couldry and Mejias, 2020; Prinsloo, 2020).

In data colonialism, social data (much like land, people, and resources in historical colonialism) are viewed as “just there” or “raw material” that can be extracted, appropriated, and commodified (Couldry and Mejias, 2020; Couldry and Mejias, 2019; Prinsloo, 2020). Making this colonization possible, through data imaginary, is the promise that data are: speedy, accessible, revealing, panoramic (data can see everything), prophetic (data can give insight and foresight), and smart (Beer, 2019; Prinsloo, 2020).

Education is one frontier that has been intensified over the last decade or two. New, large-scale data systems and infrastructures have been developed and installed by state governments, education corporations, and education institutions themselves to monitor, collect, analyze, forecast, and report data about schools, teachers, and students (Anagnostopoulous 2013; Fontaine, 2016; Williamson, 2017). These data have become a key component in developing educational policy (Williamson, 2017).

At the heart of educational data are metrics of performance and productivity from education institutions and their staff, faculty, and students (Williamson, 2017). Closely linked to performativity is that of accountability: measures of effectiveness and efficiency and focused on the quantifiable (Williamson, 2017; Fontaine, 2016). Accountability takes an instrumental view of learning and positions it as an output, allowing meaningful relationships between inputs (e.g., funding, pedagogy, and curriculum) to be made (Fontaine, 2016).

Educational policy that emphasises performance and productivity reorients educators to focus on things that can be quantified and quantified positively (Williamson, 2017). Standardized tests, for example, are a common source of data that promises to hold institutions accountable for student learning. Consequently, schools and educators are focused on raising or maintaining high test scores which can lead to “teaching to the test” by altering curriculum (Fontaine, 2016).

Intertwined within the notions of datafication and data colonialism is neoliberalism which further concentrates focus on measures of productivity, engagement, and inputs and outputs (Fawns, 2020; Ozga, 2015). Ozga (2015 and 2016) notes that neoliberal policy emphasizes the transparency of performance data to the public and data that is also comparable to other local, national, and international schools. These (inter)national data reports can result in competition and school ranking systems. Consequently, education systems can be pressured to improve their data (and rankings) which further prioritizes staff, faculty, and student data collection and policy intervention (Williamson, 2017) .

Anagnostopoulos, D., Rutledge, S.A. & Jacobsen, R. 2013. Conclusion: The Infrastructure of Accountability: Tensions, Implications and Concluding Thoughts. In, Anagnostopoulos, D., Rutledge, S.A. & Jacobsen, R. (Eds) The Infrastructure of Accountability: Data use and the transformation of American education.

Beer, D., 2018. The data gaze: Capitalism, power and perception. Sage.

Couldry, N. and Mejias, U.A., 2019. Data colonialism: Rethinking big data’s relation to the contemporary subject. Television & New Media, 20(4), pp.336-349.

Couldry, N. and Mejias, U.A., 2020. The Costs of Connection: How Data Are Colonizing Human Life and Appropriating It for Capitalism.

Fawns, T., Aitken, G. and Jones, D., 2020. Ecological teaching evaluation vs the datafication of quality: Understanding education with, and around, data. Postdigital Science and Education, pp.1-18.

Fontaine, C. 2016. The Myth of Accountability: How Data (Mis)Use is Reinforcing the Problems of Public Education, Data and Society Working Paper 08.08.2016.

Ozga, J. and Segerholm, C., 2015. Neo-liberal agenda (s) in education. Governing by inspection, pp.27-37.

Ozga, J. 2016. Trust in numbers? Digital Education Governance and the inspection process. European Educational Research Journal, 15(1) pp.69-81

Prinsloo, P. 2020. Data frontiers and frontiers of power in (higher) education: a view of/from the Global South. Teaching in Higher Education, 25(4) pp.366-383

Williamson, B. Digital Education Governance: political analytics, performativity and accountability. Chapter 4 in Big Data in Education: The digital future of learning, policy and practice. Sage.

At work, we are beginning to go through our employee evaluation period and I want to explore the types of data that could be used to measure my performance. Standardized, quantifiable, and easily comparable data are prioritized in performance and accountability policies (Anagnostopoulos, 2013; Ozga, 2016; Williamson, 2017). I decided to use my Git commit history as a metric for my performance as it satisfies the three characteristics above. At the end of each day, I recorded the total number of additions, deletions, and files modified. Also, I did not record any contextual information about the modification as quantifiable data often requires the removal of supplemental context. Using this data, the following visualization was developed.

Like other types of performance and accountability data, the process of committing is susceptible to manipulation allowing the data to be quantified positively. For example, when I make substantial changes to a file I often duplicate the file and save it as filename_old while working on the file. If I do not finish the modifications right away, I will commit both files to the repository, therefore artificially increasing the number of additions.

By reducing the history to quantifiable data and removing contextual information can potentially result in misleading conclusions as it says little about the quality of the code written, completeness of the project (which can often be difficult to assess), and the time needed to troubleshoot, find solutions, and test/play around with code. This could lead to a culture of incentivizing poorly written and purposely lengthed code, which may not be noticeable to the administrators responsible in implementing these types of policies.

Anagnostopoulos, D., Rutledge, S.A. & Jacobsen, R. 2013. Conclusion: The Infrastructure of Accountability: Tensions, Implications and Concluding Thoughts. In, Anagnostopoulos, D., Rutledge, S.A. & Jacobsen, R. (Eds) The Infrastructure of Accountability: Data use and the transformation of American education.

Ozga, J. 2016. Trust in numbers? Digital Education Governance and the inspection process. European Educational Research Journal, 15(1) pp.69-81

Williamson, B. Digital Education Governance: political analytics, performativity and accountability. Chapter 4 in Big Data in Education: The digital future of learning, policy and practice. Sage.

This week I returned to discord to track my conversations at work. During the first Discord data tracking activity I focused on recording who sent a message, when a message was sent, what type of space a message was going to (private or “public”), and how I viewed a message. For this activity I focused on the context of the messages and noted if a message was (i.e. could be perceived as) “on” or “off” task; if the message contained a question, a file, or an image; and if the message received emojis. Using this data, the following visualization was developed.

The shift to remote working has intensified and accelerated the use of surveillance software by employers on their employees and, for many, the boundaries between personal and professional lives have been blurred, if not destroyed. Employee surveillance software isn’t necessarily new, keyloggers and web traffic monitoring have been implemented in offices for quite some time. Software powered by AI are quickly being adopted and advertised as able to provide deeper insights into employees mental states and satisfaction and can be use to schedule check-in/intervention meetings, identify areas of improvement (and success), or mange employee workloads. Data such as Discord messages is a prime target for these surveillance tools as bots can be easily integrated into the platform to monitor employee communications real-time.

There are significant limitations, however, and language is complicated and nuanced which often leads to misinterpretations/false conclusions. For example, last year a chess podcast was automatically removed by the YouTube auto-moderator due to the frequency of “black”, “white”, “attack”, and “dominates” and the algorithm interpreting the conversation as potentially racist. In a professional setting misinterpretations such as these could have significantly more radical consequences.

Within the realm of education, similar language monitoring software has been suggested to assist instructors in their grading and feedback of student students writing skills (argument, vocabulary, syntax, style, etc) and develop personalized curriculum. Educational organizations such as ETS have software ready for students and educators to evaluate student write and language learning. Similar software could be integrated into discussion forums to analyze student questions and comments and provide “feedback” on student thinking, depth of understanding, and levels of engagement and satisfaction. These metrics, alongside other forms of assessment, could be used to set standards or policy and evaluate teachers. Additionally, and this could be a bit too orwellian, lectures given by teachers and student engagement levels (combination or web traffic and geolocationing) could become analyzed real-time and used as teacher performance evaluations (or monitor the pace, difficulty, or content of the course).

Last week we administered our lab skills assessments (LSA) through our in-house assessment platform. While this is our fifth time administering our LSAs virtually, I thought it would be interesting to see the types of inquiries our teaching assistants raise during assessment periods and what the response time was to begin resolve those inquiries from the teaching lab staff. The response time was recorded using the message timestamp from Slack. A few notes were recorded about the context of the message and categories were developed while drafting the visualization.

Upon quick glance, the visualization demonstrates that the lab staff are generally quick to respond to inquiries from teaching assistants and nearly half of inquiries were to ensure a student properly submitted their assessment. For many inquiries, a response from the lab staff included a resolution/answer allowing teaching assistants to relay the information quicker. However, as the visualization below indicates, the time until resolution was generally longer than the initial response time, which makes sense as additional information may be needed and conversation between multiple people emerges.

This data also highlights the areas where the implementation of the assessments could improve, and consequently influence departmental or internal lab staff policy. For example, nearly a quarter of inquiries stemmed from the shared campus server our platform is hosted on reaching our maximum allocated connections. This information could be used to justify allocating funds to purchase a dedicated server for our platform providing more control and reliability for our assessment platform. Similarly, inquiries identifying question mistakes or clarifying question wording could be reduced by requiring additional reviews from the lab staff – a policy that has already been decided on for the next term.

Education has increasingly become datafield – reducing and quantifying complex learning processes and teacher-student and student-student relationships. Student behavior and performance are measured and analyzied, marketed as insights unobtainable without the data (Knox, 2020). These insights are often presented to educators in the form of learning dashboards where student learning is summarized and personalized learning paths (or other interventions) are recommended (Brown, 2020).

Who decides what aspects of student learning is datafied and how that data is collected, analyzed, and visualized is, unfortunately, not decided by every instructor in every classroom but by the developers behind the algorithms of the learning dashboard (Raffaghelli, 2020; van Dijck, 2018). Consequently, instructors can be uncertain on how data was collected and what meaning can be drawn from it (Brown, 2020). This is especially concerning as predictive capabilities of these systems could “radically affect” a students educational career (Williamson, 2020) and educators need to possess a critical understanding of the reductive and instrumental nature of data (Raffaghelli, 2020).

Dashboards direct an instructors attention to specific areas that are perceived by the developers as ‘learning’ (Williamson, 2020; Raffaghelli, 2020) possibly distracting the instructor from other aspects of their students learning. Both Williamson (2020) and Brown (2020) note that instructors may also use these dashboards as a way to classify and categorize students to give targeted interventions, but this could possibly lead to preferential treatment (or dismissal) of certain classes of students. Algorithmic culture already reinforces the digital inequalities which may be unknown to systems dependent on these algorithms.

Instructors, too, are becoming subjected to these datafied systems. Measures of student performance are often treated as proxy measures of instructor performance (Williamson, 2020). Good instructors, then, are the producers of ‘good’ student data. Data becomes the focus and instructors urge students toward particular outcomes rather than on the learning process itself (Bulger, 2016). Instructors become to know themselves and their practice through data (Harrison, 2020).

Subjecting instructors to these datafied systems risks their autonomy as there pedagogy may be (forcibly) reshaped to be dashboard friendly (Williamson, 2020). Interestingly, Brown (2020) comments that the instructors he was surveying did not rely on the dashboard to plan their teaching. Rather, they could not identify productive strategies to incorporate them into their teaching and relegating the dashboards to a glorified polling system and an identification tool for unfamiliar students (ibid). Similarly, the role of the instructor is shifting – personalized intervention and assessing students are outsourced to the algorithm (and developers) and the teacher is assuming the role of a dashboard monitor (van Dijck, 2018).

Bulger, M. 2016. Personalized Learning: The Conversations We’re Not Having. Data & Society working paper. Available: https://datasociety.net/pubs/ecl/PersonalizedLearning_primer_2016.pdf

Brown, M. 2020. Seeing students at scale: how faculty in large lecture courses act upon learning analytics dashboard data. Teaching in Higher Education. 25(4), pp. 384-400

Harrison, M.J., Davies, C., Bell, H., Goodley, C., Fox, S & Downing, B. 2020. (Un)teaching the ‘datafied student subject’: perspectives from an education-based masters in an English university, Teaching in Higher Education, 25:4, 401-417, DOI: 10.1080/13562517.2019.1698541

Raffaghelli, J.E. & Stewart, B. 2020. Centering complexity in ‘educators’ data literacy’ to support future practices in faculty development: a systematic review of the literature, Teaching in Higher Education, 25:4, 435-455, DOI: 10.1080/13562517.2019.1696301

van Dijck, J., Poell, T., & de Waal, M. 2018. Chapter 6: Education, In The Platform Society, Oxford University Press

Williamson, B. Bayne, S. Shay, S. 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in Higher Education. 25(4), pp. 351-365.

Methodology

For the final week of the teaching block, I recorded the software/applications that I used each day to investigate which platforms/tech ecosystems I use regularly, reflect on those that I could not separate from, and identify ways I could make the apps that I use more “efficient”.

Each day, I recorded the apps that I was opening or focusing and whether the app is free, open-source, and/or I use the app primarily for work or school activities. The resulting visualization was inspired from the Dear Data “Week 50 – A week of our phones” postcard by Giorgia Lupi.

Discussion

From the visualization, I primarily use applications/services from Google and Microsoft – nearly half of these all applications and half of the applications that I used everyday were from these two companies. However, this representation ignores some of the hidden infrastructure behind the other applications:

• Amazon web services (AWS), Amazon’s on-demand cloud computing platform, dominates the web server industry controlling roughly 40 percent of the cloud market. Disney+, Reddit, Spotify, Zoom, Facebook, Twitch, LinkedIn, Instagram all depend on the AWS and other apps like Moodle and WordPress having the ability to be hosted on AWS.
• Chromium is a free and open-source browser developed initially by Google (through the Chromium project). Electron is a software framework that depends on the chromium browser to render desktop GUI applications such as visual studio code, WordPress desktop, Slack, Discord, and Teams.
• React and Angular are two popular open-source web application frameworks maintained by Facebook and Google, respectively. As such, many website and applications built in Electron (such as Discord) use one of these frameworks.

This all goes to show that the “Big Five” (van Dijck, 2018) influence more web services/applications that what is immediately obvious. It is not hard to imagine the ease activity from a group of seemingly unrelated websites/applications could be collected and analyzed to give some holistic insights to our “data doubles” (Williamson, 2020) for monetization purposes. van Dijck (2018) highlights learning data has become increasingly valuable to continue to complete our “data double”. Promises of personalization and democratization of education distract from the concerns of privacy, security, and commodification of student data and the potential of increased surveillance.

Educational platforms rely on these promises of personalization and democratization of education and are quickly being dominated by the “Big Five”. These platforms are impacting teaching and learning practices and autonomy and pose a risk of developing a “one-size-fits-all” approach to education (van Dijck, 2018) that is globalized and potentially lacks local and cultural values. Creating free and open-source educational material, platforms, and data has been offered as a way to democratizing education and push back against corporate platformization. Unfortunately, these initiatives are often quite costly and require time and expertise to develop.

van Dijck, J., Poell, T., & de Waal, M. 2018. Chapter 6: Education, In The Platform Society, Oxford University Press

Williamson, B. Bayne, S. Shay, S. 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in Higher Education. 25(4), pp. 351-365.

Methodology

Over the past week I recorded websites that I accessed that were UCR affiliated (ending in @ucr.edu; except for one of our two email accounts hosted by Gmail). At each site, I categorized the website, recorded the time the page finished loading, how I referred to the site (typed into url, link from another site, etc), and when I left the page. I also recorded if I was required to enter in a two-factor authentication password, when I connected to the campus VPN, and if I downloaded any files.

The majority of my day-to-day work is web development and we are undergoing major version updates. Subsequently I access our git repository, university databases, and test versions of our projects many times a day. I did not record this data as it would quickly become large in volume. This is a limitation for this methodology and the resulting visualization.

Discussion

In this activity, engagement was measured as visiting a website and there is little information about what I did on the site (other than total time spent and downloading files). As Brown (2020) notes, instructors are often faced with similar data and instructors often (1) assume that students who accessed the LMS regularly were to be more familiar with course rubrics, deadlines, and materials and (2) change their interactions with students based on engagement with the LMS. The latter is especially concerning as these interactions (or lack of) could “radically affect” the future of a student (Williamson, 2020).

In collecting data and developing the visualizations I omitted some data that would lead to “false positives”, e.g. duplicate two-factor authentication messages, refreshing the same page, accidentally downloading multiple files. While these false positives may be identified and algorithmically removed from learning analytic data sets, other types of false positives might not be – Brown (2020) recalls viewing students take multiple clickers out to presumably also give their friends attendance points. This raises concerns of the quality, validity, and trustworthiness of these data sets. Measures such as student attendance and student engagement are often also treated as proxy measures for instructor performance (Williamson, 2020). Instructor B from Brown (2020) notes their concern for the end of year evaluations in conjunction with a review of student data, namely attendance data, as they do not require attendance. Could these datafied performance evaluations lead to changing instructors changing policies to ensure favorable performance evaluations?

Brown, M. 2020. Seeing students at scale: how faculty in large lecture courses act upon learning analytics dashboard data. Teaching in Higher Education. 25(4), pp. 384-400

Williamson, B. Bayne, S. Shay, S. 2020. The datafication of teaching in Higher Education: critical issues and perspectives. Teaching in Higher Education. 25(4), pp. 351-365.

To start the new block, ‘Teaching’ with Data, I decided to collect my drinking habits throughout the week. For this block I will also try to transform the hand-drawn visualizations into a more “traditional” data analytics dashboard.

Methodology

Throughout the week I recorded what I drank, whether or not I purchased the beverages from a restaurant/cafe, whether I also had a snack/meal with the beverage, and a non-numerical consumption amount (e.g. a few sips, a large tea, a glass of juice, etc).

I did not fully record what I drank on Thursday – the last entry recorded is the water I had with dinner. I noticed this error Friday morning and did not append the data since I had not recorded it on Thursday. This is an attempt at modeling tracking applications as many do not allow you to append the data yourself either. This hints to the fact that data sets are often representative and limited.

The data for the week resulted in the following visualizations.

Design and Discussion

The dashboard view was developed using Metabase (dependent on a local database [SQLite]) and Adobe XD. My hand-drawn visualizations are drawn using GoodNotes on my iPad.

In designing both visualizations, I found the most difficult aspect to convey was the amount consumed. Since I did not record a numerical value, I had to estimate one for each amount, which carries some assumptions: (1) I drank the entire bottled or purchased beverage, (2) I follow volume averages for “sips” and “gulps”, and (3) the initial volume was consistent across non-bottled beverages. For the hand-drawn visualization, I settled for the dashed line system above the drink category, where: single line is 4-9 fl oz, a double line is 10-15 fl oz, and a triple line is 16 and more fl oz.

For the dashboard view, I was limited not only by the lack of precise numerical values, but also the customization capabilities of the software and the translation of data into a database table. Consequently, I decided to use the number of dashes as the amount consumed. I also kept the system relatively simple – a single table with columns: drink (text), purchased (boolean), amount (integer), food (boolean), and day (text). In a more expansive system (and more robust database queries), some of the limitations, but not all, of the customization options could be overcome. However, in the classroom teachers may not have access to modify or view the structure of the backend. This restriction hinders a true and complete understanding of how the visualization is developed and what conclusions can be drawn.

Over these past few weeks, I have attempted to quantized some aspect of my daily activity to explore the relationship between data and their ‘insights’ within the context of learning.

I started the block thinking it would be an easy task – to record data and visualize it; but, I quickly realized that it would not be. Each week I faced a unique challenge of what and how to record data for the particular activity. In deciding the what, metadata was often omitted or reduced/regrouped as the week progressed, especially while drafting visualizations. Here we can begin to understand data as being partial and selective according to the context of its collection and end use (Michael, 2016; Williamson, 2017). As Kichin states, as cited by Williamson (2017), the term ‘capta’ should be used rather than ‘data’ as data are “inherently partial, selective and representative, and the distinguishing criteria used in their capture has consequences”.

The visualization of the data has an impact of what conclusions can be drawn. For example, the design of my first visualization, a week of walking, a chat log was emulated and messages were grouped temporally. Consequently, some of metadata that I would have like to have included was omitted to keep the visualization easy to read. Rather than grouping messages temporally, I could have grouped them by category (e.g. code, simulations, lab manuals, etc) or by sender. Breaking away from the chat log style, a series of pie charts or scatter plots could have been developed where there would have been more of a numerical focus. Or, all of these visualizations could have been developed to provide a range of perspectives of the same data set in the hopes to reveal a range of insights that might not be obtainable without the collection and analysis of the data, which is an aim of learning analytics (Knox, 2020).

In regards to the numerial focus, which seems to be the trend in the collection of student data, Bulger (2016) warns that students urged towards a quantified outcome will focus more on reaching that value rather than on the process of learning itself. While I agree with Bulger, I find this to not be unique to personalized learning analytics, but rather to most educational practices where grades are assigned, which is a form of reducing learning to a data point.

Target values are often predetermined and students may vary in their approach to a specific task subsequently leading to meeting, not meeting, or surpassing these values and lead to false positive or negative outcomes (Bulger, 2016). Knox (2020) highlights a new form of hypernudge platforms that ‘nudge’ students into these predetermined values. In these systems of realiging students to predetermined values or trajectories, student agency is attacked (Bulger, 2016; Tsai, 2020).

Throughout the data collect process I was the subject and data recorder and had a deep understand to the data that was collected, but unfortunately this is not the norm. As Tsai (2020) notes, the lack of full transparency of data and algorithms can lead to a distrust of the analytics and can further remove student agency preventing students from directly challenging the precision of the analytics. How data is collected, used, manipulated, and shared needs to be transparent and open to interrogation from educators and students – the ‘black box’ needs to be opened (ibid).

Bulger, M., 2016. Personalized learning: The conversations we’re not having. Data and Society, 22(1), pp.1-29.

Knox, J., Williamson, B. and Bayne, S., 2020. Machine behaviourism: Future visions of ‘learnification’ and ‘datafication’ across humans and digital technologies. Learning, Media and Technology, 45(1), pp.31-45.

Michael, M. and Lupton, D., 2016. Toward a manifesto for the ‘public understanding of big data’. Public Understanding of Science, 25(1), pp.104-116.

Tsai, Y.S., Perrotta, C. and Gašević, D., 2020. Empowering learners with personalised learning approaches? Agency, equity and transparency in the context of learning analytics. Assessment & Evaluation in Higher Education, 45(4), pp.554-567.

Williamson, B., 2017. Big data in education: The digital future of learning, policy and practice. Sage.