The use of data instruments in performance measurement is all the rage. It seems that the continuous improvement in technology and its increasing availability has fundamentally transformed the way we think of performance-based governance mechanisms. Academia is neither spared nor silent about it. And, as it is usually the case with social scientists, the more visible or pressing something is in reality, the more it becomes an object of our curiosity and research.
Miguel Antonio Lim, a Lecturer in Education and International development at the School of Environment, Education, and Development at the University of Manchester, has recently published an article which deals with the growing use of data instruments in higher education. The article – “Governing Higher Education: The PURE Data System and the Management of the Bibliometric Self” (Higher Education Policy) – analyses the case of Elsevier’s PURE data capture system and the way its users – those working in higher education – engage with it.
In this interview, we ask Miguel to tell us a little bit more about the article and to share some of his thoughts regarding the role of data systems in academic work and management.
How would you summarise the main message of the article to someone working in academia but not especially familiar with digital infrastructures and data management systems?
I think the main message is that we should care about and be mindful of digital databases of “performance” and outputs at universities. It’s a message that is precisely meant for colleagues who aren’t familiar with these systems – and that probably refers to most people in academia. Most of us will have to engage with data management systems – it’s now practically impossible to have a web presence on university websites without giving up data about our various outputs and “performances”. We have to think about the possible repercussions of these systems. It’s not to say (and I know some will disagree) that these systems are bad in themselves – but they are tools and tools can be used in a variety of ways.
Data management systems are becoming increasingly more important for heads of departments and university managers. However, PURE seems to stand out from other such systems given that, as the article cautions, it carries a potential to become a “total” management system. What does this practically mean and what could be the implications?
I argue that PURE can become a total management system because it is an evolution beyond previous systems which were databases that just collected relatively straightforward information about published outputs. Although PURE is not the only example of new systems, it is now a widely used system that aims to track a much wider set of performance outputs. This means that it aims to capture a much wider variety of activities, including (but certainly not limited to) research management, prizes, various kinds of community service, among others.
With respect to publications, PURE also expands the amount of information it gathers because it allows users to input their publications at various stages of development. Even papers that are in relatively early stages can be entered as outputs in development. One of the features of PURE is that it can generate a CV based on the information you offer it – this is a good illustration of its aim to become what I call a total system. The implication being that PURE (and its users) could evaluate the “total” quality of an academic colleague just based on her PURE profile.
On a related note, I’d recommend that your readers take a look at Shoshanna Zuboff’s excellent new book The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. While it doesn’t directly deal with academics, it offers a pretty thoughtful reflection of where data is going as the new primary currency of what Zuboff cleverly calls surveillance capitalism.
You introduce the concept of “bibliometric self” which is inspired by the one of “quantified self”, introduced by Moore and Robinson in a 2016 issue of New Media and Society. Quantified self suggests that individuals, as you put it in the article, are “pushed to become a more productive version of themselves”. How is the bibliometric self different and what observations led you to suggest this new concept?
I coined the term bibliometric self because I thought it would capture the sense of academic life that research colleagues probably feel is the most “measured” aspect of their academic life. So in a sense it’s a specification of Moore and Robinson’s “quantified self”. Some of the early literature in this area looked at how health apps helped to shape the current desire among many to become the healthiest version of themselves by having the right body mass index, walk the right number of steps each day, eat the right number of calories. I must add that in some of their other research they highlight a specific understanding of the quantified self: the quantified self at work which is the self that I feel is being cultivated by academics in their professional lives. They try to be their best bibliometric selves. They try to publish a lot and try to publish work in the journals where they will become visible and cited by others.
In addition to being a researcher and lecturer, you are also Co-Research Coordinator and Impact Coordinator of the Manchester Institute of Education (University of Manchester). In your article you draw on your personal experience which is twofold: you are both an end-user – someone who “feeds” the system with data about your own academic achievements, and also someone who uses the data aggregated by PURE to inform the university decision making and governance. What would be the advantages and disadvantages of this double role you have?
There is a lot of advice about having a certain distance from your research object/field/data in order to maintain a level of objectivity. As many of your readers will probably be in the field of higher education studies, I think they will also realise that there are advantages with having a familiarity and understanding of the field that they analyse and seek to understand. The same could be said about this piece. I don’t claim objectivity in this piece (which some might read as a disadvantage). I do offer (what I think are) factual accounts of how PURE was developed and how it has been used.
And then I go on to use my familiarity with the system and the context in which it is used to reflect on how it could be further developed by research managers and what potential the system has to influence both users and evaluators. There are many critical studies about higher education but some are made without describing the actual (mundane) processes and mechanisms by which PURE and PURE data are used in decision-making. It will be people who have “dual roles” as you pointed out who could contribute a different – and hopefully fruitful – kind of critique.
How would you describe your experience with the journal and would you recommend Higher Education Policy to other early-career scholars?
I had a good experience with the reviewers at Higher Education Policy (HEP) and received some thoughtful feedback throughout the process. One aspect I would have considered more closely is that at the time I published it I assumed that there would be the possibly of negotiating open access to the article, given that my University has had some resources to facilitate open access and even has special “deals” with HEP’s publisher in this regard. However, I only discovered later on that HEP did not have an open access format. This means I’ve had to field a number of requests for access from colleagues for an author approved “green open access” manuscript. Of course it’s great to hear from people who really want to read my work – but I wish it wasn’t so difficult for them to access it in the first place.
Thank you so much, Miguel!
You can find the article by clicking on the following link:
Lim, M. A. (2019). Governing Higher Education: The PURE Data System and the Management of the Bibliometric Self. Higher Education Policy.
Interviewer: Jelena Brankovic
Jelena Brankovic is a postdoctoral researcher and lecturer at the Faculty of Sociology, Bielefeld University, Germany. These days she is busy trying to figure out what lies at the intersection of global fields, competition, and actorhood. She is one of the lead editors of the blog you are visiting right now. You can follow her on Twitter.