I was recently in an information briefing (which was very useful) about Web of Science and citations/searches. Here are some thoughts on how the system at present breeds conformity. Or at least, this is what I said, pretty much. very slightly odified to remove some names:
On Metrics as Tools
My concern – something I have discussed with a few others – is how there are some serious gaps in the Web of Science coverage for some areas of the social sciences and humanities. I wonder if you are interested in this discussion as well. I think there are a few important things to consider, or if they have been considered, make the thinking clear as to how they have been handled.
I work (and think) in a variety of different ways that sometimes seem to me to be specifically designed to fall between the cracks of the indexes. This started with noting that the journals I really admire, were not making it from ESCI to SSCI, or rather, some were even choosing not to. I don’t think I should say which ones, but a few I have had some reviewing or editorial exchange with have said they are pulling out of the indexing ‘game’ as metrics was both too blunt and too normative. There are also a few things, discussed especially, that were not being indexed. Smaller magazines for example, museum catalogues and artist books, visual research (I had taught ethnographic film for many years) and political pamphlets are falling by the way in the face of a normative centrifugal force.
The blunt version of the argument here is that the new Incites tools do not ‘incite’ enough – but rather encourage heading in the same direction that everyone else is heading in – collaborate with those who are most likely to collaborate with you, cite those who cite you, read those who read you etc. Sure, that perhaps has its merits in terms of group cohesion, but academic work should surely be, at one level at least, not about that at all. It is disagreement and difference we should seek, not everyone heading towards the same spiral of universal chanting “ISI ISI” as if a group of characters from a Thomas Pynchon novel had spring off the page in full riot gear. Doesn’t the tendency to seek out the most popular make it harder for new and novel ideas to get a hearing? At what point do the top citations, top metrics, top index procedures need to be disrupted by ideas might not even be recognised by ‘metrics’? Ideas that disrupt the play of uniformity, conformity, safety and repetition? Obviously, I am setting this out starkly to make the point clear, but I think there is a fundamental problem when we have 50 million papers that are there because, as you said, ‘we want to make the world a better place’ but some could argue that the world is demonstrably becoming less better, or at least a significant set of indicators would suggest that. maybe the 50 million need to not refer more and more to the centre, but seek more and more the alternative, angular, oblique and even opposite/oppositional ideas. Ahh, we are communist after all (though in communism there is also a tendency to centralisation, of course – as I said, overstating to make the point).
What mechanisms can be demonstrated within your presentation, or within the tools, that cater for the need to engage in a ‘ruthless criticism of everything’ as old beardo would have us do. The old man with a beard also saw himself as on the road to science, but that it was no easy path, there was work to be done. What could be entered into the search algorithms to ensure the critiques of normative and even hegemonic ideas in each area are challenged? What mechanisms in the search can be dysfunctional for the ongoing business model that is, frankly, no longer really fit for purpose in a degraded and entropic world…
I would love (ironic and hysterical laughter – cackle cackle hee hee hee) to see some explicit attention to how critical disruptive thinking could be built in as potions for the indexing process. I know indexing cannot be neutral, but can the biases run the other way sometimes? can you say how these questions might be addressed? And what great possibilities would be there if 100 flowers contended with 100 schools of thought in bloom…
Just to confirm that referentiality takes all kinds, my most often cited ISI works (ISI articles cited by ISI journals) show interesting trends. (All available on the download texts link in the sidebar).