The Metrics Problem
The construction of multi-parameter indices to measure important but amorphous things like quality is an exciting and embryonic research field. It is still in its infancy for the simple reason that the choice of influencing factors (“indicators”) and the weight assigned to them is highly subjective. This has led to a tsunami of criticisms of indices like college and university rankings and the credit ratings provided by financial services companies such as Standard & Poor’s, Moody’s or Fitch. What has not been tried yet is a novel approach we have developed which we call Open Metrics.
Our Approach: Open Metrics
Open Metrics is more than just the provision of free and transparent, online open access to indices. Rather than attempting to guess what parameters should be included and weighted to construct indices to measure the quality, visibility or impact of items such as a publication, a peer-review or an author or reviewer for that matter, LIBRE crowd-sources the problem appealing to the collective wisdom of expert academics themselves. Letting the academic community add and subtract indicators and weigh them according to its needs is likely to result in more objective metrics. LIBRE offers a wide array of influential factors that scholars believe are relevant to scientific quality, together with initial weights that can be finely-tuned by users themselves. In parallel, LIBRE gives the user the option of ranking items by single parameters according to their numeric count. For example the user can sort publications by the number of positive reviews, or can explore a list of potential reviewers ranked by their community rating.
In a way, Open Metrics is a pioneering experiment that we invite you to participate in. On the other hand, we hope you will find the metrics a useful and more objective tool to allow you to perform your research more efficiently.