Leading universities adopt Mendeley data to accelerate research analytics by 3 years

This week, leading academic institutions in North America, Europe, and Asia signed up to Mendeley’s new data dashboard, the Mendeley Institutional Edition. The dashboard analyses their research activity and impact on the global research community in real time – down from the 3-5 year time lag of the “Impact Factor”, the current gold standard for such evaluations. This allows academic institutions to react faster to their faculty’s research needs and provide them with quicker, more personalised support during the research process – thus accelerating the pace of scientific discovery for all of us.

Readership statitics in Mendeley Institutional EditionThe Impact Factor, a measure of the number of citations an academic journal receives, is a pivotal metric of science: Academics have to publish in high-Impact Factor journals to receive promotions, tenure, or grant funding, and universities allocate their million-dollar library budgets to those same high-Impact Factor journals. This is despite the Impact Factor’s many known flaws – the most limiting of which is that the citations it is based on take 3-5 years to accumulate.

This week’s release of Mendeley’s Institutional Edition, distributed by leading Dutch library subscriptions agent Swets, brings research impact measurement to real-time speed, while also providing more granular and social metrics of how academic research is consumed, discussed, and annotated. It allows research institutions to see detailed analytics of the journals their academics are reading, the journals they are publishing in, and how many readers those publications have. This data is built on Mendeley’s global research community of more than 1.8 million academics who are using the startup’s tools for document management, discovery, and collaboration.

The first customers of Mendeley’s data dashboard are premier international research institutions: Two prominent universities on the East Coast and in the Bay Area, the University of Pittsburgh, the University of Western Ontario, the University of Nevada, Reno, the VTT Technical Research Centre of Finland, the Korea Advanced Institute of Science and Technology, and the Agriculture, Forestry and Fisheries Research Council Japan.

Speaking of the announcement, Dr. Tod Colegrove, Head of DeLaMare Science & Engineering Library at University of Nevada, Reno, said: “Rather than spending vast amounts of staff resources attempting to quantify usage of existing library resources – remaining largely unaware of past and present use outside of the library’s current subscribed offerings – Mendeley offers a unique and immediate lens into the library’s researchers’ information behaviours. Purchase decisions can be informed directly by past and present actual use of potential library resources, rather than being left to the increasingly less relevant the-way-we’ve-always-done–it model of serials management.” His colleague Lisa Kurt added: “The collaborative nature of Mendeley is a game changer for our institution where departments and colleges are working to break through their silos and focus on the best parts of the work they do. Mendeley is solving a very real problem in a rather elegant way.”

At the University of Western Ontario, Head Librarian Joyce Garnett commented: “Western Libraries is proud to be an early adopter of Mendeley Institutional Edition, a significant addition to our digital toolbox. It will facilitate citation management for individual researchers, collaboration for research groups, and, through its analytics capacity, enable librarians to assess the relevance and use of our collections. Mendeley is unique, growing its database organically through the choices and preferences of researchers as they create and disseminate new knowledge.”

In a bid to develop alternatives to the Impact Factor, new research metrics startups such as altmetric.com and total-impact.org have already turned to Mendeley’s readership data, and several peer-reviewed studies have recently highlighted its positive correlation with the Impact Factor. Dr. Victor Henning, CEO and co-founder of Mendeley, said: “I’m excited that after receiving scientific validation from the research community, our data is now helping some of the world’s best universities work more efficiently and get to life-changing discoveries faster. My inner nerd is going: Wow, this is freaking amazing.”

Mendeley Institutional Edition screenshots: http://www.flickr.com/photos/mendeley/sets/72157630651813190/

What flavor is your research? Take our survey on grant review and tenure decision making.

Mendeley is emerging as a leading source of data on how ideas spread and which academics are the most widely read and influential in their respective fields. At Altmetrics12, a gathering of the leading researchers studying how social networks and the web are changing research, several researchers presented papers examining how Mendeley’s readership data compares with traditional research. This research provides independent third-party validation of Mendeley’s research stats and enables developers to create discovery tools to service the needs of many different types of research consumers. How do you use altmetrics? Take our survey!Read More »

Tracking Scholarly Impact on the Social Web: An #Altmetrics Workshop

New ways of getting your work noticed via the web has been a very frequent topic of our posts here. We’ve written about raising your online visibility, making your work more discoverable, and many other aspects of getting your work noticed online. That’s why it makes me very happy to announce that a workshop is being convened to discuss these very topics. At the 3rd International Conference on Web Science (14-15 June), a workshop on tracking scholarly impact on the social web has been organized. Read the post for more details.Read More »

Another researcher index? ReaderMeter looks to answer with Mendeley

Editor’s note: This is a guest crosspost from Dario Taraborelli who created the ReaderMeter application on top of the Mendeley API. Dario also blogs at Academic Productivity. He asked if he could talk a bit about citation metrics. Over to Dario….

Readers of this blog are not new to my ramblings on soft peer review, social metrics and post-publication impact measures:

  • how can we measure the impact of scientific research based on usage data from collaborative annotation systems, social bookmarking services and social media?
  • should we expect major discrepancies between citation-based and readership-based impact measures?
  • are online reference management systems more robust a data source to study scholarly readership than traditional usage factors (e.g. downloads, clickthrough rates etc.)?

Read More »

The top 10 journal articles published in 2009 by readership on Mendeley

Having seen a lot of ‘top 10 lists of 2009’ around, we thought we’d throw in our two cents and give you the top 10 most read articles on Mendeley, published in 2009!

The top paper for 2009 was written by Uri Alon, entitled: ‘How to choose a good scientific problem’, published in the journal “Molecular Cell.” Our stats tell us that there are currently 74 Mendeley users who have read this paper, even though it was only published in late 2009.

The full list of the top ten articles published in 2009 on Mendeley (as of 28th January 2010) is:

1. Uri Alon, ‘How to choose a good scientific problem’, Molecular Cell (2009), Volume: 35, Issue: 6

2. Castro Neto et al, ‘The electronic properties of graphene’, Reviews of Modern Physics (2009), Volume: 81, Issue: 1

3. Erez Lieberman-Aiden et al, ‘Comprehensive mapping of long-range interactions reveals folding principles of the human genome’, Science (2009), Volume: 326, Issue: 5950

4. Ed Bullmore & Olaf Sporns, ‘Complex brain networks: graph theoretical analysis of structural and functional systems’, Nature Reviews Neuroscience (2009), Volume: 10, Issue: 3

5. Zhong Wang, Mark Gerstein, Michael Snyder, ‘RNA-Seq: a revolutionary tool for transcriptomics’, Nature Reviews Genetics (2009), Volume: 10, Issue: 1

6. Development Core Team, ‘R: A Language and Environment for Statistical Computing’, R Foundation for Statistical Computing (2009) Volume: 2, Issue: 09/18/2009

7. Fatih Ozsolak et al, ‘Direct RNA sequencing’, Nature, Volume: 461, Issue: 7265

8. Benjamin M Bolker et al, ‘Generalized linear mixed models: a practical guide for ecology and evolution’, Trends in Ecology & Evolution (2009), Volume: 24, Issue: 3

9. Michael Schmidt & Hod Lipson, ‘Distilling free-form natural laws from experimental data’, Science (2009), Volume: 324, Issue: 592

10. Stephen J Eglen, ‘A quick guide to teaching R programming to computational biology students’, PLoS Computational Biology (2009), Volume: 5, Issue: 8

We’d like to point out that this isn’t an authoritative list of all the ‘most read articles for 2009’. Instead, these are the ones that appear in Mendeley user libraries and show some early indications of the popularity of a journal article. We will also track the evolution of these stats over the course of 2010.

Readership complementing the impact factor

With Mendeley’s growing user base, the readership count can complement other measures, such as citation metrics, adding an extra dimension to assessing a journal article’s impact.

For example, the article “How to choose a good scientific problem” is a general interest article, rather than being specific to biology which suggests it is not likely to have a high citation count in future primary research literature.

Nonetheless, it is already the most read paper on Mendeley published in 2009, a factor that would otherwise be missed. This indicates that the readership count can allude to other ways in which articles are used within a community, and therefore increase awareness of what should be read. The next step will be to anonymously track reading time and quality rating metrics to gather the most accurate data possible for our upcoming personalized recommendation engine.

Predicting research trends?

Understanding and predicting research trends is an important part of research. The citation count, used for decades as the gold standard in article-level metrics, can verify broad trends occurring within academic disciplines such as biology. While quite accurate, official citation metrics take two years to calculate. In contrast, readership statistics may be able to predict similar trends in real-time.

For example, look at The Scientist’s list of the hottest biology papers in 2009 (all published in 2007). The readership count for these papers on Mendeley correlates with ISI’s citation count at r=.76 (two-tailed, p=.13 due to the low sample size) – a near perfect correlation, even if only based on five papers and our userbase of just over 100,000 users:

Comparison of Mendeley’s most read papers with the ISI Citations

Publication ISI Citations Readers on Mendeley
A M. Werning, et al., “In vitro reprogramming of fibroblasts into a pluripotent ES-cell-like state,” Nature 448: 318-24, 2007. 512 26
E. Birney, et al., “Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project,” Nature 447: 799-816, 2007. 618 63
A. Barski, et al., “High-resolution profiling of histone methylations in the human genome,” Cell 129: 823-37, 2007. 560 33
K.A. Frazer, et al., “A second generation human haplotype map of over 3.1 million SNPs,” Nature 449: 854-61, 2007. 588 46
K. Takahashi, et al., “Induction of pluripotent stem cells from adult human fibroblasts by defined factors,” Cell 131: 861-72, 2007. 886 64
_______________________

Pearson Correlation r = .76

We look forward to comparing the top 10 list shown above to the official ISI citation metrics for 2009 publications when they are calculated and released later in 2010 or 2011.

In summary, using Mendeley’s readership figures alongside the citation metrics should make it possible in the future to evaluate the scope of a journal article within the community more effectively. Finally, further refinements and understanding of readership metrics might make it possible to identify the next big trend in the academic world.

Methodology

The top 10 list was made by noting how many times a paper appears in the libraries of individual Mendeley users (readership count) and how many distinct user tags were attributed to that paper (tag count), then we filtered the results to include only papers from 2009 – done!

Ologeez Founder joins Mendeley / Changing the Journal Impact Factor

Exciting news: Jason Hoyt, the founder of Ologeez (a semantic frontend for PubMed), is joining Mendeley! Jason holds a Ph.D. in Genetics from Stanford University. At the moment, he is still based in Palo Alto, but once the visa issues are sorted out, Jason will be joining us here in London as our new Research Director. TechCrunch broke the story today with a headline that made our geek hearts beat faster, comparing us to a Klingon battle cruiser de-cloaking in London.

To get started, Jason wrote up his reasons for joining us, and how Mendeley can help change the Impact Factor. Over to him:

———————————

Changing the Journal Impact Factor

Right, so the first thing I had to ask myself was “Why on earth would I move from San Francisco, leaJason Hoytving behind a cushy life for London, and work for a reference management start-up?” Surely any rational person would find this a bit odd.

Well, I’m not going to answer by talking about how great the team is or how enthusiastic the founders are about improving research, which is certainly all true. Rather, let’s take a real-world example of how the “tech” behind Mendeley is already making a difference with how we view the impact factors of research.

Read More »