Challenges in altmetric data collection
2 likes864 views
This document discusses inconsistencies in altmetric data reported by different data providers and aggregators. It analyzes data from Altmetric.com, Lagotto, and Mendeley.com for 30,000 DOIs collected at the same time. Results found large differences between providers for metrics like Facebook, Twitter, and Reddit counts. Reasons for inconsistencies include using different methodologies, identifiers, definitions of metrics, and data collection processes. The document calls for best practices and standards to improve consistency and quality of altmetric data reporting.
Challenges in altmetric data collection
- 3. Data provider:? source?of metrics Data aggregators:? Aggregate?and?offer?/report??metrics Mendeley.com Altmetric.com Lagotto? Mendeley?API Dump?file open?source application? Common?sources?across?different?provider/aggregators? Consistency:?having?the?same?number?as?the?source?itself?when?metric collected?at the? same?time?for?a?same?DOI
- 4. Consistency?of?altmetrics?data?among?different? providers/aggregators?is?very?necessary!? 4 document
- 9. 9 Altmetric. com,?23% Lagotto,? 68% Mendeley .com,?69% Coverage?of?DOIs: Common?metrics?(for?overlapping?DOIs): 0 10000 20000 30000 40000 50000 60000 70000 80000 Mendeley?readers Facebook Twitter CiteUlike Reddit ?Lagotto Altmetric.com Mendeley.com
- 11. Result:?Facebook? Consistency: ? Exactly?the?same?result?as?last?year:?very?different? ? Different?way?of?collecting?and?reporting?FB?counts? Differences: Lagotto:?aggregates?FB?counts:?sum?of? shares+likes+posts+comments;?search?for?DOIs?via?FB? API Altmetric.com:?reports FB?public?posts?only;?track? links?to?find?DOIs/URLs;?there?are?complexities?in?the? Facebook?search?as?shortened?URLs?don't?always? show?up?in?the?searches
- 13. Result:?other?sources ? Reddit:?huge?differences Lagotto?Reddit?counts?=posts+comments? Altmetric.com?Reddit?counts?=?posts©\comments ? CiteULike:?some?differences ? Wikipedia:?not?analyze?jet?
- 14. What?are?the?possible?reasons?for?Inconsistency? ? Using?different?methodology/approach?in? collecting?&?processing?metrics? ? Using?different?identifiers?(DOI,?PMID,?arXiv?ID) ? Differences?in?reporting?metrics?(aggregated?vs.? raw?score/public?vs.?private?posts) ? Accessibility?issues?(for?resolving?DOIs;?cookies? problems,?access?denies)?differs?across?different? publishers ? Different?updates:?possible?time?lags?in?the?data? collection?or?updating?issues 14
- 15. What?are?the?challenges:? There?is?a?need?for?both: ? Best?practices? ? Guidelines?and?standards? 15
- 16. NISO?¡®altmetrics?data?quality¡¯?working?group:? Code?of?Conduct? NISO has initiated this group to develop a draft code of conduct for collection, processing, dissemination and reuse of altmetric data that can contribute to solve many of data problems issues. 16