ݺߣ

ݺߣShare a Scribd company logo
ALTERNATIVE METRICS.

           THE END OF THE IMPACT
           FACTOR AS WE KNOW IT?
#APE2013             http://about.me/martijnroelandse
Who am I?
2                                                                                                  #APE2013

    2000 C 2003      PhD Neurobiology          Friedrich Miescher Institute, Basel


    2003 C 2005      Postdoc                   Centre for Neurogenomics and Cognitive Research, Amsterdam


    2005 C 2008      Postdoc                   Netherlands Institute for Neuroscience, Amsterdam


    2008 C 2010      Associate publisher B2B   Springer Media BV, Houten


    2010 - current   Publishing Editor STM     Springer Science+Business Media BV, Dordrecht
Research output per country
           3                                                                                                                                                                          #APE2013




           7                                                                         2.0                                                                   40
Millions




                                                                          Millions




                                                                                                                                                Millions
           6
           5                                                                         1.5                                                                   30
           4
                                                           2005                      1.0                                          2006                     20                                           2006
           3
           2                                               2009                                                                   2010                                                                  2010
                                                                                     0.5                                                                   10
           1
           0                                                                         0.0                                                                    0
                        researchers                                                                 articles                                                             citations



      Sources: OECD MSTI: all population data 2010, all research data 2009, all GERD data 2010 except Germany (2009), with extrapolation where appropriate and where World totals are the sum of
     data for all countries with available data. WIPO Statistics Database: all patents data 2009. Scival Spotlight: all Competencies data 2010. Scopus: all Articles, Citations and Highly-cited articles data
                                                                                    2010. ScienceDirect: all Usage data 2010.
The rise of the mega-journals
4                                                                             #APE2013
                                             #Articles
    -   Launched June 2006                    12000
                                              10000
    -   Biology and Medicine                   8000


    -   Rejection rate: 15%
                                               6000
                                               4000

    -   Jan 2012: Article 30.000 published     2000
                                                  0

    -   2010 Impact Factor: 4.351                        2007   2008   2009   2010   2011
#APE2013



5   Impact Factor

    Once published, the impact of their contribution to science is
    mostly assessed on the basis of out-of-date mechanisms
    such as the impact factor. However, the actual influence of
    their contribution on the journal's performance will only be
    visible for after another 2-3 years.

    A = the number of times that articles published in 2006 and 2007 were cited by indexed journals during 2008.
    B = the total number of "citable items" published by that journal in 2006 and 2007

    2008 impact factor = A/B.
#APE2013



6   Which article made a bigger impact?
    ?   Article published in a top-tier journal with 0
        citations after 2 years
    ?   Article published in a lower impact journal
        with tens of citations
Research dissemination channels have changed
7                                                               #APE2013


             ?   Scholarly citations
             ?   Non-scholarly citation
                 ?   News coverage
                 ?   Twitter, Facebook, Google+
                 ?   Blogs, Wikipedia
             ?   Post-publication recommendations
                 ?   Faculty of 1000
                 ?   Mendeley, ResearchGate, Academia.edu, Papers
Growth of non-scholarly citations
8                                                                                                            #APE2013

             10
                        7
                                      4.4             3.7                                          4.37
              5
                                                                    2.63

         %    0
                   Facebook         twitter      Pinterest         Reddit        Q&A sites        LinkedIn
              -5
                            average monthly change, 1st Sep - 1st Dec '12
             -10                                                                      -8




             200,000               187408


             150,000
                                                                            absolute numbers Sept '12.
             100,000

              50,000
                        24052
                                              2077          2195    471         833          73       174
                   0
                       Facebook twitter       Blogs    Google+ Pinterest Reddit Q&A sites LinkedIn
#APE2013



9   Which article made a bigger impact?
    ?   Article with tens of citations
    ?   Article widely discussed in the social web
    ?   Article with lots of downloads
#APE2013



10   Article Level Metrics
     Article-Level Metrics (ALMs, altmetrics, alternative metrics) are not
     just about citations and usage. The concept refers to a whole range
     of measures which might provide insight into impact or reach.
     Collectively as a suite, ALMs aims to measure research impact in a
     transparent and comprehensive manner.
ALM @ PLoS
11                                      #APE2013

     ?   Key importance:
         Findability, sharability, ci
         tability, comments
     ?   Real-time listing:
         ? Citations
         ? Usage

         ? Social Impact

         ? Post-pub review
           comments
Other ALM examples (I)
12                                                         #APE2013



                BioMed Central


                                                     IOP




                                 Nature
                                          HighWire
Other ALM examples (II)
13                             #APE2013
Scholarly vs non-scholarly citations
14                                                                                                                                                                          #APE2013



                       40
                                                                                                      Tweets can predict
                       35                                                                             highly cited articles
                       30                                                                           within the first 3 days of
                       25                                                                              article publication
               Cited




                       20
                       15
                       10
                        5
                        0
                               0           5          10          15          20          25          30           35          40          45
                                                                               Social


 Sources: Cited: Top 500 articles published in 2012 and cited in 2012 using Thompson Scientific Journal Citation Index for Springer journals in neuroscience. Social: Top 500 articles for Springer
journals in neuroscience mentioned in the social web using Altmetrics. All data 01/12/2012. Eysenbach G Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation
                                                         with Traditional Metrics of Scientific Impact J Med Internet Res 2011;13(4):e123
Concluding C Article Level Metrics
15                                                            #APE2013




               ?   A more accurate evaluation of scholarly
                   performance
               ?   Show dissemination of an article through
                   scholarly and non-scholarly communication
               ?   A new benchmark for
                   employers, funders, potential collaborators
               ?   Provide filters to select those articles most
                   relevant for their work
QUESTIONS?
#APE2013   http://about.me/martijnroelandse

More Related Content

Ape 2013 23012013

  • 1. ALTERNATIVE METRICS. THE END OF THE IMPACT FACTOR AS WE KNOW IT? #APE2013 http://about.me/martijnroelandse
  • 2. Who am I? 2 #APE2013 2000 C 2003 PhD Neurobiology Friedrich Miescher Institute, Basel 2003 C 2005 Postdoc Centre for Neurogenomics and Cognitive Research, Amsterdam 2005 C 2008 Postdoc Netherlands Institute for Neuroscience, Amsterdam 2008 C 2010 Associate publisher B2B Springer Media BV, Houten 2010 - current Publishing Editor STM Springer Science+Business Media BV, Dordrecht
  • 3. Research output per country 3 #APE2013 7 2.0 40 Millions Millions Millions 6 5 1.5 30 4 2005 1.0 2006 20 2006 3 2 2009 2010 2010 0.5 10 1 0 0.0 0 researchers articles citations Sources: OECD MSTI: all population data 2010, all research data 2009, all GERD data 2010 except Germany (2009), with extrapolation where appropriate and where World totals are the sum of data for all countries with available data. WIPO Statistics Database: all patents data 2009. Scival Spotlight: all Competencies data 2010. Scopus: all Articles, Citations and Highly-cited articles data 2010. ScienceDirect: all Usage data 2010.
  • 4. The rise of the mega-journals 4 #APE2013 #Articles - Launched June 2006 12000 10000 - Biology and Medicine 8000 - Rejection rate: 15% 6000 4000 - Jan 2012: Article 30.000 published 2000 0 - 2010 Impact Factor: 4.351 2007 2008 2009 2010 2011
  • 5. #APE2013 5 Impact Factor Once published, the impact of their contribution to science is mostly assessed on the basis of out-of-date mechanisms such as the impact factor. However, the actual influence of their contribution on the journal's performance will only be visible for after another 2-3 years. A = the number of times that articles published in 2006 and 2007 were cited by indexed journals during 2008. B = the total number of "citable items" published by that journal in 2006 and 2007 2008 impact factor = A/B.
  • 6. #APE2013 6 Which article made a bigger impact? ? Article published in a top-tier journal with 0 citations after 2 years ? Article published in a lower impact journal with tens of citations
  • 7. Research dissemination channels have changed 7 #APE2013 ? Scholarly citations ? Non-scholarly citation ? News coverage ? Twitter, Facebook, Google+ ? Blogs, Wikipedia ? Post-publication recommendations ? Faculty of 1000 ? Mendeley, ResearchGate, Academia.edu, Papers
  • 8. Growth of non-scholarly citations 8 #APE2013 10 7 4.4 3.7 4.37 5 2.63 % 0 Facebook twitter Pinterest Reddit Q&A sites LinkedIn -5 average monthly change, 1st Sep - 1st Dec '12 -10 -8 200,000 187408 150,000 absolute numbers Sept '12. 100,000 50,000 24052 2077 2195 471 833 73 174 0 Facebook twitter Blogs Google+ Pinterest Reddit Q&A sites LinkedIn
  • 9. #APE2013 9 Which article made a bigger impact? ? Article with tens of citations ? Article widely discussed in the social web ? Article with lots of downloads
  • 10. #APE2013 10 Article Level Metrics Article-Level Metrics (ALMs, altmetrics, alternative metrics) are not just about citations and usage. The concept refers to a whole range of measures which might provide insight into impact or reach. Collectively as a suite, ALMs aims to measure research impact in a transparent and comprehensive manner.
  • 11. ALM @ PLoS 11 #APE2013 ? Key importance: Findability, sharability, ci tability, comments ? Real-time listing: ? Citations ? Usage ? Social Impact ? Post-pub review comments
  • 12. Other ALM examples (I) 12 #APE2013 BioMed Central IOP Nature HighWire
  • 13. Other ALM examples (II) 13 #APE2013
  • 14. Scholarly vs non-scholarly citations 14 #APE2013 40 Tweets can predict 35 highly cited articles 30 within the first 3 days of 25 article publication Cited 20 15 10 5 0 0 5 10 15 20 25 30 35 40 45 Social Sources: Cited: Top 500 articles published in 2012 and cited in 2012 using Thompson Scientific Journal Citation Index for Springer journals in neuroscience. Social: Top 500 articles for Springer journals in neuroscience mentioned in the social web using Altmetrics. All data 01/12/2012. Eysenbach G Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact J Med Internet Res 2011;13(4):e123
  • 15. Concluding C Article Level Metrics 15 #APE2013 ? A more accurate evaluation of scholarly performance ? Show dissemination of an article through scholarly and non-scholarly communication ? A new benchmark for employers, funders, potential collaborators ? Provide filters to select those articles most relevant for their work
  • 16. QUESTIONS? #APE2013 http://about.me/martijnroelandse