ADVERTISEMENT
  About the SA Blog Network













Information Culture

Information Culture


Thoughts and analysis related to science information, data, publication and culture.
Information Culture HomeAboutContact

Discussion of scholarly information in research blogs

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



Discussion of scholarly information in research blogs

As some of you know,  Mike Thelwall, Judit Bar-Ilan (both are my dissertation advisors) and myself published an article called “Research Blogs and the Discussion of Scholarly Information” in PLoS One. Many people showed interest in the article, and I thought I’d write a “director’s commentary” post. Naturally, I’m saving all your tweets and blog posts for later research.

The Sample

We characterized 126 blogs with 135 authors from Researchblogging.Org (RB), an aggregator of blog posts dealing with peer-review research. Two over-achievers had two blogs each, and 11 blogs had two authors.

While our interest in research blogs started before we ever heard of RB, it was reading an article using RB that really kick-started the project.  Groth & Gurney (2010) wrote an article titled “Studying scientific discourse on the Web using bibliometrics: A chemistry blogging case study.” The article made for a fascinating read, because it applied bibliometric methods to blogs. Just like it says in the title, Groth & Gurney took the references from 295 blog posts about Chemistry and analyzed them the way one would analyze citations from peer-reviewed articles. They managed that because they used RB, which aggregates only posts by bloggers who take the time to formally cite their sources. Major drooling ensued at that point.  People citing in a scholarly manner out of their free will? It’s Christmas!

We didn’t choose RB because it was the best representation of science blogs in general; we chose it because of the structural citations the bloggers used. The blog posts aggregated by RB made for excellent citation-meme carriers. As Prof. Bar-Ilan says it, RB posts are the “transition phase” between the citation in formal communication and the free-form writing of blogs. It’s a bibliometrical Archaeopteryx. On hindsight, we should have emphasized our interest in the blogations more (which is why this post is called “Discussion of scholarly information in research blogs” rather than the article’s actual name).

RB also had the advantage of human editors, who decide which of the applying blogs are aggregated, so we were spared weeding the pseudo-science and spam blogs out of our data. We narrowed the sample further by focusing on blogs that had at least 20 posts in RB until January 2011, to make sure we have enough blogations “fodder” for research, and that have only one or two authors. The last rule was placed to ensure the blogs had at least basic similarities, so we wouldn’t be comparing apples and oranges.

However, choosing RB also means we chose only the bloggers who use it, a self-selecting population. There could be – there are- many bloggers that use structural citations, but either don’t bother with RB or aren’t even aware of its existence. RB has its own biases, which are very visible when one looks at its tagging system: Biology has 28 subtags and Psychology has 21, while whole disciplines like History and Sociology are only subtags under “Social Sciences”.

Most blog-cited journals

Groth and Gurney have already found that Chemistry blog posts cite research from high-impact journals, but we had to make sure the same goes for other disciplines. We extracted blogations from the last five posts of each of the 126 blogs in the sample, and you can see the results here:

Science bloggers, like most scientists, cite journals with high impact factors. Science, Nature and PNAS are the highest-impact journals in the Journal Citation Reports (JCR) “Multidisciplinary” category. We suggest several explanations:

Bloggers cite what they know – almost 60% of the bloggers were affiliated with a research institute at the time of the study. Twenty-seven percent were graduate students, 32% had a PhD. Given these journals’ prestige in the academic world, it’s expected that bloggers read and cite them often. Just because we’re in the realm of blogging doesn’t mean the bloggers weren’t influenced by academic norms (given their use of citations, we’re pretty sure they were influenced by some…)

Another explanation is the media’s preference for high-impact journals (prestige peer-reviewed journals have “authority” which the media love). If bloggers want to comment about the coverage of articles in the media, they have to cite the same articles the media use.

Distribution of bloggers’ education levels

Distribution of bloggers’ education levels

 

Affiliation with an academic institute

However, PLoS One, which is the forth in the most-cited list, is only the 12th out of 86 journals in the JCR’s “Biology” category. It’s still in the first quartile of the Biology category, but its ranking and impact factor aren’t nearly as high as those of Nature, Science and PNAS. We can assume that PLoS One‘s popularity with research bloggers is because it’s one of the best-known open access journals, and it mostly publishes biology research (RB has a Life Science bias). Also, PLoS One is a huge journal (4,403 items published in 2009, in comparison with 866 and 897 in Nature and Science, respectively), so the chances of running into PLoS One blogations were statistically higher. Our results look very much like the Mendeley ranking (a popular academic bookmarking site).

Mendeley ranking

 

Who moved my data?

Data is always a tricky part in a Web research. When dealing with articles, citations from peer-reviewed journals and so forth, the data is more-or-less stable. It’s true that articles get retracted and journals change their names, but they do so in a manageable phase. My data, on the other hand, keep running away from me.  Geologists Anne Jefferson and Chris Rowan’s joint Twitter list for Highly Allochthonous went extinct, and all that’s left are the two bloggers’ individual accounts.  Ed Yong now has 19,598 Twitter followers instead of the mere 11,638 he had when the paper was written. By the time I get this post up, he’ll undoubtedly have a few more. Krystal D’costa moved her blog to the SciAm network. Even my blog, which is part of the sample, moved! The ever-changing nature of the Web gives information scientists a lot of grief, as well as makes it very hard to replicate results.

You may have noticed I didn’t discuss the gender issue here (only 22% of the blogs had a female author). I’m saving it to a future post about women in science blogging.

ResearchBlogging.org

 

References

Shema H, Bar-Ilan J, & Thelwall M (2012). Research Blogs and the Discussion of Scholarly Information. PloS one, 7 (5) PMID: 22606239

Groth, P., & Gurney, T. (2010). Studying Scientific Discourse on the Web Using
Bibliometrics: A Chemistry Blogging Case Study
. Proceedings of the WEbSci10: Extending the Frontiers of Society On-Line.

Hadas Shema About the Author: Hadas Shema is an Information Science graduate student at Bar-Ilan University, Israel. She studies the characteristics of online scientific discourse and is a member of the European Union’s Academic Careers Understood through Measurement and Norms (ACUMEN) project. Hadas tweets at @Hadas_Shema.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 7 Comments

Add Comment
  1. 1. naishd 4:18 am 05/22/2012

    Now that we have good data on who science-bloggers are, and on what they cite, I’m wondering how long it will be before the citation behaviour of bloggers becomes ‘important’ – after all, it’s blindingly obvious that a piece of research doesn’t just have to be cited in the printed technical literature before it becomes ‘significant’ or influential. Any thoughts on that?

    On a minor point, I’m curious what you meant when you described the two individuals with two blogs as “over-achievers”. I have both recent versions of my blog registered at some aggregators; that’s because the previous version (hosted by ScienceBlogs) still gets a large amount of traffic that I can’t ignore (this due, I think, to the fact that the SciAm platform where I am now doesn’t allow anything approaching a decent blogroll in the sidebar).

    Darren Naish (Tetrapod Zoology)

    Link to this
  2. 2. Hadas Shema in reply to Hadas Shema 5:24 am 05/22/2012

    Hi Darren,

    “I’m wondering how long it will be before the citation behaviour of bloggers becomes ‘important’ – after all, it’s blindingly obvious that a piece of research doesn’t just have to be cited in the printed technical literature before it becomes ‘significant’ or influential. Any thoughts on that?”

    I believe it’s discipline-dependent at the moment. I can afford to run a blog because it’s considered (more or less) a worthy, or at least interesting, activity in my field. It’s going
    to become important to everyone as soon as we can implement data from blogs in a respected scientific index. From that on, it will be an arms race between bibliometricians and scientists, just like with the Journal Impact Factor. However, I can’t tell you when this blogation index is going to be invented (we’re working on it!).

    “On a minor point, I’m curious what you meant when you described the two individuals with two blogs as “over-achievers”.”

    I was joking, of course:) I’m very jealous of people who manage to run two separate blogs (I can barely manage one!). I didn’t factor two versions of the same blog into the research, only different blogs. Two of the blogs were Brian Switek’s Laelaps and Dinosaur Tracking, and there was another blogger with two blogs who unfortunately I can’t remember right now.

    Hadas

    Link to this
  3. 3. naishd 7:05 am 05/22/2012

    Interesting comment. Naturally I’m interested in cases where blogging is seen as part of research output, and good luck with the blogation index!

    As for the over-achievers, remember that Brian at least makes a living from freelance writing. Research and other academic commitments mean that, for others among us, blog-writing has to be done in the small hours of the morning…

    Darren

    Link to this
  4. 4. eysenbach 10:06 am 05/22/2012

    “In conclusion, the sample’s science blogs share characteristics with other means of scientific discourse. We believe that tracking and recording this communication will become a part of future research evaluation metrics.”
    Too bad that you didn’t know or didn’t cite related work on microblogs “tweetations”, which are already used by at least one journal for research evaluation (dubbed “twimpact factor”).

    Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact
    J Med Internet Res 2011;13(4):e123
    URL: http://www.jmir.org/2011/4/e123/

    Link to this
  5. 5. Hadas Shema in reply to Hadas Shema 10:32 am 05/22/2012

    Well, this study didn’t focus on Twitter as a source of references, mostly because we had our hands full with the blogs already. However, future research will definitely go there, assuming we can extract Twitter data in a satisfying manner. I did read your research (and according to PubMed, Prof. Thelwall was one of the reviewers) and plan on incorporating it in future works.

    Hadas

    Link to this
  6. 6. IgorCarron 12:00 pm 05/22/2012

    And then there is the difficult issue of defining what a research blog is. It looks as though the definition of a research blog is the electronic equivalent to Scientific American, whereby discussions take place after peer review and publication. However, that definition seems awfully narrow.

    In particular, blogs in certain areas do not rely on peer review research, rather they write about papers that have popped up on researcher’s sites or an ArXiV, i.e. much before going through a journal peer review process. These blogs are therefore unsuitable for the ResearchBlogging network for instance. Hence the issue of research or science blogs mentioning journals is somewhat missing how research blogs are used as part of the scientific process itself: Yes, some of that begins to look like peer-review.

    Igor.
    http://nuit-blanche.blogspot.com

    Link to this
  7. 7. Hadas Shema in reply to Hadas Shema 4:57 am 05/23/2012

    I’ve always defined a research blog as “blog which deals with science culture”. Though RB is technically for peer-reviewed research, I’ve definitely seen there ArXiv papers, books, etc. The use of science blogs as part of the scientific journal is very interesting, but is more in the sociology of science territory than the bibliometrics territory.

    Hadas

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Dinosaurs

Get Total Access to our Digital Anthology

1,200 Articles

Order Now - Just $39! >

X

Email this Article

X