This marks a milestone on the progress of Open Science and the promotion of Reproducibility in Scientific Research. The wide adoption of the Insight Journal by the ITK community is evidence that progress is possible, and that a community of committed individuals can trigger and sustain change.
The idea of the Insight Journal originated from the initial experience of implementing many image analysis algorithms in ITK, and realizing how insufficient traditional journal publications are when it comes to specifying the software implementation of any algorithm. At the time, it was quite evident that something has been lost in the practice of scientific research: The fundamental rule by which a researcher should communicate the outcome of her research work by describing it in enough detail for anyone else to be able to replicate it. The popularization of software as a day-to-day tool in the practice of scientific research has not been followed by a corresponding modernization in the means of publishing scientific results. The standard publication venues were still: 10 pages papers at single space, two columns with black and white static images, and it took two years for them to go from submission to publication.
The Insight Journal was conceived as a quantum leap in technical and scientific publications, by asking the fundamental question of “What would you need in order to replicate the work described by the authors? “. When attempting to answer this question, very quickly it became evident that we needed: (a) all source code, (b) input data, and (c) parameters; used in the computation of the original work, and that all these materials should be made available to both reviewers and readers.
The Journal readily adopted, what was at the time the nascent Open Access movement, and chose to enable authors to keep the copyright of their papers, and license them for any one to use, under the terms of the Creative Commons by Attribution License.
The Journal also implemented an infrastructure for systematically testing the source code accompanying every submitted paper. Since every paper is expected to be submitted with source code, data and parameters, it becomes possible for the Insight Journal infrastructure to extract the source code from a paper submission, configure and build the code, and then proceed to run it on the input data that is also provided by the authors along with the paper. The results of the entire process are then posted online as a first review of the paper.
Another novelty of the Insight Journal was to revert the order of publication and review. That is, instead of waiting for a paper to be reviewed before it gets to be published, in the Insight Journal, the paper is first published online (within 24 hours after submission), and then it is open for public peer-review by the community. The reviews are public and non-anonymous, and evolve into a forum-like conversation. Authors are also allowed and encouraged to post revisions to their papers, in particular as a response to comments posted by the reviewers.
With the visionary support of the National Library of Medicine, in the form of A2D2 funding, the Insight Journal came online in 2005. The first fully open-access, reproducible article was published in July 2005. Now, six years later, the Insight Journal has reached 500 published articles. This represents an average of over 80 published papers a year.
With 2,480 registered users, and 819 community reviews, the Insight Journal has become an integral part of the ecosystem surrounding the Insight Toolkit. This is the direct mechanism for community members to share their developments with their peers. This level of adoption clearly shows that there is a community that truly appreciates the importance of reproducibility in the practice of scientific research, and understands that the main purpose of publishing is to share information with their peers, not to be used as a surrogate for measuring productivity.
Over the past five years, the Insight Journal has become a central place for open-access and has been hosting open-access articles for the MICCAI conferences and other events. We have also extended the Insight Journal to other open-source communities with the VTK Journal and are looking to extend it further.
Continuing on this success we have recently made several improvements to the Insight Journal. In particular we are trying to keep the website up to date with the latest technology and integrate authors and readers suggestions. We are looking forward to adding another 500 articles to the Journal.
The Insight Journal was made possible thanks to the visionary support Dr. Terry Yoo, at the National Library of Medicine via the following grants and support contracts:
- A2D2 Journal – 2005
- ITK Maintenance Contract 2006-2009 – N01-LM-4-3508
- ITK Maintenance Contract 2009-2010 – HHSN276200900852P
- ITK Maintenance Contract 2010-2011 – HHSN276201000516U
This committed support to the creation of Reproducible Research Journals preceded the current renaissance of appreciation for reproducibility in Science.