In response to the concerns raised by colleagues (principally and initially in this petition, but see also Chris Blattman’s take and other responses from both sides), I wanted to clarify why I think that delaying implementation of the Journal Editors’ Transparency Statement (JETS) is a poorly thought out goal, one that will differentially disadvantage some scholars, particularly younger, less well-known scholars.
These Standards Are Already Being Implemented. To begin, and reiterate one of the arguments I made here a few days ago, journal editors already have the unilateral discretion to impose the kinds of policies that JETS is calling upon editors to implement. To wit, editors are already implementing policies along these lines. For example, see the submission/replication guidelines of the American Journal of Political Science, American Political Science Review, and the Journal of Politics, to name only three. These three vary in details, but they are consistent with JETS as they stand right now.
It’s Happening Anyway, Let’s Stay In Front of It. The point is that the JETS implementation is already under way and, indeed, was underway prior to the drafting of JETS. The DA-RT initiative is simply providing a public good: a forum for exactly the conversations that the petition signers seek. (The individuals who have contributed time to the public good that is DA-RT, and their contributions, are described here.)
The Clarifying Quality of Deadlines. The “implementation of JETS” scheduled for January 2016 is best viewed as a moment of public recognition that we as a discipline need to continue the conversations. Editorial policies are not written in stone, after all. Thus I strongly believe that delaying the implementation of JETS will do nothing other than further muddy the waters for scholars. JETS is about recognizing and shepherding the movement towards more coherent and uniform procedures to increase the transparency of social science research. Delaying it will place scholars, particularly junior and less well-known scholars, at a disadvantage. This is because implementation of the JETS will give all scholars firmer ground to stand on when seeking clarification of the details of a journal’s replication and transparency requirements.
Clear Policies Level the Playing Field and Make Editors (more) Accountable. Furthermore, scholars will be able to publicly compare and contrast these procedures, allowing more judicious selection of research design, early preparation of justifications for requests for exemptions, and finally, a counterpoint for an editorial decision that is inconsistent with the standards of peer outlets. That is, if journal X decides that one’s research is sufficiently transparent and then journal Y decides otherwise, the transparency of those journals’ standards—which JETS aims to ensure are publicly available—will ensure that the journals’ standards are fair game for comparison and debate. This is the type of conversation sought by many of the petition signers I have spoken with. Implementation of JETS will push this conversation forward, whereas delay will simply retain the status quo of an incoherent bundle of idiosyncratic policies.
Will The Sun Rise on January 15, 2016? It is important to keep in mind that the implementation of the JETS statement will in most cases result in no new policy: journal editors have been setting and fine-tuning standards like these for decades. Rather, implementing JETS binds editors—like myself—more closely to the sought-after conversations about how best to achieve transparency in the various subfields and with respect to the various methodologies of our discipline.
In other words, implementation of JETS will empower scholars to demand more transparency and accountability from the editors of the 27 journals that have signed the statement.
With that, I leave you with this.
Pingback: Patty in second post on why a delay of DA-RT is a bad idea | Dialogue on DA-RT
Hi John,
You mention that you are worried that delaying implementation will “differentially disadvantage some scholars, particularly younger, less well-known scholars.” And “This is because implementation of the JETS will give all scholars FIRMER GROUND to stand on when seeking clarification of the details of a journal’s replication and transparency requirements” (our emphasis).
The problem, as we see it, is that the journal editors’ statement does not provide such firm ground. Specifically, it says, “If cited data are restricted (e.g., classified, require confidentiality protections, were obtained under a non-disclosure agreement, or have inherent logistical constraints), authors must notify the editor at the time of submission. The editor shall have full discretion to follow their journal’s policy on restricted data, including declining to review the manuscript or granting an exemption with or without conditions.” The phrase “require confidentiality protections” is not self-explanatory. A researcher may think that some subjects require confidentiality protections but an editor may disagree, and the editor has full discretion. At the moment the profession does not give much guidance on this problem.
The papers in the last issue of the Qualitative and Multi-Method Research section newsletter http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2652097 raise these issues far better than we can in this short space. Some of these papers also make clear that several younger scholars have already experienced feeling placed at a disadvantage because of editors’ and reviewers’ actions using the JETS statement as backing. When you read these papers you may conclude, as we have, that the potential harm to junior scholars is greater from a process by which editors implement the statement now as it stands than by one in which editors hold off from implementing until some of these issues can be discussed in more detail in the profession.
Nancy Hirschmann
Mala Htun
Jenny Mansbridge
Kathy Thelen
Libby Wood
Thanks very much for the feedback. I will definitely read the QMMR newsletter, and I have been in conversations with several of our colleagues about exactly these concerns. My first take is that the key provision in (that section of) JETS is this: “The editor shall have full discretion to follow their journal’s policy on restricted data, including declining to review the manuscript or granting an exemption with or without conditions.” “Implementation” of JETS, in my mind, is satisfied only if the fourth point of the statement is met. This point commits a signatory journal to:
“Ensure that journal style guides, codes of ethics, publication manuals, and other forms of guidance are updated and expanded to include improved data access and research transparency requirements.”
Thus, editors have the (already extant) right to enforce their journal’s policies. The real change, and one that helps make the power dynamic more symmetric between scholar and editor is that an editor who has implemented JETS has provided a statement of the standards from the outset. This informs scholars ex ante, and more importantly, forces the editor(s) to write down their policies, a step that will allow the policies to be judged by the community, both specifically at the editorial board/association level and within the discipline at large.
Simply put, JETS does not give any editor any more discretion than he or she already has. Policies consistent with JETS are already in place at many journals. This is codifying a growing awareness of a need for broader discussion and progress towards more coherence across journals.