Peer Review

Double-Blind Peer Review

Most respected academic journals and university presses subject manuscripts to a rigorous process of double-blind peer review before publishing them. This process involves sending the anonymous manuscript to two experts in the field, who read and evaluate the work without knowing who produced it. The reviewers typically recommend one of three options:

  1. Accept and publish
  2. Revise and resubmit
  3. Reject

The editors share these anonymous reader’s reports wit the manuscript author(s). All of this work takes place anonymously behind the scenes, and it is a slow, arduous process.

Our site has been submitted and approved via a blind peer review process conducted by ModNets, an organization that provides a vetting community for digital modernist scholarship. This process was blind, rather than double-blind, because although the reviewers remained anonymous, our identities were not hidden.

We received reports from a content reviewer and a technical reviewer, made revisions in response to their excellent insights and suggestions, and received approval from the editors at ModNets. We are grateful to these anonymous reviewers and to the editors at ModNets for their guidance and support.

An Experiment in Public Peer Review

Even as we submitted our site for a traditional process of blind peer review, we simultaneously conducted an experiment in a public peer review. We hoped this approach would make peer review more open, collaborative, efficient, and even enjoyable. In Spring 2019, we used  Hypothes.is—a free, web-annotating tool—to enable users to comment on our site. This powerful tool provided an easy to use method for making the peer review process public, and we recommend it highly. The Hypothesis team is also extraordinarily knowledgeable, helpful, and responsive, and worked with us to set up the tool and tailor it to our needs.

We received more than 80 annotations from our Advisory Board, but none in the Open Forum, despite an active social media campaign urging people to participate. Nevertheless, the feedback we received from our Advisory Board was insightful, informative, and helpful.

screen shot of Hypothesis annotation in sidebar next to text from one of the scholarly chapters
Hypothesis annotation from Advisory Board member.

After responding to their comments and suggestions, we made an editorial decision to deactivate the Hypothesis plugin for several reasons:

  1. At the time of this project’s completion, the tool was not yet accessible for vision impaired users, and we aimed to make our site meet web accessibility standards.1
  2. The yellow highlights that appear on annotated text distract from the site’s aesthetics, which are designed to promote an immersive reading experience.

Although Hypothesis is no longer installed in the site, you can still install it on your browser and use it to annotate in public, or create your own private annotation group. We encourage you to try out this powerful tool.

Instructions for Reading Expert Peer Review and Offering Commentary on Hypothesis

During the time period when we activated the Hypothesis plugin, a sidebar would appear on the top right of every page of our site:

screen shot with arrow indicating position of Hypothesis sidebar on upper right corner of screenUsers could open the Hypothesis sidebar by clicking on the top arrow <. At the top of the sidebar, they could select either Advisory Board PR or Open Forum.

Advisory Board PR

The Advisory Group PR is a group of ten experts in the fields of Mina Loy studies, modernist/avant-garde studies, and digital studies who agreed to support our project, writing letters of commitment for our NEH grant application. These experts agreed to peer review the site using Hypothesis in the spring of 2019. We created the Advisory Board PR group for them in recognition of the need for expert peer review to verify the authority, accuracy, and value of our scholarly work. Using this group, or Advisor Board members could comment on the site, as well as read and reply to one another’s annotations. Their comments were publicly visible, but the public could not join or participate in their group.

To read expert commentary, users could select the Advisory Board PR Group at the top of the Hypothes.is sidebar. Their annotations appeared as page notes or highlighted text throughout the site.

Open Forum

We also created an Open Forum in recognition that public users also have expertise, insight, and meaningful feedback to share. Through this forum, users could also read and respond to one another’s comments. We welcome your comments and suggestions.

To participate in the Open Forum, users had to first create a free Hypothesis account, which they could do by choosing “Log in” in the Hypothesis sidebar, nd navigating to “Don’t have a Hypothesis account? Sign up.” Or they could create an account on the Hypothes.is website. Once users created and verified your account, they could login and select the Open Forum group at the top of the Hypothesis sidebar to comment on the site.

Annotations and Page Notes

To annotate a specific portion of the text, users could highlight a selection of text on the page and click the “Annotate” button that pops up:

screen shot of selected text with annotation icon highlighted.

To comment on a page as a whole, users could click on the Page Note button on the annotation sidebar:

screen shot with arrow indicating page note button on right sidebar

Whether they created an annotation or a page note, users must click the “Post to Public Forum” button before moving on.

Notes about annotations and page notes

  • Once you have created your annotation, you can edit it by clicking on the pencil icon.
  • To delete your annotation, you can click on the trash can icon.
  • If you’d like to respond directly to your annotation or someone else’s, click on the reply arrow.
  • To report inappropriate content or abuse to the group moderator (Suzanne Churchill), click the flag icon.

screen shots of annotations window with errors pointing to toolbar options.Helpful Links

Student Projects

While most of the site has been peer reviewed by professional scholars in the field, the student projects featured in New Frequencies have not been vetted by the same rigorous process, in order to give students more freedom to experiment, as well as to accommodate limitations of time and professional training. Created and owned by students, the projects featured in New Frequencies are hosted on separate websites and servers. As a result, we cannot guarantee that the content is accurate or that the projects meet web accessibility standards. Nevertheless, we believe these projects are worthy of publication. With their fresh ideas, creativity, and technological innovation, they exemplify the spirit of the en dehors garde.


 

  1. For information about the accessibility of Hypothesis, please consult Accessibility at Hypothesis.