Reviewing Guidelines

Reviewing Guidelines

NCSCM-2018 is a high­ quality conference with a competitive submission process.  As a reviewer, your work is of vital importance to the conference and we rely on your expertise and good work to generate an outstanding program. The key points while reviewing are:

  • The main criterion is a judgment of the degree to which the submitted paper contributes substantial new research or scholarship to the field of Mixed and Augmented Reality.
  • Reviewers rate the paper using a 6-point ranking scale, and provide a written evaluation.  The written evaluation needs to support the reasoning behind the numeric ranking.
  • After the review period, there is a discussion period, where all reviewers can see the other reviews.  Reviewers may adjust their evaluations.  Please use this period to seek a consensus evaluation.

First off: Reviewer Profile

Please describe your expertise in the different topic areas for submissions. List all topics where you feel qualified. Be honest, and don’t just list all areas of interest, but the ones that you feel confident about reviewing. This will help the program committee members to judge if you are the best person to do a specific review.

In your profile you can also add comments about your previous experience as a reviewer: indicate for which NCSCM-2018 conferences you have reviewed, as well as any other related conferences or journals. If it’s your first time reviewing, the program committee will be able to see this and take it into account as well.

What to look for:

The main criterion for accepting a paper is the degree to which the submitted paper contributes substantial new research or scholarship to the field of Mixed and Augmented Reality.  A good submission should also be technically correct and adhere to the standards of academic writing. Minor errors and flaws can be corrected and should not be reason to reject a paper. Be a little forgiving of poor English; it would be too bad if a truly novel paper was rejected because of poor English that could be corrected before final submission.

Numeric ratings and written evaluation

Submissions are rated in the review form on a numeric scale from 1 to 6, where 1 is “Definitely reject”, and 6 is “Definitely accept”. Be as precise in your rating of a submission as possible. Moreover, your written evaluation must support the numeric rating. The written evaluation should always indicate why the specific rating was given; if a submission should be accepted, be sure to state the reasons for it; if a submission should be rejected, again list the arguments against it. For example, your review may be discounted if you rate a paper highly but only describe its flaws, or give a very brief review.

Paper length

You should consider paper quality versus length according to a contribution-per-page judgment.  For example, a paper with eight pages should have more contribution than one with four pages. Please evaluate the paper based on the presented length and form.  Do not discount a shorter paper, because the authors could have shown more applications or evaluated against another concept or system. Similarly, if possible, do not recommend shortening a long submission, unless you can clearly articulate how shortening the paper would make it stronger.  If your judgment is that the paper cannot be accepted in its submitted form, then please explain your reasoning.

“Flawed” papers

Reviewers should be careful to not immediately dismiss a submission that appears to have a flaw (e.g., in a user study), but otherwise make a contribution. It is possible that such a paper would be accepted if the issues were addressed in some way during the shepherding process, e.g., by removing the associated prose, or adding prose (e.g., in the conclusions of the paper) that acknowledges the issue.

Presentation of the review

In writing your review, please be polite to the authors. Even if you rate a paper poorly, you can critique it in a positive voice. As part of polite reviewing practice, you should always state what is good about a paper first, followed by your criticisms. If possible, you should offer suggestions for improvement along with your criticism. Think about the type of review feedback you would like to receive on a rejected paper you submitted.

Timely reviews

Please finish your review on time. After all reviews are completed, the discussion period will commence.  Then, the senior reviewer, who invited you to review the paper, will write a meta­-review, which summarizes the reviews and provides the rationale for the suggested paper decision.  All of these tasks have tight time limits, and rely on reviews being completed by the deadline.