Below, we describe the evaluation process in an effort to improve transparency surrounding the convention.

Who are the reviewers? How are they selected?

  1. All single-presenter podium presentations and symposia submissions will be evaluated by three or more independent reviewers. Posters, research spotlight, and round table unconference submissions will be reviewed by two reviewers.
  2. SPSP members with a PhD can nominate themselves for consideration as a reviewer. During the nomination process, reviewers indicate their areas of expertise using keywords. New this year: Graduate students are eligible to review posters.*
  3. The Science Program Review Co-Chairs decide on the final pool of reviewers from the self-nominated group, seeking to ensure a range of expertise areas.
  4. Once the submission portal has closed, all submissions are matched to reviewers using keywords, primarily matching the first keyword listed. We aim to give reviewers an equal number of submissions to review.

Reviewers evaluate all submissions based on criteria specified by the Science Program Review Co-Chairs. What are those criteria and how are they used?

  1. Submissions are masked, so the reviewers cannot identify the authors; submissions are not evaluated based on the names of the people involved.
  2. Reviewers consider several dimensions when evaluating submissions:
    • Strength and rigor: Does the research reflect best practices, including issues of statistical power? Are studies well-designed to answer the research question(s)? If the session includes applied or non-empirical talks, do these present strong arguments or clear evidence toward the goals of the session?
    • Contribution: Does the submission address a question or set of questions that substantially advances our knowledge of a theoretical and/or practical/applied contribution in social and/or personality psychology?
    • Interest-value: Will the submission cut across subfields or bridge basic and applied work in an integrative way? Will it spark or make a meaningful contribution to conversations in social and/or personality psychology? Will the audience think they have learned something new? Are there other reasons to expect it to strongly appeal to social-personality psychologists?
    • Rating Scale:
      4: Excellent (top 25% of the submissions you are reviewing, top quartile)
      3: Very Good (falls in the second quartile)
      2: Good (falls in third quartile)
      1: Weak (falls in bottom quartile)

      Reviewers are asked for a rectangular distribution in which they assign only 3-4 excellent, only 3-4 very good, only 3-4 good, and only 3-4 weak. The quality of symposia submitted is usually very high, and reviewers must make relative judgments within the set of reviews assigned, even if they might be inclined to rate most of the assigned submissions as "excellent".
  3. Diversity, Equity, Inclusion, or Anti-Racism (DEIA) statement: We are extending the pilot program from last year for the 2024 Annual  Convention Submissions (learn more here). Submitters will indicate whether their work advances the DEIA goals of SPSP and if yes, they must share how. Posters, research spotlights, and roundtable unconferences are exempt and will not be asked to provide a DEIA statement. 

    Separately, reviewers evaluate the extent to which the submission advances SPSP's goal of promoting diversity and equity. Submissions advancing these goals may include (but are not limited to):
     
    • Diverse research participants (e.g., understudied or underserved populations) 
    • Diverse research methods (e.g., methodology that promotes equity or engages underserved communities or scholars)
    • Diverse members of the research team (e.g., those from underrepresented sociodemographic backgrounds, from an array of career stages, from outside the United States, or with professional affiliations that are not typical at SPSP such as predominately undergraduate serving institutions, minority-serving institutions, or outside academia. This can include providing research and training opportunities to students from underrepresented and/or marginalized backgrounds, even if they are not part of the author team)
    • Presentation topics (e.g., prejudice and discrimination, critical theories, cross-cultural research)

      Rating Scale:

      3: Exceptional- The submission clearly and strongly advances SPSP's goal of promoting diversity, equity, and inclusion.
      2: Satisfactory- The submission slightly to moderately advances SPSP's goal of promoting diversity, equity, and inclusion. We expect that this rating will be the most commonly applied rating. 
      1: Not Applicable- The submission does not advance SPSP's goal of promoting diversity, equity, and inclusion or the submitter preferred not to answer.
  4. The review process is independent each year; the content of prior conventions does not factor into the evaluation process, with one exception: If there is a significant amount of feedback during the post-convention survey indicating attendees would like to see more representation of a particular topic, the committee may increase representation of the topic at the following year's convention.
  5. When making a final decision about which submissions to accept, the Science Program Review Co-Chairs rely primarily on the reviewers' rubric scores. Submissions are unmasked at this stage, allowing the Science Program Review Co-Chairs to ensure that the program features a diverse range of speakers. When making the final selection, reviewers aim to accept high-quality submissions while creating a balanced and diverse program. There are no acceptance quotas for research subjects, but there may be general targets to ensure certain topics are adequately represented. Beyond that, any themes that may emerge do so organically based on having a high number of high-quality submissions that year.
  6. Data blitzes and single-presenter symposia that receive the highest scores from reviewers are examined carefully to create these sessions. In many cases, several high-quality submissions will cluster together in terms of content; these will be collected into a symposia and the presenters will be invited to work together to select a chair, title, and description of the program. The other highest-rated submissions that don't cluster into a symposium set will be included in one of several data blitz sessions scheduled for the convention (provided the speaker is eligible for a data blitz).

How is the final schedule determined?

  1. SPSP staff members create a grid that ensures primary keywords do not overlap on blocks of the schedule.
  2. The Convention Committee and Science Program Review Co-Chairs adjust the grid if they see sessions with overlapping content areas that are scheduled at the same time.
  3. SPSP staff members send a list of all accepted symposia to the symposium chairs, who indicate up to three sessions that should not be scheduled at the same time due to content overlap. Organizers consider this information when finalizing the grid.

What are the historic rates of acceptance for prior conferences? 

Since we have significantly revised our submission evaluation process in recent years, the data below represents up to 2024: