Guidelines and Examples for Writing an Effective Constraints on Generality Section
The examples below illustrate strong COG statements with respect to one or more of the recommendations below. To locate good examples of a particular recommendation, search this page for "Rec 1" or another number.
Recommendations
-
Clearly describe the population that the empirical findings or theoretical model are expected to apply to.
-
If generalizability is a goal of the paper, engage in a theoretical discussion of what the sample characteristics (particularly in terms of race/ethnicity, social status and power dynamics, nationality, and cultural context) might mean for the generalizability of the findings, model, and/or conclusions. Highlight any empirical and/or theoretical rationales for whether/how findings are expected to vary depending on sample characteristics.
-
If generalizability is not the goal (as in qualitative research), reflect more broadly on who the work does and does not apply to or what it is (and is not) intended to contribute.
-
Attend to multidimensionality and intersectionality of participants’ social identities and positionalities rather than basing the critical discussion just on single-axis identities.
-
Create space for the possibility that future work will find additional constraints on the findings described in the paper. Consider explaining why such research would be a valuable contribution to the literature.
Examples
Example: “Although we believe research on interpersonal comparisons has potentially broad theoretical applications, the generalizability of the findings from our qualitative study is constrained to families with one or more asthmatic children residing in St. Louis, MO and Gainesville, FL [see Simons et al. (2017)]. Nevertheless, the qualitative interviews we described represent an initial exploration of interpersonal comparisons. Qualitative interviews represent a powerful research tool in that they provide insights into thoughts and behavior. They are not meant to be representative or to provide evidence regarding the prevalence of thoughts or behavior within the population” (Shepperd et al., 2022).
Example: "Another important limitation to this work relates to constraints on generality (Simons et al., 2017). Although our findings are consistent with the broader literature, our results are only generalizable to the New Zealand context and should be considered with caution when extending to other societies. However, New Zealand is a relatively egalitarian country, and our participants were derived from a random sample from the larger population. Thus, it is reasonable to expect similar—if not stronger—effects in other contexts where ethnic-based inequities are more salient. Nevertheless, future research will need to investigate these (and related) processes in other samples to examine the generalizability of our results" (Bahamondes et al., 2022).
Example: "There are constraints on the generality of our findings (Simons, Shoda, & Lindsay, 2017). Our sample only included Black and White raters. The lack of raters from other racial groups precludes our ability to ascertain whether our findings apply to individuals who are neither White nor Black.…..To examine whether our findings replicate across various samples, future research should include individuals from additional racial groups and utilize recruitment methods that provide samples that are more representative of U.S. educational attainment (e.g., community sampling)" (Benson & Golpe, 2022).
Example: "Another important limitation of the integrative wisdom model concerns the question of its cultural generalizability. The model is based on wisdom conceptions developed by "Western" researchers and findings from studies in "Western," relatively individualistic societies. While we believe that the components of the model are also part of "Eastern" conceptions of wisdom (see, e.g., Ferrari & Alhosseini, 2019; Yang & Intezari, 2019), relative emphases may differ and/or some components may be missing from the model. One could argue that cultures as a whole differ in their levels of the different components of wisdom (e.g., Asadi et al., 2019; Atwijukire & Glück, 2020; Grossmann et al., 2012)—for example, that people in collectivistic cultures are, on average, higher on concern for others and lower on self-knowledge. While people from highly individualistic cultures become less self-focused as they develop wisdom, maybe people from highly collectivistic cultures become more aware of their personal strengths and needs. In that sense, wisdom might represent a largely culture-independent ideal of how human beings live a good life that manifests differently depending on the biological, environmental, social, and cultural conditions into which people are born" (Gluck & Weststrate, 2022).
Example: "Anti-atheist prejudice remains prevalent in the United States (Edgell et al., 2006), but is not exclusively an American phenomenon (Norenzayan & Gervais, 2015), suggesting that atheism may be similarly underreported elsewhere in the world where social pressures reinforce religiosity and its self-presentation. Self-reports yield an estimate of 500-700 million atheists worldwide (Zuckerman, 2007). Presumably, self reported atheism is less biased by social desirability concerns in more highly secularized societies like those in Scandinavia (Inglehart & Norris, 2004). Thus, it would be unwise to use the present USA data as a baseline and assume that atheism rates are uniformly much higher than self-reports suggest. Instead, we predict that underreporting of atheism covaries with cultural norms promoting religion in domains like morality and cooperation" (McKay & Whitehouse, 2014; Norenzayan et al., 2014)" (Gervais & Najle, 2017).
Example: "We did not collect demographic information across all four studies, but caregivers generally attained higher education and were recruited from urban environments which may have limited our ability to capture variability. For example, children from rural or lower-socioeconomic status backgrounds typically contribute more to household and external labor (e.g., Alcalá et al., 2014; Putnik & 712 Bornstein, 2015). Thus, children from these populations may view our "high utility" actions as more normative and may expect starker contrasts to differentiate givers. Likewise, children's own generous behaviors develop differently between cultures (e.g., Corbit et al., 2020; Cowell et 715 al., 2016; Goyal et al., 2019) and cultural variability has been observed in how children treat others based on cost information (e.g., amount of work performed; Engelmann et al., 2021, Schäfer et al., 2015). Therefore, different features and developmental trajectories may underlie children's generosity evaluations between cultures, and the consequences of these evaluations (e.g., affiliative preferences) may also vary. Future work should systematically investigate how 720 this variability may affect the use of cost and need information in evaluations and affiliative 721 choices. Until such work is conducted, these results should be generalized with caution to populations outside of the US and Canada" (Radovanovic et al., 2022).
Example: "Groups with different historical experiences of oppression may have different perceptions of cultural appropriation, with different implications. In the United States, the history of slavery and Jim Crow laws have facilitated a unique experience of being Black in America (Dubois, 1903; Sears & Savalei, 2006). As a result, Black Americans may be more perceptive to instances of cultural appropriation than other minority group members. For other minority groups that have experienced existential threat (e.g., racial genocide and cultural persecution), cultural appropriation may be especially problematic. For example, the appropriation of Native-American religious practices and artifacts is highly contentious given the group's indigenous status and history of racial annihilation (Taylor, 1997). Zou and Cheryan's (2017) model of racial positioning proposes two dimensions, perceived inferiority, and perceived foreignness (deviation from the "American" prototype), along which racial groups are perceived. Black and Native Americans are stereotyped as inferior but not foreign; Latinos are stereotyped as both inferior and foreign, Asian Americans as superior but foreign, and Whites as superior but not foreign. These dimensions may contribute in interesting ways to groups' perceptions of cultural appropriation. Racial groups that are viewed as foreign may not experience distinctiveness threat (or perceive appropriation) when out-group members use their cultural products, but perhaps only when they are also stereotyped as superior (e.g., Asian Americans) rather than inferior (e.g., Latinos). For groups such as Asian Americans, appropriation may function as acknowledgement, and lead group members to experience a sense of being seen and understood by the dominant community (Swann & Read, 1981). Additionally, as we have shown in the present research, group members stereotyped as high in superiority and low in foreignness (i.e., White Americans) also experience less threat and have less negative reactions to use of ingroup cultural products. In these cases, however, a sense of "being seen" is unlikely to follow, as the group reflects the dominant framework. This brings us back to the importance of group status in the cultural appropriation framework: Appropriation and distinctiveness threat are most likely to be perceived and experienced by cultural groups positioned and stereotyped as inferior. Direct tests of this prediction await further research" (Mosely & Biernat, 2021).
Example: "The sample did not adequately represent various groups in terms of language, gender and sub-region. Fluency in English, which is concentrated in urban and elite groups to an extent (Bansal, 2019), restricted the sample to more advantaged socio-economic groups. However, we also underscore that English is widely spoken in India, and the trend is rising among various groups (Sharma, 2020). Findings highlight several aspects of language based ethnic identity and could potentially be replicated in similar contexts where ethnic identity groups coexist along linguistic divisions. Yet this study focussed on a specific location (Karnataka) and age range (emerging adults) which could restrict generalisability beyond these demographics. Future research with more inclusive and diverse socio demographic representation will be essential to understanding ethnic identity within and beyond India. Our study also conceptualised ethnicity on the basis of language. While we did ask participants how they personally defined their ethnicity, we could not account for intersectionality which is relevant to questions on ethnicity (Malcolm & Mendoza, 2014). Future qualitative work could look into understanding more about a person's subjective experience of their ethnicity" (Lal, K. K., & Majumdar, S., 2021).
Example: "In addition, while we considered sexual orientation, we lacked a sufficient number of Black women from LGBTQ backgrounds to engage with how the Jezebel stereotype foregrounds a particular heteronormative understanding of Black women's sexuality. There is a small and growing body of psychological literature that considers Black women's diverse sexual orientations (e.g., Chmielewski, 2017), which includes critiques of scholarship on Black women that demonstrate a continued investment in heteronormativity through its focus on heterosexual women and relationship dynamics (Morgan, 2015). In particular, "scholars have often failed to reinterrogate these venerated interventions with the temporal, cultural specificity reflected in contemporary US Black women's ethnic heterogeneity, queerness and the advent of digital technologies and social media" (Morgan, 2015, p. 38). The young women in our sample were still navigating (and to varying degrees, in the process of rejecting) the heteronormative socialization that they received during childhood and adolescence and were at different stages of imagining new erotic possibilities within their Black queer sexuality" (Leath et al., 2021).
Example: "First, a significant limitation of the current studies was that both samples included participants from many different cultural and racial–ethnic backgrounds. The samples were also too small to allow for between-group comparisons. Indeed, the specific experiences of a Chinese American, a Mexican American, and an Israeli American are likely to be quite different in ways that are important for the development of an integrated bicultural identity. However, it is also the case that we applied a broadly construed identity challenge framework to BII that was expected to be meaningful across different ethnic and cultural groups, and recent BII [bicultural identity integration] work with culturally varied samples supports this assumption (Huynh et al., in press). In addition, the narrative identity orientation emphasizes the importance of the subjective interpretation of past memories, independent of the objective circumstances. Nevertheless, it will be important in future research to take the specific circumstances, historical contexts, and cultural meaning systems of different cultural groups into account in evaluating the role of narrative processing in BII (see Hammack, 2008, 2010)" (Lilgendahl et al., 2018).
Example: "The present investigation is also limited to Black women. Recent work examining identity-safety transfer has found that identity-safety cues signaling identity safety for racial and ethnic minorities, also signal identity safety for White women, and vice versa (Chaney, Sanchez, & Remedios, 2016). A Black woman role model may also encourage belonging and trust among other identities underrepresented in STEM environments, such as Black men, and White and Latina women. Also, research investigating identity-safety transfer has yet to be examined among Black women or other groups possessing multiple stigmatized identities. Although the current studies focused on STEM, Black women are underrepresented in many domains (e.g., business, management) and face unique challenges in these environments (Cook & Glass, 2014; Rosette & Livingston, 2012). The current results most likely are not limited to STEM, and future research should explore if exposure to Black women role models encourages belonging in areas outside of STEM" (Johnson et al., 2019).
Example: "Further, although prior work generally found that social class identity integration was beneficial regardless of race/ethnicity, in the present work we observed significant interactions with race/ethnicity for the first time, such that identity integration was more predictive among underrepresented minority students in college compared to ethnic majority students. Having multiple salient and/or marginalized identities (e.g., gender, race, social class, sexual orientation, religion, ability) is especially impactful, such that individuals may face additional discrimination (e.g., Cole, 2009; Rosenthal, 2016). Thus, the importance of one's identity as a first-generation student, versus other sociocultural identities, may be an important determinant of first-generation students' college experience Interactions with racial/ethnic identities may depend on several factors: first, concealability of race or ethnicity (e.g., "passing" as White), such that first-generation students who are members of a visible racial/ethnic minority group may be more likely to attribute challenges to their ethnic identity rather than college generation status. Alternatively, identifying as an immigrant or international student may supersede college generational status, thereby reducing the effects of social class bicultural identity integration. Unfortunately, the present studies had too few participants to conduct finer-grain analyses beyond ethnic majority and minority groups" (Herrmann et al., 2022).
Example: "One significant limitation is that the present work did not address issues of intersectionality. Indeed, identities are multifaceted, and individuals are simultaneously members of many groups (e.g., gender, racial, socio-economic). The role of each of these identities—and certainly their intersection—is incredibly important and necessitates further research. In the current studies, the primary reason for omission of intersectionality related to a lack of statistical power to make responsible comparisons between intersectional categories (e.g., only 16 students and 21 students in Studies 1 and 2, respectively, were doubly stigmatized [first-generation and non-White]). Thus, an important unanswered question is whether science ID differentially impacts such students or, more generally, whether science ID's effect varies as a function of intersectional identities. Given that this is one of the first studies to empirically examine the psychological mechanisms driving science ID's effect on performance, future work should aim to replicate the findings in other contexts. The present studies were implemented only in Introductory Biology which, at the current university, is comprised of majority Whites and women. Would the same results appear if the intervention was delivered in other science domains with different demographic profiles (see Binning & Unzueta, 2013; Murphy et al., 2007; Walton et al., 2015)? Substantial evidence reveals the presence of gender stereotypes in math (Nosek et al., 2002; Riegle-Crumb & King, 2010), where women may be at greater risk for stereotype threat than in the life-sciences. Here, science identity among women may be more important (Vincent-Ruz & Schunn, 2017)" (Chen et al., 2020).
Example: "Another important strength of the current research is its sample diversity, though additional research is needed, both within and outside of the United States. Do U.S. individuals of other social identities (e.g., Native, Asian, Muslim, nonbinary, transgender) conceptualize God as a White man? To what extent, and for what reason, do these conceptions vary within groups? For example, are Black Christians who attend churches with mostly White congregations more likely to conceptualize God as White? Additional research with larger and more diverse child samples would be especially informative, given that we were unable to systematically examine variation as a function of children's race, gender and age with the relatively small sample of Study 3" (Roberts et al., 2020).
Example: "Another future direction relevant to both theory and practice would be to examine how STEM stereotypes affect adolescents from underrepresented racial groups. Similar to gendered stereotypes about computer science, computer science is also stereotypically associated with Whites and Asians (Margolis, Estrella, Goode, Holme, & Nao, 2008; Walton & Cohen, 2007; see also Cvencek, Nasir, O'Connor, Wischnia, & Meltzoff, 2014), and these stereotypical representations may send messages to Black and Latino/a students that they do not belong in computer science. Girls who are Black and/or Latina may be even less likely to feel that they belong, because they possess both gender and racial identities that do not fit current stereotypes of computer scientists" (Master et al., 2016).
Example: "As a supplement to our study, we explored meta-analyses that reported selected demographic moderators (e.g., sex, race/ethnic group, age, and culture) of Agreeableness' effects (for details, see the online supplemental material and Table S24). Overall, we found no consistent evidence of replicable moderator effects; thus, constraints on the generality of our findings for any specific subpopulation are not supported by the current available evidence. Nevertheless, these questions remain open, and constraints may be warranted if replicable evidence emerges for a given subpopulation. To facilitate future meta-analyses of these matters, we urge researchers to report results by subpopulation in the article of record or in their supplemental material (for an outstanding example, see the Life Outcomes of Personality Replication Project [Soto, 2019] and its analyses by subpopulation [Soto, 2021])" (Wilmot & Ones, 2022).