Reranking partisan animosity in algorithmic social media feeds alters affective polarization

May combine two claims.

Prior randomized controlled trials on Facebook and Instagram algorithms could only proceed with negotiated support from Meta's executives, an arrangement that may limit the scope and independence of scientific work.

“While prior work made important advances in randomized controlled trials on the Facebook and Instagram algorithms (37), these experiments could only proceed with negotiated support from Meta's executives-an arrangement that may limit the scope and independence of scientific work.”

• cites [37] sourcePartly supported

  The Design and results section describes randomized experiments on Facebook and Instagram and identifies the study as a collaboration with Meta. The Acknowledgments document substantial Meta support, but our copy does not establish that support was negotiated with Meta executives or that the arrangement limited the scope or independence of scientific work; it instead notes safeguards and says Meta had no prepublication approval right. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim. May combine two claims.

The interventions may result in algorithms that reduce partisan animosity and promote greater social trust and healthier democratic discourse across party lines.

“Together, these interventions may result in algorithms that not only reduce partisan animosity but also promote greater social trust and healthier democratic discourse across party lines (38,40).”

• cites [38] sourcePartly supported

  The section on testing the AI model reports that feeds embedding democratic values reduced partisan animosity in preregistered experiments. Sections 3 and 4 discuss fostering mutual respect and promoting civil discourse as possible algorithmic goals, but the paper does not establish that these interventions increase social trust or improve discourse across party lines. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [40] sourcePartly supported

  Section 4.2.1 reports that removal and downranking feeds reduced partisan animosity compared with an engagement-based feed, supporting that part of the qualified claim. The paper lists social distrust among relevant democratic-attitude measures in Section 2.3, but the passages do not show that these feeds increased social trust or improved cross-party discourse. Section 7.3 also cautions that longitudinal, real-world effects remain untested. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

The effects correspond to a decrease and increase in affective polarization equivalent to 3 and 3.6 years of rising partisan animosity in American society.

“Like other studies (49), we contextualize these effects against the rising levels of partisan animosity in American society over time: our effects correspond to a decrease and increase in affective polarization equivalent to 3 and 3.6 years, respectively.”

• cites [49] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Technical limitations include difficulty in performing cluster-randomized experiments testing for social influence and inability to collect the full feed inventory for a given user.

“Other technical limitations of our approach include the difficulty in performing cluster-randomized experiments testing for social influence (54), and the inability to collect the full feed inventory for a given user.”

• cites [54] sourcePartly supported

  The Design section describes a limitation of graph-cluster randomization: some clustering methods can increase variance too much to justify their bias reduction. The Discussion also cautions that the clustered designs examined may not apply when the goal is to estimate peer effects themselves. These points support methodological difficulty, but the paper does not establish that such experiments are technically difficult to perform. The feed-inventory limitation is not attached to this citation. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The survey structure followed the study that introduced the eight AAPA variables.

“We followed the structure of the survey used in the study that introduced the eight AAPA variables (39).”

• cites [39] sourceSupported

May combine two claims.

The AI classifier labels expressions of the eight constructs with accuracy comparable to trained annotators and produces depolarization effects in a lab setting on a fixed feed.

“We operationalize these eight categories into an AI classifier that labels expressions of these constructs in social media posts, does so with accuracy comparable to trained annotators, and produces depolarization effects in a lab setting on a fixed feed (40).”

• cites [40] sourcePartly supported

  Study 2 Results report strong agreement between the AI model and human labels on the overall eight-variable score, but agreement was weak for two individual variables; the paper does not establish comparable accuracy for all eight separately. Study 3 Results report that an AI-ranked feed reduced partisan animosity relative to an engagement-ranked feed. The Experimental Design describes a controlled online experiment using a prepared inventory of posts, rather than a physical lab setting. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The eight AAPA factors are bipartisan threats to the healthy functioning of democracy.

“We draw, in particular, on a recent large-scale field experiment that articulated eight categories of antidemocratic attitudes and partisan animosity as bipartisan threats to the healthy functioning of democracy (39).”

• cites [39] sourceSupported

May combine two claims.

Large-scale field experiments aimed at reducing polarization by intervening on the feed algorithm have found both a decrease and an increase in polarization.

“However, large-scale field experiments aimed at reducing polarization by intervening on the feed algorithm-for example, by increasing exposure to out-party content-have found both a decrease (35) and an increase (36) in polarization.”

• cites [35] sourceNot checked — no full text of the source
• cites [36] sourcePartly supported

  Citation (36) is attached to the reported increase, not the decrease. The Abstract reports that Republicans expressed substantially more conservative views after following a liberal Twitter bot. The Research Design says participants were paid to follow bots that retweeted opposing-party messages; the study did not intervene on the feed algorithm. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

May combine two claims.

Recent large-scale experiments on Facebook and Instagram found no evidence that reduced exposure to in-party sources or a simpler reverse-chronological algorithm affected polarization and political attitudes during the 2020 U.S. election.

“Similarly, recent large-scale experiments on Facebook and Instagram found no evidence that reduced exposure to in-party sources or a simpler reverse-chronological algorithm affected polarization and political attitudes (37,23) during the 2020 U.S. election.”

• cites [37] sourceSupported
• cites [23] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Today's social media platforms (e.g., X) already intervene in people's feeds and take control over users' information stream.

“today's social media platforms (e.g., X) already intervene in people's feeds and take control over users' information stream (65).”

• cites [65] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The reweighting analysis was not preregistered, and the study minimized subjective modeling decisions by following a similar reweighting procedure as the Facebook and Instagram Election studies.

“Since we did not preregister this analysis, we minimize the number of subjective modeling decisions by considering the same participant characteristics and following a similar reweighting procedure as the Facebook and Instagram Election studies (23,37).”

• cites [23] sourceSupported
• cites [37] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

The study removed follower accounts likely to be bots using the Botometer API, filtering out accounts with Complete Automation Probability above 0.6.

“We used the Botometer API and filtered out accounts with Complete Automation Probability above 0.6 (76).”

• cites [76] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

These findings align with recent studies which have observed that affective polarization is not robustly linked to antidemocratic tendencies.

“These findings also align with recent studies which have observed that affective polarization is not robustly linked to antidemocratic tendencies (50).”

• cites [50] sourceSupported

• AssumptionRests on an assumption: The GPT-based scoring accurately measures the eight AAPA factors.
• AssumptionRests on an assumption: The browser extension's reranking effectively changes the AAPA content in participants' feeds without disrupting the user experience.

May combine two claims.

Changes to the feed algorithm caused in-feed, but not post-experiment, changes to participants' negative emotions.

“Changes to the feed algorithm caused in-feed, but not post-experiment, changes to participants' negative emotions (Fig. 3).”

May combine two claims.

The browser extension infrastructure enables real-time manipulation of what consenting users see in their feed, using LLM-based classifiers to dynamically assess and rerank posts.

“We developed a browser extension infrastructure that enables real-time manipulation of what consenting users see in their feed, using LLM-based classifiers to dynamically assess and rerank posts based on a custom ranking function.”

May combine two claims.

In the Reduced Exposure experiment, treatment participants liked 24.06 fewer posts but reposted a similar number.

“liked -24.06 fewer posts (95% CI: [-39.53,-8.59]), but reposted a similar number of posts (estimate: -4.18; 95% CI: [-9.89, 1.53]).”

May combine two claims.

In the Reduced Exposure experiment, treatment participants had a 0.61% higher like rate and a 0.058% higher repost rate on viewed posts.

“On the other hand, they engaged with the posts they viewed at higher rates, with a 0.61% higher like rate (95% CI: [0.11%, 1.12%]) and a 0.058% higher repost rate (95% CI: [0.009%, 0.11%]).”

May combine two claims.

The intervention does not disproportionately impact any specific political leaning or demographic group, suggesting the approach is bipartisan.

“Importantly, our results indicate that the intervention does not disproportionately impact any specific political leaning or demographic group, suggesting that the approach is bipartisan.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants answered an average of 13 in-feed affective polarization questions in Reduced Exposure and 14.7 in Increased Exposure.

“Over the full ten days of the experiment, participants answered an average of 13 (median = 9) in-feed affective polarization questions in the Reduced Exposure experiment and 14.7 (median = 11) in the Increased Exposure experiment.”

May combine two claims.

The Reduced Exposure treatment downranked 85.4 AAPA posts on average (median = 22) per user per day, and Increased Exposure upranked 10.9 AAPA posts on average (median = 5.43) per user per day.

“The Reduced Exposure treatment downranked 85.4 AAPA posts on average (median = 22) per user per day, and the Increased Exposure upranked 10.9 AAPA posts on average (median = 5.43) per user per day (Appendix S3.2.3).”

Classified as probably not a substantial claim.

The study used a browser extension for Google Chrome and Microsoft Edge that reranked participants' feeds in real-time.

“we developed a browser extension for Google Chrome and Microsoft Edge that reranked participants' feeds in real-time (41).”

• cites [41]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

On average, participants viewed 154 posts per day and spent 24.1 minutes per day with a browser tab in foreground on X feed.

“On average, participants viewed 154 posts per day (median = 92) and spent 24.1 minutes (median = 13.1 minutes) per day with a browser tab in the foreground on their X feed.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants answered an average of 13.2 in-feed emotion surveys in Reduced Exposure and 15.0 in Increased Exposure.

“Similarly, they answered an average of 13.2 (median = 9) infeed emotion surveys in the Reduced Exposure experiment and 15.0 (median = 12) in the Increased Exposure experiment, with each survey asking them to rate two emotions.”

Classified as probably not a substantial claim. May combine two claims.

N=1,256 completed the full study: 727 in Reduced Exposure and 529 in Increased Exposure.

“Of the qualified participants, N=1,256 completed the full study by responding to a followup survey at the end of the experiment: 727 in the Reduced Exposure experiment and 529 in the Increased Exposure experiment”

Probably not a substantial claim: may not pass S3 (load-bearing).

A total of 1,662 participants completed the initial survey and qualified for inclusion.

“a total of 1,662 participants completed the initial survey and qualified for inclusion in the study.”

May combine two claims.

The treatment effect of reduced exposure to AAPA on postexperiment affective polarization has a PATE point estimate similar to the SATE, but its confidence interval includes zero after reweighting.

“The only exception is the treatment effect of reduced exposure to AAPA postexperiment: although the PATE point estimate is similar to the SATE, its confidence interval now includes zero, reflecting a loss of precision from reweighting.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Around 73% of the participants qualified and were invited to the full study.

“Around 73% of the participants qualified and were invited to the full study.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Increased Exposure to AAPA content increases the Opposition to Bipartisan Cooperation.

“We find that Reduced Exposure to AAPA content reduces the Biased Evaluation of Politicized Facts while increasing the exposure increases the Opposition to Bipartisan Cooperation (Fig. S16).”

Probably not a substantial claim: may not pass S3 (load-bearing).

During the experiment, the presidential nomination of Kamala Harris was followed by an increase in AAPA content, particularly in Republican feeds.

“For instance, during the experiment, external events (such as the presidential nomination of Kamala Harris) were followed by an increase in the proportion of AAPA content, particularly in the feeds of Republican participants.”

• AssumptionRests on an assumption: The LLM accurately classifies posts as AAPA based on the eight factors.

Probably not a substantial claim: may not pass S3 (load-bearing).

Some AAPA factors have a high chance of occurring together.

“Fig. S5 shows the correlation between these factors, showing that some of them have a high chance of occurring together.”

• AssumptionRests on an assumption: The LLM accurately classifies posts as AAPA based on the eight factors.

Probably not a substantial claim: may not pass S3 (load-bearing).

Reduced Exposure to AAPA content reduces the Biased Evaluation of Politicized Facts.

“We find that Reduced Exposure to AAPA content reduces the Biased Evaluation of Politicized Facts while increasing the exposure increases the Opposition to Bipartisan Cooperation (Fig. S16).”

May combine two claims.

Some event indicators (e.g., the assassination attempt on Trump and the presumptive nomination of Harris) reach the conventional 5% significance threshold, but these effects do not survive Bonferroni correction for multiple comparisons (five hypotheses).

“While some event indicators-such as the assassination attempt on Trump and the presumptive nomination of Harris-reach the conventional 5% significance threshold, these effects do not survive Bonferroni correction for multiple comparisons (five hypotheses).”

May combine two claims.

The political classifier model achieves an F1 score of 93.6% on the GPT-4 labels and an F1 score of 91.3% on the human ground truth.

“The model achieves an F1 score of 93.6% on the GPT-4 labels and an F1 score of 91.3% on the human ground truth.”

May combine two claims.

Support for political violence appears in 5.3% of political posts and 1.62% of the posts seen by participants in their feeds.

“However, their effects may not reach statistical significance, possibly because they are rare (appearing in only 5.3% of political posts and 1.62% of the posts seen by participants in their feeds), and we are not powered to detect effects for these rarer factors when analyzed individually.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The GPT-4 annotation step classified 34.2% of the posts as political and 65.8% as non-political.

“This step annotated 34.2% of the posts as political and 65.8% as non-political.”

May combine two claims.

The reweighted sample corresponds to a 23.3% smaller sample size in the Reduced Exposure experiment and a 17.3% smaller sample size in the Increased Exposure experiment.

“We find that our reweighted sample corresponds to a 23.3% smaller sample size in the Reduced Exposure experiment and a 17.3% smaller sample size in the Increased Exposure experiment.”

Classified as probably not a substantial claim.

AAPA content was already prevalent during the study period.

“Because AAPA content was already prevalent during the study period, reducing exposure was achieved by down-ranking many posts, whereas increasing exposure required up-ranking fewer but more intense AAPA posts, making this condition a comparatively harder test.”

May combine two claims.

The time difference between the last exposure and the start of the post-survey is not a significant predictor of changes in affective polarization and does not alter the significance of the treatment effect.

“We find that the time difference between the last exposure and the start of the post-survey is not a significant predictor of changes in affective polarization and does not alter the significance of the treatment effect.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The median delay gap between the last exposure and the start of the post-survey was 25.5 hours (mean = 42.3).

“The median delay gap was 25.5 hours (mean = 42.3).”

May combine two claims.

For the increased experiment, the BERT-based classification predicting the experimental condition achieved an F1 score of 50.5% (± 4.7%), performance no better than random guessing.

“For the reduced experiment, the F1 score was 47.4% (± 6.8%), and for the increased experiment, it was 50.5% (± 4.7%); performance no better than random guessing.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Among the remaining responses, 10.3% were manually coded as reporting a "positive experience".

“Among the remaining responses, we manually coded 10.3% as reporting a "positive experience,"”

May combine two claims.

For the reduced experiment, the BERT-based classification predicting the experimental condition achieved an F1 score of 47.4% (± 6.8%), performance no better than random guessing.

“For the reduced experiment, the F1 score was 47.4% (± 6.8%), and for the increased experiment, it was 50.5% (± 4.7%); performance no better than random guessing.”

May combine two claims.

Compared to the X population, the study sample included fewer Republicans (Reduced Exposure sample: 31.6%, Increased Exposure sample: 33%, X population: 48.4%), fewer participants with less than a college degree (Reduced Exposure sample: 43.7%, Increased Exposure sample: 50.9%, X population: 65.6%), and a similar proportion of white participants (Reduced Exposure sample: 75.7%, Increased Exposure:

“Compared to the X population, our sample included fewer Republicans (Reduced Exposure sample: 31.6%,Increased Exposure sample: 33%, X population: 48.4%), fewer participants with less than a college degree (Reduced Exposure sample: 43.7%, Increased Exposure sample: 50.9%,X population: 65.6%), and a similar proportion of white participants (Reduced Exposure sample: 75.7%, Increased Exposure sample:”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The PATE estimate for affective polarization in-feed is slightly larger than the SATE in the Increased Exposure experiment, while in the Reduced Exposure experiment, it is slightly smaller.

“the PATE estimate for affective polarization in-feed is slightly larger than the SATE, while in the Reduced Exposure experiment, it is slightly smaller.”

• AssumptionRests on an assumption: The GPT-based scoring accurately measures the eight AAPA factors.
• AssumptionRests on an assumption: The browser extension's reranking effectively changes the AAPA content in participants' feeds without disrupting the user experience.

Probably not a substantial claim: may not pass S3 (load-bearing).

The PATE estimates of the treatment effects on emotions are similar to the SATE estimates.

“The PATE estimates of the treatment effects on emotions are similar to the SATE estimates.”

• AssumptionRests on an assumption: The GPT-based scoring accurately measures the eight AAPA factors.
• AssumptionRests on an assumption: The browser extension's reranking effectively changes the AAPA content in participants' feeds without disrupting the user experience.

Probably not a substantial claim: may not pass S3 (load-bearing).

1,218 participants (96.9%) provided a non-empty response to the open-ended post-experiment survey question about whether the Chrome extension affected their experience on Twitter/X.

“This question was open-ended, and 1,218 participants (96.9%) provided a non-empty response.”

May combine two claims.

Supplementary exploratory analysis reweighting the sample to match the population of X users reduced statistical power but replicated the results, indicating substantial robustness.

“Supplementary exploratory analysis reweighting our sample to match the population of X users reduced our statistical power but replicated our results, indicating substantial robustness (Appendix S10).”

• AssumptionRests on an assumption: The GPT-based scoring accurately measures the eight AAPA factors.
• AssumptionRests on an assumption: The browser extension's reranking effectively changes the AAPA content in participants' feeds without disrupting the user experience.

Probably not a substantial claim: may not pass S3 (load-bearing).

2.8% of participants mentioned noticing a slowdown in the website's performance.

“Finally, 2.8% of participants mentioned noticing a slowdown in the website's performance.”

Probably not a substantial claim: may not pass S3 (load-bearing).

4.1% of participants reported observing more political content.

“Additionally, 4.1% reported observing more political content;”

May combine two claims.

For both experiments, the treatment effect remains statistically significant in the sample of participants who completed only the post-experiment survey or only at least one in-feed survey.

“For both experiments, the treatment effect remains statistically significant.”

• AssumptionRests on an assumption: The GPT-based scoring accurately measures the eight AAPA factors.
• AssumptionRests on an assumption: The browser extension's reranking effectively changes the AAPA content in participants' feeds without disrupting the user experience.

Probably not a substantial claim: may not pass S3 (load-bearing).

Measures of absolute engagement are highly skewed: most participants view, like, and repost a small number of posts and a small number of participants view, like, and repost many posts.

“measures of absolute engagement are highly skewed: most participants view, like, and repost a small number of posts and a small number of participants view, like, and repost many posts.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

In the affective polarization analysis, the treatment effect was 3.24 (95% CI [1.21, 5.27]) in the Reduced Exposure experiment.

“Condition: Treatment 3.24 * -2.56 * [1.21; 5.27] [-4.53; -0.59]”

• AssumptionRests on an assumption: The GPT-based scoring accurately measures the eight AAPA factors.
• AssumptionRests on an assumption: The browser extension's reranking effectively changes the AAPA content in participants' feeds without disrupting the user experience.

May combine two claims.

The variation in negative emotions is statistically significant in both experiments: increased in the increased exposure experiment and reduced in the reduced exposure experiment.

“The results are qualitatively similar, the variation in negative emotions is statistically significant in both experiments: increased in the increased exposure experiment and reduced in the reduced exposure experiment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The observation of more political content was uniformly distributed across conditions.

“however, this observation was uniformly distributed across conditions”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The observed drop in participation was primarily due to the use of unsupported devices or browsers (e.g., mobile).

“The observed drop in participation was primarily due to the use of unsupported devices or browsers (e.g., mobile).”

May combine two claims.

The BERT-based classification predicting the assigned experiment achieved an F1 score of 53.9% (± 4.7%) on a balanced dataset, which was statistically indistinguishable from random guessing.

“The classification achieved an F1 score of 53.9% (± 4.7%) on a balanced dataset, performance statistically indistinguishable from random guessing.”

Classified as probably not a substantial claim.

The reported treatment effects are based on participants who completed both the post-experiment survey and at least one in-feed survey for the corresponding outcome (N=1,090).

“To maintain a consistent sample across the postexperiment and in-feed analyses, the reported treatment effects are based on participants who completed both the post-experiment survey and at least one in-feed survey for the corresponding outcome (N = 1,090).”

May combine two claims.

Both treatment and control groups were well balanced across covariates, and there was no evidence of differential attrition between conditions.

“Both treatment and control groups were well balanced across covariates, and there was no evidence of differential attrition between conditions (Appendix S1.3).”

Classified as probably not a substantial claim.

The approach enables scientists to test a much broader set of interventions beyond those platforms might agree to test.

“Our approach enables scientists to test a much broader set of interventions beyond those that platforms might agree to test, fostering a more robust ecosystem for independent social media research.”

• AssumptionRests on an assumption: The browser extension's real-time reranking of the feed is equivalent to platform-level feed ranking for the purpose of studying algorithmic effects.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study was preregistered.

“We conducted a preregistered field experiment on X, the most used social media platform for political discourse in the U.S. (42), using our extension to dynamically rerank participants' social media content by either increasing or decreasing exposure to content that expresses these eight factors of antidemocratic attitudes and partisan animosity (AAPA) over the course of a week.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Describing the causal relation between social media and polarization as a unified phenomenon remains challenging because platforms are complex socio-technical systems and small variations in design or exogenous factors can lead to different outcomes.

“Describing the causal relation between social media and polarization as a unified phenomenon remains challenging: platforms are complex socio-technical systems, and small variations in design or exogenous factors can lead to different outcomes (56).”

• cites [56] sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

The study used Bernoulli randomization to assign participants to either the Reduced or Increased Exposure experiment, and subsequently to treatment and control groups with equal probability.

“We used Bernoulli randomization to assign participants to either the Reduced or Increased Exposure experiment, and subsequently to treatment and control groups with equal probability (Section S1.3).”

May combine two claims.

The method was used to conduct a preregistered 10-day field experiment with 1,256 participants on X during the 2024 U.S. presidential campaign.

“We developed a platform-independent method that reranks participants' feeds in real-time and used this method to conduct a preregistered 10day field experiment with 1,256 participants on X during the 2024 U.S. presidential campaign.”

Probably not a substantial claim: may not pass S2 (checkable).

The findings provide a potential pathway for platforms to address political polarization and societal division through algorithmic interventions.

“As political polarization and societal division become increasingly linked to social media activity, our findings provide a potential pathway for platforms to address these challenges through algorithmic interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Reliance on in-feed surveys to assess immediate responses follows established practices of ecological momentary assessment.

“Our reliance on in-feed surveys to assess immediate responses to stimuli follows established practices of ecological momentary assessment (46).”

• cites [46] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The experiment examines short-term effects and does not address whether these effects persist over time.

“Additionally, the experiment examines short-term effects-immediate in-feed responses and post-survey responses measured soon after the experiment-but does not address whether these effects persist over time.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Neither experiment observed differential attrition rates across conditions.

“Neither experiment observed differential attrition rates across conditions.”

May combine two claims.

Post-experiment emotion effects were not significant, suggesting the effects were likely short-lived.

“Post-experiment emotion effects were not significant, suggesting that the effects were likely short-lived.”

Probably not a substantial claim: may not pass S2 (checkable).

The method can be integrated into social media AIs to mitigate harmful personal and societal consequences.

“The success of this method shows that it can be integrated into social media AIs to mitigate harmful personal and societal consequences.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

In scenarios where this ranking approach is integrated into the feed algorithm, the intervention would be continuously applied, and any effects may dissipate once it is turned off.

“We also note that in scenarios where this ranking approach is integrated into the feed algorithm, the intervention would be continuously applied, and any effects may dissipate once it is turned off.”

May combine two claims.

Engagement analyses indicate a practical trade-off: interventions that downrank AAPA may reduce short-term engagement volume, posing challenges for engagement-driven business models.

“At the same time, our engagement analyses indicate a practical trade-off: interventions that downrank AAPA may reduce short-term engagement volume, posing challenges for engagement-driven business models”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors preregistered two sample size targets that, based on a pilot study, would enable them to detect effects of 2 and 2.5 degrees on the feeling thermometer scale with 95% power.

“Due to the uncertainty in the recruitment process, we preregistered two sample size targets that, based on a pilot study, would enable us to detect effects of 2 and 2.5 degrees on the feeling thermometer scale with 95% power.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The final completion rates were 69.1% (798) for Bovitz and 90.3% (458) for CloudResearch.

“The final completion rates were 69.1% (798) for Bovitz and 90.3% (458) for CloudResearch.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Our work does not introduce substantial additional risks compared to existing algorithmic ranking methods.

“our work does not introduce substantial additional risks.”

Classified as probably not a substantial claim.

Mixed results reveal the difficulty in identifying what, if any, algorithmic intervention might help reduce polarization, especially during politically charged times.

“These mixed results reveal the difficulty in identifying what, if any, algorithmic intervention might help reduce polarization, especially during politically charged times.”

Probably not a substantial claim: may not pass S3 (load-bearing).

X is the most used social media platform for political discourse in the U.S.

“We conducted a preregistered field experiment on X, the most used social media platform for political discourse in the U.S. (42)”

• cites [42]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors used randomization inference to test whether the imbalance in background characteristics of participants assigned to treatment vs. control in each of the two experiments was larger than expected from chance.

“We used randomization inference to test whether the imbalance in the background characteristics of the participants assigned to treatment vs. control in each of the two experiments was larger than we would expect from chance (61).”

• cites [61]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The study targeted participants who used X, were 18 or older, resided in the United States, and self-identified as either Democrats or Republicans.

“We targeted participants who used X, were 18 or older, resided in the United States, and self-identified as either Democrats or Republicans (Appendix S5).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The reranking process took approximately three seconds and was consistent across all experimental conditions.

“The reranking process took approximately three seconds and was consistent across all experimental conditions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants could withdraw consent or discontinue participation at any time without penalty.

“Participants were given the option to withdraw their consent or discontinue participation at any time without penalty.”

Probably not a substantial claim: may not pass S3 (load-bearing).

False Discovery Rate adjustment was applied to account for testing multiple hypotheses.

“To account for testing multiple hypotheses, we applied False Discovery Rate adjustment (Section S6).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors optimized for power in detecting effects on their primary dependent variables, i.e., the in-feed and post-experiment effects on affective polarization.

“We optimized for power in detecting effects on our primary dependent variables, i.e., the in-feed and post-experiment effects on affective polarization.”

Classified as probably not a substantial claim. May combine two claims.

Participants with fewer than ten feed load events after ten days were excluded from the study and did not receive the post-experiment survey.

“After ten days, participants with minimal participation, i.e., those for whom we recorded fewer than ten feed load events, were excluded from the study and did not receive the post-experiment survey.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All procedures were approved by the university's Institutional Review Board.

“All procedures were approved by our university's Institutional Review Board.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Quasi-Poisson models were used to estimate effects on engagement volume and Beta models to estimate effects on engagement rates, to appropriately model the nature of the outcomes.

“Finally, to appropriately model the nature of the outcomes, we used quasi-Poisson models to estimate the effects on engagement volume and Beta models to estimate the effects on engagement rates (Section S9).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors used simulations (using the simR package) to conduct the power analyses, with models corresponding to those described in Section S1.4, because of the complexity of the mixed-effects models.

“Due to the complexity of the mixed-effects models, we used simulations to conduct the power analyses (using the simR package) with models that correspond to those described in Section S1.4.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The order of the survey questions other than the political feeling question was randomized.

“The political feeling question was asked first, whereas the order of the other questions was randomized.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All participants provided informed consent before installing the browser extension and participating in the study.

“All participants were provided with informed consent before installing the browser extension and participating in the study.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

The content-focused hypothesis has been difficult to operationalize into interventions, making studies that intervene on cross-partisan exposure and reverse-chronological ranking attractive but more diffuse in their impact and thus more likely to observe mixed results.

“This content-focused hypothesis has been difficult to operationalize into interventions, making studies that intervene on cross-partisan exposure and reverse-chronological ranking attractive but more diffuse in their impact and thus more likely to observe mixed results.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The numeric value of the slider was not displayed to prevent anchoring effects.

“The numeric value was not displayed to prevent anchoring effects”

May combine two claims.

Results are similar when restricting to participants who completed the post-survey for the post-experiment effect and those who completed at least one in-feed survey for the in-feed effect.

“We observe similar results when we consider only those who completed the post-survey when analyzing the post-experiment effect and only those who completed at least one in-feed survey when analyzing the in-feed effect (Sections S11.1 and S11.2).”

Probably not a substantial claim: may not pass S3 (load-bearing).

All analyses follow the preregistration plan filed at the Open Science Foundation.

“All analyses follow our preregistration plan filed at the Open Science Foundation (48).”

• cites [48]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The primary moderators for heterogeneous treatment effects were preregistered as party identification, party strength, and proportion of political content in the baseline feed; secondary moderators were gender, race, age, income, and socioeconomic ladder.

“As preregistered, we considered party identification, party strength, and the proportion of political content in the participant's feed during the baseline period as primary moderators, and gender, race, age, income, and socioeconomic ladder as secondary moderators.”

Classified as probably not a substantial claim. May combine two claims.

The study used a large language model to score each political post from zero to eight based on the number of eight AAPA factors expressed in its text, classifying a post as AAPA if it reflected at least four factors.

“We then scored each political post from zero to eight based on the number of the eight AAPA factors expressed in its text, and we classified a post as AAPA if it reflected at least four of these eight factors.”

Classified as probably not a substantial claim.

The in-feed surveys were shown at an equivalent rate and position in control conditions.

“For both experiments, the in-feed surveys were shown at an equivalent rate and position in the control conditions (Appendix S1.2).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The browser extension added a latency of about three seconds when opening the feed for the first time, consistent across all experimental conditions.

“This process added a latency of about three seconds when opening the feed for the first time, which was consistent across all experimental conditions.”

May combine two claims.

No evidence of covariate imbalance was found in the Reduced Exposure (p = 0.59) or the Increased Exposure experiment (p = 0.18).

“We did not find evidence of covariate imbalance in the Reduced Exposure (𝑝 = 0.59) or the Increased Exposure experiment (𝑝 = 0.18).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The survey was placed consistently in treatment and control conditions.

“As in the Reduced Exposure experiment, the survey was placed consistently in the treatment and control conditions.”

Classified as probably not a substantial claim.

The analysis of in-feed survey outcomes used linear mixed-effects models to account for multiple observations per participant, controlling for the participant's average baseline response and recruitment platform.

“To estimate the effects on the outcomes measured with in-feed surveys, we used linear mixed-effects models to account for the multiple observations per participant.We regressed the outcome on the treatment assignment, the participant's average response on the same question during the three-day baseline period, and the recruitment platform indicator.”

Classified as probably not a substantial claim.

Participants were allocated to the experiments using Bernoulli randomization.

“We used Bernoulli randomization to allocate participants to the experiments with either Reduced or Increased Exposure to AAPA content”

Classified as probably not a substantial claim. May combine two claims.

The analysis of post-experiment outcomes was preregistered and used ordinary least squares, controlling for the participant's pre-survey response and recruitment platform.

“As preregistered, to estimate the effects of the interventions on the post-experiment outcomes, we used ordinary least squares, regressing the outcome variable on the treatment assignment indicator, the participant's response to the same question on the pre-survey, and a recruitment platform indicator (Section S1.4).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Return rate and time spent on the platform are key metrics for social media platforms.

“Our analysis investigates the return rate and time spent on the platform, key metrics for social media platforms (62)”

• cites [62] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

At the end of the experiment, participants received financial compensation and a debriefing statement.

“At the end of the experiment, participants received financial compensation through the recruitment platform and a debriefing statement via email.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Allocation continued until the respective target quotas from power analyses were reached.

“until we reached the respective target quotas from our power analyses (48)”

• cites [48]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Survey questions were adapted based on participants' political leanings to ensure contextual relevance.

“To ensure contextual relevance, survey questions were adapted based on participants' political leanings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The survey was not shown with every post to avoid survey fatigue.

“To avoid survey fatigue, the survey was not shown with every post.”

May combine two claims.

No evidence of differential attrition patterns was found in either experiment: Reduced Exposure p = 0.34; Increased Exposure p = 0.72.

“We did not find any evidence of differential attrition patterns in either of the two experiments: Reduced Exposure, 𝑝 = 0.34; and Increased Exposure, 𝑝 = 0.72.”

Probably not a substantial claim: may not pass S3 (load-bearing).

LASSO regression selected only pre-experiment survey answers as control covariates.

“We also preregistered that we would consider including other control covariates selected via LASSO regression on the outcomes, but found that only the pre-experiment survey answers were selected.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study used a preregistered target of at least 1,100 participants (600 Reduced, 500 Increased) based on power analyses on pilot studies.

“We preregistered a target of at least 1,100 participants to complete the full study (600 in the Reduced Exposure experiment; 500 in the Increased Exposure experiment) based on power analyses performed on pilot studies (48).”

• cites [48]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

An algorithm that upranks content reflecting genuine political dialogue is less likely to polarize than one that upranks demagoguery.

“An algorithm that upranks content reflecting genuine political dialogue is less likely to polarize than one that upranks demagoguery.”

May combine two claims.

No evidence of differential attrition rates was found in either experiment: Reduced Exposure p = 0.38; Increased Exposure p = 0.92.

“We found no evidence of differential attrition rates in either of the two experiments: Reduced Exposure, 𝑝 = 0.38; and Increased Exposure, 𝑝 = 0.92.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Volume and rate of likes and reposts play a significant role in how X's feed algorithm ranks posts.

“along with volume and rate of likes and reposts, which play a significant role in how X's feed algorithm ranks posts, as documented in the open-source code released by X (63).”

• cites [63]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Participants were allocated to treatment or control with equal probability.

“and to a treatment arm (treatment or control) with an equal probability.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors considered all questions in the pre-experiment survey as covariates in the balance test.

“We considered all questions in the pre-experiment survey as covariates”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors ran a power analysis based on data from a pilot study with 70 participants that followed the same experimental design and analysis procedure, to inform the sample size for their experiments.

“To inform the sample size for our experiments, we ran a power analysis based on data from a pilot study with 70 participants that followed the same experimental design and analysis procedure.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Changing the gap between sessions from one hour to 45 or 90 minutes, or adjusting the maximum session length from two hours to 1.5 or 2.5 hours, did not substantively affect the results.

“We also note that changing the gap between sessions from one hour to 45 or 90 minutes, or adjusting the maximum session length from two hours to 1.5 or 2.5 hours, did not substantively affect the results reported below.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The engagement rate analysis uses Beta regression models to account for the fact that the outcomes are rates.

“We use Beta regression models to appropriately account for the fact that the outcomes are rates.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The data collected is limited to actions on the main feed and does not include any private content.

“This data is limited to the actions on the main feed and does not include any private content.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study used a held-out dataset composed of 5,000 posts annotated by GPT-4 and a random subset of 300 posts annotated by humans to evaluate the model.

“We evaluate the model on a held-out dataset composed of 5,000 posts annotated by GPT-4 and a random subset of 300 posts annotated by humans.”

Probably not a substantial claim: may not pass S3 (load-bearing).

When the same averaging approach is applied to the unweighted data and an ordinary linear regression is run, the resulting point estimates and confidence intervals are similar to those obtained with the mixed-effects models.

“We note that when we apply the same averaging approach to the unweighted data and run an ordinary linear regression, the resulting point estimates and confidence intervals are similar to those obtained with the mixed-effects models.”

Classified as probably not a substantial claim.

The reweighting analysis included participants who completed the post-experiment survey and at least one affective polarization or emotions in-feed survey during the intervention period (N=1,112).

“We consider the participants who completed the post-experiment survey and at least one affective polarization or emotions in-feed survey during the intervention period (N=1,112).”

Probably not a substantial claim: may not pass S4 (based).

Mirroring the same procedure for the control group ensures the similarity of the two conditions in all respects, including potential server latency.

“Despite not adding the post, mirroring the same procedure for the control group ensures the similarity of the two conditions in all respects, including potential server latency.”

Probably not a substantial claim: may not pass S3 (load-bearing).

To accommodate rates of zero or one, the outcomes are transformed using the transformation by Smithson and Verkuilen (78).

“To accommodate rates of zero or one, we transform the outcomes using the commonly used transformation by Smithson and Verkuilen (78).”

• cites [78] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The analysis did not trim the weights because there were no extreme values.

“We did not trim the weights since there were no extreme values.”

Classified as probably not a substantial claim. May combine two claims.

The design required two control conditions because the Reduced Exposure intervention placed the in-feed survey instrument where down-ranked content had previously been, and the Increased Exposure intervention placed the in-feed assessment instrument where up-ranked content would have been inserted.

“The design required two control conditions since the Reduced Exposure intervention placed the in-feed survey instrument where down-ranked content had previously been, and the Increased Exposure intervention placed the in-feed assessment instrument where up-ranked content would have been inserted.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study randomly split the dataset into 45,000 posts for training and 5,000 posts for testing.

“We later randomly split this dataset into 45,000 posts for training and 5,000 posts for testing.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study used a stratified sample of 300 accounts, selecting 50 users from each of six groups.

“Finally, we took a stratified sample of 300 accounts, selecting 50 users from each of the six groups defined above.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The scoring service uses a fixed seed for reproducibility.

“We use a fixed seed for reproducibility.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The analysis adjusts for participants' baseline activity and recruitment platform in the engagement analyses.

“To estimate the effects of our interventions, we adjust for participants' baseline activity and recruitment platform, similar to our main analysis (Tables S6 and S7).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The percentage of AAPA content varies over time and can be influenced by major political events.

“These percentages vary over time and can be influenced by major political events Fig. S7.”

• AssumptionRests on an assumption: The LLM accurately classifies posts as AAPA based on the eight factors.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The experiments were designed to detect meaningful effects on affective polarization and are not necessarily powered to detect small effects on engagement.

“Finally, as we discuss in Section S1.3, our experiments were designed to detect meaningful effects on affective polarization and are not necessarily powered to detect small effects on engagement.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Using linear models with log-transformed outcomes, Poisson models, or quasi-Poisson models yields similar results to the reported linear models.

“We report results from linear models but note that using linear models with log-transformed outcomes, Poisson models, or quasi-Poisson models yields similar results.”

Classified as probably not a substantial claim.

The analysis used raking to generate weights that calibrate to population estimates of party affiliation, education, and race.

“We used raking to generate the weights that calibrate to the population estimates of these participant characteristics.”

May combine two claims.

The experiments were designed to detect effects on affective polarization and are likely not as well-powered to detect effects on the seven remaining attitudes.

“We note that our experiments were designed to detect effects on affective polarization (Appendix S1.3), and are likely not as well-powered to detect effects on these outcomes.”

Classified as probably not a substantial claim.

The implementation ensures that the selection logic for the survey position is equivalent for the two experimental conditions, differing only in the down-ranking of AAPA posts for the treatment group.

“This implementation ensures that the selection logic for the survey position is equivalent for the two experimental conditions, differing only in the down-ranking of AAPA posts for the treatment group.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study reweighted the sample to reflect the X user population in terms of party affiliation, education, and race to examine generalizability.

“To examine the generalizability of our treatment effect estimates, we reweighted our sample to reflect the X user population in terms of party affiliation, education, and race.”

May combine two claims.

The treatment effects do not significantly vary over time and there is no evidence of decay effects within the week of the intervention.

“These results suggest that the treatment effects do not significantly vary over time and that there is no evidence of decay effects within the week of the intervention.”

Classified as probably not a substantial claim.

The proportions of X users across party affiliation, education, and race were determined using the 2024 National Public Opinion Reference Survey conducted by Pew Research.

“To determine the proportion of X users across these dimensions, we relied on the 2024 National Public Opinion Reference Survey conducted by Pew Research.”

May combine two claims.

Participants continued to respond with a similar degree of variation, providing no evidence of convergence.

“This suggests that participants continued to respond with a similar degree of variation, providing no evidence of convergence.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In addition to the Beta regression models, regression tables from an equivalent analysis using linear models are reported.

“In addition to the results from the Beta regression models, we also report regression tables from an equivalent analysis using linear models (Tables S14-S17).”

Classified as probably not a substantial claim. May combine two claims.

The analysis performs False Discovery Rate (FDR) adjustment using the method described in (77), considering each experiment separately and grouping variables into primary, secondary, and tertiary outcomes.

“We perform False Discovery Rate (FDR) adjustment using the method described in (77), considering each experiment (Reduced and Increased Exposure) separately and grouping the variables as:”

• cites [77] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

For each post, three labels were collected using Prolific and the value was assigned based on majority voting.

“For each post, we collect three labels using Prolific and assign the value based on majority voting.”

Classified as probably not a substantial claim. May combine two claims.

Because there is no well-established method for incorporating raking weights into mixed-effect models, the study averaged each participant's responses to the in-feed surveys and ran weighted regression using svyglm.

“Since we are unaware of any well-established method for incorporating raking weights into mixed-effect models, we first averaged each participant's responses to the infeed surveys and then ran weighted regression using svyglm.”

May combine two claims.

Together, these results suggest that the experimental condition did not significantly impact participants' experience, as they did not perceive or report notable differences.

“Together, these results suggest that the experimental condition did not significantly impact participants' experience, as they did not perceive or report notable differences.”

• AssumptionRests on an assumption: Participants' self-reports about whether the extension affected their experience accurately reflect the actual impact of the extension on their experience.
• AssumptionRests on an assumption: The BERT-based classifier and open-ended response coding are sufficiently sensitive to detect a meaningful impact on participants' experience if one existed.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study computed the effective sample size of the reweighted sample to estimate the proportional increase in variance due to reweighting.

“We compute the effective sample size (79) of the reweighted sample to estimate the proportional increase in variance due to reweighting.”

• cites [79]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The experiment was conducted during a period of heightened political activity preceding the 2024 US election.

“The experiment was conducted during a period of heightened political activity preceding the 2024 US election.”

Classified as probably not a substantial claim.

The analysis used the svyglm function from the survey R package to incorporate raking weights.

“To incorporate the raking weights in our analyses, we used the svyglm function from the survey R package (80,81).”

• cites [80]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [81] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The 2024 National Public Opinion Reference Survey was fielded between February 1 and June 10, 2024, just before the start of the experiment (July 7, 2024) and after the rebranding of Twitter to X.

“The survey was fielded between February 1 and June 10, 2024, just before the start of our experiment (July 7, 2024) and after the rebranding of Twitter to X.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Manual review of a random sample of 100 responses found complete agreement with GPT-4's annotations.

“We manually reviewed a random sample of 100 responses and found complete agreement with GPT-4's annotations.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Quasi-Poisson regression effectively modeled the overdispersion in the engagement data and was less sensitive to outliers than a negative binomial model.

“We chose a quasi-Poisson regression as it effectively modeled the overdispersion in the data and was less sensitive to outliers than a negative binomial model.”

Classified as probably not a substantial claim.

The pilot study revealed that the Reduced Exposure group required more participants to detect the effect of the intervention.

“as our pilot study revealed that the first group required more participants to detect the effect of the intervention (48).”

• cites [48]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.