What common statistical errors are found in meta-analyses within strength and conditioning research?

Common errors include ignoring outliers, miscalculating effect sizes by using standard errors instead of standard deviations, ignoring within-study correlations, focusing on within-group rather than between-group results, and failing to account for within-study variance. A study by Kadlec and colleagues found 85% of top meta-analyses contained at least one such error.

Why is it problematic to use standard errors instead of standard deviations when calculating effect sizes?

Standard errors are mathematically smaller than standard deviations, especially with larger sample sizes. Plugging a standard error into an effect size formula like Cohen's D can massively inflate the true effect size, leading to an overestimation of the intervention's impact.

What is the historical origin of meta-analyses and what does it reveal about scientific discourse?

The first modern attempt at a meta-analysis involved a 'beef' between Almroth Wright (who hated statistics) and Karl Pearson (a statistician) over the typhoid vaccine. This history shows that scientific discourse often involves heated debates and that early meta-analyses were conducted with limited tools and imperfect data.

What are the common misconceptions about motivation, according to Self-Determination Theory?

Many people view motivation as a binary 'on or off' state, or something that needs to be manufactured through sheer willpower. Self-Determination Theory suggests motivation exists on a spectrum of quality (extrinsic to intrinsic) and that intrinsic motivation can be cultivated by creating the right conditions, rather than forced.

How do goal hierarchies improve upon traditional SMART goals?

SMART goals are often isolated and decontextualized. Goal hierarchies provide a tiered system with a stable superordinate goal reflecting personal values, supported by intermediate and subordinate goals. This structure offers resilience, equifinality (multiple paths to a goal), and multifinality (one goal supporting multiple higher-level goals), making motivation more robust.

What are the three core psychological needs in Self-Determination Theory, and how do they foster intrinsic motivation?

The three needs are autonomy (authentic choice and volition), competence (feeling effective and capable), and relatedness (feeling connected and understood by others). Fulfilling these needs creates conditions for intrinsic motivation to organically manifest, rather than needing to be manufactured.

How can the COM-B model of behavior change be used in conjunction with goal setting and motivation?

The COM-B model (Capability, Opportunity, Motivation, Behavior) helps determine if a lack of desired behavior change stems from issues in capability (skills, knowledge, physical/psychological capacity) or opportunity (resources, environment, social support), in addition to motivation. It provides a framework to troubleshoot why goals aren't being met, often starting with easier-to-modify capability and opportunity factors.

What should one consider if struggling with workout motivation despite having clear goals?

First, audit your psychological needs: check if your approach supports your autonomy (choice in workouts), competence (appropriate challenge level), and relatedness (social connection). If these are met, revisit your goal hierarchy to ensure it still aligns with your current values and desires.

How can people enhance their 'opportunity' for behavior change, according to the COM-B model?

Opportunities can be enhanced by addressing external factors like access to resources, creating a conducive environment, or seeking social support. Examples include switching to a gym with better equipment, using a tracking app like MacroFactor to reduce friction, or joining a group exercise class for social connection.

Key Moments

Meta-Analysis Errors; Motivation and Behavior Change (Episode 107)

Stronger By Science

Sports3 min read112 min video

Dec 12, 2022|4,832 views|162|18

Save to Pod

Key Moments

TL;DR

Errors in meta-analyses are common; focus on goal hierarchies and self-determination for motivation and behavior change.

Key Insights

85% of highly cited meta-analyses in strength and conditioning contain statistical errors, challenging their status as infallible evidence.

Common meta-analysis errors include ignoring outliers, miscalculating effect sizes, ignoring within-study correlations, focusing on within-group results, and failing to account for within-study variance.

Effective goal setting involves a hierarchy anchored by a stable, self-tied superordinate goal, supported by intermediate and subordinate goals, offering resilience and equifinality.

Motivation exists on a spectrum from amotivation to intrinsic motivation, with Self-Determination Theory highlighting autonomy, competence, and relatedness as key psychological needs for fostering intrinsic motivation.

The COM-B model (Capability, Opportunity, Motivation, Behavior) provides a framework for behavior change, emphasizing that motivation, capability, and opportunity all influence behavior.

Troubleshooting motivation and behavior change involves auditing psychological needs (autonomy, competence, relatedness) first, then examining the goal hierarchy, and finally assessing capability and opportunity.

THE PREVALENCE OF ERRORS IN META-ANALYSES

A recent study revealed that 85% of highly cited meta-analyses in strength and conditioning research contain at least one statistical error. This finding challenges the common perception of meta-analyses as the undisputed gold standard of evidence. While meta-analyses are valuable for synthesizing research, their reliability hinges on the accuracy of the underlying data and statistical methods. Historically, meta-analysis emerged from complex statistical debates rather than a perfect, universally accepted method, underscoring the need for critical evaluation.

COMMON STATISTICAL ERRORS IN META-ANALYSES

Researchers identified several recurring statistical errors. These include ignoring outliers that significantly skew results, miscalculating effect sizes (often by using standard errors instead of standard deviations, leading to inflated effect sizes), failing to account for correlations within studies where multiple outcomes are measured from the same participants, focusing on within-group changes instead of between-group comparisons, and neglecting to properly weight studies based on sample size and variance. The prevalence of these errors, even in highly cited works, necessitates a more critical approach to interpreting meta-analytic findings.

EFFECTIVE GOAL SETTING: THE GOAL HIERARCHY

Moving beyond simplistic SMART goals, a more effective approach involves a goal hierarchy. This structure starts with a superordinate goal deeply connected to one's values and sense of self, providing stability and resilience. Underneath this primary goal are intermediate goals, which are then supported by more specific, yet contextualized, subordinate goals. This hierarchy offers equifinality (multiple paths to a higher goal) and multifinality (a single action supporting multiple goals), enhancing motivation and adaptability when facing challenges.

UNDERSTANDING AND CULTIVATING MOTIVATION

Motivation is not a simple on/off switch; it exists on a spectrum from amotivation to intrinsic motivation. Self-Determination Theory posits that this spectrum is influenced by the fulfillment of three core psychological needs: autonomy (a sense of choice and volition), competence (feeling effective and capable), and relatedness (meaningful connection with others). Rather than manufacturing motivation, the focus should be on creating conditions that allow intrinsic motivation to organically manifest by supporting these fundamental needs.

THE COM-B MODEL FOR BEHAVIOR CHANGE

To translate motivation into action, the COM-B model (Capability, Opportunity, Motivation, Behavior) offers a robust framework. This model highlights that behavior is influenced by an individual's capability (skills, capacity), opportunity (environmental factors, resources), and motivation. Importantly, capability and opportunity can also impact motivation, creating a dynamic interplay. By assessing all these components, individuals can better understand the barriers and facilitators to changing their behaviors and achieving their goals.

A UNIFIED APPROACH TO GOAL ACHIEVEMENT

Achieving goals requires a systematic approach that integrates goal setting, motivation, and behavior change. The process begins with establishing a goal hierarchy, followed by ensuring the fulfillment of psychological needs (autonomy, competence, relatedness) to cultivate intrinsic motivation. Subsequently, the COM-B model is used to assess and optimize capability and opportunity for desired behaviors. When challenges arise during goal pursuit, troubleshooting involves working backward through this model, starting with capability and opportunity, then motivation, and finally re-evaluating the goal hierarchy if necessary.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Concepts

●People Referenced

Common Questions

Meta-analyses are typically at the top of the hierarchy of evidence, but their quality depends on effective execution. Critics argue that many meta-analyses in the field contain statistical errors, undermining their perceived infallibility.

Topics

Statistical Errors Evidence-based Fitness COM-B Model Relatedness Strength And Conditioning

Mentioned in this video

Concepts

SMART Goals

An acronym (Specific, Measurable, Achievable, Relevant, Time-bound) for goal setting, criticized as not empirically great for health and fitness.

COM-B Model

A model of behavior change where motivation is central, alongside capability and opportunity, all influencing behavior.

Self-Determination Theory

A psychological theory by Ryan and Deci that posits motivation exists on a spectrum from low to high quality, based on intrinsic factors and fulfillment of psychological needs (autonomy, competence, relatedness).

Hierarchy of Evidence

A framework that places systematic reviews and meta-analyses at the top, but assumes effective execution.

People

Almroth Wright

An early 1900s physician who developed the typhoid vaccine but despised statistics.