RESUMO

Null hypothesis significant testing (NHST) is the dominant statistical approach in the geriatric and rehabilitation fields. However, NHST is routinely misunderstood or misused. In this case, the findings from clinical trials would be taken as evidence of no effect, when in fact, a clinically relevant question may have a "non-significant" p-value. Conversely, findings are considered clinically relevant when significant differences are observed between groups. To assume that p-value is not an exclusive indicator of an association or the existence of an effect, researchers should be encouraged to report other statistical analysis approaches as Bayesian analysis and complementary statistical tools alongside the p-value (eg, effect size, confidence intervals, minimal clinically important difference, and magnitude-based inference) to improve interpretation of the findings of clinical trials by presenting a more efficient and comprehensive analysis. However, the focus on Bayesian analysis and secondary statistical analyses does not mean that NHST is less important. Only that, to observe a real intervention effect, researchers should use a combination of secondary statistical analyses in conjunction with NHST or Bayesian statistical analysis to reveal what p-values cannot show in the geriatric and rehabilitation studies (eg, the clinical importance of 1kg increase in handgrip strength in the intervention group of long-lived older adults compared to a control group). This paper provides potential insights for improving the interpretation of scientific data in rehabilitation and geriatric fields by utilizing Bayesian and secondary statistical analyses to better scrutinize the results of clinical trials where a p-value alone may not be appropriate to determine the efficacy of an intervention.

Assuntos

RESUMO

Introducción. La prueba de significancia de la hipótesis nula (PSHN) constituye la herramienta más usada para evaluar hipótesis científicas y tomar decisiones al respecto, en especial en ciencias de la salud. Sin embargo, por décadas ha estado en el centro del debate, ya que se han identificado varios problemas conceptuales y de interpretación. Se realizó una revisión de artículos científicos que ilustran las críticas de esta controversia y su relevancia en el ámbito de la investigación en salud. Algunas alternativas para la PSHN son una adecuada interpretación del valor p, uso de intervalos de confianza, incluir el tamaño del efecto y adoptar un marco de inferencia bayesiana. En todos los casos en que se utilice PSHN, su uso debe ser claramente justificado.

Background. Null hypothesis significance testing (NSHT) constitutes the most widely applied tool for the evaluation of scientific hypotheses and decision making in health sciences. However, the method has been the centre of a heated debate where various criticisms related to conceptual and interpretational problems. A review of scientific articles that illustrate the criticisms of this controversy and its relevance in the field of health research was carried out. Some alternatives for the NSHT are an adequate interpretation of the p-value, use of confidence intervals, including the effect size and adopting a Bayesian inference framework. In all cases where NSHT is used, its use should be clearly justified.

RESUMO

Given the limitations of frequentist method for null hypothesis significance testing, different authors recommend alternatives such as Bayesian inference. A poor understanding of both statistical frameworks is common among clinicians. The present is a gentle narrative review of the frequentist and Bayesian methods intended for physicians not familiar with mathematics. The frequentist p-value is the probability of finding a value equal to or higher than that observed in a study, assuming that the null hypothesis (H0) is true. The H0 is rejected or not based on a p threshold of 0.05, and this dichotomous approach does not express the probability that the alternative hypothesis (H1) is true. The Bayesian method calculates the probability of H1 and H0 considering prior odds and the Bayes factor (Bf). Prior odds are the researcher's belief about the probability of H1, and the Bf quantifies how consistent the data is concerning H1 and H0. The Bayesian prediction is not dichotomous but is expressed in continuous scales of the Bf and of the posterior odds. The JASP software enables the performance of both frequentist and Bayesian analyses in a friendly and intuitive way, and its application is displayed at the end of the paper. In conclusion, the frequentist method expresses how consistent the data is with H0 in terms of p-values, with no consideration of the probability of H1. The Bayesian model is a more comprehensive prediction because it quantifies in continuous scales the evidence for H1 versus H0 in terms of the Bf and the

Dadas las limitaciones del método de significancia frecuentista basado en la hipótesis nula, diferentes autores recomiendan alternativas como la inferencia bayesiana. Es común entre los médicos una comprensión deficiente de ambos marcos estadísticos. Esta es una revisión narrativa amigable de los métodos frecuentista y bayesiano dirigida quienes no están familiarizados con las matemáticas. El valor de p frecuentista es la probabilidad de encontrar un valor igual o superior al observado en un estudio, asumiendo que la hipótesis nula (H0) es cierta. La H0 se rechaza o no con base en un umbral p de 0.05, y este enfoque dicotómico no expresa la probabilidad de que la hipótesis alternativa (H1) sea verdadera. El método bayesiano calcula la probabilidad de H1 y H0 considerando las probabilidades a priori y el factor de Bayes (fB). Las probabilidades a priori son la creencia del investigador sobre la probabilidad de H1, y el fB cuantifica cuán consistentes son los datos con respecto a H1 y H0. La predicción bayesiana no es dicotómica, sino que se expresa en escalas continuas del fB y de las probabilidades a posteriori. El programa JASP permite realizar análisis frecuentista y bayesiano de una forma simple e intuitiva, y su aplicación se muestra al final del documento. En conclusión, el método frecuentista expresa cuán consistentes son los datos con H0 en términos de valores p, sin considerar la probabilidad de H1. El modelo bayesiano es una predicción más completa porque cuantifica en escalas continuas la evidencia de H1 versus H0 en términos del fB y de las probabilidades a posteriori.

Assuntos

RESUMO

Although null hypothesis testing (NHT) is the primary method for analyzing data in many natural sciences, it has been increasingly criticized. Recently, approaches based on information theory (IT) have become popular and were held by many to be superior because it enables researchers to properly assess the strength of the evidence that data provide for competing hypotheses. Many studies have compared IT and NHT in the context of model selection and stepwise regression, but a systematic comparison of the most basic uses of statistics by ecologists is still lacking. We used computer simulations to compare how both approaches perform in four basic test designs (t-test, ANOVA, correlation tests, and multiple linear regression). Performance was measured by the proportion of simulated samples for which each method provided the correct conclusion (power), the proportion of detected effects with a wrong sign (S-error), and the mean ratio of the estimated effect to the true effect (M-error). We also checked if the p-value from significance tests correlated to a measure of strength of evidence, the Akaike weight. In general both methods performed equally well. The concordance is explained by the monotonic relationship between p-values and evidence weights in simple designs, which agree with analytic results. Our results show that researchers can agree on the conclusions drawn from a data set even when they are using different statistical approaches. By focusing on the practical consequences of inferences, such a pragmatic view of statistics can promote insightful dialogue among researchers on how to find a common ground from different pieces of evidence. A less dogmatic view of statistical inference can also help to broaden the debate about the role of statistics in science to the entire path that leads from a research hypothesis to a statistical hypothesis.

RESUMO

Most randomized trials are designed and analyzed using frequentist statistical approaches such as null hypothesis testing and P values. Conceptually, P values are cumbersome to understand, as they provide evidence of data incompatibility with a null hypothesis (e.g., no clinical benefit) and not direct evidence of the alternative hypothesis (e.g., clinical benefit). This counterintuitive framework may contribute to the misinterpretation that the absence of evidence is equal to evidence of absence and may cause the discounting of potentially informative data. Bayesian methods provide an alternative, probabilistic interpretation of data. The reanalysis of completed trials using Bayesian methods is becoming increasingly common, particularly for trials with effect estimates that appear clinically significant despite P values above the traditional threshold of 0.05. Statistical inference using Bayesian methods produces a distribution of effect sizes that would be compatible with observed trial data, interpreted in the context of prior assumptions about an intervention (called "priors"). These priors are chosen by investigators to reflect existing beliefs and past empirical evidence regarding the effect of an intervention. By calculating the likelihood of clinical benefit, a Bayesian reanalysis can augment the interpretation of a trial. However, if priors are not defined a priori, there is a legitimate concern that priors could be constructed in a manner that produces biased results. Therefore, some standardization of priors for Bayesian reanalysis of clinical trials may be desirable for the critical care community. In this Critical Care Perspective, we discuss both frequentist and Bayesian approaches to clinical trial analysis, introduce a framework that researchers can use to select priors for a Bayesian reanalysis, and demonstrate how to apply our proposal by conducting a novel Bayesian trial reanalysis.

Assuntos

RESUMO

Resumen ANTECEDENTES: El valor de p es el método más empleado para estimar la significación estadística de cualquier hallazgo; sin embargo, en los últimos años se ha intensificado su debate al respecto, debido a la baja credibilidad y reproducibilidad de diversos estudios. OBJETIVO: Describir el estado actual del concepto del valor de p y la significación estadística (prueba de significación de la hipótesis nula [por sus siglas en inglés: Null Hypothesis Significance Testing: NHST]), especificar los problemas más importantes y puntualizar las soluciones propuestas para la mejor utilización de los conceptos. METODOLOGÍA: Se llevó a cabo la búsqueda bibliográfica en MEDLINE y Google Scholar, con los términos: "NHST", "Statistical significance; P value" en idioma inglés y español, de 2018-2019, limitándose a la selección de artículos publicados entre 2005 y 2019, mediante la revisión de tipo narrativo con búsqueda manual; sobre todo estudios de metodología. RESULTADOS: La búsqueda global reportó 1411 artículos: 875 de PubMed y 536 de Google Scholar. Se excluyeron 817 por duplicación, 155 sin acceso completo y 414 ensayos clínicos (sin metodología estadística); los 25 restantes fueron el motivo del análisis. CONCLUSIONES: El concepto del valor de p no es simple, tiene varias falacias y malas interpretaciones que deben considerarse para evitarlas en lo posible. Se recomienda no usar el término "estadísticamente significativo" o "significativo", sustituir el umbral de 0.05 por 0.005, informar valores de p precisos y con IC95%, riesgo relativo, razón de momios, tamaño del efecto o potencia y métodos bayesianos.

Abstract BACKGROUND: The P value is the most widely used method of estimating the statistical significance of any finding, however, in recent years the debate over the P value has been increasingly intensified due to the low credibility and reproducibility of many studies. OBJECTIVE: To describe the current state of the concept of the value of P and the statistical significance (Null Hypothesis Significance Testing (NHST), specify the most important problems and point out the solutions proposed in the literature for their best use. METHODOLOGY: Search in MEDLINE and Google Scholar, with the terms: "NHST", "Statistical significance; P value "in English and Spanish, carried out from 2018-2019, limited to articles published from 2005 to 2019, and a narrative-type review with manual search. Articles on methodology were preferably selected. RESULTS: The global search yielded 1411 articles, 875 from PubMed and 536 from Google Scholar. 817 were excluded by duplication, 155 without full access, 414 from clinical trials, without statistical methodology. The 25 selected articles were the reason for the analysis. CONCLUSIONS: The concept of the value of P is not simple, and it has several fallacies and misinterpretations that must be taken into account to avoid them as much as possible. Recommendations: Do not use "statistically significant" or "significant", replace the threshold of 0.05 with 0.005, report accurate P values with 95% CI, relative risk, odds ratio, effect size or power, and Bayesian methods.

Assuntos

RESUMO

Currently, it has been observed a growing number of publications in all fields of Dentistry. These publications act as scientific evidence, as well as a basis for clinical decision-making in dental care routine. It is important to note that the results and conclusions in articles are based on the p-value that is a purely probabilistic and statistical parameter, and it assists the researcher to accept or reject the null hypothesis being tested. The p-value was proposed by Fisher in 1925, and in Dentistry, it is usual to adopt the p-value stated in 0.05.1 In practical terms, when a statistical test results in p-value less than 0.05, the null hypothesis must be rejected (equality between groups), assuming that there is a difference between the assessed groups.2 In other words, p<0.05 indicates statistically significant difference between groups. Under a critical look, the researcher and reader should keep in mind that a statistical difference is not always reflecting a true clinical importance. In addition, a lack of statistical significance does not necessarily relate to the absence of clinical significance. The clinical importance is far beyond statistical calculations based on the p-value.3 A study presents clinical importance when the one being tested presents clinical effect capable to change the behavior of the dentist in daily routine. This judgment should be done by the researcher based on the results of his/her research, clinical experience and actual knowledge. In addition, estimates of effect sizes should be presented. This facilitates assessment of how large or small the observed effect could actually be in the population of interest, and hence how clinically important it could be. Kassab et al. (2006)4 compared periodontal parameters in groups with and without chemical biomodifciation of the root prior surgical coverage in cases of gingival recession. The group, that used edetic acid, statistically improved the periodontal parameters in relation to the group without surface biomodification. However, this difference was imperceptible to both dentist and patient. That is, the clinical result of root coverage will be the same when using or not acid biomodification of the root. In other words, there was not an important clinical effect of this step, although there was a significant difference. In the above example, it is clear that just because a statistic test is significant doesn't mean the effect it measures is significant or clinically important. Then, researchers

Assuntos

RESUMO

The validity of a study depends on its proper planning, execution and analysis. If these are sufficiently correct, the decision to apply the recommendations issued depends on the expected clinical effect. This effect may have random variations, hence the need to use statistical inference. For years the p-value has been used to determine this statistical significance and the confidence intervals to measure the magnitude of the effect. In this review we present a proposal of how to interpret the 95 % confidence intervals (CI 95 %) as estimators of the expected effect variability based on considering the threshold or value of clinical significance and the null value of the difference or rejection of statistical significance. Thus, an association or effect where the CI 95 % includes the null value (no effect or difference) is interpreted as inconclusive; one between the null value and the clinical threshold (without including them) as possibly inconsequential; one that does not include the null value but the clinical threshold as yet not conclusive and one beyond the clinical threshold as conclusive.

La validez de un estudio depende de su adecuada planeación, ejecución y análisis. Si estas son suficientemente correctas, la decisión de aplicar las recomendaciones emitidas depende del efecto clínico esperado. Este efecto puede tener variaciones aleatorias, de ahí la necesidad de usar la inferencia estadística. Durante años se ha usado el valor de p para determinar esta significancia estadística y los intervalos de confianza para medir la magnitud del efecto. En esta revisión se presenta una propuesta de cómo interpretar los intervalos de confianza a 95 % (IC 95 %) como estimadores de la variabilidad del efecto esperado, con base en considerar el umbral o valor de significancia clínica y el valor nulo de diferencia o rechazo de significancia estadística. Con ello, una asociación en la cual el IC 95 % incluye el valor nulo (no efecto o diferencia) es interpretado como no concluyente; uno entre el valor nulo y el umbral clínico (sin incluirlos) como posiblemente intrascendente; uno que no incluye al valor nulo, pero sí al umbral clínico como aún no contundente y uno más allá del umbral clínico como contundente.

Assuntos