P-Hacking: Critical Response
The knowledge of the existence of p-hacking habit among researchers destroys the confidence of readers and believers who look up to science as the solution and answer to most life questions and problems. It makes science to look like a fallacy. Toying around with hypothesis, variables, and values to achieve a statistically significant p-value define the practice of p-hacking. The tendency is destructive to science and as such should be discouraged. Several methods can be adopted such as changing practice behavior, incorporating other analysis methods alongside p-value or replacing it in totality with other methods. This paper will argue the evils of p-hacking and make recommendations to be adopted in an effort to eradicate the practice of p-hacking.
P-Hacking In Brief
P-hacking is a term used to define the following of logically unacceptable methodology to probe data and obtain a significant p-value. P-value refers to a statistically significant coefficient around 0.05 to warrant a research as worth publishing (Gupta, 2018). Scientist resort to this unscrupulous method in order to avoid the stress of generating a hypothesis: A hypothesis that is built on a plain observation of cause and effect. Scientists’ urge and pressure to hack the p-value may be due to the temptation to conduct an analysis midway before collecting all the data (Gupta, 2018). A scientist may do this to decide whether to go on with the research or not. Secondly, anybody may hack the p-value in case of ghost or unaccounted for variables that come up while doing a research. Thirdly, the researcher’s biases can also contribute to this act where the researcher is debating whether to record all variables and which to report after the research (Gupta, 2018). Finally, a scientist may decide to hack the p-value in the event that he has achieved a statistically significant p-value and does not desire to carry on with the collection of more data (Gupta, 2018). Therefore, the reasons for p-hacking may lead one to question the authenticity of scientific publications.
Someone may doubt the entire scientific world with its results. For the scientists, this does not mean that all scientists engage in p-hacking, stayed on course in the attainment of p-values, reproducible results could have been attained to back their work. Presently a lack of reproducibility and forged data is causing many to retract (Kitzes, Turek & Deniz, 2017). The correct p-value assists in determining whether the published scientific research is reliable. A genuine research should proceed from the hypothesis forward to p-value and not the other way operating from a p-value to the hypothesis. However, this applies to the habit of hacking the p-value as a way of designing a hypothesis that takes care of the hacked p-value.
Recommendations for Scientists and Public Readings Relying on P-Values
Flexibility in the Use of P-Value
The significance of the p-value to scientific publication is immense since it introduces a sense of rationality. However, insisting on a rigidly fixed p-value may be the actual cause in tempting scientists to manipulate their values. A more flexible p-value can help avert the danger of scientists being prompted to manipulate their p-values. For example, a rationale can be introduced allowing values at a certain range within 0.05 to be acceptable. These values may occur before or after the standard 0.05 (Sprent & Smeeton, 2016). This will be an incentive to encourage scientists to obtain actual p-values. Think of scientists who have not been able to publish their work because they were 0.00009 away from the standard value of 0.05 (Sprent & Smeeton, 2016). This creates a system of frustration. Alternatively, in case the p-value is not the expected value thereby turning down a research, the researcher should not give up, this is an indication that the research design should be corrected or there is a need for conducting a larger research since smaller researches do not generate a statistically significant p-value (Sprent & Smeeton, 2016).
“Strike It Out”
A more radical way would be expunging the whole technique as a prerequisite in determining the value of a research. An infectious habit like p-hacking has made p-value almost valueless. One cannot be completely certain that the work they are previewing is justifiable or is a result of some shortcuts. In fact, after learning about the reality of p-hacking, there is a constant nagging question in mind about the authenticity of a research work that keeps lingering in the mind while reading a research. This limitation has robbed the p-value of its dignity and scrapping it off and replacing it with a method that will inhibit or reduce the tendency to twist data would be a better approach. A school of thought suggests that replacing p-value with a method like estimation of the research model would be more effective (Perna, Pratesi, Ruiz-Gazen, 2018). The same school of thought suggests that p-value should be used in determining whether a model is appropriate or requires modifications.
Combining the Use of P-Value with Other Techniques
A more gentle way to remedy the habit of p-hacking can be the amalgamation of the p-value with other reliable testing methods. An amalgamation will introduce a system of checks and balances. Other methods such as determination of median p-value, Bayesian tool, and blinded analysis, which involves concealing information so that the researcher is not tempted to hack the results, have been used alongside p-value to reach conclusions with some degree of success (Perna, Pratesi, Ruiz-Gazen, 2018).
Another technique would involve the use of model selection. Model selection operates on the principle that the models for the null and alternative hypothesis are different. In case the model for alternative hypothesis is preferred then the null hypothesis is rejected. An advantage of this method is that it is simple and more objective (Perna, Pratesi, Ruiz-Gazen, 2018). Combining the use of p-value with other techniques can make the process simple. For example, in case, two or more models agree but in disagreement with the P-value, a researcher may opt for the majority, which is more reliable and justifiable. Therefore, it should not be a do or die for researchers to use the p-value. P-value should give way for more relevant measures or work with other methods.
Behavioral Change
The simplest way to solve the menace of p-hacking without modifying it, expunging it, or incorporating it with other techniques is the behavioral change among researchers using p-values. One can make a personal commitment to operate ethically and avoid unscientific conjectures before finishing a research. Researchers can adopt preregistration techniques (Gupta, 2018). Preregistration involves an advance definition of the plan to be followed in the research process. A researcher can stick to the preregistration plan and make it public so that readers and other users of the work have confidence in the results obtained (Gupta, 2018). By committing to follow the preregistered plan, the researcher will record the values even if they are unacceptable as the facts (Gupta, 2018). Such studies may generate research ideas that can be very profound.
Conclusion
P-hacking destroys the credibility of scientific papers and science as a means to solving human problems. The process is unscientific since it involves manipulation of variables or data to achieve a statistically significant p-value. Scientists should work patiently in observing and collecting results and stick to their findings, which may be a path to making a great discovery. Alternatively, the use of p-value can be substituted with other methods such as the use of the median p-value, Bayesian model, and other models. The technique can also be replaced and instead be used to determine the fitness of a model and need of modifying a research model. In case the scientific world will adopt these recommendations, the unnecessary evil of p-hacking plaguing the science world will be controlled or stumped out.
References
Gupta, T. (2018). Psychoinformatics. Redshine Publication
Kitzes, J., Turek, D., & Deniz, F. (Eds.). (2017). The practice of reproducible research: case studies and lessons from the data-intensive sciences. Univ of California Press.
Perna, C., Pratesi, M., & Ruiz-Gazen, A. (Eds.). (2018). Studies in Theoretical and Applied Statistics: SIS 2016, Salerno, Italy, June 8-10 (Vol. 227). Springer.
Sprent, P., & Smeeton, N. C. (2016). Applied nonparametric statistical methods. Chapman and Hall/CRC.