Is it good science to keep adding participants/manipulating data until you find an effect?

Right last one, the final hurdle and all that jazz. Is it good science to manipulating the data and adding participants until you find an effect? We could in a way compare this question to a picture. An artist spends their hours attempting to perfect their creation, constant tweaking, adding colours here, more shading there, all in the pursuit of creating a masterpiece. But can we really think like this for something such as research? Sure given in a context similar to that it might seem fine, but let’s look at this more closely.

I’m going revisit some themes and arguments covered in my last blog, but both blogs are related to finding significant results so I guess some things are bound to crop up more than once.  Let’s use the work of Allers et al as an example of some of the dangers of manipulating data. This research discusses the use of stem cell research as a potential cure for HIV. HIV is an incredibly prominent disease within certain cultures and is responsible for the death of millions of people every year. Now this research is only in starting stages, with treatment only being given to single patients. Now if this research is progressed this could maybe one day lead to a real cure to HIV, but what if the data is manipulated? Now I understand in this case patients will need to be added so that this research can be progressed further, but manipulating the data specifically to find significant results would be beyond unethical. The results need to be 100% certain on a condition as wide spread. Even if the condition is for a handful of people, it is the job of clinicians and professions to provide the highest level of care as possible, and with that honesty is needed. The clinician needs to have 100% confidence in their claims to patients, not say “well actually, this might not be a cure, we don’t know whether it works or not because the evidence was changed.”

In that sense, if we are changing data then all we are doing is kidding ourselves and others. At some point or another the truth will come to light, so why waste others time? Ok, I guess on an individual basis it’s better to publish significant results as it reflects better on them, but how can anyone truly be that selfish when it comes to the welfare of someone’s life? We have to think about the greater good of the whole thing and with this honesty is the best policy. By all means, if a researcher believes there is something significant within their area of study, then repeat the study, but do not simply an existing one to fit for purpose. This is both unethical and deceptive. We turn towards science for truth, facts about life and the universe, from the smallest components such as atoms, to how we exist on this planet. At the very least we want to adhere to this and carry on discovering new answers, and if we manipulate data and have to wriggle our way to get what we want, then that really isn’t good science. We can have confidence in the things that just stand out, not what we dig and dig for. Right blog (or should I say rant) finished, at last there done with!

http://bloodjournal.hematologylibrary.org/content/117/10/2791.short

Advertisements

5 thoughts on “Is it good science to keep adding participants/manipulating data until you find an effect?

  1. Your altered HIV data example is a definate possibility, as researchers have altered data in studies before. Cyril Burt did a lot for education and intelligence, and his research helped form the 11 plus test in Britain. After his death many looked into his data and started to question whether or not his studies into twins and heritability were as perfect as he said they were. Some discovered that a lot of his data had been altered, and that his research assistants were even given false names to protect them if anyone discovered his fraud! So is it possible that our education system was built on lies? Maybe. Although some have argued against the idea of Burt commiting fraud it is still a very worrying possibility that may not be unique to Burt. ‘The science and politics of I.Q’ by Leon Kamin looks this case in more detail

  2. I understand your point concerning HIV and the importance of providing patients with 100% correct information about treatment and I agree that it would be unfair to give false hope. However even if the drug only had small chance of reducing or eliminating symptoms then some people may like to have that option, particularly if they have tried other treatments. But although you mention that patients may be given false hope by researchers increasing the sample size in drugs trials, this could also happen with careful selection of participants i.e. ones that are likely to have a strong reaction to the drug being tested (Sinclair & Haynes, 2011). This leads me to feel that this situation is not actually that much different to that of removing outliers where it is usually acceptable to do so as long as it is stated in the results and the reasons for doing do are practical. Therefore I feel that the answer to this question is contingent on reasons for manipulating your data, which should be available for judgement in research reports.
    Ref
    Sinclair, J.C. & Haynes, R.B. (2011). Selecting participants that raise a clinical trials population attributable fraction can increase the treatment effect within the trial and reduce the required sample size. Journal of Clinical Epidemiology, 64(8), 893-902. Retrieved from: http://ac.els-cdn.com/S0895435611000023/1-s2.0-S0895435611000023-main.pdf?_tid=73a45f84ee88c195c35b582848b3df36&acdnat=1333568323_da240c946685aa99154880595e3977e7

  3. Pingback: Homework for my TA week 10 « fr4nw

  4. I am very strong in my opinion that it is very wrong to keep adding participants in order for the researcher to yield the results they want. At the beginning of the research, researchers should calculate scientifically how many participants are necessary for their research. This figure should then be stuck to unless the attrition rate is very high or there were errors in the data collection. If a finding is to be significant and one of use for adaption into the real world, then this significance level should be found. It is understandable that researcher’s can spend sometimes years researching one area and it can be very dissapointing to then find the results didn’t show anything. But at the same time, that is a finding in itself and thus finding a significant result isn’t everything in psychology. If a researcher keeps adding participants or keeps carrying out any sort of statistical analyses on data purely to find a signigicant result, then they are misleading others into potentially thinking this result is astonishing. If the significant result is few and far between, then it’s adapted into the real world (which is the whole point of research at the end of the day) may not yield such pleasing results as expected, thus potentially wasting lots of time and money. Overall, it is understandable as to why researchers want significant results, however adding participants for this cause and carrying out analysis till one test shows significance is wrong and can be detrimental with its effects.

  5. Pingback: Homework for my TA :) | dy bannee diu

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s