The news is filled with pronouncements of scientific advancements and information that turn out to be mistaken. Many times this reporting is based on experiments that fail to bear out the same results when replicated or reproduced. Sometimes, that’s because the initial experiments lacked a good hypothesis. That’s the topic of this module in Experimental Design. [credits] Good science is not a fishing expedition. It’s not randomly measuring tons of variables in an attempt to find the inklings of a pattern. Robust experiments, reproducible experiments, are those that are designed to answer a specific question. That question is conveyed in your hypothesis. And your experiment should be designed to support or refute it. A hypothesis is a clear and understandable explanation for a set of observations. A good experiment is one that is designed to test whether that explanation (or hypothesis) is correct or not. Let’s consider an animal model, in this case the activity of mice. Sometimes mice are active; sometimes they’re not. Why is that? A bad experiment would be just to go watch the mice and collect a ton of data in an attempt to guess why. But that’s what too much science does these days. It’s trying to use lots of data to look for associations, and then passing that off as robust answers ready for promotion to the world. Now it’s important to note that those types of studies are fine as pilot studies, sometimes called “hypothesis generating” studies. Those studies are fine as long as you recognize that they’re designed to give you information that you can use to form hypotheses for much better, reproducible studies down the line. The results of pilot work shouldn’t be broadcast as definitive, though. They’re not designed to be used in that manner. In this case, a pilot study watching the mice might give us some ideas, or help us develop hypotheses, for what is really affecting mice activity. For example, one hypothesis might be that mice activity varies depending on the time of the day. Another might be that the food they eat affects their activity. Another might be that their activity depends on how many other mice are around. Experiments are often constructed around a null hypothesis, which is the belief that there is no relationship and that the supposed variable of interest is not the cause of a difference in activity. A study is then designed to test the null hypothesis. So if our hypothesis is that activity depends on the time of the day, our null hypothesis would be that the time of the day has no relationship on activity. To test that, we might measure the activity of a number of mice at different times of the day. If we can’t, though statistical tests, see any relationship, then the null hypothesis isn’t refuted. We often say that it can’t be “rejected”. That’s one of the tricky things about science. We rarely get to “prove” stuff. Everything has some uncertainty to it. We just try to come up with an estimation of how likely something is to be right or wrong. A study to test the time of day hypothesis will be designed differently than the food hypothesis or the presence of other mice hypothesis. It might even be possible to test more than one of these hypotheses at a time. But for each hypothesis, we want to make sure that our study is designed to answer that particular question. If not, then the results we get might not be answering the question we think they are. And therefore our results may not be reproducible. The same applies to human subjects research, of course. Good science is careful and involves planning. Good hypotheses are well thought out before starting an experiment and they’re questioned and reasoned. Good research will test those hypotheses. It will recognize the limitations of the methods used and try to minimize them. It will understand the implications of findings. You should be prepared and ready to answer what it means if your hypothesis is supported by evidence. And you should be prepared to answer what it means if your hypothesis isn’t. Good research uses good methods to test good questions. It all comes down to the hypothesis. Make sure you have one. Make sure it’s clear. Make sure the methods test it. That’s how you can be sure what you’re doing is robust, and reproducible.