Research reports and some of the lingo.

I have some experience with research reports, both writing and reading. In this forum, we come across many and are trying to make sense of them. One of the best tips I ever got about research is the following 4 steps.
1. Practical problem (does X effect Y)?
2. Statistical problem (does a sample population have a relationship between X and Y)?
3. Statistical solution (what is the correlation between X and Y)?
4. Practical solution (eliminate X due to the relationship with Y).

A couple of key thoughts on the generic research report. Most of the research that I have seen on PC is correlational research, which looks at is there a relationship between two variables. There are two VERY important things to consider when looking at correlation results, the sample population, and the significance of the relationship.

Each correlation study has a sample group of men, 1,000, 2,000, etc. and the hope of the researcher is this sample represents the population. The gold standard of research is “pure random sample of sample participants” and most samples are not pure, some are better than others. Age, comorbidity, the region where participants live, ethnicity, etc. all affect how well the sample represents the population. Remember the end the statistic solution refers to the larger population, or we make inference about the variable in question to the population.

The second critical factor is the significance of the relationship. A perfect correlation is 1, no correlation is 0. See https://onlinecourses.science.psu.edu/stat100/node/35 for more on significance. The real key and most often in research there is a positive correlation meaning based on the statistical test there is a relationship. Now I have often heard and read there is a statistical correlation but not a practical one.

One other thought is bias, researcher bias, sample bias and many other biases. I am not suggesting intentional bias. When I did my dissertation, I had a strong hypothesis. The results came in completely opposite of what I believed. At first, I thought I failed. However, disproving or looking for another variable that could explain the result is as valuable as my initial belief about the relationship between variables. I suggest most researchers have bias and the good ones declare it and ensure the research plan accounts for it.

In this source are some good thoughts and definitions of many of the terms in research reports. http://researchbasics.education.uconn.edu/correlation/

I was taught to read research reports with a healthy skepticism. Look for flaws, look for the makeup of the sample and how well the sample represents the population. Also look for the significance of the findings. One key to research is where is it published. Peer-reviewed is the gold standard but there are different levels of peers. Being published in the AMA is more rigorous than a local group that formed a website to publish their thoughts.

I am not a statistician but do have some experience with research design and interpretation. Please feel free to add your thoughts and experience to this thread. The better we are at understanding the importance of a study, the better decisions we will make. Thanks, Denis

Hi Subdenis,

I also work with mathematical analysis every day, and give frequent ppt presentations using stats (usually just correlation stats)...often to people who have typically had some statistics training, but often way-back in their college days.

Anyhow, I clicked on the UConn url you posted, and then clicked on the link to the ppt presentation on correlations...just because I am always on the hunt for slick ideas on how to better "tell the story."

I got a chuckle because I have pretty much the exact same "basic training" on correlations in my oft-used slide deck to give a primer on what correlations are, and strength (r)...which I review in my presentations just before laying out some of the results/analysis/recommendations I have made. The slides from UConn used height and foot size as an illustrative example of having a positive strong correlation.

In my slide deck, I cover the same thing, but in a single slide. I tell people I'm going to give them a stats class on a single slide, but it's just to remind them that we all use this stuff intuitively in every day life...so I don't totally blow them away/lose them with by graphical analysis on the next slide. My slide's illustrative examples are:
** positive, strong correlation -> child's age and vocabulary
** negative, strong correlation -> price and demand for computer tablets
** unrelated (scattered) -> height and beer consumption...which always gets a laugh


Anyhow, you are correct that there is a lot of correlation analysis in medicine, but you know the old saying: correlation is not causation. Right? So, cause and effect is not proven by correlation. For that, the randomized control trial is required. This is, for example, what the FDA requires for drug approvals. More specifically, they require large, well-designed, well-conducted, randomized, double-blinded, placebo-controlled human clinical trials, or RCTs for short, for "proof."

And as you may also realize, some things simply cannot practically be "proven." My best example of this is the effect of one single food in the complex human diet; correlation, yes, but "proof," no. This is an example of where we just have to be smart!


Thanks for sharing.


Groton?

Post Edited (NKinney) : 9/22/2017 12:58:12 PM (GMT-6)

This is why we have levels of evidence. Causation can only be proved definitively by a large randomized clinical trial, preferably corroborated independently (Level 1a). Researchers publishing in the top journals are usually careful about how they state their conclusions. It's the mainstream media that usually draws unsupported inferences.