Research Bias

Research bias, also called experimenter bias, is a process where the scientists performing the research influence the results, in order to portray a certain outcome.

Some bias in research arises from experimental error and failure to take into account all of the possible variables.

Other bias arises when researchers select subjects that are more likely to generate the desired results, a reversal of the normal processes governing science.

Bias is the one factor that makes qualitative research much more dependent upon experience and judgment than quantitative research.

For example, when using social research subjects, it is far easier to become attached to a certain viewpoint, jeopardizing impartiality.

The main point to remember with bias is that, in many disciplines, it is unavoidable. Anyexperimental design process involves understanding the inherent biases and minimizing the effects.

In quantitative research, the researcher tries to eliminate bias completely whereas, inqualitative research, it is all about understanding that it will happen.

Design Bias

Design bias is introduced when the researcher fails to take into account the inherent biases liable in most types of experiment.

Some bias is inevitable, and the researcher must show that they understand this, and have tried their best to lessen the impact, or take it into account in the statistics and analysis.

Another type of design bias occurs after the research is finished and the results analyzed. This is when the original misgivings of the researchers are not included in the publicity, all too common in these days of press releases and politically motivated research.

For example, research into the health benefits of Acai berries may neglect the researcher’s awareness of limitations in the sample group. The group tested may have been all female, or all over a certain age.

Selection/Sampling Bias

Sampling bias occurs when the process of sampling actually introduces an inherent bias into the study. There are two types of sampling bias, based around those samples that you omit, and those that you include:

Omission Bias

This research bias occurs when certain groups are omitted from the sample. An example might be that ethnic minorities are excluded or, conversely, only ethnic minorities are studied.

For example, a study into heart disease that used only white males, generally volunteers, cannot be extrapolated to the entire population, which includes women and other ethnic groups.

Omission bias is often unavoidable, so the researchers have to incorporate and account for this bias in the experimental design.

Inclusive Bias

Inclusive bias occurs when samples are selected for convenience.

This type of bias is often a result of convenience where, for example, volunteers are the only group available, and they tend to fit a narrow demographic range.

There is no problem with it, as long as the researchers are aware that they cannot extrapolate the results to fit the entire population. Enlisting students outside a bar, for a psychological study, will not give a fully representative group.

Procedural Bias

Procedural bias is where an unfair amount of pressure is applied to the subjects, forcing them to complete their responses quickly.

For example, employees asked to fill out a questionnaire during their break period are likely to rush, rather than reading the questions properly.

Using students forced to volunteer for course credit is another type of research bias, and they are more than likely to fill the survey in quickly, leaving plenty of time to visit the bar.

Measurement Bias

Measurement bias arises from an error in the data collection and the process of measuring.

In a quantitative experiment, a faulty scale would cause an instrument bias and invalidate the entire experiment. In qualitative research, the scope for bias is wider and much more subtle, and the researcher must be constantly aware of the problems.

• Subjects are often extremely reluctant to give socially unacceptable answers, for fear of being judged. For example, a subject may strive to avoid appearing homophobic or racist in an interview.
  This can skew the results, and is one reason why researchers often use a combination of interviews, with an anonymous questionnaire, in order to minimize measurement bias.
• Particularly in participant studies, performing the research will actually have an effect upon the behavior of the sample groups. This is unavoidable, and the researcher must attempt to assess the potential effect.
• Instrument bias is one of the most common sources of measurement bias in quantitative experiments. This is the reason why instruments should be properly calibrated, and multiple samples taken to eliminate any obviously flawed or aberrant results.

Interviewer Bias

This is one of the most difficult research biases to avoid in many quantitative experiments when relying upon interviews.

With interviewer bias, the interviewer may subconsciously give subtle clues in with body language, or tone of voice, that subtly influence the subject into giving answers skewed towards the interviewer’s own opinions, prejudices and values.

Any experimental design must factor this into account, or use some form of anonymous process to eliminate the worst effects.

See how to avoid this:
Double Blind Experiment

Response Bias

Conversely, response bias is a type of bias where the subject consciously, or subconsciously, gives response that they think that the interviewer wants to hear.

The subject may also believe that they understand the experiment and are aware of the expected findings, so adapt their responses to suit.

Again, this type of bias must be factored into the experiment, or the amount of information given to the subject must be restricted, to prevent them from understanding the full extent of the research.

Reporting Bias

Reporting Bias is where an error is made in the way that the results are disseminated in the literature. With the growth of the internet, this type of bias is becoming a greater source of concern.

The main source of this type of bias arises because positive research tends to be reported much more often than research where the null hypothesis is upheld. Increasingly, research companies bury some research, trying to publicize favorable findings.

Unfortunately, for many types of studies, such as meta-analysis, the negative results are just as important to the statistics.

Reference

https://explorable.com/research-bias

When to Use a Particular Statistical Test

Statistics:When which and why?

Multiple Regression Analysis

Multiple regression analysis is a powerful technique used for predicting the unknown value of a variable from the known value of two or more variables- also called the predictors.

More precisely, multiple regression analysis helps us to predict the value of Y for given values of X1, X2, …, Xk.

For example the yield of rice per acre depends upon quality of seed, fertility of soil, fertilizer used, temperature, rainfall. If one is interested to study the joint affect of all these variables on rice yield, one can use this technique.

An additional advantage of this technique is it also enables us to study the individual influence of these variables on yield.

Dependent and Independent Variables

By multiple regression, we mean models with just one dependent and two or more independent (exploratory) variables. The variable whose value is to be predicted is known as the dependent variable and the ones whose known values are used for prediction are known independent (exploratory) variables.

The Multiple Regression Model

In general, the multiple regression equation of Y on X1, X2, …, Xk is given by:

Y = b0 + b1 X1 + b2 X2 + …………………… + bk Xk

Interpreting Regression Coefficients

Here b0 is the intercept and b1, b2, b3, …, bk are analogous to the slope in linear regression equation and are also called regression coefficients. They can be interpreted the same way as slope. Thus if bi = 2.5, it would indicates that Y will increase by 2.5 units if Xi increased by 1 unit.

The appropriateness of the multiple regression model as a whole can be tested by the F-test in the ANOVA table. A significant F indicates a linear relationship between Y and at least one of the X's.

How Good Is the Regression?

Once a multiple regression equation has been constructed, one can check how good it is (in terms of predictive ability) by examining the coefficient of determination (R2). R2 always lies between 0 and 1.

R2 - coefficient of determination

All software provides it whenever regression procedure is run. The closer R2 is to 1, the better is the model and its prediction.

A related question is whether the independent variables individually influence the dependent variable significantly. Statistically, it is equivalent to testing the null hypothesis that the relevant regression coefficient is zero.

This can be done using t-test. If the t-test of a regression coefficient is significant, it indicates that the variable is in question influences Y significantly while controlling for other independent explanatory variables.

Assumptions

Multiple regression technique does not test whether data are linear. On the contrary, it proceeds by assuming that the relationship between the Y and each of Xi's is linear. Hence as a rule, it is prudent to always look at the scatter plots of (Y, Xi), i= 1, 2,…,k. If any plot suggests non linearity, one may use a suitable transformation to attain linearity.

Another important assumption is non existence of multicollinearity- the independent variables are not related among themselves. At a very basic level, this can be tested by computing the correlation coefficient between each pair of independent variables.

Other assumptions include those of homoscedasticity and normality.

Multiple regression analysis is used when one is interested in predicting a continuous dependent variable from a number of independent variables. If dependent variable is dichotomous, then logistic regression should be used.

Reference

https://explorable.com/multiple-regression-analysis

Statistical Correlation

Statistical correlation is a statistical technique which tells us if two variables are related.

For example, consider the variables family income and family expenditure. It is well known that income and expenditure increase or decrease together. Thus they are related in the sense that change in any one variable is accompanied by change in the other variable.

Again price and demand of a commodity are related variables; when price increases demand will tend to decreases and vice versa.

If the change in one variable is accompanied by a change in the other, then the variables are said to be correlated. We can therefore say that family income and family expenditure, price and demand are correlated.

Relationship Between Variables

Correlation can tell you something about the relationship between variables. It is used to understand:

1. whether the relationship is positive or negative
2. the strength of relationship.

Correlation is a powerful tool that provides these vital pieces of information.

In the case of family income and family expenditure, it is easy to see that they both rise or fall together in the same direction. This is called positive correlation.

In case of price and demand, change occurs in the opposite direction so that increase in one is accompanied by decrease in the other. This is called negative correlation.

Coefficient of Correlation

Statistical correlation is measured by what is called coefficient of correlation (r). Its numerical value ranges from +1.0 to -1.0. It gives us an indication of the strength of relationship.

In general, r > 0 indicates positive relationship, r < 0 indicates negative relationship while r = 0 indicates no relationship (or that the variables are independent and not related). Here r = +1.0 describes a perfect positive correlation and r = -1.0 describes a perfect negative correlation.

Closer the coefficients are to +1.0 and -1.0, greater is the strength of the relationship between the variables.

As a rule of thumb, the following guidelines on strength of relationship are often useful (though many experts would somewhat disagree on the choice of boundaries).

Value of r	Strength of relationship
-1.0 to -0.5 or 1.0 to 0.5	Strong
-0.5 to -0.3 or 0.3 to 0.5	Moderate
-0.3 to -0.1 or 0.1 to 0.3	Weak
-0.1 to 0.1	None or very weak

Correlation is only appropriate for examining the relationship between meaningful quantifiable data (e.g. air pressure, temperature) rather than categorical data such as gender, favorite color etc.

Disadvantages

While 'r' (correlation coefficient) is a powerful tool, it has to be handled with care.

1. The most used correlation coefficients only measure linear relationship. It is therefore perfectly possible that while there is strong non linear relationship between the variables, r is close to 0 or even 0. In such a case, a scatter diagram can roughly indicate the existence or otherwise of a non linear relationship.
2. One has to be careful in interpreting the value of 'r'. For example, one could compute 'r' between the size of shoe and intelligence of individuals, heights and income. Irrespective of the value of 'r', it makes no sense and is hence termed chance or non-sense correlation.
3. 'r' should not be used to say anything about cause and effect relationship. Put differently, by examining the value of 'r', we could conclude that variables X and Y are related. However the same value of 'r' does not tell us if X influences Y or the other way round. Statistical correlation should not be the primary tool used to study causation, because of the problem with third variables.

Reference

https://explorable.com/statistical-correlation

Experimental design

Reference
Experimental Design

Completely Randomized Design & Completely Randomized Design

Completely Randomized Design

A completely randomized design is probably the simplest experimental design, in terms of data analysis and convenience. With this design, subjects are randomly assigned to treatments.

Treatment
Placebo	Vaccine
500	500

A completely randomized design layout for a hypothetical medical experiment is shown in the table to the right. In this design, the experimenter randomly assigned subjects to one of two treatment conditions. They received a placebo or they received a cold vaccine. The same number of subjects (500) are assigned to each treatment condition (although this is not required). The dependent variable is the number of colds reported in each treatment condition. If the vaccine is effective, subjects in the "vaccine" condition should report significantly fewer colds than subjects in the "placebo" condition.

A completely randomized design relies on randomization to control for the effects of extraneous variables. The experimenter assumes that, on averge, extraneous factors will affect treatment conditions equally; so any significant differences between conditions can fairly be attributed to the independent variable.

Randomized Block Design

With a randomized block design, the experimenter divides subjects into subgroups called blocks, such that the variability within blocks is less than the variability between blocks. Then, subjects within each block are randomly assigned to treatment conditions. Compared to a completely randomized design, this design reduces variability within treatment conditions and potential confounding, producing a better estimate of treatment effects.

The table below shows a randomized block design for a hypothetical medical experiment.

Gender	Treatment
	Placebo	Vaccine
Male	250	250
Female	250	250

Subjects are assigned to blocks, based on gender. Then, within each block, subjects are randomly assigned to treatments (either a placebo or a cold vaccine). For this design, 250 men get the placebo, 250 men get the vaccine, 250 women get the placebo, and 250 women get the vaccine.

It is known that men and women are physiologically different and react differently to medication. This design ensures that each treatment condition has an equal proportion of men and women. As a result, differences between treatment conditions cannot be attributed to gender. This randomized block design removes gender as a potential source of variability and as a potential confounding variable.

Reference 

http://stattrek.com/statistics/dictionary.aspx?definition=Completely%20randomized%20design

book on statistics

Reference
statistics bbok

SD and SEM

It is easy to be confused about the difference between the standard deviation (SD) and the standard error of the mean (SEM). Here are the key differences:

•	The SD quantifies scatter — how much the values vary from one another.

•	The SEM quantifies how precisely you know the true mean of the population. It takes into account both the value of the SD and the sample size.

•	Both SD and SEM are in the same units -- the units of the data.

•	The SEM, by definition, is always smaller than the SD.

•

The SEM gets smaller as your samples get larger. This makes sense, because the mean of a large sample is likely to be closer to the true population mean than is the mean of a small sample. With a huge sample, you'll know the value of the mean with a lot of precision even if the data are very scattered.

•

The SD does not change predictably as you acquire more data. The SD you compute from a sample is the best possible estimate of the SD of the overall population. As you collect more data, you'll assess the SD of the population with more precision. But you can't predict whether the SD from a larger sample will be bigger or smaller than the SD from a small sample. (This is not strictly true. It is the variance -- the SD squared -- that doesn't change predictably, but the change in SD is trivial and much much smaller than the change in the SEM.)

Note that standard errors can be computed for almost any parameter you compute from data, not just the mean. The phrase "the standard error" is a bit ambiguous. The points above refer only to the standard error of the mean.

Reference 

http://www.graphpad.com/guides/prism/6/statistics/index.htm?stat_semandsdnotsame.htm

One-Way Analysis of Variance (ANOVA) example

Reference

ANNOVA Example

How Do You Interpret P Values?

In technical terms, a P value is the probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis.

For example, suppose that a vaccine study produced a P value of 0.04. This P value indicates that if the vaccine had no effect, you’d obtain the observed difference or more in 4% of studies due to random sampling error.

P values address only one question: how likely are your data, assuming a true null hypothesis? It does not measure support for the alternative hypothesis. This limitation leads us into the next section to cover a very common misinterpretation of P values.

P Values Are NOT the Probability of Making a Mistake

Incorrect interpretations of P values are very common. The most common mistake is to interpret a P value as the probability of making a mistake by rejecting a true null hypothesis (a Type I error).

There are several reasons why P values can’t be the error rate.

First, P values are calculated based on the assumptions that the null is true for the population and that the difference in the sample is caused entirely by random chance. Consequently, P values can’t tell you the probability that the null is true or false because it is 100% true from the perspective of the calculations.

Second, while a low P value indicates that your data are unlikely assuming a true null, it can’t evaluate which of two competing cases is more likely:

• The null is true but your sample was unusual.
• The null is false.

Determining which case is more likely requires subject area knowledge and replicate studies.

Let’s go back to the vaccine study and compare the correct and incorrect way to interpret the P value of 0.04:

• Correct: Assuming that the vaccine had no effect, you’d obtain the observed difference or more in 4% of studies due to random sampling error.
   
• Incorrect: If you reject the null hypothesis, there’s a 4% chance that you’re making a mistake.

To see a graphical representation of how hypothesis tests work, see my post: Understanding Hypothesis Tests: Significance Levels and P Values.

Refernce

http://blog.minitab.com/blog/adventures-in-statistics/how-to-correctly-interpret-p-values

Experimental design

An Experimental Design Example

Consider the following hypothetical experiment. Acme Medicine is conducting an experiment to test a new vaccine, developed to immunize people against the common cold. To test the vaccine, Acme has 1000 volunteers - 500 men and 500 women. The participants range in age from 21 to 70.

In this lesson, we describe three experimental designs - a completely randomized design, a randomized block design, and a matched pairs design. And we show how each design might be applied by Acme Medicine to understand the effect of the vaccine, while ruling out confounding effects of other factors.

Completely Randomized Design

The completely randomized design is probably the simplest experimental design, in terms of data analysis and convenience. With this design, participants are randomly assigned to treatments.

Treatment
Placebo	Vaccine
500	500

A completely randomized design layout for the Acme Experiment is shown in the table to the right. In this design, the experimenter randomly assigned participants to one of two treatment conditions. They received a placebo or they received the vaccine. The same number of participants (500) were assigned to each treatment condition (although this is not required). The dependent variable is the number of colds reported in each treatment condition. If the vaccine is effective, participants in the "vaccine" condition should report significantly fewer colds than participants in the "placebo" condition.

A completely randomized design relies on randomization to control for the effects of extraneous variables. The experimenter assumes that, on averge, extraneous factors will affect treatment conditions equally; so any significant differences between conditions can fairly be attributed to the independent variable.

Randomized Block Design

With a randomized block design, the experimenter divides participants into subgroups calledblocks, such that the variability within blocks is less than the variability between blocks. Then, participants within each block are randomly assigned to treatment conditions. Because this design reduces variability and potential confounding, it produces a better estimate of treatment effects.

Gender	Treatment
	Placebo	Vaccine
Male	250	250
Female	250	250

The table to the right shows a randomized block design for the Acme experiment. Participants are assigned to blocks, based on gender. Then, within each block, participants are randomly assigned to treatments. For this design, 250 men get the placebo, 250 men get the vaccine, 250 women get the placebo, and 250 women get the vaccine.

It is known that men and women are physiologically different and react differently to medication. This design ensures that each treatment condition has an equal proportion of men and women. As a result, differences between treatment conditions cannot be attributed to gender. This randomized block design removes gender as a potential source of variability and as a potential confounding variable.

In this Acme example, the randomized block design is an improvement over the completely randomized design. Both designs use randomization to implicitly guard against confounding. But only the randomized block design explicitly controls for gender.

Note 1: In some blocking designs, individual participants may receive multiple treatments. This is called using the participant as his own control. Using the participant as his own control is desirable in some experiments (e.g., research on learning or fatigue). But it can also be a problem (e.g., medical studies where the medicine used in one treatment might interact with the medicine used in another treatment).

Note 2: Blocks perform a similar function in experimental design as strata perform in sampling. Both divide observations into subgroups. However, they are not the same. Blocking is associated with experimental design, and stratification is associated with survey sampling.

Matched Pairs Design

Pair	Treatment
	Placebo	Vaccine
1	1	1
2	1	1
...	...	...
499	1	1
500	1	1

A matched pairs design is a special case of the randomized block design. It is used when the experiment has only two treatment conditions; and participants can be grouped into pairs, based on some blocking variable. Then, within each pair, participants are randomly assigned to different treatments.

The table to the right shows a matched pairs design for the Acme experiment. The 1000 participants are grouped into 500 matched pairs. Each pair is matched on gender and age. For example, Pair 1 might be two women, both age 21. Pair 2 might be two women, both age 22, and so on.

For the Acme example, the matched pairs design is an improvement over the completely randomized design and the randomized block design. Like the other designs, the matched pairs design uses randomization to control for confounding. However, unlike the others, this design explicitly controls for two potential lurking variables - age and gender.

Reference

http://stattrek.com/experiments/experimental-design.aspx?Tutorial=AP