Descriptive Statistics is that branch of statistics which is concerned with describing the population under study. It summarizes the data, which is already known. Descriptive statistics are visual in form. Means, medians, variances, standard deviations, correlation, coefficients, etc. can be communicated via charts and graphs. 

"Descriptive Statistics are used to present quantitative descriptions in a manageable form. In a research study we may have lots of measures. Or we may measure a large number of people on any measure. Descriptive statistics help us to simplify large amounts of data in a sensible way. Each descriptive statistic reduces lots of data into a simpler summary" (Research Methods Knowledge Base > Descriptive Statistics).

"Descriptive statistics convey two basic aspects of a sample: central tendency and dispersion. The former describes the most representative or common or central observation of the sample, and the latter how the sample is distributed around the common variate" (Statistics for Anthropology)

Inferential Statistics makes statements about the population from which samples were obtained. It is the branch of statistics dealing with conclusions, generalizations, predictions, and estimations based on data from samples. 

"With inferential statistics, you are trying to reach conclusions that extend beyond the immediate data alone. For instance, we use inferential statistics to try to infer from the sample data what the population might think. Or, we use inferential statistics to make judgments of the probability that an observed difference between groups is a dependable one or one that might have happened by chance in this study. Thus, we use inferential statistics to make inferences from our data to more general conditions; we use descriptive statistics simply to describe what’s going on in our data" (Research Methods Knowledge Base > Inferential Statistics).

There are many types of inferential statistics and which one(s) to use will be specific to the research design and sample characteristics. However, most inferential statistics are based on the principle that a test-statistic value is calculated on the basis of a particular formula. 

Hypothesis Testing is the use of statistics to determine the probability that a given hypothesis is true. The usual process of hypothesis testing consists of four steps:

1. The null (H0) and alternative (H1) hypotheses are stated.

2. The level of statistical significance,* (criteria) is established for a decision .

3. Collect the sample / Compute the test statistic** and compare the produced p-value to the criterion set in step 2.

• The p-value is the probability value.
  • "P value is the probability, given that the null hypothesis is true, of obtaining data as extreme or more extreme than that observed" (Oxford Handbook of Medical Statistics) 
  • "The probability of any event can only vary between 0 and 1 (which correspond to 0 and 100%). If an event is certain to occur, it has a probability of 1; while, if it is certain the event will not occur, it has a probability of 0" (Statistics explained : an introductory guide for life scientists).

4. Make a decision based on the probability. The sample is compared with the null hypothesis’ parameters, and a conclusion is reached about which hypothesis to accept. The decision to reject or retain the H0 is called significance.***​

When testing a hypothesis, two errors may be committed: 1) Rejection of the null hypothesis when in fact it is true (type I error), or 2) Failure to reject a false null hypothesis (type II error).

Definitions:

*"Level of significance, or significance level, refers to a criterion of judgment upon which a decision is made regarding the value stated in a null hypothesis. The criterion is based on the probability of obtaining a statistic measured in a sample if the value stated in the null hypothesis were true. The most commonly used significance level is 5%, (0.05).

**The test statistic is a mathematical formula that allows researchers to determine the likelihood of obtaining sample outcomes if the null hypothesis were true. The value of the test statistic is used to make a decision regarding the null hypothesis.

***Significance, or statistical significance, describes a decision made concerning a value stated in the null hypothesis. When the null hypothesis is rejected, we reach significance. When the null hypothesis is retained, we fail to reach significance" (Introduction to hypothesis testing, (Ch. 8).

Also see: Formulate a Research Question or Hypothesis

Note: A test statistic is a quantity, derived from the sample, used to determine how well the model/means fits the data. 

Parametric Tests (regression, comparison and correlation) are used where a Normal distribution of variables is assumed (think "bell-shaped curve":

• Regression Models are used to test cause-and-effect relationships.
  • Simple Regression is used to estimate the relationship between two quantitative variables.
    • The Coefficient of Determination (or R2 value) is the test statistic used to evaluate the goodness-of-fit of the model. R² = 1 - unexplained variation / total variation
  • Multiple Linear Regression is used to describe relationships between two or more independent variables and one dependent variable.
  • Logistic Regression is similar to multiple linear regression, except the response/outcome variable is binary ("0" or "1") which is used to represent "Yes" or "No." "Pass" or "Fail," for example.
• Comparison Tests are used to look for differences among group means. Expected outputs will be the t-value, degrees of freedom, p-value
  • T-tests are used to compare the means of two groups. To chose which t-test to use, ask whether the groups being compared derive from a single population or two different ones, and whether you want to test the difference in a specific direction. The t-value (or t-statistic) is the test statistic used to determine whether the null hypothesis is supported or rejected.
  • ANOVA & MANOVA: Analysis of variance (ANOVA) is the statistical procedure of comparing the means of a variable across several groups (more than two groups). ANOVA uses F-tests to assess the equality of the means. The F-value (of F-statistic) is the ratio of two variances (between group variation to within group variation). Used as the test statistic, the larger the value, the more likely it is that the variation associated with the independent variable is real and not due to chance. Multivariate analysis of variance (MANOVA) is used to compare means of several variables simultaneously across several groups (more than two groups). 
• Correlation Tests check whether two variables are related without assuming cause-and-effect relationships.
  • Pearson' r is used to express the correlation between two variables X and Y.
  • Looking for an association between two categorical variables? A Chi-squared test (or X²) "calculates the frequencies that would be expected if there were no association (i.e. null hypothesis is true)" (Oxford handbook of medical statistics).

Non-parametric Tests do not make as many assumptions about the data. Useful when one or more of the common statistical assumptions are violated (i.e. Normal distribution and a linear relationship do not hold), but the inferences that are made aren't as strong as with parametric tests:

• Rank Correlation Tests: Assumptions for these tests (Spearman or Kendall) are that variables can be ranked and the relationship between the variables either increases or decreases. Use in place of regression and correlation tests.
• "Kruskal-Wallis Test is a nonparametric approach to the one-way ANOVA. The procedure is used to compare three or more groups on a dependent variable that is measured on at least an ordinal level" (Comprehensive Clinical Psychology).
• Wilcoxon Rank-Sum Test is the nonparametric version of the independent t-test, commonly used for the comparison of two groups of nonparametric data.
• Use the Wilcoxon Signed Rank Test (or Wilcoxon Matched Pairs Test) for a repeated measure design where the same sujects are evaluated under two different conditions.