Statistics: sampling
1.1 Definitions of Statistics, Probability, and Key Terms
Statistics, probability, and the basic vocabulary
Statistics is the study of data — the actual values of variables collected from a population or sample. It has two main branches:
- Descriptive statistics organizes and summarizes data, typically using graphs or numerical summaries such as the mean (average) or proportion.
- Inferential statistics uses formal methods to draw conclusions about a population based on sample data, relying on probability to quantify confidence in those conclusions.
Probability is the mathematical framework for quantifying the likelihood of events, especially under randomness. It connects long-run patterns to short-run uncertainty:
- A single coin toss is unpredictable, but over many repetitions (e.g., 4,000 tosses), the proportion of heads converges to the expected probability of $\frac{1}{2}$ or 0.5.
- Empirical results (e.g., 996 heads in 2,000 tosses, or 12,012 heads in 24,000 by Karl Pearson) closely match theoretical probabilities, illustrating the law of large numbers.
Population refers to the entire group of interest — all first-year students at ABC College, all cars with front-seat dummies, or all medical doctors in a directory. A sample is a subset drawn from the population to study its characteristics. For a sample to be useful, it must be representative — reflecting the population’s key traits.
Key terms: population, sample, parameter, statistic, variable, and data.
| Term | Definition | Example |
|---|---|---|
| Population | The entire group under study | All first-year students at ABC College this term |
| Sample | A subset of the population used for analysis | All students enrolled in one section of a beginning statistics course |
| Parameter | A numerical characteristic of the entire population | The average amount of money spent by first-year students at ABC College this term |
| Statistic | A numerical characteristic of the sample | The average amount of money spent by first-year students in the sample |
| Variable | A characteristic measured for each member of the population | The amount of money spent (excluding books) by one first-year student |
| Data | The observed values of a variable | $150, $200, and $225 |
- Variable
- A characteristic or measurement that can be determined for each member of a population, denoted by capital letters such as X or Y. Variables are either numerical (e.g., points earned, money spent) or categorical (e.g., party affiliation).
- Data
- The actual observed values of a variable, which can be numbers (e.g., exam scores, dollar amounts) or words (e.g., party affiliations, yes/no responses). A single value is called a datum.
Mean vs. proportion: two common ways to summarize numerical and categorical data.
| Mean | Proportion | |
|---|---|---|
| Definition | The arithmetic average of numerical data | The fraction of a sample or population with a specific categorical attribute |
| Example | Average exam score: $(86 + 75 + 92) / 3 = 84.3$ | Proportion of men in a class: $22 / 40$ |
| Use case | Summarizing numerical variables | Summarizing categorical variables |
1.2 Data, Sampling, and Variation in Data and Sampling
Data are either qualitative (categorical) or quantitative.
Types of data and their properties
| Qualitative (categorical) | Quantitative discrete | Quantitative continuous | |
|---|---|---|---|
| Definition | Result of categorizing or describing attributes; described by words or letters | Result of counting; takes on only certain numerical values | Result of measuring; may include fractions, decimals, or irrational numbers |
| Examples | Hair color, blood type, ethnic group, car model | Number of books, number of phone calls, number of students | Weight, height, time, amount of money |
| Mathematical analysis | Not easily averaged (e.g., average hair color) | Can be averaged, summed, etc. | Can be averaged, summed, etc. |
Data can be measured on different scales:
- Nominal scale: Qualitative categories with no order (e.g., favorite food, yes/no responses).
- Interval scale: Quantitative data with meaningful differences but no true zero (e.g., temperature in °C).
Sampling methods
Common random sampling techniques
| Simple random | Stratified | Cluster | Systematic | Convenience | |
|---|---|---|---|---|---|
| How it works | Every member has equal chance; every sample of size n equally likely | Population divided into strata; proportionate random sample from each | Population divided into clusters; randomly select clusters; include all members | Random start point; select every nth member thereafter | Use readily available results |
| Pros | Simple, fair | Ensures representation across groups | Efficient for large populations | Simple to implement | Easy to collect |
| Cons | May miss subgroups if sample small | Requires clear strata definitions | Clusters may not represent population | Periodic patterns may bias results | High risk of bias |
Sampling can be done with replacement (member can be chosen more than once) or without replacement (member chosen only once). Most practical sampling is without replacement, especially for large populations.
Sampling and nonsampling errors
Sampling errors arise from the sampling process itself (e.g., sample too small). Nonsampling errors come from external factors (e.g., defective measuring device). Sampling bias occurs when some population members are less likely to be chosen, leading to incorrect conclusions.
Common problems with samples include self-selected responses, small sample sizes, undue influence on responses, non-response bias, misleading data presentation, and confounding variables.
Variation in data and samples
Variation is inherent in data (e.g., 16-ounce cans may contain slightly more or less liquid) and in samples (even random samples from the same population differ). Larger samples tend to yield results closer to the population average, but variability remains.
Sample size matters: small samples may be unreliable, but even large samples can be biased (e.g., call-in surveys). Random, well-designed surveys of 1,200–1,500 observations are often sufficient for valid conclusions.
1.3 Frequency, Frequency Tables, and Levels of Measurement
Levels of measurement
- Nominal scale
- Qualitative (categorical) data without any meaningful order; examples include categories, colors, names, labels, yes/no responses, or smartphone brands. Cannot be used in calculations.
- Ordinal scale
- Qualitative data that can be ordered but differences between values are not measurable; examples include rankings of national parks or survey responses like "excellent" to "unsatisfactory". Cannot be used in calculations.
- Interval scale
- Quantitative data with a definite order and measurable differences, but no true zero; examples include temperature in Celsius or Fahrenheit. Differences make sense, but ratios do not.
- Ratio scale
- Quantitative data with a definite order, measurable differences, and a true zero; ratios are meaningful. Examples include exam scores or heights. The minimum value is zero, and ratios like "twice as much" are valid.
Data can be organized by frequency, the count of how often a value occurs, or by relative frequency, the proportion of times a value occurs relative to the total number of observations. Relative frequencies can be expressed as fractions, decimals, or percentages.
Example of frequency and relative frequency for student work hours.
| Data value | Frequency | Relative frequency |
|---|---|---|
| 2 | 3 | $$\frac{3}{20}$$ or 0.15 |
| 3 | 5 | $$\frac{5}{20}$$ or 0.25 |
| 4 | 3 | $$\frac{3}{20}$$ or 0.15 |
| 5 | 6 | $$\frac{6}{20}$$ or 0.30 |
| 6 | 2 | $$\frac{2}{20}$$ or 0.10 |
| 7 | 1 | $$\frac{1}{20}$$ or 0.05 |
Cumulative relative frequency is the running total of relative frequencies. Each entry is the sum of all previous relative frequencies plus the current row’s relative frequency. The final entry equals one (or close to one, accounting for rounding).
1.4 Experimental Design and Ethics
A randomized experiment tests whether a change in one variable causes a change in another. The variable that is deliberately changed is the explanatory variable (also called the independent variable). The variable that is measured for an effect is the response variable (also called the dependent variable).
Each distinct value of the explanatory variable is a treatment, and every individual or object receiving a treatment is an experimental unit.
Lurking variables are outside factors that could influence the response variable. To isolate the effect of the explanatory variable, researchers use random assignment of experimental units to treatment groups. This spreads all lurking variables evenly across groups, so any difference in the response variable can be attributed to the treatments themselves.
Random assignment is what allows a study to support a cause-and-effect conclusion. Without it, lurking variables may bias the results, making it impossible to claim that the explanatory variable is responsible for changes in the response variable.
Glossary
- statistics
- Statistics is the science that deals with the collection, analysis, interpretation, and presentation of data.
- descriptive statistics
- Descriptive statistics is the organization and summarization of data, using graphs or numerical measures such as averages.
- inferential statistics
- Inferential statistics uses probability to draw conclusions from data and to determine how confident we can be that those conclusions are correct.
- probability
- Probability is a mathematical tool used to study randomness and the chance that an event occurs.
- population
- A population is the collection of persons, things, or objects under study.
- sample
- A sample is a portion or subset of a population that is selected and studied to gain information about the population.
- sampling
- Sampling is the selection of a portion of a population so that the sample can be used to learn about the population.
- statistic
- A statistic is a number that represents a property of a sample.
- parameter
- A parameter is a numerical characteristic of the whole population that can be estimated by a statistic.
- representative sample
- A representative sample is a sample that contains the characteristics of the population.
- variable
- A variable is a characteristic or measurement that can be determined for each member of a population; variables may be numerical or categorical.
- data
- Data are the actual values of a variable; they may be numbers or words. A datum is a single value.
- mean
- Mean and proportion are common statistical quantities: mean is found by adding values and dividing by the number of values, and proportion is the fraction of the sample or population in a category.
- average
- The words mean and average are often used interchangeably in practice, although the technical term is arithmetic mean.
- qualitative data
- Qualitative data are categorical data that describe attributes of a population and are generally expressed with words or letters.
- quantitative data
- Quantitative data are numerical data produced by counting or measuring attributes of a population.
- quantitative discrete data
- Quantitative discrete data are counting data that take only certain numerical values.
- quantitative continuous data
- Quantitative continuous data are measurement data that may include fractions, decimals, or irrational numbers.
- Pareto chart
- A Pareto chart is a bar chart whose bars are sorted from largest to smallest by category size.
- simple random sample
- In simple random sampling, every group of size n is equally likely to be chosen, so each sample of the same size has an equal chance of being selected.
- stratified sample
- A stratified sample is formed by dividing the population into strata and then taking a proportionate simple random sample from each stratum.
- cluster sample
- A cluster sample is formed by dividing the population into clusters, randomly selecting some clusters, and including all members from the selected clusters.
- systematic sample
- A systematic sample is formed by randomly selecting a starting point and then taking every nth item from a list of the population.
- convenience sampling
- Convenience sampling uses results that are readily available and is a non-random method that may be highly biased.
- sampling with replacement
- Sampling with replacement means that after a member is selected, it is returned to the population and may be chosen again.
- simple random sampling without replacement
- Simple random sampling without replacement means that a member may be chosen only once; for large populations and small samples it is approximately the same as sampling with replacement.
- sampling error
- Sampling error is variation caused by the sampling process, whereas nonsampling error is caused by factors not related to the sampling process.
- sampling bias
- Sampling bias occurs when some members of the population are less likely than others to be chosen, which can lead to incorrect conclusions.
- biased sample
- A sample that is not representative of the population is biased and produces inaccurate, invalid results.
- variation
- Variation is present in any set of data, and different measurements of the same kind of object may differ naturally.
- level of measurement
- The level of measurement of data can be nominal, ordinal, interval, or ratio, and correct statistical procedures depend on knowing the level.
- nominal scale
- Nominal-scale data are qualitative categories with no meaningful order and cannot be used in calculations.
- ordinal scale
- Ordinal-scale data can be ordered, but the differences between values cannot be measured and they cannot be used in calculations.
- interval scale
- Interval-scale data are quantitative data with a definite order and meaningful differences, but no true zero or minimum value.
- ratio scale
- Ratio-scale data are interval-scale data with a minimum value, so differences and ratios are meaningful.
- frequency
- A frequency is the number of times a data value occurs, and the sum of the frequencies equals the sample size.
- relative frequency
- A relative frequency is the frequency divided by the total number of data values, and it may be written as a fraction, decimal, or percent.
- cumulative relative frequency
- Cumulative relative frequency is the running total of the relative frequencies, and the final cumulative relative frequency should equal 1, up to rounding.
- explanatory variable
- An experiment investigates the relationship between two variables; the explanatory variable is the variable that causes change, and the response variable is the affected variable.
- response variable
- In a randomized experiment, the researcher manipulates the explanatory variable and measures the response variable; the different values of the explanatory variable are treatments.
- experimental unit
- An experimental unit is a single object or individual to be measured.
- lurking variable
- Lurking variables are additional variables that can cloud a study and make it hard to determine cause and effect.
- random assignment
- Random assignment spreads lurking variables among treatment groups so that the planned treatments are the only systematic difference between groups.