Statistics Optional: Syllabus, Sources & Strategy
Learning Objectives
- Understand the fundamental concepts of Statistics Optional: Syllabus, Sources & Strategy
- Apply key formulas and techniques to solve problems
- Practice with exam-level questions to build speed and accuracy
1. Overview of the Optional and its Place in the Examination
1.1 Why Statistics Attracts Mathematics and Science Graduates
Statistics as an optional subject draws candidates who have already encountered probability, calculus and linear algebra during their undergraduate studies because the syllabus builds directly on those foundations. The subject offers a rare combination of theoretical rigour and practical relevance; the same tools that appear in Paper I are routinely used in economic modelling, demographic analysis and data‑driven policy evaluation, which are core components of General Studies Paper III. Consequently, a well‑prepared candidate can leverage the optional to score high marks while simultaneously strengthening the analytical sections of the general studies papers.
1.2 Structure of the Two Papers and Marks Distribution
The examination consists of two papers, each carrying two hundred and fifty marks and lasting three hours. Paper I is devoted to probability theory, distribution theory, sampling distributions, estimation and hypothesis testing, together with non‑parametric methods and order statistics. Paper II shifts the focus to applied areas: linear models and regression, design of experiments, sample survey theory, statistical quality control and operations research. Both papers contain a mixture of short‑answer questions that test conceptual clarity and long‑answer questions that require complete derivations, proofs or numerical work. The marking scheme rewards logical step‑by‑step exposition, so a candidate who can present a coherent argument with correct notation gains a decisive advantage.
1.3 Overlap with General Studies and Essay
Topics such as sampling techniques, index numbers, time series analysis and linear programming appear explicitly in the General Studies syllabus under economy, statistics and technology sections. Moreover, the essay paper often invites discussion on data‑driven governance, evidence‑based policy or the role of statistical thinking in development; a statistics optional aspirant can draw on concrete theorems and real‑world examples to enrich such essays. This overlap reduces the marginal effort required for the optional and creates a positive feedback loop between the two parts of the examination.
2. Paper I: Probability and Statistical Inference – Detailed Syllabus Map
2.1 Probability Foundations and Random Variables
The opening segment of Paper I establishes the axiomatic basis of probability: sample space, events, the three Kolmogorov axioms, conditional probability and Bayes’ theorem. Independence of events and of random variables is defined precisely, because later results on joint distributions and convergence rely on it. Random variables are introduced as measurable functions on the sample space; the distinction between discrete and continuous variables is made through the probability mass function and the probability density function respectively. Expectation, variance, moments, moment generating functions, characteristic functions and probability generating functions are developed as tools for summarising distributions and for proving limit theorems. A thorough grasp of these concepts is essential because every subsequent topic — from the derivation of the Poisson limit of the binomial to the construction of the likelihood function — uses them as building blocks.
2.2 Standard Discrete and Continuous Distributions
The syllabus enumerates a core set of distributions that candidates must be able to recognise, derive moments for, and apply in modelling. Among the discrete family, the Bernoulli, binomial, Poisson, geometric, negative binomial and hypergeometric distributions cover the spectrum from a single trial to sampling without replacement. The continuous family includes the uniform, exponential, gamma, beta, normal, Cauchy, Laplace, logistic, Student’s t, chi‑square and F distributions. Each distribution is characterised by its support, parameters, moment generating function (where it exists) and relationships to other distributions — for instance, the gamma distribution generalises the exponential and the chi‑square, while the beta distribution arises naturally as the distribution of order statistics from a uniform sample. Understanding these interconnections allows a candidate to simplify complex problems by transforming them into a known form.
2.3 Joint Distributions, Convergence and Sampling Distributions
Joint probability mass or density functions lead to marginal and conditional distributions, covariance, correlation and the concept of regression of one variable on another. The best linear predictor emerges from minimising mean squared error, a result that foreshadows the linear models of Paper II. Convergence concepts — convergence in probability, almost sure convergence and convergence in distribution — are developed rigorously, culminating in the weak and strong laws of large numbers and the central limit theorem in its Lindeberg‑Lévy and Lyapunov forms. These limit theorems justify the use of normal approximations for sample means and sums, which in turn underpins the derivation of the sampling distributions of the sample mean, sample variance, sample proportion and the associated t, chi‑square and F statistics. Mastery of these sampling distributions is a prerequisite for the confidence interval and hypothesis testing sections that follow.
2.4 Estimation Theory: Point and Interval Estimation
Point estimation begins with the method of moments and maximum likelihood estimation. The properties of estimators — unbiasedness, consistency, efficiency, sufficiency (via the factorisation theorem), completeness and the Rao‑Blackwell and Lehmann‑Scheffé theorems — are examined in depth. The concept of a uniformly minimum variance unbiased estimator (UMVUE) ties these properties together. Interval estimation covers confidence intervals for a single mean, a single proportion, the difference of two means, the difference of two proportions and the ratio of two variances, each derived from the appropriate sampling distribution. Candidates must be comfortable manipulating pivotal quantities and recognising when a statistic follows a t, chi‑square or F distribution under the null hypothesis.
2.5 Testing of Hypotheses and Non‑Parametric Methods
The hypothesis testing framework introduces null and alternative hypotheses, Type I and Type II errors, size and power. The Neyman‑Pearson lemma provides the most powerful test for simple hypotheses, while the monotone likelihood ratio property extends it to uniformly most powerful tests for one‑sided composite hypotheses. The likelihood ratio test offers a general procedure for multi‑parameter problems, and its asymptotic chi‑square distribution under the null is a key result. Non‑parametric tests — sign test, Wilcoxon signed‑rank test, Wilcoxon rank‑sum (Mann‑Whitney U) test, Kruskal‑Wallis test, runs test, Kolmogorov‑Smirnov test, chi‑square goodness‑of‑fit and independence tests, and Fisher’s exact test — are required for situations where distributional assumptions are untenable. Order statistics, including the distribution of the sample range, median and quantiles, complete the paper and provide tools for constructing distribution‑free confidence intervals.
3. Paper II: Applied Statistics – Detailed Syllabus Map
3.1 Linear Models, Regression and Multivariate Analysis
Paper II opens with the general linear model Y = Xβ + ε, where the design matrix X may be of less than full rank. The Gauss‑Markov theorem establishes the best linear unbiased estimator (BLUE) of β under the assumptions of linearity, unbiasedness and homoscedastic uncorrelated errors. Estimation by ordinary least squares, the normal equations, and the properties of the hat matrix are examined. Hypothesis testing for linear restrictions uses the F statistic derived from the residual sum of squares. Multiple regression diagnostics — multicollinearity (variance inflation factor), heteroscedasticity (Breusch‑Pagan test), autocorrelation (Durbin‑Watson statistic) and influential observations (Cook’s distance) — are required. The syllabus also covers principal component analysis, factor analysis and canonical correlation as multivariate techniques, with emphasis on interpretation of eigenvalues and loadings.
3.2 Design of Experiments
Experimental design focuses on controlling variability through randomisation, replication and blocking. The completely randomised design (CRD), randomised block design (RBD) and Latin square design (LSD) are the three fundamental structures. For each design the analysis of variance table, the expected mean squares, and the efficiency relative to CRD are derived. Factorial experiments — 2ⁿ and 3ⁿ series — introduce main effects, interactions and confounding. The concept of orthogonal contrasts, Yates’ algorithm for 2ⁿ factorials, and the construction of fractional factorial designs using defining relations are covered. Split‑plot and strip‑plot designs address situations where some factors are harder to randomise. Missing plot techniques and the analysis of covariance (ANCOVA) for adjusting treatment effects with concomitant variables complete this section.
3.3 Sample Survey Theory
Sample survey theory begins with the distinction between probability and non‑probability sampling. Simple random sampling with and without replacement (SRSWR, SRSWOR) provides the baseline for estimating population totals, means and proportions, together with their variances and confidence intervals. Stratified random sampling introduces allocation problems — proportional, Neyman and optimum allocation — and the gain in precision measured by the design effect. Systematic sampling, cluster sampling (one‑stage and two‑stage) and multi‑stage sampling are developed with their variance estimators. Ratio and regression estimators exploit auxiliary information to improve efficiency. The Horvitz‑Thompson estimator provides a unified framework for unequal probability sampling. Non‑sampling errors — non‑response, measurement error and coverage error — are discussed qualitatively, as they affect the credibility of survey results.
3.4 Statistical Quality Control
Quality control distinguishes between chance causes and assignable causes of variation. Control charts for variables — X̄ chart, R chart, S chart — and for attributes — p chart, np chart, c chart, u chart — are derived from the sampling distribution of the relevant statistic under the assumption of a stable process. The concepts of average run length (ARL), operating characteristic (OC) curves and the design of control charts with specified Type I and Type II error probabilities are examined. Acceptance sampling by attributes (single, double and multiple sampling plans) and by variables (using the normal distribution) are covered, including the construction of OC curves and the determination of sample size and acceptance numbers for given producer’s and consumer’s risks. Process capability indices Cp and Cpk link the control chart limits to specification limits.
3.5 Operations Research
Operations research in the syllabus is limited to deterministic models. Linear programming formulation, graphical solution for two variables, the simplex algorithm (including the two‑phase and big‑M methods), duality theory, sensitivity analysis and the transportation and assignment problems are required. The Hungarian method for the assignment problem, the Vogel’s approximation method for initial feasible solutions in transportation, and the concept of degeneracy are tested. Network models — shortest path, maximum flow, minimum cost flow and project scheduling with PERT/CPM — round off the section. Candidates must be able to formulate real‑world decision problems as linear programmes and interpret the dual variables as shadow prices.
| Design | Randomisation | Blocking Structure | Degrees of Freedom for Error | Typical Use Case |
|---|---|---|---|---|
| Completely Randomised Design (CRD) | All treatments randomised to all experimental units | No blocking | N − t | Homogeneous experimental material, e.g., laboratory trials |
| Randomised Block Design (RBD) | Treatments randomised within each block | One blocking factor | (b − 1)(t − 1) | Field experiments with a known fertility gradient |
| Latin Square Design (LSD) | Treatments randomised subject to row and column constraints | Two orthogonal blocking factors | (t − 1)(t − 2) | Greenhouse or plot experiments with two sources of heterogeneity |
4. Standard References and How to Use Them
4.1 Core Texts for Probability and Inference
The classic Indian textbook “Fundamentals of Mathematical Statistics” by S. C. Gupta and V. K. Kapoor covers the entire Paper I syllabus with worked examples that match the examination style. For a more rigorous treatment of probability theory, “A First Course in Probability” by Sheldon Ross provides clear explanations of the axioms, conditional probability and limit theorems. “Statistical Inference” by George Casella and Roger Berger is the standard reference for estimation, hypothesis testing and asymptotic theory; its chapters on sufficiency, completeness and the Rao‑Blackwell theorem are indispensable. Candidates should read these texts selectively, focusing on theorems that appear explicitly in the syllabus and solving the end‑of‑chapter exercises that mirror previous question patterns.
4.2 Applied Statistics References
For linear models and regression, “Applied Linear Statistical Models” by Kutner, Nachtsheim, Neter and Li offers a comprehensive treatment of diagnostics, model selection and multicollinearity. “Design and Analysis of Experiments” by Douglas Montgomery is the definitive source for CRD, RBD, LSD, factorial and split‑plot designs, with numerous numerical examples. “Sampling Techniques” by William Cochran remains the authoritative text for survey sampling, covering stratification, cluster sampling and ratio estimation in depth. “Introduction to Statistical Quality Control” by Montgomery again serves well for control charts and acceptance sampling. Finally, “Operations Research: An Introduction” by Hamdy Taha covers the simplex method, duality, transportation, assignment and network models with step‑by‑step algorithms.
4.3 Supplementary Materials and Practice Resources
Previous years’ question papers are the most reliable guide to the depth and style of questions; they should be attempted under timed conditions after each major topic is completed. The “Statistics Optional” modules on reputable online platforms provide concise video explanations of difficult proofs such as the Lindeberg‑Feller central limit theorem or the derivation of the F distribution from independent chi‑square variables. A personal formula notebook — organised by topic, with each entry containing the statement, assumptions and a one‑line derivation — becomes a high‑yield revision tool in the final months. Candidates should also maintain a separate notebook for non‑parametric test decision rules, because these are often asked as direct‑recall questions.
5. Structured Study Plan Across the Preparation Timeline
5.1 Foundation Phase (Months 1‑2)
The first two months should be devoted to mastering the probability and distribution theory of Paper I. Begin with the axioms, Bayes’ theorem and random variables, solving at least thirty problems on each sub‑topic. Proceed to discrete and continuous distributions, deriving moment generating functions and establishing relationships (e.g., Poisson as a limit of binomial, gamma‑exponential‑chi‑square chain). Allocate one week to joint distributions, covariance and regression, and another week to convergence concepts and the central limit theorem. Conclude the phase with sampling distributions, constructing tables of pivotal quantities for the mean, variance and proportion. Daily revision of the formula notebook ensures that derivations remain fresh.
5.2 Core Inference and Applied Phase (Months 3‑4)
Month three focuses on estimation and hypothesis testing. Work through the method of moments, maximum likelihood estimation, and the proofs of the Rao‑Blackwell and Lehmann‑Scheffé theorems. Practice constructing confidence intervals for all parameters listed in the syllabus. In hypothesis testing, master the Neyman‑Pearson lemma, the monotone likelihood ratio property, and the likelihood ratio test with its asymptotic distribution. Simultaneously, begin Paper II with linear models: derive the normal equations, understand the hat matrix, and perform regression diagnostics on sample datasets. Month four covers design of experiments and sample survey theory. For each design, write out the ANOVA table, expected mean squares and efficiency formula. In sampling, derive variance expressions for SRSWR, SRSWOR, stratified and cluster sampling, and practise ratio and regression estimation with numerical data.
5.3 Specialised Topics and Integration Phase (Month 5)
The fifth month is reserved for statistical quality control, operations research and non‑parametric methods. Draw control charts by hand for given datasets, compute ARL and OC curves, and solve acceptance sampling plan problems. In operations research, formulate at least ten linear programming problems from diverse contexts (diet, production, blending), solve them by the simplex method, and interpret the dual. For non‑parametric tests, create a decision flowchart that maps data type (paired, independent, ordinal, nominal) to the appropriate test. Integrate the two papers by attempting full‑length mock papers that mix questions from both syllabi; this builds stamina and highlights cross‑topic connections such as the use of the F distribution in both ANOVA and regression.
5.4 Revision and Answer‑Writing Refinement (Month 6)
The final month should be split into three cycles of ten days each. Cycle one revisits Paper I topics in the order of the syllabus, with emphasis on writing complete proofs within the allotted time. Cycle two does the same for Paper II, paying special attention to the presentation of ANOVA tables, simplex tableaus and control chart diagrams. Cycle three consists of two full‑length simulated examinations under strict exam conditions, followed by detailed self‑evaluation using the official marking scheme. During this period, the formula notebook and the non‑parametric flowchart are the only reference materials allowed, mimicking the closed‑book environment of the actual test.
6. Answer Writing Technique and Presentation
6.1 Structuring Long‑Answer Questions
A well‑structured answer begins with a clear statement of the problem, followed by the assumptions and the relevant theorem or formula. Each mathematical step should be written on a separate line, with brief explanatory phrases in plain language — for example, “By the factorisation theorem, T(X) is sufficient because the joint density factors as g(T(x),θ)h(x).” Diagrams such as the ANOVA table, the simplex tableau or the control chart should be drawn with a ruler, labelled fully, and referenced in the text. Concluding the answer with a one‑sentence interpretation of the result (e.g., “Hence the 95% confidence interval for the population mean is (12.3, 15.7).”) signals completeness to the examiner.
6.2 Managing Derivations and Proofs
When a proof is requested, the candidate must decide whether to present a full rigorous derivation or a concise sketch. The syllabus expects full derivations for standard results such as the moment generating function of the normal distribution, the distribution of the sample variance, or the Neyman‑Pearson lemma. For less central results — for instance, the asymptotic distribution of the sample median — a sketch that cites the relevant theorem (Bahadur representation) and states the final expression is acceptable. In all cases, the logical flow must be uninterrupted; skipping steps to save time often leads to loss of marks because the examiner cannot award partial credit for missing links.
6.3 Time Allocation and Question Selection
Each paper contains eight questions of which five must be attempted. A prudent strategy is to spend the first ten minutes scanning the paper, identifying the questions that align with the candidate’s strongest topics, and allocating approximately thirty minutes per selected question. Questions that require only a definition or a short formula (e.g., “Define the hazard function of the exponential distribution”) should be answered in five minutes, leaving more time for derivation‑heavy questions. If a question appears to require an excessively long computation (e.g., a 4×4 simplex tableau with multiple iterations), it may be wiser to choose an alternative question that tests the same concept with a smaller numerical example. Regular timed practice develops the intuition needed for this real‑time decision making.
| Distribution | Support | Parameters | Mean | Variance | Typical Application |
|---|---|---|---|---|---|
| Binomial | {0,1,…,n} | n (trials), p (success prob.) | np | np(1−p) | Number of successes in fixed trials |
| Poisson | {0,1,2,…} | λ (rate) | λ | λ | Rare events over fixed interval |
| Normal | (−∞,∞) | μ (mean), σ² (variance) | μ | σ² | Measurement errors, sums of i.i.d. variables |
| Exponential | (0,∞) | θ (scale) or λ=1/θ (rate) | θ | θ² | Inter‑arrival times in Poisson process |
| Gamma | (0,∞) | α (shape), β (scale) | αβ | αβ² | Waiting time for α events, Bayesian conjugate prior |
| Beta | (0,1) | α, β (shape) | α/(α+β) | αβ/[(α+β)²(α+β+1)] | Proportions, order statistics from uniform |
Consistent application of the study plan, disciplined answer writing and strategic use of the standard references will enable a candidate to master the Statistics optional syllabus and convert that mastery into a high score in the examination.
Asymptotic Theory and Large‑Sample Inference
Asymptotic theory supplies the bridge between exact finite‑sample results and the practical approximations used in most applied work. The central limit theorem (CLT) in its Lindeberg‑Feller form guarantees that for a sequence of independent, not necessarily identically distributed random variables with finite variances, the standardized sum converges in distribution to a standard normal law provided the Lindeberg condition holds. This condition is the examiner’s favourite test of conceptual clarity: a candidate must be able to state it precisely and recognise when it fails, for example in heavy‑tailed distributions where the Lyapunov condition is violated. The delta method extends the CLT to smooth functions of estimators; if √n(θ̂ − θ) → N(0, Σ) and g is differentiable at θ with non‑zero gradient, then √n(g(θ̂) − g(θ)) → N(0, ∇g(θ)ᵀ Σ ∇g(θ)). Mastery of the delta method is essential for deriving asymptotic variances of transformed parameters such as the log‑odds in logistic regression or the correlation coefficient via Fisher’s z‑transformation.
Consistency and asymptotic normality of maximum likelihood estimators (MLEs) rest on regularity conditions: identifiability, interior true parameter, twice continuous differentiability of the log‑likelihood, and the existence of a dominating integrable function for the score. The information matrix equality I(θ) = E[−∂²ℓ/∂θ²] = Var(∂ℓ/∂θ) underpins the asymptotic covariance I(θ)⁻¹. Examiners often ask for a proof sketch of the Cramér‑Rao lower bound and its attainment by the MLE under these conditions. A common twist is to present a model where the support depends on the parameter (e.g., uniform on (0, θ)) and ask why the regularity conditions break down, leading to a non‑standard n‑rate of convergence and a non‑normal limiting distribution. Understanding the distinction between √n‑consistency and n‑consistency, and being able to cite the Bahadur representation for sample quantiles, separates a competent answer from an outstanding one.
Higher‑order asymptotics such as Edgeworth expansions and saddlepoint approximations are rarely required in full, but the concept of Bartlett correction for likelihood ratio statistics appears in the syllabus. The correction factor 1 + b/n adjusts the χ² approximation so that the error in the tail probability is of order n⁻² instead of n⁻¹. Candidates should be able to write the expression for the Bartlett factor in a simple exponential family and explain its impact on small‑sample inference. Finally, the notion of stochastic equicontinuity and the functional delta method underpin the asymptotic theory of empirical processes; a brief mention of Donsker’s theorem and its role in the Kolmogorov‑Smirnov test demonstrates breadth without requiring a full proof.
Bayesian Inference and Decision Theory
Bayesian methods treat unknown parameters as random variables with a prior distribution π(θ) that encodes pre‑experimental information. The posterior π(θ | x) ∝ L(x | θ) π(θ) follows from Bayes’ theorem. Conjugate families — Beta‑Binomial, Gamma‑Poisson, Normal‑Normal — yield closed‑form posteriors and are the workhorses of exam questions because they allow rapid computation of posterior means, variances, and credible intervals. The posterior predictive distribution p(y | x) = ∫ p(y | θ) π(θ | x) dθ is frequently asked in the context of model checking; candidates should be comfortable deriving it for the normal‑normal model and interpreting a posterior predictive p‑value.
Decision theory formalises the choice of an estimator δ(x) through a loss function L(θ, δ). Under squared‑error loss, the Bayes estimator is the posterior mean; under absolute‑error loss it is the posterior median; under 0‑1 loss it is the posterior mode. The risk function R(θ, δ) = E[L(θ, δ(X))] and the Bayes risk r(π, δ) = ∫ R(θ, δ) π(θ) dθ provide a frequentist‑Bayesian bridge. A standard exam problem asks to show that the Bayes estimator minimises the posterior expected loss and, when the prior is proper, also minimises the Bayes risk among all estimators. The concept of admissibility — an estimator that is not dominated by any other — is illustrated by the James‑Stein estimator, which dominates the sample mean in dimensions ≥ 3 under squared‑error loss, thereby proving the sample mean inadmissible. Candidates must be able to state the James‑Stein shrinkage factor (1 − (p − 2)/‖X‖²) and explain why the result does not contradict the Cramér‑Rao bound.
Hierarchical (multilevel) models extend the Bayesian framework by placing hyperpriors on prior parameters, e.g., θᵢ | μ, τ² ∼ N(μ, τ²) with μ ∼ N(μ₀, σ₀²) and τ² ∼ Inv‑Gamma(a, b). Gibbs sampling or Metropolis‑Hastings algorithms are the standard computational tools; the full conditional distributions are often of standard form, enabling a straightforward Gibbs sampler. Examiners may ask for the derivation of the full conditionals for a normal‑normal‑inverse‑gamma hierarchy and a brief description of convergence diagnostics such as trace plots, Gelman‑Rubin R̂, and effective sample size. Understanding the distinction between prior sensitivity and robustness, and being able to conduct a prior‑posterior overlap check, completes the Bayesian toolkit expected at this level.
Time Series Modelling and Forecasting
Time series analysis in the optional focuses on linear stochastic processes that are covariance‑stationary after suitable differencing. A process {Yₜ} is weakly stationary if its mean μ is constant, its variance γ(0) is finite, and its autocovariance γ(h) = Cov(Yₜ, Yₜ₊ₕ) depends only on lag h. The autocorrelation function (ACF) ρ(h) = γ(h)/γ(0) and the partial autocorrelation function (PACF) are the primary diagnostic tools. For an AR(p) process Yₜ = φ₁Yₜ₋₁ + … + φₚYₜ₋ₚ + εₜ with εₜ ∼ WN(0, σ²), the ACF decays exponentially or with damped sine waves while the PACF cuts off after lag p. Conversely, an MA(q) process Yₜ = εₜ + θ₁εₜ₋₁ + … + θ_qεₜ₋q has an ACF that cuts off after lag q and a PACF that decays. The ARMA(p,q) model combines both behaviours; its ACF and PACF both tail off, and identification relies on the extended ACF (EACF) table or information criteria such as AIC = −2 log L + 2k and BIC = −2 log L + k log n.
Estimation of ARMA parameters can be performed by conditional least squares (CLS), exact maximum likelihood via the Kalman filter, or the innovations algorithm. The asymptotic distribution of the MLE is √n(θ̂ − θ) → N(0, I(θ)⁻¹) where I(θ) is the Fisher information matrix derived from the spectral density. Forecasting uses the conditional expectation Ŷₜ₊ₕ|ₜ = E[Yₜ₊ₕ | Y₁,…,Yₜ]; for an ARMA model the h‑step‑ahead forecast converges to the unconditional mean μ as h → ∞, and the forecast error variance approaches the process variance. Prediction intervals are constructed as Ŷₜ₊ₕ|ₜ ± zα/2 √(MSEₕ), where MSEₕ is the mean‑squared forecast error obtained recursively from the ARMA coefficients. Examiners often test the ability to compute one‑step‑ahead forecasts for a given AR(2) model and to write the recursion for the forecast error variance.
Non‑stationary series are handled by differencing: if ∇ᵈYₜ is stationary, the original series is integrated of order d, denoted I(d). The ARIMA(p,d,q) model applies ARMA to the differenced series. Unit‑root tests — Dickey‑Fuller (DF), augmented DF (ADF), Phillips‑Perron — are essential; the null hypothesis is a unit root (non‑stationarity) against the alternative of stationarity. The test statistic does not follow the standard t‑distribution; critical values are tabulated by Dickey and Fuller. A common trap is to apply the standard t‑test to the coefficient of Yₜ₋₁ in an AR(1) regression; candidates must cite the non‑standard distribution and the need for DF critical values. Seasonal ARIMA (SARIMA) extends the framework with seasonal AR and MA polynomials at lag s, written as ARIMA(p,d,q)×(P,D,Q)_s. Model building follows the Box‑Jenkins cycle: identification, estimation, diagnostic checking (Ljung‑Box Q‑test on residuals), and forecasting.
Survival Analysis and Reliability Theory
Survival analysis deals with time‑to‑event data where censoring is pervasive. The survival function S(t) = P(T > t) and the hazard function h(t) = f(t)/S(t) are the fundamental quantities. For a non‑negative random lifetime T with density f and distribution F, the cumulative hazard H(t) = ∫₀ᵗ h(u) du satisfies S(t) = exp(−H(t)). The exponential distribution with constant hazard λ yields S(t)=e-λt and is the only continuous distribution with the memoryless property. The Weibull distribution generalises the exponential by allowing a monotone hazard h(t)=αβtβ-1; β>1 gives increasing hazard (wear‑out), β<1 gives decreasing hazard (infant mortality). The log‑normal and gamma families provide flexible hazard shapes including bathtub curves.
Right censoring occurs when the event time is known only to exceed the observation time C. The likelihood contribution for an observed failure at tᵢ is f(tᵢ); for a right‑censored observation it is S(cᵢ). The Kaplan‑Meier (product‑limit) estimator Ŝ(t) = ∏tᵢ ≤ t (1 − dᵢ/nᵢ) where dᵢ is the number of events at tᵢ and nᵢ the number at risk just before tᵢ, is non‑parametric and handles arbitrary censoring patterns. Its variance is estimated by Greenwood’s formula Var(Ŝ(t)) ≈ Ŝ(t)² ∑tᵢ ≤ t dᵢ/(nᵢ(nᵢ−dᵢ)). The log‑rank test compares two or more survival curves; the test statistic is χ² = (∑(Oᵢ−Eᵢ))² / ∑Vᵢ where Oᵢ, Eᵢ, Vᵢ are observed, expected events and variance in group i at each distinct failure time. The Cox proportional hazards model h(t|X) = h₀(t) exp(βᵀX) separates baseline hazard from covariate effects; partial likelihood L(β) = ∏i: event exp(βᵀXᵢ) / ∑j∈R(tᵢ) exp(βᵀXⱼ) eliminates h₀(t). Estimation of β via maximising the partial likelihood yields asymptotically normal estimates with covariance given by the observed information matrix.
In reliability theory, the system lifetime is a function of component lifetimes. For series systems, the system fails at the minimum component lifetime; for parallel systems, at the maximum. The reliability function of a coherent system can be expressed via the structure function φ(x) and component reliabilities pᵢ = P(component i works). The Birnbaum importance measure Iᵢ = ∂R/∂pᵢ quantifies the marginal contribution of component i to system reliability. Examiners may ask to compute system reliability for a bridge network or to derive the hazard function of a series system with independent Weibull components. The concept of accelerated life testing, where stress levels accelerate failure according to an Arrhenius or inverse power law, links survival analysis to experimental design and is a frequent source of applied questions.
Statistical Computing: Simulation, Bootstrap and MCMC
Modern statistical practice relies heavily on computational methods, and the optional syllabus expects familiarity with the principles behind them. Monte Carlo integration approximates an expectation E[g(X)] by (1/N)∑i=1^N g(Xᵢ) where Xᵢ are i.i.d. draws from the target distribution. Variance reduction techniques — antithetic variates, control variates, importance sampling, and stratified sampling — improve efficiency. For importance sampling, the estimator is (1/N)∑ g(Xᵢ) w(Xᵢ) with weights w(x)=f(x)/q(x) where f is the target density and q the proposal; the optimal proposal is proportional to |g|f, which is rarely attainable but guides choice.
The bootstrap resamples the observed data with replacement to approximate the sampling distribution of a statistic. The non‑parametric bootstrap draws B samples of size n from the empirical distribution; the parametric bootstrap draws from a fitted model. The bootstrap estimate of standard error is the standard deviation of the bootstrap replicates; percentile and bias‑corrected accelerated (BCa) intervals improve coverage. A typical exam question asks to describe the bootstrap algorithm for the median and to explain why the naive percentile interval may be biased for skewed statistics. The jackknife, a leave‑one‑out linear approximation, provides a quick bias estimate and variance estimate but is less accurate for non‑smooth statistics.
Markov chain Monte Carlo (MCMC) generates dependent draws from a target posterior π(θ|x) when direct sampling is infeasible. The Metropolis‑Hastings algorithm proposes θ* from a proposal q(θ*|θ) and accepts with probability α = min{1, π(θ*|x) q(θ|θ*) / [π(θ|x) q(θ*|θ)]}. The Gibbs sampler is a special case where the proposal is the full conditional distribution, yielding acceptance probability 1. Convergence diagnostics include trace plots, autocorrelation plots, the Gelman‑Rubin R̂ statistic (values <1.1 indicate convergence), and effective sample size (ESS). Candidates should be able to write the full conditionals for a normal‑normal‑inverse‑gamma hierarchical model and to outline a Gibbs sampler. Understanding the difference between geometric ergodicity (fast convergence) and sub‑geometric ergodicity (slow, heavy‑tailed targets) helps in diagnosing poor mixing. Finally, the concept of Hamiltonian Monte Carlo (HMC) and the No‑U‑Turn Sampler (NUTS) is mentioned in advanced texts; while not required for derivation, awareness that HMC uses gradient information to propose distant states with high acceptance probability demonstrates up‑to‑date knowledge.
- Confusing the Lindeberg condition with the Lyapunov condition and stating the wrong sufficient condition for the CLT.
- Assuming the MLE is always asymptotically normal without checking regularity conditions such as parameter‑dependent support.
- Using the standard t‑distribution critical values for the Dickey‑Fuller test instead of the non‑standard DF tables.
- Treating the Kaplan‑Meier estimator as if it were a parametric MLE and applying the usual Fisher information formula for its variance.
- Writing the Metropolis‑Hastings acceptance ratio without the proposal density ratio when the proposal is asymmetric.
- Reporting a bootstrap percentile interval for a highly skewed statistic without applying the BCa correction.
- Ignoring the need for a proper prior when claiming that the Bayes estimator minimises the Bayes risk among all estimators.
Generalized Linear Models: Theory, Link Functions, and Diagnostics
Generalized linear models (GLMs) extend the classical linear model by allowing the response distribution to belong to the exponential family and by linking the mean to a linear predictor through a monotone differentiable link function. The three components — random component, systematic component, and link — are specified as: (i) Yᵢ ∼ F(μᵢ, φ) with density f(y;θ,φ)=exp{[yθ−b(θ)]/a(φ)+c(y,φ)}; (ii) ηᵢ = xᵢᵀβ; (iii) g(μᵢ)=ηᵢ where g is the link. Canonical links (θ=η) simplify the score equations to Xᵀ(y−μ)=0, yielding the familiar iteratively re‑weighted least squares (IRLS) algorithm. For non‑canonical links the Fisher scoring step uses the expected information matrix I(β)=XᵀWX with weights wᵢ = (∂μᵢ/∂ηᵢ)² / Var(Yᵢ). Common families include binomial with logit, probit, or complementary log‑log links; Poisson with log link; gamma with inverse or log link; and inverse Gaussian with inverse‑square link. Overdispersion is handled by introducing a dispersion parameter φ estimated via Pearson χ²/df or by quasi‑likelihood, which replaces the true likelihood with a variance function V(μ) and retains the same estimating equations. Model diagnostics rely on deviance residuals rᴰᵢ = sign(yᵢ−μᵢ)√{2[ℓ(yᵢ;yᵢ)−ℓ(yᵢ;μᵢ)]}, Pearson residuals rᴾᵢ = (yᵢ−μᵢ)/√{V(μᵢ)}, and leverage values from the hat matrix H = W1/2X(XᵀWX)−1XᵀW1/2. Influence measures such as Cook’s distance Dᵢ = (rᴾᵢ)² hᵢᵢ / [p(1−hᵢᵢ)] flag observations that disproportionately affect β̂. Goodness‑of‑fit is assessed by the scaled deviance D/φ (≈ χ²n−p) and by the Hosmer–Lemeshow test for grouped binary data. Examiners often ask to derive the score equations for a binomial GLM with a probit link, to show that the canonical link for the gamma family is the reciprocal, or to explain why the Pearson χ² statistic may be preferred over the deviance when the response is sparse.
High‑Dimensional Statistics: Regularization, Sparsity, and Model Selection
When the number of predictors p exceeds the sample size n, ordinary least squares fails because XᵀX is singular. Regularization adds a penalty to the loss function, producing estimators that trade bias for variance reduction and often perform variable selection. The Lasso (least absolute shrinkage and selection operator) solves min_β {½‖y−Xβ‖₂² + λ‖β‖₁}. The ℓ₁ penalty yields sparse solutions because the constraint region is a diamond with corners on the axes; geometrically, the solution hits a corner when the contour of the RSS touches an axis. The Karush‑Kuhn‑Tucker conditions give the soft‑thresholding rule for orthogonal designs: β̂ⱼ = sign(β̂ⱼOLS) (|β̂ⱼOLS|−λ)_+. For correlated predictors the Lasso tends to select one variable from a group; the elastic net adds an ℓ₂ term α‖β‖₂² to encourage grouping. The ridge estimator β̂ridge = (XᵀX+λI)−1Xᵀy shrinks coefficients but never sets them exactly to zero. Model selection consistency requires the irrepresentable condition: ‖XS^cᵀX_S (X_SᵀX_S)−1 sign(β_S)‖_∞ ≤ 1−η for some η>0, where S is the true support. Under this condition and with λ ∼ √(log p / n), the Lasso recovers the exact support with probability tending to one. Information criteria adapted to high dimensions include the extended BIC EBIC = n log(RSS/n) + |S| log n + 2γ log(p choose |S|) with γ∈[0,1] controlling the penalty for model space size. Cross‑validation remains the workhorse for choosing λ; however, the one‑standard‑error rule often selects a sparser model with comparable predictive error. Recent theory on de‑biased Lasso constructs asymptotically normal estimators for individual coefficients by adding a correction term Θ Xᵀ(y−Xβ̂)/n where Θ approximates (XᵀX/n)−1. This enables valid confidence intervals and hypothesis tests even when p≫n. Examiners may ask to prove the soft‑thresholding solution for orthogonal X, to state the irrepresentable condition, or to compare the prediction risk of ridge versus Lasso under a sparse true β.
Causal Inference: Potential Outcomes, Directed Acyclic Graphs, and Identification
Causal inference formalises the question “what would happen if we intervened?” using the potential outcomes framework (Neyman‑Rubin). For each unit i let Yᵢ(1) and Yᵢ(0) denote the outcomes under treatment and control. The individual causal effect is τᵢ = Yᵢ(1)−Yᵢ(0); the average treatment effect (ATE) is τ = E[Y(1)−Y(0)]. The fundamental problem of causal inference is that only one potential outcome is observed: Yᵢ = TᵢYᵢ(1)+(1−Tᵢ)Yᵢ(0). Identification requires assumptions: (i) Consistency: observed outcome equals the potential outcome under the received treatment; (ii) Conditional exchangeability (ignorability): Y(1),Y(0) ⟂ T | X; (iii) Positivity: 0 < P(T=1|X) < 1 for all X. Under these, τ = E[ E[Y|T=1,X] − E[Y|T=0,X] ]. Directed acyclic graphs (DAGs) encode conditional independencies via d‑separation. A back‑door path from T to Y is any path that starts with an arrow into T; a set Z satisfies the back‑door criterion if it blocks all back‑door paths and contains no descendants of T. Then τ = Σ_z {E[Y|T=1,Z=z]−E[Y|T=0,Z=z]} P(Z=z). The front‑door criterion applies when a mediator M intercepts all directed paths from T to Y and there is no unblocked back‑door path from T to M; identification then uses τ = Σ_m P(M=m|T=1) Σt' E[Y|T=t',M=m] P(T=t'). Instrumental variables (IV) exploit a variable Z that affects T, has no direct effect on Y, and is independent of unmeasured confounders; the Wald estimand τIV = Cov(Y,Z)/Cov(T,Z) identifies the local average treatment effect (LATE) for compliers. Estimation methods include inverse‑probability weighting (IPW) with weights 1/e(X) for treated and 1/(1−e(X)) for controls, where e(X)=P(T=1|X) is the propensity score; doubly robust estimators combine IPW with outcome regression and remain consistent if either model is correct. Sensitivity analysis quantifies how strong an unmeasured confounder must be to overturn the conclusion; Rosenbaum’s Γ bounds the odds ratio of treatment assignment between matched pairs. Examiners often ask to draw a DAG for a given scenario, to state the back‑door criterion, or to derive the IPW estimator and prove its double robustness.
Advanced Sampling Designs: Adaptive, Ranked Set, and Two‑Phase Sampling
Standard probability sampling (SRS, stratified, cluster) assumes a fixed design. Adaptive sampling modifies the selection probabilities based on observed values, improving efficiency for rare or clustered populations. In adaptive cluster sampling, an initial sample of units is drawn; if a unit satisfies a condition (e.g., yᵢ > c), its neighbours are added to the sample, and the process repeats. The Hansen‑Hurwitz estimator for the population total is Ŷ = Σi∈s yᵢ / πᵢ where πᵢ are the inclusion probabilities computed from the adaptive network. Variance estimation uses the conditional variance given the initial sample plus the variance of the conditional expectation. Ranked set sampling (RSS) exploits cheap ranking information: draw m sets of size r, rank units within each set by a concomitant variable, then measure the unit with rank k from the k‑th set. The RSS mean ȳRSS = (1/m) Σk=1^r ȳ(k) is unbiased for the population mean and has variance Var(ȳRSS) = (1/mr) Σk=1^r σ(k)² where σ(k)² is the variance of the k‑th order statistic. RSS outperforms SRS when ranking is imperfect but positively correlated with the variable of interest. Two‑phase (double) sampling collects auxiliary information on a large first‑phase sample (n₁) and the study variable on a subsample (n₂). The regression estimator Ŷreg = N ȳ₂ + β̂ (X̄₁−x̄₂) uses the first‑phase mean X̄₁ to improve precision; its variance is approximately (1−n₂/N) S_y²/n₂ + (1−n₁/N) β² S_x²/n₁. Optimal allocation minimises cost C = c₁n₁ + c₂n₂ subject to a variance constraint, yielding n₁/n₂ = √(c₂ S_x² / c₁ β² S_y²). Examiners may ask to derive the inclusion probabilities for adaptive cluster sampling, to prove unbiasedness of the RSS mean, or to obtain the optimal n₁,n₂ for a given cost function.
- Confusing the canonical link with the identity link for non‑Gaussian families and applying the wrong score equations.
- Assuming the Lasso selects the true model without verifying the irrepresentable condition or the β‑min condition.
- Stating that IPW alone yields a doubly robust estimator; double robustness requires an outcome model as well.
- Using the standard SRS variance formula for ranked set sampling and ignoring the order‑statistic variances.
- Neglecting the positivity assumption in causal inference and reporting an ATE estimate when some propensity scores are near zero.
- Treating the front‑door criterion as a substitute for the back‑door criterion without checking the no‑unblocked‑back‑door‑to‑mediator condition.
- Applying the Wald IV estimand when the instrument is weak (Cov(T,Z) ≈ 0) and not reporting the resulting large finite‑sample bias.
These advanced topics deepen the aspirant’s ability to tackle non‑standard questions that combine theory with practical implementation.