Flashcards in Simple Linear Regression Model; Estimation and Inference Deck (17)
The independent variable, explanatory variable, predictor variable or regressor , exogenous.
The dependent variable,
explained variable, response variable or regressand, endogenous.
What is a statistical relationship?
Statistical relationships among variables deal with random or stochastic variables, that is, variables that have probability distributions.
What is a deterministic relationship?
Functional or deterministic dependency, we also deal with variables, but these variables are not random or stochastic.
Regression versus causation
Although regression analysis deals with the dependence of one variable on other variables, it does not necessarily imply causation. A statistical relationship, however strong and however suggestive, can never establish causal connection.
Regression versus correlation
Correlation Analysis: the primary objective is to measure the strength or degree of linear association between two variables (both are assumed to be random)
Regression Analysis: we try to estimate or predict the average value of one variable (dependent, and assumed to be stochastic) on the basis of the fixed values of other variables (independent, and non-stochastic)
Population Regression Function
It states merely that the expected value of the distribution of Y given Xi is functionally related to Xi.
If we join conditional mean values, we obtain what is known as the population regression line (PRL), or more generally, the population regression curve. More simply, it is the regression of Y on X.
Geometrically, then, a population regression curve is simply the locus of the conditional means of the dependent variable for the fixed values of the explanatory variable(s).
conditional expectation function (CEF) or population regression function (PRF) or population regression (PR) for short.
• In simple terms, it tells how the mean or average response of Y varies with X.
Linearity in variables
The first and perhaps more “natural” meaning of linearity is that the conditional expectation of Y is a linear function of Xi,
Linearity in parameters
The second interpretation of linearity is that the conditional expectation of
Y, E(Y | Xi), is a linear function of the parameters, the β’s; it may or may not be linear in the variable X.
Of the two interpretations of linearity, linearity in the parameters is relevant for the development of the regression theory.
Properties of Expected Values
•The expected value of a constant is the constant itself. Thus, if b is a constant,
E(b) = b.
• If a and b are constants,
E(aX + b) = aE(X) + b
• If X and Y are independent random variables, then E(XY) = E(X)E(Y)
• If X is a random variable with PDF f(x) and if g(X) is any function of X, then
Let X be a random variable and let
E(X) = μ. The distribution, or spread, of the X values around the expected value can be measured by the variance, which is defined as
Var (X) = σ^2 = E(X − μ)^2
E(X^2) - μ^2
E(X^2) - [E(X)]^2
Deterministic component of the function
The part of the variation in Y explained by the changes in X
Random component of the function
The variation in Y that is explained by other factors other than X.
The end result is a point estimate, which is a single number that serves as an intelligent guess of the value of the parameter.
The estimate consists of a range of values thought to contain an unknown parameter.
The disturbance term u is a surrogate for all those variables that are omitted from the model but that collectively affect Y.