Chapter 3 · Section 7 practice
Multiple Linear Regression and Multicollinearity
Work from question 1 to 10. Edit your Python box, click Run Code, review the result, then click Submit attempt. Reference answers unlock after all ten attempts are submitted. This records completion, not correctness. Each box runs independently. Progress is saved in this browser.
1. Count usable cities
Warm-up. Print the number of raw rows, observed-profit cities and residual degrees of freedom.
Write your answer, then run it.
2. Read a partial slope
Warm-up. Print the disposable-income slope and the fitted profit difference for US$1,000 higher disposable income, holding the other inputs fixed.
Write your answer, then run it.
3. Predict the proposal
Warm-up. Supply the six proposed-city inputs and print its full-model predicted annual Profit.
Write your answer, then run it.
4. Reconstruct adjusted R-squared
Build your skills. Reconstruct adjusted R-squared from SSE, SST, n and the six predictors; compare with statsmodels.
Write your answer, then run it.
5. Read regression RMSE
Build your skills. Calculate course RMSE s in annual US dollars using the correct degrees of freedom.
Write your answer, then run it.
6. Inspect predictor dependence
Build your skills. Print the correlation between social-security recipients and the age-65+ share.
Write your answer, then run it.
7. Calculate an auxiliary VIF
Build your skills. Regress social-security recipients on the other five demographics and calculate its VIF.
Write your answer, then run it.
8. Read joint significance
Challenge. Print the overall F statistic and its p-value.
Write your answer, then run it.
9. Read individual uncertainty
Challenge. Print the social-security t statistic, p-value and coefficient confidence interval.
Write your answer, then run it.
10. Recognise perfect dependence
Challenge. Create a predictor equal to twice Income. Print the matrix rank and number of columns for the intercept plus these two income columns.
Write your answer, then run it.
Reference answers are locked until all 10 attempts are submitted.
1. Count usable cities
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
print(len(raw), len(observed), int(model.df_resid))Expected output
111 110 103The file includes one proposal with no Profit; it is excluded from the 110-row fit.
2. Read a partial slope
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
print(model.params["Disposable Income"])
print(model.params["Disposable Income"] * 1000)Expected output
2.535033685877819
2535.0336858778187The slope has units of annual profit dollars per income dollar.
3. Predict the proposal
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
new = raw.loc[raw["Profit"].isna(), features].copy()
new["const"] = 1.0
new = new[["const"] + features]
print(model.predict(new).iloc[0])Expected output
185509.19914483736The missing proposed outcome is not used in model estimation.
4. Reconstruct adjusted R-squared
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
n, k = len(observed), len(features)
sse = (model.resid ** 2).sum()
sst = ((observed["Profit"] - observed["Profit"].mean()) ** 2).sum()
print(1 - (sse / (n - k - 1)) / (sst / (n - 1)))
print(model.rsquared_adj)Expected output
0.7415425112215834
0.7415425112215834There are six predictors and one fitted intercept.
5. Read regression RMSE
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
n, k = len(observed), len(features)
print(((model.resid ** 2).sum() / (n - k - 1)) ** 0.5)Expected output
13547.405851410773The denominator is 103; the unit is the Profit unit.
6. Inspect predictor dependence
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
print(observed["Soc Security (per 1,000)"].corr(observed["% 65 or Older"]))Expected output
0.9380039475543935This is dependence among two predictors, not their correlations with Profit.
7. Calculate an auxiliary VIF
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
name = "Soc Security (per 1,000)"
others = [column for column in features if column != name]
auxiliary = sm.OLS(observed[name], sm.add_constant(observed[others])).fit()
print(1 / (1 - auxiliary.rsquared))Expected output
10.03817284743602Profit is excluded from this auxiliary regression.
8. Read joint significance
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
print(model.fvalue)
print(model.f_pvalue)Expected output
53.12213306024511
2.4060224708208015e-29The joint null sets all six slopes to zero.
9. Read individual uncertainty
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
name = "Soc Security (per 1,000)"
print(model.tvalues[name], model.pvalues[name])
print(model.conf_int().loc[name])Expected output
-0.431131118541678 0.6672739996214572
0 -266.097397
1 171.065013
Name: Soc Security (per 1,000), dtype: float64The coefficient is conditional on the other five demographics; a large p-value can reflect imprecision.
10. Recognise perfect dependence
import pandas as pd
import statsmodels.api as sm
raw = pd.read_csv("https://busanalytics-book.pages.dev/data/profit.csv")
features = [
"Income", "Disposable Income", "Birth Rate (per 1,000)",
"Soc Security (per 1,000)", "CV Death (per 100,000)", "% 65 or Older"
]
# The proposed Kansas City location has no observed Profit.
observed = raw.dropna(subset=["Profit"]).copy()
X = sm.add_constant(observed[features])
model = sm.OLS(observed["Profit"], X).fit()
import numpy as np
observed["IncomeTwice"] = 2 * observed["Income"]
design = sm.add_constant(observed[["Income", "IncomeTwice"]])
print(np.linalg.matrix_rank(design), design.shape[1])Expected output
2 3Rank two for three columns means the two income slopes are not uniquely identifiable.