《多元回归分析.ppt》由会员分享，可在线阅读，更多相关《多元回归分析.ppt（70页珍藏版）》请在三一文库上搜索。
1、多元迴歸分析,遺漏變數偏誤 多元迴歸模型 多元迴歸模型的估計 多元迴歸模型: 實例 變異數分析與參數檢定 多元迴歸模型的幾個重要議題,遺漏變數偏誤,我們將不再假設解釋變數為固定值, 而是隨機變數 在簡單迴歸模型中, 只有一個解釋變數, 然而, 在大多數的情形下, 被解釋變數Y 通常可被一個以上的變數所解釋。舉例來說, 所得水準除了受到教育程度的影響之外, 亦可能受到工作經驗等其他變數所影響,遺漏變數偏誤,此外, 只考慮一個解釋變數時, 可能會產生遺漏變數偏誤(omitted variable bias) 考慮解釋變數(如教育程度) 與另外一個變數(如父母所得水準) 具相關性,(一般來說, 父母
2、所得越高, 子女能夠得到的教育越好, 教育程度自然越高) 且該變數(父母所得水準) 本身亦會直接影響被解釋變數(所得水準), (一般來說, 父母所得越高, 投注在子女身上的其他資源越多, 子女的所得也因而越高),遺漏變數偏誤,如果我們在迴歸模型中忽略了此變數, 就會造成遺漏變數偏誤 假設原有解釋變數為X, 遺漏變數為Z, 而被解釋變數為Y 。換句話說, 一個變數是否為迴歸模型中的遺漏變數, 必須符合以下兩條件: 該變數與模型原有的解釋變數相關: Corr (X, Z) 0。 該變數 Z 亦會直接影響被解釋變數Y 。,Suppose the true model is The estimated
3、 model is The covariance between Xi and error term is,6,Therefore, Since 0, we have,7,遺漏變數的影響,遺漏變數偏誤不會隨樣本增加而變小 簡言之, 如果我們忽略了遺漏變數, 將使原有的解釋變數的估計式 不是參數 的一致估計式 遺漏變數偏誤決定於|Cov(X, Z)| 的大小 若Cov(X, Z) 0, 則存在正向偏誤(高估欲估計的參數); 反之, 若Cov(X, Z) 0, 則存在負向偏誤(低估欲估計的參數)。,An example of omitted variable bias:,Mozart Effect
4、? Listening to Mozart for 10-15 minutes could raise IQ by 8 or 9 points. (Nature 1993) Students who take optional music or arts courses in high school have higher English and math test scores than those who dont.,9,多元迴歸模型,我們將只考慮一個解釋變數的簡單迴歸模型擴充為如下的多元迴歸模型: 其中, X = X1, . . . , Xk 就是模型中的k 個解釋變數, ei 為隨機干
5、擾項, 且,y,x1,b,0,Response,Plane,(Observed y),e,i,Population Multiple Regression Model,Bivariate model:,x2,(x1i , x2i),是未知參數, 其意義為 亦即在控制其他變數影響之情況下, 第j 個解釋變數對於Y 的淨影響,多元迴歸模型: 薪資所得, 教育程度與工作經驗,多元迴歸模型為 薪資所得= 0+ 1教育程度+ 2工作經驗+ei , 簡單迴歸模型為 薪資所得= + 教育程度+ ei , 可以確定的是, 1 與 都是用來探討教育程度對於薪資所得的影響, 但是1 與 的詮釋卻不相同, 單純地衡
6、量教育程度如何影響薪資所得, 亦即, 教育程度增加一單位(譬如說增加一年), 薪資所得將增加 單位 然而, 我們知道影響薪資所得的解釋變數應該不只一個, 因此, 一旦我們將其他可能的解釋變數考慮進來(本例中的工作經驗), 則1 詮釋為:在給定相同的工作經驗下, 教育程度增加一單位, 薪資所得將增加1 單位,多元迴歸模型,這就是在經濟學的研究中, 我們時常探討所謂的其他情況不變下(ceteris paribus), 變數之間的關係 譬如說, 其他情況不變下, 價格如何影響需求量。或者是, 其他情況不變下, 工資率如何影響勞動供給,多元迴歸模型的估計,欲估計迴歸模型中的未知參數, 我們知道 相互獨
7、立, 最小平方法為,多元迴歸模型的估計,因此, 尋找 來極大 透過 我們可以得到k + 1 條標準方程式, 進而解出 許多商業軟體如EXCEL 都能夠輕易地幫你找出這些估計值,Estimation of 2,For a model with k independent variables,多元迴歸模型: 實例,阿中為一物流送貨員, 時常在外奔波運送貨品。阿中的老板懷疑阿中利用在外送貨的空檔開小差, 因此,阿中的老板將他以前的送貨行程記錄調出,根據多元迴歸模型: 其中, Y =在外奔波時數, X1 =送貨路程, 而X2 =送貨點個數 阿中的老板估計出如下的迴歸模型,在固定的送貨點個數下, 阿中的
8、送貨路程每多一公里, 在外奔波時數增加0.066 小時; 在相同的送貨路程下, 阿中的送貨點每多一個,在外奔波時數增加0.694 小時 其中,在本例中, 以及,根據自由度為n (k + 1) = 10 (2 + 1) = 7的t 分配, 在顯著水準 =1%, 5% 以及10%的臨界值分別為3.499, 2.365 以及1.895 因此, 在1% 的顯著水準下具顯著性, 而 則是在10% 的顯著水準下具顯著性 送貨路程與送貨點個數無論是在經濟上或是統計上均具顯著性 亦即, 都是在外奔波時數的重要解釋變數,在得到以上的估計後, 阿中的老板一旦知道阿中今天有5 個送貨點得跑, 總路程為110 公里,
9、 則阿中的老板可以預測阿中今天在外奔波時數為0.39 + 0.066 110 + 0.694 5 = 10.35 小時 如果阿中今天在外奔波了12 個小時, 則阿中的老板就能夠合理地懷疑阿中利用2 小時開小差 這個例子清楚地說明迴歸模型的兩大重要功能:解釋與預測,23.1 The Multiple Regression Model,A chain is considering where to locate a new restaurant. Is it better to locate it far from the competition or in a more affluent are
10、a? Use multiple regression to describe the relationship between several explanatory variables and the response. Multiple regression separates the effects of each explanatory variable on the response and reveals which really matter.,Copyright 2011 Pearson Education, Inc.,3 of 47,23.2 Interpreting Mul
11、tiple Regression,Example: Womens Apparel Stores Response variable: sales at stores in a chain of womens apparel (annually in dollars per square foot of retail space). Two explanatory variables: median household income in the area (thousands of dollars) and number of competing apparel stores in the s
12、ame mall.,Copyright 2011 Pearson Education, Inc.,7 of 47,23.2 Interpreting Multiple Regression,Example: Womens Apparel Stores Begin with a scatterplot matrix, a table of scatterplots arranged as in a correlation matrix. Using a scatterplot matrix to understand data can save considerable time later w
13、hen interpreting the multiple regression results.,Copyright 2011 Pearson Education, Inc.,8 of 47,23.2 Interpreting Multiple Regression,Scatterplot Matrix: Womens Apparel Stores,Copyright 2011 Pearson Education, Inc.,9 of 47,23.2 Interpreting Multiple Regression,Example: Womens Apparel Stores The sca
14、tterplot matrix for this example Confirms a positive linear association between sales and median household income. Shows a weak association between sales and number of competitors.,Copyright 2011 Pearson Education, Inc.,10 of 47,23.2 Interpreting Multiple Regression,Correlation Matrix: Womens Appare
15、l Stores,Copyright 2011 Pearson Education, Inc.,11 of 47,23.2 Interpreting Multiple Regression,Partial Slopes: Womens Apparel Stores,Copyright 2011 Pearson Education, Inc.,16 of 47,23.2 Interpreting Multiple Regression,Marginal and Partial Slopes Partial slope: slope of an explanatory variable in a
16、multiple regression that statistically excludes the effects of other explanatory variables. Marginal slope: slope of an explanatory variable in a simple regression.,Copyright 2011 Pearson Education, Inc.,15 of 47,23.2 Interpreting Multiple Regression,Partial Slopes: Womens Apparel Stores,Copyright 2
17、011 Pearson Education, Inc.,16 of 47,Inference in Multiple Regression,Inference for One Coefficient The t-statistic is used to test each slope using the null hypothesis H0: j = 0. The t-statistic is calculated as,Copyright 2011 Pearson Education, Inc.,31 of 47,Inference in Multiple Regression,t-test
18、 Results for Womens Apparel Stores The t-statistics and associated p-values indicate that both slopes are significantly different from zero.,Copyright 2011 Pearson Education, Inc.,32 of 47,Prediction Intervals An approximate 95% prediction interval is given by . For example, the 95% prediction inter
19、val for sales per square foot at a location with median income of $70,000 and 3 competitors is approximately $545.47 $136.06 per square foot.,Copyright 2011 Pearson Education, Inc.,33 of 47,Partial Slopes: Womens Apparel Stores The slope b1 = 7.966 for Income implies that a store in a location with
20、a higher median household of $10,000 sells, on average, $79.66 more per square foot than a store in a less affluent location with the same number of competitors. The slope b2 = -24.165 implies that, among stores in equally affluent locations, each additional competitor lowers average sales by $24.16
21、5 per square foot.,Copyright 2011 Pearson Education, Inc.,17 of 47,Marginal and Partial Slopes Partial and marginal slopes only agree when the explanatory variables are uncorrelated. In this example they do not agree. For instance, the marginal slope for Competitors is 4.6352. It is positive because
22、 more affluent locations tend to draw more competitors. The MRM separates these effects but the SRM does not.,Copyright 2011 Pearson Education, Inc.,18 of 47,Checking Conditions,Conditions for Inference Use the residuals from the fitted MRM to check that the errors in the model are independent; have
23、 equal variance; and follow a normal distribution.,Copyright 2011 Pearson Education, Inc.,21 of 47,Checking Conditions,Calibration Plot Calibration plot: scatterplot of the response on the fitted values . R2 is the correlation between and ; the tighter data cluster along the diagonal line in the cal
24、ibration plot, the larger the R2 value.,Copyright 2011 Pearson Education, Inc.,22 of 47,23.3 Checking Conditions,Calibration Plot: Womens Apparel Stores,Copyright 2011 Pearson Education, Inc.,23 of 47,23.3 Checking Conditions,Residual Plots Plot of residuals versus fitted y values is used to identif
25、y outliers and to check for the similar variances condition. Plot of residuals versus each explanatory variable are used to verify that the relationships are linear.,Copyright 2011 Pearson Education, Inc.,24 of 47,23.3 Checking Conditions,Residual Plot: Womens Apparel Stores This plot of residuals v
26、ersus fitted values of y has no evident pattern.,Copyright 2011 Pearson Education, Inc.,25 of 47,23.3 Checking Conditions,Residual Plot: Womens Apparel Stores This plot of residuals versus Income has no evident pattern.,Copyright 2011 Pearson Education, Inc.,26 of 47,Checking Conditions,Check Normal
27、ity: Womens Apparel Stores The quantile plot indicates nearly normal condition is satisfied.,Copyright 2011 Pearson Education, Inc.,27 of 47,變異數分析與參數檢定,我們可以輕易地將簡單迴歸模型中的變異數分析表擴展為多元迴歸架構下的變異數分析表。其中, UV的自由度變成n k 1 係因估計參數 而損失了(k + 1) 個自由度。,F 檢定,一如前一章的討論, 對於 的虛無假設, 我們可以採用F 檢定: 在顯著水準為 下, 當 我們拒絕虛無假設,我們可以算出判
28、定係數R2 為 亦即被解釋變數Y 的總變異中, 有多少比例可被迴歸模型所解釋 然而, 每增加一個解釋變數進入多元迴歸模型,UV 亦會隨之減少(或著不變), 進而使得R2 增加(或著不變),為什麼每增加一個解釋變數, UV 就會隨之減少?,假設你本來考慮兩個解釋變數, 如今欲增加一個解釋變數, 因此, 極小化問題變成 如果找到的 恰好為零, 則此時的UV 就會等於只考慮兩個解釋變數時的 : 若找到的 不為零, 代表,亦即, 多增加一個解釋變數會使UV 降低, 進而造成R2 增加 因此, 以R2 來判斷多元迴歸模型會有一個糟糕的問題: 考慮的解釋變數越多, 模型的解釋能力越好 如此一來, 我們若是
29、在原來的模型中無止境的增加解釋變數, 或是放入一些不相干的變數, 模型的解釋力不會降低(亦即增加或是不變), 但是這樣做毫無意義,修正的判定係數(adjusted coefficient of determination),為了彌補判定係數的這個缺陷, 我們採用修正的判定係數:,在 中, 我們對於增加解釋變數予以懲罰, 當解釋變數增加, 雖然R2 會增加或不變, 但是懲罰項增加, 進而拉低 。 因此, 利用修正的判定係數來衡量模型的配適度, 並不會得到解釋變數多多益善的結論,Example: Womens Apparel Stores,Response variable: sales at s
30、tores in a chain of womens apparel (annually in dollars per square foot of retail space). Two explanatory variables: median household income in the area (thousands of dollars) and number of competing apparel stores in the same mall.,Copyright 2011 Pearson Education, Inc.,7 of 47,R-squared and se,The
31、 equation of the fitted model for estimating sales in the womens apparel stores example is = 60.3587 + 7.966 Income -24.165 Competitors,Copyright 2011 Pearson Education, Inc.,12 of 47,R-squared and se,R2 indicates that the fitted equation explains 59.47% of the store-to-store variation in sales. For
32、 this example, R2 is larger than the r2 values for separate SRMs fitted for each explanatory variable; it is also larger than their sum. For this example, se = $68.03.,Copyright 2011 Pearson Education, Inc.,13 of 47,R-squared and se,is known as the adjusted R-squared. It adjusts for both sample size
33、 n and model size k. It is always smaller than R2. The residual degrees of freedom (n-k-1) is the divisor of se. and se move in opposite directions when an explanatory variable is added to the model ( goes up while se goes down).,Copyright 2011 Pearson Education, Inc.,14 of 47,Inference for the Mode
34、l: F-test,F-test: test of the explanatory power of the MRM as a whole. F-statistic: ratio of the sample variance of the fitted values to the variance of the residuals.,Copyright 2011 Pearson Education, Inc.,28 of 47,23.4 Inference in Multiple Regression,Inference for the Model: F-test The F-Statisti
35、c is used to test the null hypothesis that all slopes are equal to zero, e.g., H0: .,Copyright 2011 Pearson Education, Inc.,29 of 47,F-test Results in Analysis of Variance Table,The F-statistic has a p-value of 0.0001; reject H0. Income and Competitors together explain statistically significant vari
36、ation in sales.,Copyright 2011 Pearson Education, Inc.,30 of 47,Steps in Fitting a Multiple Regression,What is the problem to be solved? Do these data help in solving it? Check the scatterplots of the response versus each explanatory variable (scatterplot matrix). If the scatterplots appear straight
37、 enough, fit the multiple regression model. Otherwise find a transformation. Obtain the residuals and fitted values from the regression.,Copyright 2011 Pearson Education, Inc.,34 of 47,Steps in Fitting a Multiple Regression,Use residual plot of e vs. to check for similar variance condition. Construc
38、t residual plots of e vs. explanatory variables. Look for patterns. Check whether the residuals are nearly normal. Use the F-statistic to test the null hypothesis that the collection of explanatory variables has no effect on the response. If the F-statistic is statistically significant, test and int
39、erpret individual partial slopes.,Copyright 2011 Pearson Education, Inc.,35 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Motivation A banking regulator would like to verify how lenders use credit scores to determine the interest rate paid by subprime borrowers. The regulator would like to separate its
40、effect from other variables such as loan-to-value (LTV) ratio, income of the borrower and value of the home.,Copyright 2011 Pearson Education, Inc.,36 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Method Use multiple regression on data obtained for 372 mortgages from a credit bureau. The explanatory var
41、iables are the LTV, credit score, income of the borrower, and home value. The response is the annual percentage rate of interest on the loan (APR).,Copyright 2011 Pearson Education, Inc.,37 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Method Find correlations among variables:,Copyright 2011 Pearson Edu
42、cation, Inc.,38 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Method Check scatterplot matrix (like APR vs. LTV ) Linearity and no obvious lurking variables conditions satisfied.,Copyright 2011 Pearson Education, Inc.,39 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Mechanics Fit model and check conditions.
43、,Copyright 2011 Pearson Education, Inc.,40 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Mechanics Residuals versus fitted values. Similar variances condition is satisfied.,Copyright 2011 Pearson Education, Inc.,41 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Mechanics Nearly normal condition is not satisf
44、ied; data are skewed.,Copyright 2011 Pearson Education, Inc.,42 of 47,4M Example 23.1: SUBPRIME MORTGAGES,Message Regression analysis shows that the characteristics of the borrower (credit score) and loan LTV affect interest rates in the market. These two factors together explain almost half of the variation in interest rates. Neither income of the borrower nor the home value improves a model with these two variables.,Copyright 2011 Pearson Education, Inc.,43 of 47,