http://ordination.okstate.edu/MULTIPLE.htm
The more variables you have, the higher the amount of variance you can explain. Even if each variable doesn't explain much, adding a large number of variables can result in very high values of R2. This is why some packages provide "Adjusted R2," which allows you to compare regressions with different numbers of variables.
The same holds true for polynomial regression. If you have N data points, then you can fit the points exactly with a polynomial of degree N-1.
The degrees of freedom in a multiple regression equals N-k-1, where k is the number of variables. The more variables you add, the more you erode your ability to test the model (e.g. your statistical power goes down).
Statistical power problems with moderated multiple regression in management research
Journal of Management, Nov-Dec, 1995 by Herman Aguinis
http://findarticles.com/p/articles/mi_m4256/is_n6_v21/ai_17792511
Artifacts Influencing the Power of MMR
Factors identified as detrimental to statistical power in MMR hypothesis tests are related to (1) variable distributions (predictor variable range restriction, error variance heterogeneity), (2) operationalizations of criterion and predictor variable (measurement error, inappropriate metrics, artificial dichotomization or polychotomization), (3) sample size (total sample size, sample size across moderator-based subgroups), and (4) predictor intercorrelation.(2) Next, each of these factors and their impact on the power of MMR are described.
In MMR analyses, predictor scores (X and Z) are used to compute the product term ([X.sup.*]Z) which carries information about the interaction. Thus, X and [X.sup.*]Z, and Z and [X.sup.*]Z tend to be highly correlated (i.e., multicollinear). Some researchers (Morris et al., 1986; Smith & Sasaki, 1979) have argued that the presence of multicollinearity in MMR leads to an ill-conditioned solution in which the regression coefficients are unstable, error terms are larger, and power is decreased. In high multicollinearity situations, small observed score changes due to measurement error may be magnified and result in large changes in B (i.e., the vector of unstandardized regression coefficients) and, consequently, there is a larger capitalization on chance. Thus, because multicollinearity is virtually guaranteed in MMR, and is known to lead to unstable coefficients (including [b.sub.3]), it has been posited that the power of MMR is not sufficient to detect moderating effects (Morris et al., 1986).
Given this supposed power problem, two strategies have been proposed to mitigate multicollinearity: (a) "centering" predictor variables. The most common centering approach is to subtract the mean from each score (i.e., [Mathematical Expression Omitted]; [Mathematical Expression Omitted]; [Mathematical Expression Omitted]) (cf. Tate, 1984). Tate (1984, p. 253) illustrates how centering reduces collinearity by noting that the slope of [X.sup.*]Z on X for a central value (mean) of Z is [Mathematical Expression Omitted], whereas the slope of [Mathematical Expression Omitted] on [Mathematical Expression Omitted] at [Mathematical Expression Omitted] is zero. (b) Morris et al. (1986) introduced the use of principal-components regression (PCR), which was advocated as not being as affected by multicollinearity and, consequently, as being a more powerful method than MMR for tests of moderator hypotheses.
Fortunately, recent developments suggest that concerns about the detrimental impact of multicollinearity on power are unwarranted. Cronbach (1987) stated that the effects of multicollinearity on MMR analyses are: (1) increased rounding error, (2) increased regression coefficient sampling errors, and (3) difficulty in regression coefficient interpretation, especially for lower-order terms (see also Aiken & West, 1991). However, Cronbach asserted that multicollinearity is not detrimental to the power of MMR, as Morris et al. contended. The reasons for the apparent loss of power are that (1) the number of predictors reduces the degrees of freedom for the numerator of the F ratio, and (2) in the presence of multicollinearity, additional predictors contribute little to the sum of squares for regression. Thus, in one example provided by Morris et al., it seemed that decreasing the number of predictors from 13 to 10 reduced collinearity and increased power. However, Cronbach showed that in the typical MMR analysis, there are two measured predictors, in addition to one derived from them. In this more typical situation, collinearity does not adversely affect power. It should be noted, however, that high multicollinearity may cause computational problems and, thus, it is recommended that predictors be centered before computing the product term (Cronbach, 1987; Jaccard et al., 1990).
