What does it mean when I add a new variable to my linear model and the R^2 stays the same?In what cases (if any) does r^2 remain unchanged on adding a new variable?How can I predict values from new inputs of a linear model in R?Does a stepwise approach produce the highest $R^2$ model?F test and t test in linear regression modelCompare linear regression models (same and different response variable)In linear model, if you add one more variable, then what happens to the constant?Getting estimate and CI for dummy variable in linear modelCircularity in Linear Regression: Independent variable used as dependent in the same modelWhat is the difference between generalized linear models and generalized least squaresPCA without response variable to get linearly dependent set of linear (mixed) model inputswhy does adding new variables to a regression model keep R squared unchanged

What is it called when someone votes for an option that's not their first choice?

How do you say "Trust your struggle." in French?

Connection Between Knot Theory and Number Theory

Not hide and seek

Mortal danger in mid-grade literature

Calculate Pi using Monte Carlo

Center page as a whole without centering each element individually

Why is implicit conversion not ambiguous for non-primitive types?

Are all namekians brothers?

How do you justify more code being written by following clean code practices?

Do people actually use the word "kaputt" in conversation?

How do I prevent inappropriate ads from appearing in my game?

How can I, as DM, avoid the Conga Line of Death occurring when implementing some form of flanking rule?

What is the period/term used describe Giuseppe Arcimboldo's style of painting?

What (if any) is the reason to buy in small local stores?

Asserting that Atheism and Theism are both faith based positions

Why can't I get pgrep output right to variable on bash script?

Hashing password to increase entropy

Can you take a "free object interaction" while incapacitated?

What is the purpose of using a decision tree?

How to get directions in deep space?

Friend wants my recommendation but I don't want to give it to him

What should be the ideal length of sentences in a blog post for ease of reading?

Why would five hundred and five same as one?



What does it mean when I add a new variable to my linear model and the R^2 stays the same?


In what cases (if any) does r^2 remain unchanged on adding a new variable?How can I predict values from new inputs of a linear model in R?Does a stepwise approach produce the highest $R^2$ model?F test and t test in linear regression modelCompare linear regression models (same and different response variable)In linear model, if you add one more variable, then what happens to the constant?Getting estimate and CI for dummy variable in linear modelCircularity in Linear Regression: Independent variable used as dependent in the same modelWhat is the difference between generalized linear models and generalized least squaresPCA without response variable to get linearly dependent set of linear (mixed) model inputswhy does adding new variables to a regression model keep R squared unchanged













4












$begingroup$


I'm inclined to think that the new variable is not correlated to the response. But could the new variable be correlated to another variable in the model?










share|cite|improve this question









$endgroup$











  • $begingroup$
    It depends, could you provide us with some reduced data lines or output from your linear models. Without more information it's hard to assist you
    $endgroup$
    – OliverFishCode
    Mar 7 at 19:54






  • 5




    $begingroup$
    It shouldn't stay exactly the same unless it is perfectly orthogonal to your response, or is a linear combination of the variables already included. It may be that the change is smaller than the number of decimal places displayed.
    $endgroup$
    – gung
    Mar 7 at 20:02






  • 5




    $begingroup$
    @gung What you can infer is that the new variable is orthogonal to the response modulo the subspace generated by the other variables. That's more general than the two options you mention.
    $endgroup$
    – whuber
    Mar 7 at 20:12










  • $begingroup$
    @whuber, yes, I suppose so.
    $endgroup$
    – gung
    Mar 7 at 20:17










  • $begingroup$
    Test your variables for multicollinearity en.wikipedia.org/wiki/Multicollinearity probably some features are linearly connected. Use caret package and vif() in R sthda.com/english/articles/39-regression-model-diagnostics/…
    $endgroup$
    – Tom Zinger
    Mar 7 at 22:33
















4












$begingroup$


I'm inclined to think that the new variable is not correlated to the response. But could the new variable be correlated to another variable in the model?










share|cite|improve this question









$endgroup$











  • $begingroup$
    It depends, could you provide us with some reduced data lines or output from your linear models. Without more information it's hard to assist you
    $endgroup$
    – OliverFishCode
    Mar 7 at 19:54






  • 5




    $begingroup$
    It shouldn't stay exactly the same unless it is perfectly orthogonal to your response, or is a linear combination of the variables already included. It may be that the change is smaller than the number of decimal places displayed.
    $endgroup$
    – gung
    Mar 7 at 20:02






  • 5




    $begingroup$
    @gung What you can infer is that the new variable is orthogonal to the response modulo the subspace generated by the other variables. That's more general than the two options you mention.
    $endgroup$
    – whuber
    Mar 7 at 20:12










  • $begingroup$
    @whuber, yes, I suppose so.
    $endgroup$
    – gung
    Mar 7 at 20:17










  • $begingroup$
    Test your variables for multicollinearity en.wikipedia.org/wiki/Multicollinearity probably some features are linearly connected. Use caret package and vif() in R sthda.com/english/articles/39-regression-model-diagnostics/…
    $endgroup$
    – Tom Zinger
    Mar 7 at 22:33














4












4








4


1



$begingroup$


I'm inclined to think that the new variable is not correlated to the response. But could the new variable be correlated to another variable in the model?










share|cite|improve this question









$endgroup$




I'm inclined to think that the new variable is not correlated to the response. But could the new variable be correlated to another variable in the model?







linear-model r-squared






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Mar 7 at 19:46









Chance113Chance113

362




362











  • $begingroup$
    It depends, could you provide us with some reduced data lines or output from your linear models. Without more information it's hard to assist you
    $endgroup$
    – OliverFishCode
    Mar 7 at 19:54






  • 5




    $begingroup$
    It shouldn't stay exactly the same unless it is perfectly orthogonal to your response, or is a linear combination of the variables already included. It may be that the change is smaller than the number of decimal places displayed.
    $endgroup$
    – gung
    Mar 7 at 20:02






  • 5




    $begingroup$
    @gung What you can infer is that the new variable is orthogonal to the response modulo the subspace generated by the other variables. That's more general than the two options you mention.
    $endgroup$
    – whuber
    Mar 7 at 20:12










  • $begingroup$
    @whuber, yes, I suppose so.
    $endgroup$
    – gung
    Mar 7 at 20:17










  • $begingroup$
    Test your variables for multicollinearity en.wikipedia.org/wiki/Multicollinearity probably some features are linearly connected. Use caret package and vif() in R sthda.com/english/articles/39-regression-model-diagnostics/…
    $endgroup$
    – Tom Zinger
    Mar 7 at 22:33

















  • $begingroup$
    It depends, could you provide us with some reduced data lines or output from your linear models. Without more information it's hard to assist you
    $endgroup$
    – OliverFishCode
    Mar 7 at 19:54






  • 5




    $begingroup$
    It shouldn't stay exactly the same unless it is perfectly orthogonal to your response, or is a linear combination of the variables already included. It may be that the change is smaller than the number of decimal places displayed.
    $endgroup$
    – gung
    Mar 7 at 20:02






  • 5




    $begingroup$
    @gung What you can infer is that the new variable is orthogonal to the response modulo the subspace generated by the other variables. That's more general than the two options you mention.
    $endgroup$
    – whuber
    Mar 7 at 20:12










  • $begingroup$
    @whuber, yes, I suppose so.
    $endgroup$
    – gung
    Mar 7 at 20:17










  • $begingroup$
    Test your variables for multicollinearity en.wikipedia.org/wiki/Multicollinearity probably some features are linearly connected. Use caret package and vif() in R sthda.com/english/articles/39-regression-model-diagnostics/…
    $endgroup$
    – Tom Zinger
    Mar 7 at 22:33
















$begingroup$
It depends, could you provide us with some reduced data lines or output from your linear models. Without more information it's hard to assist you
$endgroup$
– OliverFishCode
Mar 7 at 19:54




$begingroup$
It depends, could you provide us with some reduced data lines or output from your linear models. Without more information it's hard to assist you
$endgroup$
– OliverFishCode
Mar 7 at 19:54




5




5




$begingroup$
It shouldn't stay exactly the same unless it is perfectly orthogonal to your response, or is a linear combination of the variables already included. It may be that the change is smaller than the number of decimal places displayed.
$endgroup$
– gung
Mar 7 at 20:02




$begingroup$
It shouldn't stay exactly the same unless it is perfectly orthogonal to your response, or is a linear combination of the variables already included. It may be that the change is smaller than the number of decimal places displayed.
$endgroup$
– gung
Mar 7 at 20:02




5




5




$begingroup$
@gung What you can infer is that the new variable is orthogonal to the response modulo the subspace generated by the other variables. That's more general than the two options you mention.
$endgroup$
– whuber
Mar 7 at 20:12




$begingroup$
@gung What you can infer is that the new variable is orthogonal to the response modulo the subspace generated by the other variables. That's more general than the two options you mention.
$endgroup$
– whuber
Mar 7 at 20:12












$begingroup$
@whuber, yes, I suppose so.
$endgroup$
– gung
Mar 7 at 20:17




$begingroup$
@whuber, yes, I suppose so.
$endgroup$
– gung
Mar 7 at 20:17












$begingroup$
Test your variables for multicollinearity en.wikipedia.org/wiki/Multicollinearity probably some features are linearly connected. Use caret package and vif() in R sthda.com/english/articles/39-regression-model-diagnostics/…
$endgroup$
– Tom Zinger
Mar 7 at 22:33





$begingroup$
Test your variables for multicollinearity en.wikipedia.org/wiki/Multicollinearity probably some features are linearly connected. Use caret package and vif() in R sthda.com/english/articles/39-regression-model-diagnostics/…
$endgroup$
– Tom Zinger
Mar 7 at 22:33











2 Answers
2






active

oldest

votes


















5












$begingroup$

Seeing little to no change in $R^2$ when you add a variable to a linear model means that the variable has little to no additional explanatory power to the response over what is already in your model. As you note, this can be either because it tells you almost nothing about the response or it explains the same variation in the response as the variables already in the model.






share|cite|improve this answer









$endgroup$




















    1












    $begingroup$

    As others have alluded, seeing no change in $R^2$ when you add a variable to your regression is unusual. In finite samples, this should only happen when your new variable is a linear combination of variables already present. In this case, most standard regression routines simply exclude that variable from the regression, and your $R^2$ will remain unchanged because the model was effectively unchanged.



    As you notice, this does not mean the variable is unimportant, but rather that you are unable to distinguish its effect from that of the other variables in your model.



    More broadly however, I (and many here at Cross Validated) would caution against using R^2 for model selection and interpretation. What I've discussed above is how the $R^2$ could not change and the variable still be important. Worse yet, the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable. Broadly, using $R^2$ for model selection fell out of favor in the 70s, when it was dropped in favor of AIC (and its contemporaries). Today -- a typical statistician would recommend using cross validation (see the site name) for your model selection.



    In general, adding a variable increases $R^2$ -- so using $R^2$ to determine a variables importance is a bit of a wild goose chase. Even when trying to understand simple situations you will end up with a completely absurd collection of variables.






    share|cite|improve this answer









    $endgroup$












    • $begingroup$
      Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
      $endgroup$
      – Richard Hardy
      Mar 7 at 21:38











    Your Answer





    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "65"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396220%2fwhat-does-it-mean-when-i-add-a-new-variable-to-my-linear-model-and-the-r2-stays%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    5












    $begingroup$

    Seeing little to no change in $R^2$ when you add a variable to a linear model means that the variable has little to no additional explanatory power to the response over what is already in your model. As you note, this can be either because it tells you almost nothing about the response or it explains the same variation in the response as the variables already in the model.






    share|cite|improve this answer









    $endgroup$

















      5












      $begingroup$

      Seeing little to no change in $R^2$ when you add a variable to a linear model means that the variable has little to no additional explanatory power to the response over what is already in your model. As you note, this can be either because it tells you almost nothing about the response or it explains the same variation in the response as the variables already in the model.






      share|cite|improve this answer









      $endgroup$















        5












        5








        5





        $begingroup$

        Seeing little to no change in $R^2$ when you add a variable to a linear model means that the variable has little to no additional explanatory power to the response over what is already in your model. As you note, this can be either because it tells you almost nothing about the response or it explains the same variation in the response as the variables already in the model.






        share|cite|improve this answer









        $endgroup$



        Seeing little to no change in $R^2$ when you add a variable to a linear model means that the variable has little to no additional explanatory power to the response over what is already in your model. As you note, this can be either because it tells you almost nothing about the response or it explains the same variation in the response as the variables already in the model.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered Mar 7 at 19:55









        TrynnaDoStatTrynnaDoStat

        5,57211335




        5,57211335























            1












            $begingroup$

            As others have alluded, seeing no change in $R^2$ when you add a variable to your regression is unusual. In finite samples, this should only happen when your new variable is a linear combination of variables already present. In this case, most standard regression routines simply exclude that variable from the regression, and your $R^2$ will remain unchanged because the model was effectively unchanged.



            As you notice, this does not mean the variable is unimportant, but rather that you are unable to distinguish its effect from that of the other variables in your model.



            More broadly however, I (and many here at Cross Validated) would caution against using R^2 for model selection and interpretation. What I've discussed above is how the $R^2$ could not change and the variable still be important. Worse yet, the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable. Broadly, using $R^2$ for model selection fell out of favor in the 70s, when it was dropped in favor of AIC (and its contemporaries). Today -- a typical statistician would recommend using cross validation (see the site name) for your model selection.



            In general, adding a variable increases $R^2$ -- so using $R^2$ to determine a variables importance is a bit of a wild goose chase. Even when trying to understand simple situations you will end up with a completely absurd collection of variables.






            share|cite|improve this answer









            $endgroup$












            • $begingroup$
              Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
              $endgroup$
              – Richard Hardy
              Mar 7 at 21:38
















            1












            $begingroup$

            As others have alluded, seeing no change in $R^2$ when you add a variable to your regression is unusual. In finite samples, this should only happen when your new variable is a linear combination of variables already present. In this case, most standard regression routines simply exclude that variable from the regression, and your $R^2$ will remain unchanged because the model was effectively unchanged.



            As you notice, this does not mean the variable is unimportant, but rather that you are unable to distinguish its effect from that of the other variables in your model.



            More broadly however, I (and many here at Cross Validated) would caution against using R^2 for model selection and interpretation. What I've discussed above is how the $R^2$ could not change and the variable still be important. Worse yet, the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable. Broadly, using $R^2$ for model selection fell out of favor in the 70s, when it was dropped in favor of AIC (and its contemporaries). Today -- a typical statistician would recommend using cross validation (see the site name) for your model selection.



            In general, adding a variable increases $R^2$ -- so using $R^2$ to determine a variables importance is a bit of a wild goose chase. Even when trying to understand simple situations you will end up with a completely absurd collection of variables.






            share|cite|improve this answer









            $endgroup$












            • $begingroup$
              Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
              $endgroup$
              – Richard Hardy
              Mar 7 at 21:38














            1












            1








            1





            $begingroup$

            As others have alluded, seeing no change in $R^2$ when you add a variable to your regression is unusual. In finite samples, this should only happen when your new variable is a linear combination of variables already present. In this case, most standard regression routines simply exclude that variable from the regression, and your $R^2$ will remain unchanged because the model was effectively unchanged.



            As you notice, this does not mean the variable is unimportant, but rather that you are unable to distinguish its effect from that of the other variables in your model.



            More broadly however, I (and many here at Cross Validated) would caution against using R^2 for model selection and interpretation. What I've discussed above is how the $R^2$ could not change and the variable still be important. Worse yet, the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable. Broadly, using $R^2$ for model selection fell out of favor in the 70s, when it was dropped in favor of AIC (and its contemporaries). Today -- a typical statistician would recommend using cross validation (see the site name) for your model selection.



            In general, adding a variable increases $R^2$ -- so using $R^2$ to determine a variables importance is a bit of a wild goose chase. Even when trying to understand simple situations you will end up with a completely absurd collection of variables.






            share|cite|improve this answer









            $endgroup$



            As others have alluded, seeing no change in $R^2$ when you add a variable to your regression is unusual. In finite samples, this should only happen when your new variable is a linear combination of variables already present. In this case, most standard regression routines simply exclude that variable from the regression, and your $R^2$ will remain unchanged because the model was effectively unchanged.



            As you notice, this does not mean the variable is unimportant, but rather that you are unable to distinguish its effect from that of the other variables in your model.



            More broadly however, I (and many here at Cross Validated) would caution against using R^2 for model selection and interpretation. What I've discussed above is how the $R^2$ could not change and the variable still be important. Worse yet, the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable. Broadly, using $R^2$ for model selection fell out of favor in the 70s, when it was dropped in favor of AIC (and its contemporaries). Today -- a typical statistician would recommend using cross validation (see the site name) for your model selection.



            In general, adding a variable increases $R^2$ -- so using $R^2$ to determine a variables importance is a bit of a wild goose chase. Even when trying to understand simple situations you will end up with a completely absurd collection of variables.







            share|cite|improve this answer












            share|cite|improve this answer



            share|cite|improve this answer










            answered Mar 7 at 20:13









            user5957401user5957401

            29727




            29727











            • $begingroup$
              Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
              $endgroup$
              – Richard Hardy
              Mar 7 at 21:38

















            • $begingroup$
              Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
              $endgroup$
              – Richard Hardy
              Mar 7 at 21:38
















            $begingroup$
            Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
            $endgroup$
            – Richard Hardy
            Mar 7 at 21:38





            $begingroup$
            Could you elaborate on the $R^2$ could change somewhat (or even dramatically) when you include an irrelevant variable, specifically on the case of a dramatical change? In which sense would the variable then be irrelevant?
            $endgroup$
            – Richard Hardy
            Mar 7 at 21:38


















            draft saved

            draft discarded
















































            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396220%2fwhat-does-it-mean-when-i-add-a-new-variable-to-my-linear-model-and-the-r2-stays%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Identity Server 4 is not redirecting to Angular app after login2019 Community Moderator ElectionIdentity Server 4 and dockerIdentityserver implicit flow unauthorized_clientIdentityServer Hybrid Flow - Access Token is null after user successful loginIdentity Server to MVC client : Page Redirect After loginLogin with Steam OpenId(oidc-client-js)Identity Server 4+.NET Core 2.0 + IdentityIdentityServer4 post-login redirect not working in Edge browserCall to IdentityServer4 generates System.NullReferenceException: Object reference not set to an instance of an objectIdentityServer4 without HTTPS not workingHow to get Authorization code from identity server without login form

            2005 Ahvaz unrest Contents Background Causes Casualties Aftermath See also References Navigation menue"At Least 10 Are Killed by Bombs in Iran""Iran"Archived"Arab-Iranians in Iran to make April 15 'Day of Fury'"State of Mind, State of Order: Reactions to Ethnic Unrest in the Islamic Republic of Iran.10.1111/j.1754-9469.2008.00028.x"Iran hangs Arab separatists"Iran Overview from ArchivedConstitution of the Islamic Republic of Iran"Tehran puzzled by forged 'riots' letter""Iran and its minorities: Down in the second class""Iran: Handling Of Ahvaz Unrest Could End With Televised Confessions""Bombings Rock Iran Ahead of Election""Five die in Iran ethnic clashes""Iran: Need for restraint as anniversary of unrest in Khuzestan approaches"Archived"Iranian Sunni protesters killed in clashes with security forces"Archived

            Can't initialize raids on a new ASUS Prime B360M-A motherboard2019 Community Moderator ElectionSimilar to RAID config yet more like mirroring solution?Can't get motherboard serial numberWhy does the BIOS entry point start with a WBINVD instruction?UEFI performance Asus Maximus V Extreme