Sample size for A/B testing done to measure difference in NPS











up vote
0
down vote

favorite












I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.



Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?










share|improve this question









New contributor




ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
























    up vote
    0
    down vote

    favorite












    I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.



    Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?










    share|improve this question









    New contributor




    ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.






















      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.



      Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?










      share|improve this question









      New contributor




      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.



      Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?







      statistics sampling ab-testing






      share|improve this question









      New contributor




      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question









      New contributor




      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question








      edited Nov 18 at 1:10









      Nick

      19.1k51433




      19.1k51433






      New contributor




      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked Nov 18 at 1:02









      ChronoZero

      1




      1




      New contributor




      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





























          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });






          ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.










           

          draft saved


          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53357002%2fsample-size-for-a-b-testing-done-to-measure-difference-in-nps%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown






























          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.










           

          draft saved


          draft discarded


















          ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.













          ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.












          ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.















           


          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53357002%2fsample-size-for-a-b-testing-done-to-measure-difference-in-nps%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Create new schema in PostgreSQL using DBeaver

          Deepest pit of an array with Javascript: test on Codility

          Fotorealismo