Sample size for A/B testing done to measure difference in NPS
up vote
0
down vote
favorite
I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.
Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?
statistics sampling ab-testing
edited Nov 18 at 1:10
Nick
19.1k51433
asked Nov 18 at 1:02
ChronoZero
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
0
down vote
favorite
I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.
Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?
statistics sampling ab-testing
edited Nov 18 at 1:10
Nick
19.1k51433
asked Nov 18 at 1:02
ChronoZero
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.
Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?
statistics sampling ab-testing
edited Nov 18 at 1:10
Nick
19.1k51433
asked Nov 18 at 1:02
ChronoZero
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I'm doing an A/B testing where we're sending an extra SMS with information about a certain process to our customers, while the control group isn't receiving the SMS. This SMS is after they purchase, so i'm not trying to measure a difference in conversion, but rather in "loyalty". Because we already do an NPS survey, I figured that a good way to measure the impact of the SMS would be to compare the NPS of both groups.
Here comes the question: How do I calculate the sample size for this experiment? I know that when you work with conversion rates you need a big sample size, because if your conversion is say 5%, you need to at least survey 1000 people just to get 50 that will convert. But with NPS it seems to me that I don't have that problem, but I'm very confused in how to proceed to estimate the sample size. Any tips?
statistics sampling ab-testing
statistics sampling ab-testing
edited Nov 18 at 1:10
Nick
19.1k51433
asked Nov 18 at 1:02
ChronoZero
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Nov 18 at 1:10
Nick
19.1k51433
asked Nov 18 at 1:02
ChronoZero
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Nov 18 at 1:10
Nick
19.1k51433
edited Nov 18 at 1:10
Nick
19.1k51433
edited Nov 18 at 1:10
Nick
19.1k51433
19.1k51433
asked Nov 18 at 1:02
ChronoZero
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 18 at 1:02
ChronoZero
1
asked Nov 18 at 1:02
ChronoZero
1
1
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
ChronoZero is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.
ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.
ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.
ChronoZero is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53357002%2fsample-size-for-a-b-testing-done-to-measure-difference-in-nps%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
