From the conversion glossary
Concepts referenced in this article, defined.
A/B Testing SMS Campaigns
Start lifting conversions today.
Run rigorous A/B tests and personalize every visit on Shopify or any storefront โ no engineers required.
Concepts referenced in this article, defined.
Run rigorous A/B tests and personalize every visit on Shopify or any storefront โ no engineers required.
A/B testing SMS campaigns means sending two message variants to randomly split segments of your SMS subscriber list and measuring which generates more clicks, sessions, and purchases. SMS is the highest-open-rate channel available to ecommerce brands โ Indian D2C marketers consistently report 90%+ open rates โ but that reach is wasted without optimizing the message. Systematic SMS testing typically improves CTR by 15โ30% and campaign revenue by 10โ20% within 90 days.
SMS has a 95%+ open rate in India, where most shopping happens on mobile. Every subscriber receives your message. But SMS also has the lowest tolerance for irrelevance โ one bad message can trigger an opt-out, which is irreversible on most platforms.
The brands winning on SMS โ Boat, Sugar Cosmetics, Mamaearth โ are not sending random campaigns and hoping for the best. They test message framing, link placement, urgency language, and send times with the same discipline they apply to their paid ads.
The stakes are clear:
Step 1: Choose one test variable
SMS has fewer testable elements than email or push, but each one is high impact:
Step 2: Form a clear hypothesis
Example: "I believe leading with 'โน300 off' (absolute discount) will generate higher CTR than '30% off' (percentage discount) because absolute savings feel more concrete to price-conscious Indian buyers."
Step 3: Randomly split your list
Use a 50/50 split. Most Indian SMS platforms (Exotel, Gupshup, WebEngage, CleverTap) support random A/B splits. Ensure the split is randomized โ not split by last name, city, or signup date, all of which introduce bias.
Step 4: Send at the same time
Send both variants simultaneously. A 15-minute delay between sends is enough to introduce significant time-of-day bias in click patterns.
Step 5: Measure for 24โ48 hours
SMS clicks concentrate in the first 2 hours but continue for up to 24 hours. Wait the full 24 hours before comparing CTR.
Step 6: Apply learnings
Use the winning variant as your new control. Your next test builds from the winner, not from a fresh baseline.
The first 40 characters determine whether the recipient reads further or deletes. Test:
How you frame the same offer changes click rates dramatically:
For Indian D2C buyers, absolute โน discounts typically outperform percentage discounts for baskets under โน1,500. Above โน2,000, percentage discounts often perform better. Test for your specific AOV.
Standard SMS does not support clickable buttons, so the CTA is embedded in text. Test:
Shorter is not always better. Test:
Example short: "Mamaearth sale ends midnight. โน400 off. [link]" Example long: "Hey! Mamaearth's Diwali sale ends tonight at midnight. Get โน400 off your next order โ free COD on all orders. Claim now: [link]"
This is one of the highest-variance variables for Indian D2C SMS:
| Time Window | Best For |
|---|---|
| 9โ11 AM | Wellness, supplements, morning routines |
| 12โ1 PM | Fashion, accessories (lunch break browsing) |
| 6โ8 PM | All categories โ highest purchase intent |
| Saturday 10 AMโ12 PM | Weekend browsing โ strong for premium products |
Test your audience specifically. A beauty brand's subscribers may peak at different times than a fitness brand's.
"Hi [Name]," at the start of an SMS typically improves CTR by 8โ12% for Indian audiences. Test:
Because SMS lists tend to be smaller than email lists, reaching significance requires careful planning:
| List size (per variant) | Min detectable CTR difference |
|---|---|
| 500 | ~10 percentage points |
| 1,000 | ~7 percentage points |
| 2,500 | ~4 percentage points |
| 5,000+ | ~3 percentage points |
For most Indian D2C brands with 5,000โ20,000 SMS subscribers, run 50/50 tests and aim for 95% confidence before calling a winner.
TRAI regulations require explicit DND checking, sender ID registration (e.g., "KPVSMS" for Kapiva), and content template pre-approval for promotional SMS. Ensure both variants of your A/B test use pre-approved templates โ unapproved templates will be blocked before delivery, invalidating your test.
Use only transactional SMS pathways for order updates and promotional pathways for offers. Mixing these pathways violates TRAI guidelines and can result in sender ID suspension.
Testing without a registered template. In India, promotional SMS requires pre-approved templates. Testing an unregistered template variant wastes money when it fails to deliver.
Declaring a winner after 2 hours. SMS click rates spike in the first 90 minutes and then continue slowly. Declare winners after 24 hours.
Ignoring opt-out rate. A message variant with higher CTR but 3x the opt-out rate is a net loss. Track both.
Not tracking to purchase. Click-through rate is a proxy โ revenue per SMS is the real metric. Use UTM parameters on your links so you can attribute purchases back to each variant.
Sending every day. SMS frequency directly impacts opt-out rate. Test frequency as a separate variable (2x per week vs. 3x per week) before scaling winning messages.
Use UTM parameters on every link. Without UTMs, you can measure clicks but not purchases. Tag every variant link with utm_campaign=sms_test_a and utm_campaign=sms_test_b to track revenue in Google Analytics or your Shopify dashboard.
Test festive messages before the festival. During Diwali or Holi, you want to deploy your best-performing message format from day one. Run festive format tests 2โ3 weeks in advance.
Build a brand vocabulary. After 20+ SMS tests, you will know which words drive clicks for your audience. Common high-performers in Indian D2C: "free COD," "midnight," "only for you," product category specifics ("gut health," "hair fall," "brightening").
Segment before testing. Buyers vs. non-buyers, high-AOV vs. low-AOV, and recent purchasers vs. lapsed customers all respond to different messaging. Test within segments rather than across your whole list.
Related reading: