What Is AB Testing Of Notification?

At PushEngage, we constantly work to make the product feature rich. We understand the importance of improving the push notification effectiveness for your business and so we have now updated our AB testing feature.

This means you can do AB testing based on:

  • Notification content
  • Landing page URL
  • Large Image & Notification Image
  • Multi-Action¬†Button
  • Interaction Types

AB Test Notification settings

You can also put different UTM parameters for each of the notification. This way you can even track the performance using your own analytics tool like Google Analytics.

How AB Testing of Push Notification Works?

Using AB notification is very simple and straightforward. AB test feature is available under the New Notification section. In AB test notification, the number of subscribers is divided equally by the system. When you click on send, the half of your subscribers will receive Notification A the other half will receive Notification B. After a couple of hours, check the analytics to see which notification performed well. Based on the result, next time when you send the notification, follow the format of the winning notifications to get better results. See the below steps and screenshots for easy understanding of the feature.

Basic Content of Notification

It will have all the details which users will see.

  • Title Of the Notification
  • Message of the Notification
  • Notification Icon
  • Large Image
  • Landing page URL

Notification Settings

In AB Test Notification, you can test notification interaction type:

  • Keep open 20 seconds
  • Keep notification open until interaction

AB Test Notification Based on Notification Title

Keep all the other attributes similar and change only the title. Below is the sample:

AB Testing Notification based on Title

AB Test Notification Based on Notification Message

Keep everything same and change only the Notification Message. Below is the sample:

AB Testing Notification based on Message

AB Test Notification Based on Small Image

You can change the notification icon image and keep all the parameters same.

AB Testing Notification based on Icon Image

AB Test Notification Based on Large Image

In large image notification testing, you can test in two different ways:

Two Different Large Image:

AB Testing Notification based on different large image

One With Large Image and other without Large Image:

AB Testing Notification based on image and without image

AB Test Notification Based on Landing Page URL

If you want to test the landing page, you can put different URL for each notification. Then check which landing lead to a better response.

AB Test Notification Based on Multi-Action Button

In PushEngage, we offer multiple action button for each notification. Under AB Test, you can use 4 different Call To Action Button Text. This way you can see which Call To Action Button Text leads to higher click rates. Consider below example with 4 different types of notification: “Grab Now“, “Last Chance“, “Click Now“, “Check Offer

AB Testing Notification based on multi-action button

AB Test Notification Based on Notification Settings

Under Notification Duration, you have two options to choose to expire the notification:

  • Keep 20 Seconds
  • Keep open until interaction

You can test the above two settings with AB test notifications. This way you will know, whether to Expire the notification after 20 seconds of display or wait until a user clicks on it.

AB Test Notification Analytics

After you send the Notification, you can check the Analytics data to see which notification has performed well. Below is the analytics snapshot to get more clarity:

AB Testing Push Notifications Analytics Data

Benefits of using AB testing of notifications

Like any other marketing tool, you need to optimize your push notifications. Earlier, to understand what kind of notification will get a good response, you were sending notifications to all users and then analyzing all the notifications. To test each parameter, you need to send a notification to all and this will increase your testing time.

With AB testing of notifications, you can test each of the parameters with single sending. For e.g., if you want to see whether a user will like the large image notification or not, you will create Notification A with a large image and Notification B without a large image.

  • Saves time: With single notification, you can test customer’s response to specific attributes.
  • Can test multiple attributes: You can also test a customer’s response to change in multiple attributes.
  • Fair chance to each test notifications: System divides your customer base equally between Notification A & B, hence gives fair chance to test each notification
  • Analytics data to validate: Analytics data shows the report separately for each notification. You can easily compare and see which performed well.
  • Different UTM to track custom analytics tool: You can also use different UTM parameters for each notification. This way you can track the notification on any analytics tool.

Free trial for push notification