GemX allows you to create two types of experiments:
-
Template Testing: focuses on optimizing individual pages.
-
Multipage Testing: focuses on optimizing the entire sales funnel.
This article will walk you through how to create a Template Testing experiment step by step. To learn how to set up a Multipage Testing experiment, please refer to this article: How to Create a Multipage Testing Experiment in GemX.
Please make sure that the GemX: CRO & A/B Testing app is already installed on your Shopify store.
If not, follow this guide first: How to Install GemX on Your Shopify Store.
What is Template Testing in GemX?
Template Testing allows you to compare two versions of the same page: your original (Control) and a new one (Variant): to identify which version performs better.
It helps users optimize their website's performance by running experiments that test different variations of web pages, enhancing the user experience and boosting conversion rates. Consequently, it increases opportunities for conversions and drives revenue growth.
How to Create a Template Testing Experiment?
Template Testing in GemX lets you A/B test different templates (Control vs. Variant) to see which one performs better based on key metrics like conversion or revenue.
To create a template testing experiment in GemX, follow these steps:
Step 1: Access the GemX Dashboard
From your Shopify Admin, locate the GemX: CRO & A/B Testing under the Apps section. Click on Create new experiment.
Step 2: Choose Experiment Type
Select Create Template Experiment.
Step 3: Select the control template
The Control is your original or default template. It serves as the baseline for comparison against the experimental version (Variant).
Step 4: Select the variant template
The Variant is the alternative version you’ll test against the Control. This could involve a different design, layout, or messaging approach.
There are two scenarios:
-
If your Control is a Shopify template, you will have one option: Select template for variant, which allows you to choose an existing template as the Variant.
-
If your Control is a GemPages template, you have two options for creating the Variant:
-
Select an existing template: Choose another existing template to use as the Variant.
-
Create Variant based on Control: Create a duplicate of the Control template as the Variant.
Note: The option “Create Variant based on Control” is only available in GemPages version 7.
If you choose the “Create Variant based on Control” option, a new tab will open in the GemPages Editor where you can customize the duplicated template.
This duplicated template is used exclusively within GemX and will not appear in your GemPages page listing.
Note:
-
You can change, preview, and remove templates. However, removal is only allowed for the Variant template, not the Control.
-
You can view the specific Pages/Products/Collections/Articles(*) to test within the Template pair in the campaign by clicking the Pages/Products/Collections/Articles to test button:
Step 4: Configure Advanced Settings
1. Set winning metric
Choose the primary metric that’s most important to you for this test. It helps to identify which version wins.
2. Set device type
You can choose to target specific device types, including desktop, tablet, or mobile or All devices. This ensures that the variant templates are shown only to visitors on the selected devices. The default is All devices.
-
All devices
-
Desktop
-
Tablet
-
Mobile
3. Set visitor type
-
All Visitors: This option includes both new and returning visitors, allowing you to test the templates across your entire audience.
-
New Visitors: Target only first-time visitors to your website. This is useful if you want to see how your templates perform with visitors who have no prior experience with your site.
-
Returning Visitors: Focus on visitors who have previously visited your website. This can help you understand how familiar visitors react to the changes in the template compared to their past experiences.
4. Set traffic source
-
All Sources: Include all traffic sources in your A/B test. This option is ideal if you want to get a comprehensive view of how your templates perform across all channels.
-
Specific Source: Target traffic from specific channels to see how your templates perform with visitors coming from particular sources. The specific sources you can target include:
-
Referral: Visitors who arrive at your site through links from other websites.
-
Organic Social: Traffic from unpaid social media posts or shares.
-
Organic Search: Visitors who find your site through unpaid search engine results.
-
Email: Traffic generated from email campaigns.
-
Paid Social: Visitors coming from paid social media ads.
-
Paid Search: Traffic from paid search engine ads.
-
SMS: Visitors who access your site via links sent through SMS campaigns.
Note: These traffic sources are currently fixed and cannot be customized. If you need additional options, feel free to send us a Feature Request.
5. Set traffic split
You'll allocate the percentage of traffic that each variant will receive during the A/B test. This distribution determines how many visitors will see the origin template versus the variant template when they visit the associated page.
6. Set Market and Language
Show page only to visitors coming from selected markets and languages.
Step 5: Activate your experiment
You'll activate your experiment, making it live and operational based on the settings you've configured.
Once you click the start button, your experiment will go live. The live page will start functioning according to the settings you've established, such as the selected templates, traffic allocation, targeted devices, visitor types, and traffic sources.
Even after activating an experiment, you can still edit the traffic distribution — allowing you to adjust how visitors are split across versions in real time.
To modify other experiment settings (e.g., goals, device targeting, or content), you’ll need to pause the experiment first.
Typically, data becomes available for analysis after approximately 2 weeks of running the experiment or once there are around 2,000 visitors.
To learn how to read experiment data in GemX, please see this article: How to view and read data of an experiment in GemX.
Frequently asked questions
1. How do I know if my A/B test is running?
You can check by visiting the experiment link in two different browsers or opening it in Incognito mode. This allows you to simulate different user sessions.
2. Does GemX change my ad URL during A/B testing?
No. GemX keeps your original link intact. Visitors are redirected to the Control URL, and either the Control or Variant template is dynamically loaded in the background.
3. Do both templates need to be published?
Yes. Both the Control and Variant templates must be published for the test to work correctly. GemX uses Shopify's rendering system to display the templates, so unpublished ones cannot be shown.
4. Why do users keep seeing the same version?
GemX assigns each visitor a template version for consistency throughout their session. This means that a visitor will always see the same version to avoid confusion and maintain a consistent experience.
To test the other variant:
-
Use Incognito mode or another browser
-
Clear cookies/cache
-
Try on a different device
