- An A/B testing tool runs a script in the head of your website and can change any element on the page. Its security review covers account access, evidence, script delivery, contract and operations.
- The 25 checks below are grouped by area. Each one names who in your company should review it: IT, data protection or procurement.
- Ask for evidence in writing. When a certification is in preparation, record that status together with the date you received the answer.
- On the Pro and Enterprise plans, Varify lets the account owner make MFA mandatory for all users, records every account change in an activity log, and completes security and privacy questionnaires on the Enterprise plan. SOC 2 and SSO (SAML) are in preparation.
Evaluating the security of an A/B testing tool means checking two things: who can change your website through the tool, and how the tool's script gets onto your pages. A testing tool sits in the head of every page it runs on. Its users can change content, layout and code for a share of your visitors. That makes it a tool IT security should review before it is bought.
This page gives you 25 checks to run before you buy, grouped into five areas, with a column that says who should review each check. The last section shows how Varify answers each check today, with a link to the source. The privacy side of the review has its own 8-point privacy evaluation checklist.
What an A/B testing tool can touch on your site
Before you check the vendor, check the tool's reach. Three things define the attack surface of a client-side A/B testing tool:
- The script in the head. The testing script loads on every page it is installed on, usually before the page renders, so that visitors do not see the original before the variation. Whoever controls that script controls what runs on those pages.
- Changes to the page (DOM). Variations change text, images, layout and links. Many tools also allow custom JavaScript and CSS per variation. A variation can therefore do anything a script on your page can do.
- Access to the editor. Users of the tool decide which variation goes live for which visitors. A compromised account, or an account with too many rights, can publish changes to your production website.
The 25 checks follow from these three points: who may log in and publish, what evidence the vendor provides, how the script is delivered, what the contract says, and how the service is operated.
The 25 checks
Copy the table into your evaluation. The column "Who reviews" names the team that should own the answer.
| # | Check | Who reviews |
|---|---|---|
| Account security | ||
| 1 | Multi-factor authentication (MFA) is available for every user and can be made mandatory for all users of the account. | IT |
| 2 | Single sign-on (SSO) through your identity provider is available. | IT |
| 3 | The login is protected against automated attacks. | IT |
| 4 | Roles and permissions can be defined per user and per workspace. | IT |
| 5 | Publishing an experiment can be limited to specific roles. | IT |
| 6 | An audit trail records who changed what, and when. | IT, Data protection |
| Evidence | ||
| 7 | SOC 2 report, or the current status of the SOC 2 process. | IT, Procurement |
| 8 | ISO 27001 certificate, or the vendor's statement on it. | IT, Procurement |
| 9 | Summary of the latest penetration test, or the vendor's statement on it. | IT |
| 10 | Your security questionnaire, answered in writing. | IT, Procurement |
| 11 | Technical and organizational measures (TOMs), documented in the DPA. | Data protection |
| Script and delivery | ||
| 12 | Content Security Policy (CSP) requirements are documented: directives and domains. | IT |
| 13 | Whether the script needs unsafe-eval, and how its scope can be limited. | IT |
| 14 | The script can be delivered from your own subdomain (first-party context). | IT |
| 15 | Which further scripts the tool loads at runtime, and from where. | IT |
| 16 | Which code a variation may contain, and who reviews custom JavaScript before it goes live. | IT |
| Data protection and contract | ||
| 17 | A data processing agreement (DPA) is available, including return and deletion of data at contract end. | Data protection, Procurement |
| 18 | All sub-processors are listed with their place of processing. | Data protection |
| 19 | The procedure for new sub-processors, including the objection deadline, is defined. | Data protection |
| 20 | Optional features that add sub-processors, for example AI features, are named, with how they are activated. | Data protection |
| Operations | ||
| 21 | Data is encrypted in transit. | IT |
| 22 | Stored data is encrypted (encryption at rest). | IT |
| 23 | Outages are monitored and published. | IT |
| 24 | Failover is tested regularly. | IT |
| 25 | Recovery targets (RPO, RTO) are defined. | IT |
Account security
Many risks with marketing tools start with an account: a shared password, a former agency login that still works, or a user with publishing rights who should only read reports. Four controls close these gaps:
- MFA, enforced. Optional MFA protects only the users who turn it on. Ask whether an administrator can make it mandatory for everyone in the account.
- SSO. With single sign-on, access ends when a user leaves your identity provider. Without it, offboarding depends on someone remembering the testing tool. If SSO is not available yet, document the offboarding step in your own process.
- Roles. Separate the people who build experiments from the people who publish them, and give agencies only the workspaces they work in.
- Audit trail. When a variation changes your checkout, you need to know who launched it and when. An audit trail also answers the question auditors ask about every system with write access to production.
Evidence: what to request and what counts
Security teams usually ask for three kinds of evidence: a SOC 2 report, an ISO 27001 certificate, and a penetration test summary. Each tells you something different:
- SOC 2 is an attestation report by an auditor on the vendor's controls. A Type II report covers a period of time; a Type I report covers a point in time. Ask for the report period and the scope.
- ISO 27001 certifies an information security management system. Ask for the certificate and the scope statement: which parts of the company and which services it covers.
- A penetration test shows how an external tester attacked the application at a given date. Ask for a summary and the date.
If a vendor has these in preparation and none completed, the review has to rely on other material: a completed security questionnaire, the TOMs in the DPA, the account security controls you can test yourself during a trial, and a public status page. Record the status the vendor states, the date of the statement, and the documents you received in its place.
Script and delivery
The script is where an A/B testing tool meets your own security controls. Three questions belong in every review:
- Content Security Policy. Which directives and domains does the tool need, and does it require
unsafe-eval? The page on Content Security Policy for A/B testing tools explains the directives and how to limit the risk. - First-party context. Can the script be delivered from a subdomain of your own website? The page on first-party context (CNAME) explains the setup.
- Runtime scripts. Which further scripts does the tool load after the main script, and from which domains? Every domain on that list belongs in your CSP and in your review.
Also agree internally who may publish a variation that contains custom JavaScript. That is a process question, but it decides how much of your website's code passes through the testing tool.
Data protection and contract
The data protection part of the security review rests on the data processing agreement. It lists the data categories, the TOMs and the sub-processors, and it defines what happens when a sub-processor is added. The page Varify sub-processors shows Varify's list from the DPA.
For the operations checks, ask for written answers to all five points: encryption in transit, encryption at rest, outage monitoring, failover tests, and recovery targets (RPO and RTO). Where the vendor has no public answer, the questionnaire is the place to get one.
How Varify answers each check today
The table lists only the checks for which Varify has a public source. Checks without a public source are left out; ask for them in your questionnaire.
| Check | Status | Source |
|---|---|---|
| MFA mandatory for all users (1) | Available on Pro and Enterprise | Docs, Plans |
| Login protection (3) | Available: Cloudflare Turnstile | Docs |
| Passkey login | Available since May 30, 2026 | Changelog |
| SSO (SAML) (2) | In preparation | Plans |
| Roles and permissions per user and workspace (4) | Enterprise: own roles with individual permissions per user and workspace | Plans |
| Audit trail (6) | Available: activity log; account history on Pro and Enterprise | Docs, Plans |
| SOC 2 (7) | In preparation | Plans |
| Security questionnaire (10) | Completed during procurement on Enterprise | Plans |
| TOMs (11) | Available, as part of the order processing contract | DPA (PDF) |
| First-party context, CNAME (14) | Available on Enterprise | Plans |
| Encryption in transit (21) | Available: encrypted connections such as SFTP or HTTPS | DPA |
| Outage monitoring (23) | Public service status page | status.varify.io |
In Varify's own words:
- "As the account owner, you can also make MFA mandatory for all users in the account."
- "The Varify login area is protected by Cloudflare Turnstile."
- "The activity log records all changes within your account - a complete audit trail. This allows you to see who made which change and when at any time."
- "Advanced roles & permission settings: Create your own roles and assign individual permissions per user and workspace."
- "Questionnaires on cyber security or data protection: We complete your security and privacy assessments during procurement."
- SOC 2 is in preparation. SSO (SAML), single sign-on through your identity provider such as Okta, Entra ID or Google Workspace, is in preparation.
Varify also provides the order processing contract including the appendix of the technical and organizational measures (TOMs) for download: DPA (PDF).
Run your security review with Varify
MFA you can enforce, an activity log for every change, and completed security questionnaires on the Enterprise plan.