Applitools migrationEarly access

Migrate a Playwright suite from Applitools Eyes to Delta-QA

A Playwright suite written for Applitools Eyes runs on Delta-QA after two changes: its import and its environment variables. Its eyes.open, check, close, BatchInfo, addBrowser and regions stay.

Install, and change the import

The Eyes API of @delta-qa/vision takes the names and the meaning of Applitools Eyes for Playwright. It needs Node.js 22 or later and @playwright/test 1.62 or later.

The Vision SDK is first available in early access: contact us to use it on your suite. Request access to Vision

Applitools	Delta-QA
`import { test } from "@applitools/eyes-playwright/fixture";`	`import { test } from "@delta-qa/vision/eyes/fixture";`
`import type { EyesFixture } from "@applitools/eyes-playwright/fixture";`	`import type { EyesFixture } from "@delta-qa/vision/eyes/fixture";`
`import { Eyes, Target, … } from "@applitools/eyes-playwright";`	`import { Eyes, Target, … } from "@delta-qa/vision/eyes";`

eyesConfig in playwright.config stays as it is. With the fixture, a test changes its import only:

// import { test } from "@applitools/eyes-playwright/fixture";
import { test } from "@delta-qa/vision/eyes/fixture";

test("checkout", async ({ page, eyes }) => {
  await page.goto("https://shop.example/cart");
  await eyes.check("Cart", { fully: true });
}); // opened before the test, closed by the fixture

Set the environment variables

Variable	Read for
`DELTAQA_API_KEY`	The API key of your Delta-QA workspace (`dqa_…`), with the “Can edit” role, created in Settings → Workspace → API keys. Store it in a secret of your CI.
`APPLITOOLS_API_KEY`	Read when `DELTAQA_API_KEY` is not set: keep the variable, give it a Delta-QA key.
`DELTAQA_SERVER_URL`	Your Delta-QA server, for an On-Premise installation; Delta-QA's cloud when not set. `APPLITOOLS_SERVER_URL` is never read.
`APPLITOOLS_BATCH_ID`	The version tested — the commit. The batch `id` given in the code comes first.
`APPLITOOLS_BRANCH`	The branch. `setBranchName` in the code comes first.
`APPLITOOLS_PARENT_BRANCH` `APPLITOOLS_BASELINE_BRANCH`	The base branch, whose reference is used while the branch has none. `APPLITOOLS_BASELINE_BRANCH` comes before `APPLITOOLS_PARENT_BRANCH`; the code comes first.

The _NAME forms of the branch variables (APPLITOOLS_BRANCH_NAME, APPLITOOLS_PARENT_BRANCH_NAME, APPLITOOLS_BASELINE_BRANCH_NAME) are read too. An empty variable counts as a variable not set. Without a key, opening a test fails and names the variable to set.

These Applitools variables are not read, and do not warn: APPLITOOLS_SERVER_URL, APPLITOOLS_BATCH_NAME, APPLITOOLS_DONT_CLOSE_BATCHES, APPLITOOLS_SHOW_LOGS.

Configure your CI

The SDK reads neither git nor the variables of your CI: the lines below set the branch, the base branch and the commit from them, in the three Applitools variables.

GitHub Actions

On a push, github.head_ref is empty: the branch is then github.ref_name. On a pull request, github.sha is the merge commit GitHub makes, not the commit pushed: the head commit comes first.

env:
  APPLITOOLS_BRANCH: ${{ github.head_ref || github.ref_name }}
  APPLITOOLS_PARENT_BRANCH: ${{ github.base_ref || github.event.repository.default_branch }}
  APPLITOOLS_BATCH_ID: ${{ github.event.pull_request.head.sha || github.sha }}

GitLab CI

In a merge request pipeline, CI_MERGE_REQUEST_TARGET_BRANCH_NAME is the target branch; in a branch pipeline, it is empty.

script:
  - export APPLITOOLS_BRANCH="${CI_MERGE_REQUEST_SOURCE_BRANCH_NAME:-$CI_COMMIT_REF_NAME}"
  - export APPLITOOLS_PARENT_BRANCH="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-$CI_DEFAULT_BRANCH}"
  - export APPLITOOLS_BATCH_ID="$CI_COMMIT_SHA"
  - npx playwright test

Bitbucket Pipelines

BITBUCKET_PR_DESTINATION_BRANCH is set in a pull request pipeline only.

script:
  - export APPLITOOLS_BRANCH="$BITBUCKET_BRANCH"
  - export APPLITOOLS_PARENT_BRANCH="$BITBUCKET_PR_DESTINATION_BRANCH"
  - export APPLITOOLS_BATCH_ID="$BITBUCKET_COMMIT"
  - npx playwright test

Azure Pipelines

The System.PullRequest variables are set on a pull request build only, as refs/heads/… for Azure Repos. Build.SourceBranchName keeps only the last segment of a branch (feature/cart gives cart), so the branch is read from the full ref.

- script: |
    branch="${SYSTEM_PULLREQUEST_SOURCEBRANCH:-$BUILD_SOURCEBRANCH}"
    base="${SYSTEM_PULLREQUEST_TARGETBRANCH:-}"
    export APPLITOOLS_BRANCH="${branch#refs/heads/}"
    export APPLITOOLS_PARENT_BRANCH="${base#refs/heads/}"
    export APPLITOOLS_BATCH_ID="${SYSTEM_PULLREQUEST_SOURCECOMMITID:-$BUILD_SOURCEVERSION}"
    npx playwright test

A base branch equal to the branch is not used: on the default branch, these lines give no base.

Close in the teardown, never in the test body

With the fixture, eyes is closed at the test's teardown. Without it, call eyes.close() in an afterEach, and runner.getAllTestResults() in an afterAll.

Never close in the body of the test: waiting for the verdict takes the time of the test — 30 seconds by default in Playwright Test — and a step that fails before close() leaves the test open. The teardown has its own timeout.

import { test } from "@playwright/test";
import { Eyes, Target, VisualGridRunner } from "@delta-qa/vision/eyes";

const runner = new VisualGridRunner();
let eyes: Eyes;

test.beforeEach(async ({ page }) => {
  eyes = new Eyes(runner);
  await eyes.open(page, "Shop", test.info().title);
});
test("checkout", async ({ page }) => {
  await page.goto("https://shop.example/cart");
  await eyes.check("Cart", Target.window().fully());
});
test.afterEach(async () => {
  await eyes.close(); // throws DiffsFoundError when a difference awaits a judgment
});
test.afterAll(async () => {
  await runner.getAllTestResults();
});

Every Applitools setting, and what it becomes

Every setting of Applitools Eyes is accepted: the suite compiles and runs. Each one is either applied by Delta-QA, or accepted without effect, with a warning. A few calls do not exist.

Applied

Setting	In Delta-QA
`appName`	The application of the test; with the test name, it identifies the test. In `eyesConfig`, `My App` when not set, as with Applitools' fixture.
`testName`	The name of the test, its key: a changed name is a new test. In `eyesConfig`, it can also be a function of the test; without it, the test's title path without its file (`describe > test`), as with Applitools' fixture.
`apiKey` `setApiKey`	The key, instead of `DELTAQA_API_KEY` or `APPLITOOLS_API_KEY` — a Delta-QA key. It never appears in a message, a log or a link.
`branchName`	The branch of the run; without it, `APPLITOOLS_BRANCH`, then `APPLITOOLS_BRANCH_NAME`.
`parentBranchName` `baselineBranchName`	The base branch, `baselineBranchName` before `parentBranchName`; without them, their variables, in the same order.
`batch`	Its `id` is the version tested — the commit; without it, `APPLITOOLS_BATCH_ID`; without either, one version per process (see what differs, below).
`saveNewTests`	Whether the pages of a new test become its reference; `true` when not set.
`viewportSize`	The size given to the page when the test opens.
`browsersInfo` `addBrowser(width, height)`	Each size is an environment, for Chrome or without a browser name. Its language, time zone, theme and screen density are read in the test's browser.
`fully` `fully(true)`	The whole page — what Delta-QA checks in any case.
`withName` `name`	The name of the check.
`Target.window`	The page of the test.
`Target.region` `CheckSettings.region` `checkRegion` `checkRegionBy` `checkRegionByElement` `checkElement` `checkElementBy`	The only part of the page compared: the first element a locator, a selector or an element handle designates, as at Applitools; the last region given wins. The regions to ignore it holds are left out.
`ignoreRegions` `ignoreRegion` `ignore` `ignores`	Each element a region designates — a locator, a selector, `{ selector, type }`, an element handle — is left out of the comparison: every element of a locator, as at Applitools. Also the form `{ region }` and the `ignoreRegions` key of the object form.
`eyesConfig.failTestsOnDiff`	When a verdict that is not `PASSED` fails, in the fixture: `'afterEach'` fails the test at its teardown; `'afterAll'` fails the worker once its tests are done; `false` fails nothing, the verdict still awaited. Without it, `'afterAll'` when the `CI` variable is set and `'afterEach'` otherwise, as the code of Applitools' fixture decides.
`eyesConfig.afterAll`	Called once a worker's tests are done, with the result of each.

The calls

Call	In Delta-QA
`open(page, appName, testName, viewportSize?)`	The test is identified by its application and its name: a changed name is a new test. The viewport size is given to the page.
`check(name?, Target.window())` `checkWindow(tag?)`	Checks the page as it is, in each environment. A check without a name is matched by the address of its page; two checks of the same name are accepted. It answers a `MatchResult` without a verdict, as Applitools' `check` does: the verdict comes at the close.
`close(throwErr = true)`	Waits for the verdict: `DiffsFoundError` on a difference to judge, `NewTestError` on a new test to judge (`saveNewTests(false)`), `TestFailedError` on a difference judged a bug or on a failure. The message gives the link of the version in Delta-QA.
`closeAsync()` `runner.getAllTestResults(throwErr = true)`	`closeAsync()` closes without waiting; the runner waits for every verdict, closes a test left open, and throws on one that did not pass — the error of a `closeAsync()` is kept for it.
`abort()` `abortIfNotClosed()` `abortAsync()`	Closes without waiting for the verdict, and fails nothing: the results are marked aborted, without a result; `null` when no test is open. Delta-QA still compares the runs, and the version shows their result.

Accepted without effect

A setting Delta-QA does not apply does not fail the test: the test runs without it. Its first use in a process emits a Node.js warning that names it, of code DELTAQA_IGNORED_SETTING; later uses say nothing more, and node --disable-warning=DELTAQA_IGNORED_SETTING silences them. An empty value — setX(process.env.VARIABLE) outside the CI — is a setting not given, and does not warn. The names below are the ones the warning gives.

Comparison settings

Match levels, regions by match level, floating regions and displacements tune Applitools' comparison. Delta-QA compares the page the way a human eye would, with no tolerance level to set: every difference is shown, and a person judges it in Delta-QA.

matchLevel
defaultMatchSettings
enablePatterns
ignoreDisplacements
ignoreCaret
useDom
sendDom
CheckSettings.matchLevel
CheckSettings.layout
CheckSettings.exact
CheckSettings.strict
CheckSettings.content
CheckSettings.ignoreColors
CheckSettings.dynamic
CheckSettings.enablePatterns
CheckSettings.ignoreDisplacements
CheckSettings.ignoreCaret
CheckSettings.ignoreMismatch
CheckSettings.useDom
CheckSettings.sendDom
CheckSettings.layoutRegion
CheckSettings.layoutRegions
CheckSettings.strictRegion
CheckSettings.strictRegions
CheckSettings.contentRegion
CheckSettings.contentRegions
CheckSettings.floating
CheckSettings.floatings
CheckSettings.floatingRegion
CheckSettings.floatingRegions
CheckSettings.dynamicRegion
CheckSettings.dynamicRegions

Screenshot settings

Stitching, scrolling, cropping, waiting and debug screenshots shape Applitools' screenshot. Delta-QA checks the whole page as the test left it when check is called: to wait for content or to load a lazy section, do it in the test, before the check.

hideCaret
hideScrollbars
forceFullPageScreenshot
stitchMode
stitchOverlap
scrollRootElement
cut
rotation
waitBeforeScreenshots
waitBeforeCapture
matchTimeout
saveDebugScreenshots
debugScreenshotsPath
debugScreenshotsPrefix
fully(false)
CheckSettings.fully(false)
CheckSettings.scrollRootElement
CheckSettings.stitchMode
CheckSettings.timeout
CheckSettings.waitBeforeCapture
CheckSettings.lazyLoad
CheckSettings.useSystemScreenshot

Browsers, devices and environments

Delta-QA renders in Chromium: the size given to addBrowser counts, not the browser named, so two browsers of one size are one environment. A device of Applitools' grid (DeviceName, IosDeviceName, emulation, multi-device) adds no environment. The language, time zone, theme and screen density are read in the test's browser, and environment labels change nothing. ClassicRunner and VisualGridRunner behave the same.

addBrowser(chrome-one-version-back)
addBrowser(chrome-two-versions-back)
addBrowser(firefox)
addBrowser(firefox-one-version-back)
addBrowser(firefox-two-versions-back)
addBrowser(safari)
addBrowser(safari-one-version-back)
addBrowser(safari-two-versions-back)
addBrowser(edgechromium)
addBrowser(edgechromium-one-version-back)
addBrowser(edgechromium-two-versions-back)
addBrowser(edgelegacy)
addBrowser(ie)
addBrowser(ie10)
addBrowser(device)
addDeviceEmulation
addMultiDeviceTarget
mobileOptions
captureStatusBar
layoutBreakpoints
CheckSettings.layoutBreakpoints
environmentName
envName
baselineEnvName
deviceInfo
hostApp
hostAppInfo
hostOS
hostOSInfo
scaleRatio
CheckSettings.densityMetrics
eyesConfig.type

Applitools' rendering grid

Rendering options, hooks, fetching, caches and concurrency of Applitools' grid have no counterpart: the page is checked as the test left it. A script you ran in a hook runs in the test, before the check.

visualGridOptions
CheckSettings.ufgOption
CheckSettings.ufgOptions
CheckSettings.visualGridOption
CheckSettings.visualGridOptions
CheckSettings.variationGroupId
CheckSettings.hook
CheckSettings.beforeRenderScreenshotHook
CheckSettings.webHook
disableBrowserFetching
CheckSettings.disableBrowserFetching
disableNMLUrlCache
autProxy
bypassCSP
skipRootHtmlResource
concurrentSessions
RunnerOptions.testConcurrency
RunnerOptions.fetchConcurrency
RunnerOptions.removeDuplicateTests

Batches, baselines and results

A version carries the commit: it has no name, no sequence, no properties, no notification and nothing to close. A reference changes only when a person judges a difference in Delta-QA; a test without a reference is “New”. Checks are matched by their name, else by the address of their page. isDisabled: a test meant to run without a check opens no Eyes.

BatchInfo.name
BatchInfo.sequenceName
BatchInfo.startedAt
BatchInfo.notifyOnCompletion
BatchInfo.properties
BatchInfo.buildId
dontCloseBatches
BatchClose
properties
removeSession
saveDiffs
saveFailedTests
ignoreBaseline
displayName
sessionType
CheckSettings.pageId
isDisabled

Git

The SDK reads no git: give the base branch by parentBranchName or baselineBranchName, and the commit by the batch id or APPLITOOLS_BATCH_ID.

compareWithParentBranch
ignoreGitMergeBase
gitMergeBaseTimestamp
latestCommitInfo

Server, connection and logs

They concern Applitools' server, its connection and its logs. Delta-QA's server is read from DELTAQA_SERVER_URL; the SDK writes no Applitools log and emits no Applitools event.

serverUrl
proxy
agentId
httpVersion
connectionTimeout
Eyes.on
Eyes.setLogHandler
Eyes.addSessionEventHandler
eyesConfig.logConfig
RunnerOptions.maskLog

Features outside Delta-QA

Delta-QA does not validate accessibility, does not read Figma and has no natural-language check; a Playwright page has no web view.

accessibilityValidation
CheckSettings.accessibilityRegion
CheckSettings.accessibilityRegions
figmaBaselines
figmaOptions
nlpOptions
Target.webview
CheckSettings.webview

Regions without an element

Delta-QA spots the elements that differ: a region given by coordinates ({ x, y, width, height }, { left, top, width, height }) has no equivalent: the page is compared without it — whole, for Target.region. The zone is the element, without padding; a regionId names a region in Applitools' dashboard. With Target.frame or Target.shadow, the page is compared whole, and a region then given by a selector fails the check with a TypeError: a zone is an element of the page itself.

CheckSettings.ignoreRegions(coordinates)
Target.region(coordinates)
CodedRegion.padding
CodedRegion.regionId
Target.frame
Target.shadow
CheckSettings.frame
CheckSettings.shadow

An unknown key of the object form of a configuration or of a check warns under its name (CheckSettings.<key> in a check).

Absent: what does not compile

These are not settings: they return data Delta-QA does not have, or check an image rather than a page. A suite that uses them does not compile, rather than receive an invented value.

Absent	Why
`eyes.locate` `eyes.extractText` `eyes.extractTextRegions`	They return regions or text that Applitools reads in its screenshot.
`eyes.checkImage` `Target.image` `Target.buffer` `Target.base64` `Target.path` `Target.url`	Delta-QA checks pages, not images.
`eyes.getLogger` `eyes.logger`	Applitools' log.
`eyes.run`	In Applitools' fixture, its natural-language check.

What differs from Applitools

Deliberate choices of Delta-QA, to know before the first run.

Situation	Delta-QA
Matching a check to its reference	By the check's name when each name is unique in the test; otherwise in order, by name, else by page address: a check inserted or removed does not shift the others.
A change accepted on a branch	No automatic merge of baselines from one branch into another. When the same difference appears on another branch, Delta-QA offers “Already intended on … — apply?”.
Accepting a difference	“Intended” applies to the branches you tick.
No `APPLITOOLS_BATCH_ID` nor batch `id`	One version per process: each Playwright Test worker is a process, so several workers make several versions — with the fixture too. Set `APPLITOOLS_BATCH_ID` to the commit of your CI to share one version.
A region to ignore that matches no element	Left out, and the check is sent without it, as Applitools does — and a Node.js warning of code `DELTAQA_IGNORED_REGION` names it, once per region and process (`node --disable-warning=DELTAQA_IGNORED_REGION` silences it). A region to check that matches nothing fails the check with `ZoneNotFoundError`, as at Applitools.
`runner.getAllTestResults()` and a test left open	The test is closed and its verdict given; Applitools aborts it.
Fixture: `eyesConfig` at the top of `playwright.config` and in a project	The project's replaces the top one, as Playwright Test gives an option; `test.use` adds its keys to the project's. Applitools' fixture merges both key by key.
Fixture: `type: 'classic'` with `browsersInfo`	Each size is an environment, as with `ClassicRunner` outside the fixture; Applitools' fixture drops the sizes. `type` warns.
Fixture: `expect(page).toHaveScreenshot()`	Playwright Test's own: `expect` is the one of `@playwright/test`. Applitools' fixture turns it into an Eyes check.
A test left without a check for 30 minutes	Delta-QA closes it; `close()` then reads its verdict.

The data you send

What the SDK sends — the pages your tests check and the files they use — serves only to compare them. Testing on production personal data is your responsibility: run the suite on test data.

The Vision SDK: Delta-QA checks in your Playwright tests

Applitools, Applitools Eyes and Eyes are trademarks of Applitools Ltd. Delta-QA is not affiliated with, endorsed or sponsored by Applitools.

Run your Eyes suite on Delta-QA

To use Vision, contact us: it is first available in early access. Tell us which tests you want checked, and we open your access.

Request access to Vision Create a free account