Visual testing is an automated verification method that compares screenshots of a web interface between two versions to detect graphical regressions (a moved element, a modified color, a component overflowing its container).
We use GitLab every day at Delta-QA. Our repos, our pipelines, our merge requests, our CI/CD: everything runs on GitLab. It is not a default choice: it is a deliberate choice for a platform that offers a natively integrated pipeline, a container registry, and a DevOps philosophy that is coherent from end to end.
When we talk about visual testing in GitLab CI, we are not regurgitating documentation we read the day before. We are sharing what we learned by actually using it. This article covers the available approaches, the GitLab CI specifics that affect visual testing, and how to get a reliable visual regression pipeline.
The essentials on one page (definition, no-code automation, tool selection grid): Automated regression testing, no-code.
Not ready to touch your
.gitlab-ci.ymlyet? Get started with visual testing outside the pipeline: Delta-QA compares your screens with no-code, with a free account and no credit card. Try Delta-QA for free →
Why GitLab CI Lends Itself Well to Visual Testing
GitLab CI has characteristics that make it particularly suited to visual testing, characteristics that GitHub Actions or Jenkins users do not have by default.
Native Artifacts
GitLab CI natively manages pipeline artifacts. When a visual test produces HTML reports, diff images, or screenshots, you declare them as artifacts and they are accessible directly from the merge request interface. No external service is needed to store and view the results.
It is an underrated advantage. With GitHub Actions, you have to configure a separate artifact upload or use a third-party service to make the results accessible. With GitLab, it is native.
Review Environments
GitLab offers Review Apps: ephemeral environments deployed automatically for each merge request. Your application runs in a dedicated environment, accessible through a temporary URL. It is the ideal environment for visual testing: stable, isolated, and representative of production.
Self-Managed Runners
GitLab lets you use runners hosted on your own infrastructure. For visual testing, that is crucial: you control the rendering environment (OS, fonts, GPU), which reduces false positives caused by environment differences. We use dedicated runners with a frozen configuration to guarantee capture reproducibility.
The Built-In Container Registry
Every GitLab project has a container registry. You can store a Docker image preconfigured for visual testing there (with the right fonts, the right browser, the right dependencies) and use it as the base of your test jobs. That eliminates a major source of instability.
Approaches to Visual Testing in GitLab CI
Playwright in a GitLab CI Job
Playwright is the most robust open source tool for visual testing in CI. Its native toHaveScreenshot() method handles capture, comparison, and automatic retries.
Integration with GitLab CI. You create a job in your .gitlab-ci.yml that uses Playwright's official Docker image, runs your tests, and publishes the results as artifacts. Playwright's HTML reports can be viewed directly in GitLab.
For the exact structure of the YAML file, ask your AI copilot (it is unbeatable on GitLab CI syntax, about the only thing it does not hallucinate on, yet). More seriously, Playwright's official documentation for CI environments is your best reference.
What works well. Playwright's official Docker image includes all the browsers and the necessary system dependencies. Combined with the GitLab container registry, you can create a derived image with your fonts and specific configurations. GitLab artifacts make test reports accessible with no extra configuration.
Limits in the GitLab context. Baselines must be generated in the CI environment, not locally. It is a universal rule of visual testing, but GitLab makes things easier: you can trigger a pipeline manually to regenerate the baselines. Managing baselines in Git remains a challenge (binary files, merge conflicts), but GitLab LFS is natively integrated.
Percy With GitLab CI
Percy (BrowserStack) offers an official integration with GitLab CI. The Percy SDK captures snapshots during your pipeline and sends them to the Percy cloud for comparison and review.
What works. Percy automatically detects the GitLab branch and merge request. Results appear as an external status on your MR. Cross-browser is handled on the cloud side.
Limits. Per-snapshot pricing remains an issue. And you add an external dependency to your pipeline: if Percy is unavailable, your check stays pending. For a team that chose GitLab precisely to control its infrastructure, this external dependency can contradict the philosophy.
BackstopJS in GitLab CI
BackstopJS works in GitLab CI through its official Docker image. HTML reports are perfectly suited to GitLab's artifact system.
What works. BackstopJS's HTML report is one of the most visual and readable in the ecosystem. Published as a GitLab artifact, it can be viewed directly in the browser from the MR interface. JSON scenario configuration is explicit and versionable.
Limits. The project has gone through periods of intermittent maintenance (check recent activity before committing). Configuration can become verbose for complex applications, and you must manually manage iteration over your pages or components.
Delta-QA in GitLab CI: The Vision SDK
Delta-QA enters a GitLab CI pipeline through your Playwright tests, with the Vision SDK.
How it works. You store the API key in a masked CI/CD variable (DELTAQA_API_KEY) and add vision.check() to the pages your tests must verify. The job runs npx playwright test as usual. For each check, Delta-QA compares the page with its reference as a human eye would: it spots the elements that changed and names them in the report.
What is different. No baselines to store in your repo: references live in your Delta-QA space, one per branch; the SDK reads neither git nor the CI: a line of your job gives it the branch, the base branch and the commit. An unreviewed difference fails the job, and its log gives the link to the report, which says which element changed; there you judge "Intended" or "Bug". The tests that give the same commit form one version, even when spread across several parallel jobs.
And without code. Journeys recorded in Delta-QA by the QA team are judged in the same space. The SDK covers Playwright in TypeScript and JavaScript, on Chromium; it is first offered in early access, with access requested from its page.
Before writing a single CI job, measure what visual testing detects on your pages. Delta-QA captures and compares your screens in a few clicks, with a free account and no credit card. Try Delta-QA for free →
GitLab CI Specifics to Know for Visual Testing
Cache vs Artifacts
It is a distinction many people confuse. The GitLab CI cache is shared between the pipelines of the same project (it speeds up jobs: npm dependencies, Playwright browsers). Artifacts are the outputs of a specific job: test reports, screenshots, diffs.
For visual testing, use the cache for browsers and dependencies, and artifacts for test results. Never cache the baselines (they must live in the repo to be versioned with the code).
Stages and Dependencies Between Jobs
GitLab CI organizes jobs into sequential stages. The visual test must be in a stage that runs after the deployment of your Review App. A common pattern:
- build: compiling the application
- deploy-review: deploying the Review App
- test-visual: visual test on the Review App
- cleanup: deleting the environment
GitLab CI's needs directive lets you create explicit dependencies between jobs without going through sequential stages, which can speed up your pipeline.
Protected Environment Variables
API tokens for cloud services (Percy, Chromatic) must be stored as CI/CD variables in GitLab. Watch out: protected variables are only available on protected branches. If your feature branches are not protected, visual testing with a cloud service will fail silently, a classic trap.
Time and Memory Limits
Visual testing is resource-hungry. Rendering pages in a headless browser consumes memory, and image comparison takes time. GitLab.com's shared runners have time limits (generally one hour per job) and memory limits. For substantial visual test suites, self-managed runners with dedicated resources are recommended.
Building a Robust Visual Testing Pipeline
Here is the approach we recommend, based on our own experience.
Start Small and Targeted
Do not visually test all your pages on day one. Identify the five to ten most critical pages (the ones your users see first, the ones where a visual regression has the most business impact). Then expand gradually.
Use Review Apps as the Target
Rather than starting a local server in your CI job, deploy a Review App and test it. The advantage: the environment is stable, accessible, and representative. The drawback: you add the deployment duration to the pipeline. The trade-off is worth it.
Stabilize Your Rendering Environment
Create a Docker image dedicated to visual testing, stored in your GitLab registry. Include the fonts used by your application, the exact browser version, and all system dependencies. Version this image. When your rendering environment is frozen, false positives drop drastically.
Start in Non-Blocking Mode
Configure your visual testing job with allow_failure: true for the first few weeks. This lets your team get familiar with the results without visual testing blocking merge requests. Once confidence is established and false positives are under control, switch to blocking mode.
Notify Intelligently
GitLab CI can send notifications on different channels (Slack, email, webhook). Configure visual testing notifications to alert only on failures, not on successes. The goal is to draw attention when needed, not to drown your team in messages.
Using Artifacts as a Reference System
GitLab artifacts are not only for publishing reports: well organized, they become the backbone of your reference strategy. Two practices make the difference.
Structure Artifacts Readably
Do not pile all your screenshots flat. Create a clear tree: one folder per tested page, containing the reference screenshot, the current screenshot, and the diff. Place an HTML report at the root to navigate between results. When a test fails, the reviewer opens GitLab's built-in artifact browser, navigates to the page concerned, and immediately sees the difference without leaving the interface.
Use main Artifacts as References
Rather than storing baselines in the repo, you can use the artifacts of the last successful pipeline on the main branch as the reference set. GitLab CI lets you download the artifacts of a specific job on a specific branch through its API.
Concretely, your feature branch's visual test job starts by downloading the reference screenshots from the artifacts of the latest main pipeline. It then captures the feature branch screenshots, compares the two sets, and stores its own results as artifacts of the current pipeline. No binary baseline pollutes your Git history.
A Two-Tier Retention Policy
The retention period of GitLab artifacts is configurable, and for visual testing a two-tier policy works well:
- Main branch: 30 days (or more). These artifacts serve as stable references for all comparisons.
- Feature branches: 7 days. Long enough for the merge request to be handled and merged.
Adjust to your rhythm: a team with long cycles (branches that live for several weeks) will need longer retention on main, and will have to rebase regularly or re-download the latest references on each run to avoid comparing against stale baselines.
FAQ
Is GitLab CI as good as GitHub Actions for visual testing?
For visual testing specifically, GitLab CI has native advantages: built-in artifacts, Review Apps, the container registry, and self-managed runners. These features make it easier to set up a stable, reproducible visual testing environment. GitHub Actions has its own rich marketplace, but some of these features require additional configuration.
Do you need self-managed runners for visual testing in GitLab CI?
It is not mandatory, but it is strongly recommended. GitLab.com's shared runners work, but their variable hardware configuration can introduce rendering differences. A self-managed runner with a frozen configuration (fonts, browser, resolution) eliminates this source of false positives and generally offers better performance.
How do you manage visual test baselines in a GitLab repo?
If you use Playwright or BackstopJS, baselines (PNG files) live in your repo. Enable GitLab LFS to prevent binary files from bloating your Git history. For merge conflicts on baselines, the simplest strategy is to accept the source branch's new baselines and regenerate if needed. With Delta-QA, baselines are managed by the tool and do not pollute your repo.
Can you use GitLab's Review Apps as a visual testing environment?
Yes, and it is even the approach we recommend. The Review App provides a stable, isolated environment for each merge request. Your visual testing job waits for the Review App to be deployed, captures the screenshots on the temporary URL, and compares them with the references. It is more reliable than a server started on the fly in the CI job.
Does visual testing in GitLab CI work with monorepos?
Yes. GitLab CI supports conditional rules that let you trigger visual testing only if front-end files were modified. Combined with the only:changes directive, you avoid running pointless visual tests when only the backend changed. It is essential to keep a pipeline fast in a monorepo.
What is the best Docker image for visual testing in GitLab CI?
Playwright's official image (mcr.microsoft.com/playwright) is an excellent starting point. It includes the browsers and the system dependencies. For an even more stable environment, create a derived image that adds your application's fonts and pins exact versions. Store it in your GitLab project's container registry for fast access.
Conclusion
GitLab CI is a platform naturally suited to visual testing. Its native features (artifacts, Review Apps, container registry, self-managed runners) solve problems that other CI platforms leave you to handle manually.
But the platform does not do everything. Visual testing in CI remains an engineering challenge: stabilizing rendering, managing baselines, handling false positives, and building a review workflow that works for the whole team.
If you are on GitLab and want to add visual testing to your pipeline without the complexity of test scripts and manual baseline management, Delta-QA is designed to fit natively into your GitLab CI workflow.
Ready to make your merge requests more reliable with visual testing? Launch your first comparison with Delta-QA, free and with no credit card, before even touching your
.gitlab-ci.yml. Try Delta-QA for free →
