# Continuous E2E testing on GitHub Actions: run the suite all day at a flat price

> Run your end-to-end suite on every push and on a schedule all day, so regressions from agent commits and flaky tests show up within minutes. On GitHub's 4-core runner a runner busy all month is $500.40; on Skipbox S it is a flat $39, with the browsers kept on your own machine between runs.

Coding agents push code around the clock, and your end-to-end suite is the check that sees what their unit tests don't: whether sign-in, checkout and the pages around a change still work together. A team that runs E2E once a night finds a regression merged in the morning only after the next night's run, with a day of merges on top of it. Running the suite all day fixes that, and on per-minute runners it is the most expensive thing in CI: one 4 vCPU runner busy all month is $500.40 on GitHub.

Skipbox runs the loop for a flat $39 a month with unlimited minutes, on your own machine that keeps the browsers and packages installed between runs. Moving is one line, `runs-on: skipbox`.

- A 4 vCPU runner busy all month: $500.40 on GitHub, $39 on Skipbox
- The same bill at a run every 15 minutes or every 5, so the loop can run as often as it is useful
- Browsers, npm's cache and Docker images stay on your machine, so 96 runs a day don't reinstall them 96 times

**GitHub Actions at a flat price, unlimited minutes** Skipbox runs your GitHub Actions jobs on your own dedicated machine: $39 a month, however many minutes you run. Docker layers and dependency caches stay between jobs, so the next job starts right away.

A 4 vCPU runner, 20,000 minutes a month: GitHub $222 → Skipbox $39. You save $183 every month.

Moving takes one line: `runs-on: skipbox`.

[Start my 7-day free trial](https://app.skipbox.dev/sign-in?ref=g-continuous-e2e-testing-github-actions-en)

## Run it on every push, and on a loop

Two triggers in one workflow:

- **On every push to a pull request**, so a regression is caught before it merges.
- **On a schedule against main or staging**, every 15 minutes, so flaky tests, regressions that slipped through, and breakage from outside your code (a payment sandbox, an auth provider, a browser update) show up within minutes.

GitHub's shortest schedule is every 5 minutes, on the default branch's latest commit. A concurrency group keeps the loop from stacking up, and Playwright's retries mark tests that fail and then pass as flaky, so a test that flakes once in fifty runs shows up twice a day instead of hiding for a week.

## What the loop costs on per-minute runners

A loop every 15 minutes is 96 runs a day. With a few shards, the runner is busy most of the day, every day:

| A month of E2E on 4 vCPU | GitHub | Skipbox S | You save each month |
|---|---|---|---|
| 10,000 minutes | $102 | $39 | **$63** |
| 20,000 minutes | $222 | $39 | **$183** |
| 30,000 minutes | $342 | $39 | **$303** |
| Busy all month | $500.40 | $39 | **$461.40** |

GitHub's figures are after GitHub Team's included usage. Four shards on the loop means four rows like the last one. Try your own minutes in the [calculator on the home page](https://skipbox.dev/?ref=g-continuous-e2e-testing-github-actions-en#cost).

That bill is why most teams don't run E2E all day, and they pay for it in hours: a regression an agent merged at night is found in the morning, after more commits landed on it, and bisecting it takes the time the loop would have saved. Agent-started pull requests keep rising (GitHub reported more than 40% in September 2026, up from 17% in March), and a budget that caps the per-minute bill stops the jobs mid-month, which turns the loop off exactly when it is needed.

## Running the loop yourself

To keep it on GitHub-hosted runners and affordable, teams end up doing this:

- Thinning the schedule to hourly or nightly and accepting slower detection
- Tuning which tests run on which trigger so the minutes fit the budget
- Sharding to finish faster, then paying each shard's setup again
- Or running their own self-hosted runner for the loop, and keeping its browsers, OS and disk healthy

Skipbox removes the trade-off. The loop costs the same however often it runs, and nobody looks after the machine.

## What Skipbox gives the loop

**A flat price for all-day runs.** Each plan runs its jobs around the clock with unlimited minutes. Skipbox jobs use no GitHub minutes, so a GitHub budget never turns the loop off.

| Plan | Machine | Price |
|---|---|---|
| S | 4 vCPU · 16 GB | $39 a month |
| M | 8 vCPU · 16 GB | $79 a month |
| L | 16 vCPU · 32 GB | $149 a month |

Your plans' vCPU set how many shards run at once: four `runs-on: skipbox` shards run together on four S plans or one L. Shards past that wait their turn.

**Setup that doesn't repeat.** Each plan comes with your account's own machine, kept between jobs. Playwright's browsers in `~/.cache/ms-playwright`, npm's cache, and the Docker images your app and its services use are on the disk for the next run.

**Each run starts clean.** The runner's credentials, processes and every container are wiped after each job, and a health check runs. A machine that fails it is rebuilt from a clean image. From 13 November 2026, your checkout and its build output (Cargo's `target/`) stay on your own machine for your next jobs; Clean every job gives each job a fresh machine.

## Moving is one line

```yaml
name: E2E
on:
  pull_request:
  schedule:
    - cron: "*/15 * * * *"
concurrency:
  group: e2e-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
  e2e:
    runs-on: skipbox # was ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: lts/*
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
```

1. Sign in to Skipbox with GitHub.
2. Install the Skipbox GitHub App and pick the repositories.
3. Change `runs-on` and push.

## Security

- The machine is your account's alone. No other company's jobs run on it.
- After every job, the runner's credentials, processes and containers are wiped. From 13 November 2026, your checkout and its build output (Cargo's `target/`) stay on your own machine for your next jobs; Clean every job gives each job a fresh machine.
- Code from people without write access (such as pull requests from forks) runs on a fresh, throwaway machine.

The details are in [how jobs are isolated](https://docs.skipbox.dev/isolation).

## Try it

The trial is free for 7 days or 3,000 job-minutes, whichever comes first, with no card. It runs one `runs-on: skipbox` job at a time: move the E2E job, turn on the schedule, and watch a day of runs on a flat price.

## FAQ

### Should I run end-to-end tests on every push?

Yes, when agents write a large share of your commits. A unit test checks the function that changed; an end-to-end test checks that sign-in, checkout and the pages around them still work together. On a flat-price runner like Skipbox, running the suite on every push and on a schedule costs the same as running it once a night.

### How often can a GitHub Actions workflow run on a schedule?

At most every 5 minutes. Scheduled workflows run on the latest commit of the default branch and can start late when GitHub is busy. Every 15 minutes is a common loop: 96 runs a day. On Skipbox the bill is the same at every 15 minutes as at every 5.

### What does running E2E tests all day cost on GitHub Actions?

One 4 vCPU runner busy for the whole month is $500.40 on GitHub's 4-core runner, after GitHub Team's included usage. On Skipbox, plan S runs one 4 vCPU job at a time around the clock for a flat $39 a month, $461.40 less.

### How do I run Playwright shards at the same time on Skipbox?

Your plans' vCPU set how many jobs run at once. Four runs-on: skipbox shards of 4 vCPU run together on four S plans or one L plan; with fewer, the extra shards wait their turn and start as soon as one finishes.

### How do I find flaky end-to-end tests?

Turn on retries in CI. Playwright marks a test that fails and then passes on a retry as flaky, separate from passed and failed, and the report lists them. A suite on a loop shows the same flaky test many times a day, so it stands out in a day instead of a sprint.

Updated 2026-11-11 · https://skipbox.dev/guides/continuous-e2e-testing-github-actions