Flutter logo

How to execute Flutter integration tests in GitHub Action ?

published at Apr 17, 2022
#flutter#github action

When you write Flutter integration tests, you have different options to run them on Github Action. You can use a third-party service like the Firebase Test Lab or you can run these tests on emulators (iOS or Android) that you set up. In my last personal project, I chose this last option and I searched a lot on the internet before finding a solution. So here is an article that demonstrates what I learned and what I did to make it work.

Prepare your environment

To follow this tutorial, you must have Flutter installed. If so, please run flutter doctor to verify that your environment is set up correctly.

flutter doctor screenshot

Obviously, to run the tests on Github Action, you need a Github account.

Create your project

Create a Flutter project by using the skeleton template. It scaffolds an application with a List View and a Detail View:

flutter create flutter_project --template=skeleton
cd flutter_project

Create an empty repository on Github:

create project on Github

Then, push the project's source code to Github:

git init
git checkout -b main
git add .
git commit -m "create flutter project"
git remote add origin git@github.com:<username>/<project_name>.git
git push -u origin main

Write some integration tests

We need the integration_test and flutter_test packages to write integration tests. Add these dependencies to the dev_dependencies section of the app’s pubspec.yaml file:

# pubspec.yaml

dev_dependencies:
  integration_test:
    sdk: flutter
  flutter_test:
    sdk: flutter

Then install the dependencies:

flutter pub get

Create a new directory, integration_test, with an empty app_test.dart file. Now we can write some tests:

// integration_test/app_test.dart

import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';

import 'package:flutter_project/main.dart' as app;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  group('end-to-end test', () {
    testWidgets('go to the list and detail views', (WidgetTester tester) async {
      app.main();
      await tester.pumpAndSettle();

      expect(find.text('Sample Items'), findsOneWidget);
      expect(find.textContaining('SampleItem'), findsNWidgets(3));

      final Finder sampleItem = find.text('SampleItem 1');

      await tester.tap(sampleItem);

      await tester.pumpAndSettle();

      expect(find.text('Item Details'), findsOneWidget);
    });
  });
}

Run these tests to verify that they pass:

flutter test integration_test/

integration tests results

Create a Github workflow

Now, we need to configure our CI on Github Action. For this, we will define a workflow, composed of 2 different jobs: one responsible for running tests on iOS devices and the second for running them on Android devices.

Create a file named main.yml in the .github/workflowsdirectory:

# .github/workflows/main.yml

# Name of your workflow.
name: main

# Trigger the workflow manually or after push a commit
on: [push, workflow_dispatch]

# 2 jobs are configured.
# The first one runs tests on iOS devices.
# The second runs tests on Android devices
jobs:

  # job responsible for running Flutter tests on iOS devices
  ios:
    # Creates a build matrix for your jobs. You can define different variations of an environment to run each job
    strategy:
      matrix:
        device:
          # The available simulators are listed by the "xcrun xctrace list devices" command
          - "iPhone 11 Simulator (15.2)" # the name of the simulator could be different depending on the macos version you are using
      # if one of the jobs in the matrix expansion fails, the rest of the jobs will be cancelled
      fail-fast: true
    runs-on: macos-11 # or macos-latest if you prefer, but be aware that the available simulators could be different if you run a different version
    steps:
      - name: "List all simulators"
        run: "xcrun xctrace list devices"
      - name: "Start Simulator"
        # the command "xcrun simctl boot" expects a device identifier
        # the assignment of the UDID variable consists of retrieving the ID of the simulator
        # by extracting it from the command "xcrun xctrace list devices"
        run: |
          UDID=$(xcrun xctrace list devices | grep "^${{ matrix.device }}" | awk '{gsub(/[()]/,""); print $NF}')
          echo $UDID
          xcrun simctl boot "${UDID:?No Simulator with this name found}"
      - uses: actions/checkout@v3
      - name: Setup Flutter SDK
        uses: subosito/flutter-action@v2
        with:
          channel: stable
          # instead of "channel: stable", you could be more precise by specifying the exact version of Flutter you're using:
          # flutter-version: '<FLUTTER_VERSION>'
      - name: Install Flutter dependencies
        run: flutter pub get
      - name: Run integration tests
        run: flutter test integration_test --verbose

  # job responsible for running Flutter tests on Android devices
  android:
    runs-on: macos-11
    strategy:
      matrix:
        api-level:
          - 29
          # you can add more API level if you want to run your tests on different API
      fail-fast: true
    steps:
      - uses: actions/checkout@v3
      - name: Setup Flutter SDK
        uses: subosito/flutter-action@v2
        with:
          channel: stable
          # instead of "channel: stable", you could be more precise by specifying the exact version of Flutter you're using:
          # flutter-version: '<FLUTTER_VERSION>'
      - name: Install Flutter dependencies
        run: flutter pub get
      - name: Run integration tests
        # more info on https://github.com/ReactiveCircus/android-emulator-runner
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: ${{ matrix.api-level }}
          arch: x86_64
          profile: Nexus 6
          script: flutter test integration_test --verbose

Then, commit and push your work:

git add .
git commit -m "write some tests and setup CI"
git push

Finally go to the “Actions” tab on Github. You can see your actions are running and performed successfully. Congrats ! 🥳

successful CI