How to execute Flutter integration tests in GitHub Action ?
How to execute Flutter integration tests in GitHub Action ?
When you write Flutter integration tests, you have different options to run them on Github Action. You can use a third-party service like the Firebase Test Lab or you can run these tests on emulators (iOS or Android) that you set up. In my last personal project, I chose this last option and I searched a lot on the internet before finding a solution. So here is an article that demonstrates what I learned and what I did to make it work.
Prepare your environment
To follow this tutorial, you must have Flutter installed. If so, please run flutter doctor to verify that your environment is set up correctly.
Obviously, to run the tests on Github Action, you need a Github account.
Create your project
Create a Flutter project by using the skeleton template. It scaffolds an application with a List View and a Detail View:
Create an empty repository on Github:
Then, push the project's source code to Github:
Write some integration tests
We need the integration_test and flutter_test packages to write integration tests. Add these dependencies to the dev_dependencies section of the app’s pubspec.yaml file:
# pubspec.yamldev_dependencies: integration_test: sdk: flutter flutter_test: sdk: flutterThen install the dependencies:
Create a new directory, integration_test, with an empty app_test.dart file. Now we can write some tests:
// integration_test/app_test.dartimport 'package:flutter_test/flutter_test.dart';import 'package:integration_test/integration_test.dart';import 'package:flutter_project/main.dart' as app;void main() { IntegrationTestWidgetsFlutterBinding.ensureInitialized(); group('end-to-end test', () { testWidgets('go to the list and detail views', (WidgetTester tester) async { app.main(); await tester.pumpAndSettle(); expect(find.text('Sample Items'), findsOneWidget); expect(find.textContaining('SampleItem'), findsNWidgets(3)); final Finder sampleItem = find.text('SampleItem 1'); await tester.tap(sampleItem); await tester.pumpAndSettle(); expect(find.text('Item Details'), findsOneWidget); }); });}Run these tests to verify that they pass:
Create a Github workflow
Now, we need to configure our CI on Github Action. For this, we will define a workflow, composed of 2 different jobs: one responsible for running tests on iOS devices and the second for running them on Android devices.
Create a file named main.yml in the .github/workflowsdirectory:
# .github/workflows/main.yml# Name of your workflow.name: main# Trigger the workflow manually or after push a commiton: [push, workflow_dispatch]# 2 jobs are configured.# The first one runs tests on iOS devices.# The second runs tests on Android devicesjobs: # job responsible for running Flutter tests on iOS devices ios: # Creates a build matrix for your jobs. You can define different variations of an environment to run each job strategy: matrix: device: # The available simulators are listed by the "xcrun xctrace list devices" command - "iPhone 11 Simulator (15.2)" # the name of the simulator could be different depending on the macos version you are using # if one of the jobs in the matrix expansion fails, the rest of the jobs will be cancelled fail-fast: true runs-on: macos-11 # or macos-latest if you prefer, but be aware that the available simulators could be different if you run a different version steps: - name: "List all simulators" run: "xcrun xctrace list devices" - name: "Start Simulator" # the command "xcrun simctl boot" expects a device identifier # the assignment of the UDID variable consists of retrieving the ID of the simulator # by extracting it from the command "xcrun xctrace list devices" run: | UDID=$(xcrun xctrace list devices | grep "^${{ matrix.device }}" | awk '{gsub(/[()]/,""); print $NF}') echo $UDID xcrun simctl boot "${UDID:?No Simulator with this name found}" - uses: actions/checkout@v3 - name: Setup Flutter SDK uses: subosito/flutter-action@v2 with: channel: stable # instead of "channel: stable", you could be more precise by specifying the exact version of Flutter you're using: # flutter-version: '<FLUTTER_VERSION>' - name: Install Flutter dependencies run: flutter pub get - name: Run integration tests run: flutter test integration_test --verbose # job responsible for running Flutter tests on Android devices android: runs-on: macos-11 strategy: matrix: api-level: - 29 # you can add more API level if you want to run your tests on different API fail-fast: true steps: - uses: actions/checkout@v3 - name: Setup Flutter SDK uses: subosito/flutter-action@v2 with: channel: stable # instead of "channel: stable", you could be more precise by specifying the exact version of Flutter you're using: # flutter-version: '<FLUTTER_VERSION>' - name: Install Flutter dependencies run: flutter pub get - name: Run integration tests # more info on https://github.com/ReactiveCircus/android-emulator-runner uses: reactivecircus/android-emulator-runner@v2 with: api-level: ${{ matrix.api-level }} arch: x86_64 profile: Nexus 6 script: flutter test integration_test --verboseThen, commit and push your work:
Finally go to the “Actions” tab on Github. You can see your actions are running and performed successfully. Congrats ! 🥳
