CI/CD for Android Guide: Bitbucket Pipelines and Gradle Play Publisher

Motivation

You can use Fastlane.

1. Fastlane needs to be installed on each build machine. With multiple developers or if you use multiple computers to develop, you need to setup and maintain it multiple times. Fastlane’s documentation isn’t great imo and especially the setup on Linux is not straight-forward (time consuming). Having multiple build machines also impacts the reproducibility of the build, differences in the Fastlane dependencies (Ruby libs) can result in different results.
2. Signing and deploying requires the signing/upload key and also the Google Play API key. As few people as possible should have access to those keys. It’s possible to create new keys every time an employee with access to the keys leaves the company (provided upload keys are used to sign the app) but it’s certainly not an ideal process.
3. One of my requirements was to start the build process upon merge. Fastlane isn’t the tool for that unless it’s combined with other tools which leads us to…

You can use Jenkins.

• Trigger upon merge into a specific branch (master in my case)
• Run unit tests (I have ui tests as well but they aren’t part of the pipeline yet)
• Run lint and fail the build if errors are detected
• Build a free and a paid version of the app as bundles
• Sign the app
• Automatically increment the build number
• Upload app metadata (release notes, description, screen shots) if needed
• Deploy the app to the internal testing track
• Manual promotion from the internal testing track to the production track by app flavor (free/paid in my case)
• Secrets (signing key, Google Play API key, passwords) can’t be retrieved by developers with access to the repo

Overview

1. We need programmatic access to Google Play to publish and promote apps (plus manage the meta data) -> we need an API key for the Google Play Developer API.
2. We need to configure the Gradle build to use a signing configuration that reads the signing information from environment variables.
3. We need to configure the Gradle Play Publisher plugin to interact with Google Play (upload, publish apps and manage meta data).We need to configure Bitbucket Pipeline to tie everything together.
4. The pipeline configuration defines the trigger (upon merge), the build steps (Gradle), the deploy steps (Gradle Play Publisher) and injects the environment variables (secrets, build number).

Google Play Developer API

TL;TLTR (too long; too lazy to read):

• Go to your Google Play Developer Console as owner of the account and open the API access page:
  
• Accept the Terms of Service (if not done yet)

• Create a new Google Cloud project if you haven’t created one already (otherwise link an existing one).

• Under Service Accounts click on “Create new service account” and open the link that leads to the Google Cloud Platform:
  

• In Google Cloud Platform click on “CREATE SERVICE ACCOUNT”:
  

• Pick a meaningful name and description before hitting the “CREATE” button:
  

• The account needs the role “Service Account User”:

• You don’t need to grant user access to the new service account, Google Cloud adds the required users automatically with the correct permissions (a Google Play service and your own user) so just hit “DONE”:
  

• Next, you need to create an API key for the account.
  Open the actions menu (the three dots) and select “Manage keys”:
  

• Under “ADD Key” select “Create new key”:
  

• Create a JSON key:
  

• After hitting the “CREATE” button, the key file will be downloaded to your computer. I recommend to rename the file to make its purpose more obvious:
  

• Now you’re done in Google Cloud Platform and you can go back to the Google Play Console (to the API access screen). The newly created account should appear under “Service accounts” (hit the “Refresh service account” button). Click on “Grant access”:

• Click on “Add app” and select all apps you want to manage with this service account:
  

• The Account permissions are already set correctly so that the service can manage all release related activities (create releases including publication to production, management of meta data etc.).

• Click on “Invite user” and you’re done.
  We will use the Gradle Play Publisher plugin to validate API key setup later on.

Gradle Build

KEYSTORE_FILE=/path to the keystore file/playstore.keystore
KEYSTORE_PASSWORD=keystore password
KEYSTORE_KEY_ALIAS=key alias
KEYSTORE_KEY_PASSWORD=key password

signingConfigs {
    release {
        storeFile file(KEYSTORE_FILE)
        storePassword KEYSTORE_PASSWORD
        keyAlias KEYSTORE_KEY_ALIAS
        keyPassword KEYSTORE_KEY_PASSWORD
    }
}

buildTypes {
   debug {
        // debug build type configuration ...
   }

   release {
        // release build type configuration ...
        signingConfig signingConfigs.release
   }
}

./gradlew bundleRelease

Build Number

versionCode 124

versionCode project.hasProperty('BUILD_NUMBER') ? project['BUILD_NUMBER'].toInteger() : 124

Gradle Play Publisher

• Add the plugin to the app’s build.gradle file:

plugins {
    id 'com.android.application'
    id 'com.github.triplet.play' version '3.3.0'
    // other plugins...
}

• Add a configuration block to the app’s build.gradle file (after the android block):

android { ... }
play {
    serviceAccountCredentials = file(GOOGLE_PLAY_API_KEY)
}

• You’ll notice the GOOGLE_PLAY_API_KEY parameter. It’s a reference to the api key file we got when setting up the key for the service account -> google-play-api-key.json. The parameter needs to be defined in the ~/.gradle/gradle.properties file (analogous the signing config parameters):

GOOGLE_PLAY_API_KEY=/path to the api key file/google-play-api-key.json

• If everything was setup properly, the following command when run from the root directory of your app will download the app’s meta data:

./gradlew bootstrap

Bitbucket Pipeline

1. Create secret files (keystore and API key file) and make then available to the pipeline
2. Run unit tests
3. Build the app bundle(s) and deploy them to the internal testing track
4. Promote the free version of the app to the production track (manual step)
5. Promote the pro/paid version of the app to the production track (manual step)

1. Create secret files

• KEYSTORE_FILE
• KEYSTORE_PASSWORD
• KEYSTORE_KEY_ALIAS
• KEYSTORE_KEY_PASSWORD
• GOOGLE_PLAY_API_KEY

base64 google-play-api-key.json
base64 playstore.keystore

image: androidsdk/android-30

pipelines:
  branches:
    master:
      - step:
          name: Create keystore and API key
          script:
            # create the keystore file and the google play api key file
            - mkdir keys
            - echo $KEYSTORE_FILE_BASE64 | base64 --decode > keys/$KEYSTORE_FILE
            - echo $GOOGLE_PLAY_API_KEY_BASE64 | base64 --decode > keys/$GOOGLE_PLAY_API_KEY
          artifacts:
            - keys/**

branches:
    master:

• Create a new directory:

mkdir keys

• Decode the keystore file and write it to ./keys/$KEYSTORE_FILE ($KEYSTORE_FILE_BASE64 being the base64 encoded keystore file, $KEYSTORE_FILE being the name of the keystore file to create in the Docker container):

echo $KEYSTORE_FILE_BASE64 | base64 --decode > keys/$KEYSTORE_FILE

• Decode the api key file and write it to ./keys/$GOOGLE_PLAY_API_KEY ($GOOGLE_PLAY_API_KEY_BASE64 being the base64 encoded api key file, $GOOGLE_PLAY_API_KEY being the name of the api key file to create in the Docker container):

echo $GOOGLE_PLAY_API_KEY_BASE64 | base64 --decode > keys/$GOOGLE_PLAY_API_KEY

artifacts:
            - keys/**

2. Run unit tests

./gradlew -PKEYSTORE_FILE=../keys/$KEYSTORE_FILE
          -PKEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
          -PKEY_ALIAS=$KEY_ALIAS
          -PKEY_PASSWORD=$KEY_PASSWORD
          -PGOOGLE_PLAY_API_KEY=../keys/$GOOGLE_PLAY_API_KEY
          test

step:
    name: Run unit tests
    caches:
      - gradle
    script:
      - export GRADLE_OPTS='-XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -Dorg.gradle.jvmargs="-Xmx2048m -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=false -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true'
      - "./gradlew -PKEYSTORE_FILE=../keys/$KEYSTORE_FILE
                   -PKEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                   -PKEY_ALIAS=$KEY_ALIAS
                   -PKEY_PASSWORD=$KEY_PASSWORD
                   -PGOOGLE_PLAY_API_KEY=../keys/$GOOGLE_PLAY_API_KEY
                   test"
    artifacts:
      - app/build/outputs/**
      - app/build/reports/**

Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

3. Build and deploy the bundle

- step:
    name: Build & deploy
    caches:
      - gradle
    script:
      - export GRADLE_OPTS='-XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -Dorg.gradle.jvmargs="-Xmx2048m -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=false -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true'
      - "./gradlew -PKEYSTORE_FILE=../keys/$KEYSTORE_FILE
                   -PKEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                   -PKEY_ALIAS=$KEY_ALIAS
                   -PKEY_PASSWORD=$KEY_PASSWORD
                   -PGOOGLE_PLAY_API_KEY=../keys/$GOOGLE_PLAY_API_KEY
                   -PBUILD_NUMBER=$BITBUCKET_BUILD_NUMBER
                   clean :app:publishFreeReleaseBundle :app:publishProReleaseBundle"
    artifacts:
      - app/build/outputs/

• We pass in -PBUILD_NUMBER=$BITBUCKET_BUILD_NUMBER to set the versionCode of the app

• The plugin publishes to the internal testing track by default, use the --track argument to publish to a different track (internal, alpha, beta, production)

4. Promote to production

- parallel:
    - step:
        name: Promote free version
        caches:
          - gradle
        trigger: manual
        script:
          - export GRADLE_OPTS='-XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -Dorg.gradle.jvmargs="-Xmx2048m -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=false -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true'
          - "./gradlew -PKEYSTORE_FILE=../keys/$KEYSTORE_FILE
                   -PKEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                   -PKEY_ALIAS=$KEY_ALIAS
                   -PKEY_PASSWORD=$KEY_PASSWORD
                   -PGOOGLE_PLAY_API_KEY=../keys/$GOOGLE_PLAY_API_KEY
                   -PBUILD_NUMBER=$BITBUCKET_BUILD_NUMBER
                        promoteFreeReleaseArtifact --from-track internal --promote-track production --release-status completed"
    - step:
        name: Promote pro version
        caches:
          - gradle
        trigger: manual
        script:
          - export GRADLE_OPTS='-XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -Dorg.gradle.jvmargs="-Xmx2048m -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=440m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8" -Dorg.gradle.parallel=false -Dorg.gradle.daemon=false -Dorg.gradle.configureondemand=true'
          - "./gradlew -PKEYSTORE_FILE=../keys/$KEYSTORE_FILE
                   -PKEYSTORE_PASSWORD=$KEYSTORE_PASSWORD
                   -PKEY_ALIAS=$KEY_ALIAS
                   -PKEY_PASSWORD=$KEY_PASSWORD
                   -PGOOGLE_PLAY_API_KEY=../keys/$GOOGLE_PLAY_API_KEY
                   -PBUILD_NUMBER=$BITBUCKET_BUILD_NUMBER
                        promoteProReleaseArtifact --from-track internal --promote-track production --release-status inProgress --user-fraction .5"

promoteFreeReleaseArtifact --from-track internal --promote-track production --release-status completed

promoteProReleaseArtifact --from-track internal --promote-track production --release-status inProgress --user-fraction .5

trigger: manual