CI/CD, Signing & Release Explained
BUILD & TOOLING › Delivery
Every Android release you ship to the Play Store passes through two different keys, and understanding why is the whole foundation of release engineering. The upload key is one you generate and hold yourself, on your laptop or in a CI secret store, and you use it to sign the app bundle you send to Google. The app signing key is different: only Google holds it, and it is the key that actually signs the APKs a real device downloads and installs.
When you upload a signed bundle, Play verifies your upload signature, strips it off, and re-signs the split APKs it generates with the app signing key before delivery. That split exists on purpose. The key that faces the outside world, sitting on a machine that could be stolen or a CI runner that could be compromised, is disposable. The key users actually trust to install updates never has to leave Google's infrastructure at all.
The two keys fail very differently, and that asymmetry is exactly what interviewers are testing when they ask about lost keys. Say your laptop with the upload keystore on it gets stolen. You're fine: open Play Console under Setup, then App signing, and request an upload key reset. Register a new upload certificate and keep shipping. The app signing key never moved, so existing users update with zero disruption and no package name change.
Losing the app signing key itself is a different story, and it can only happen if you opted out of Play App Signing and self-manage that key. If it's gone, there is no reset available. You cannot publish another update to that app, ever. The only option is launching a brand new app under a new package name, which strands every existing install. That asymmetry is exactly why Play App Signing, where Google is the one holding the app signing key, is effectively mandatory for new apps today.
New apps on Google Play must ship as an Android App Bundle, not a standalone APK, and the reason is worth understanding rather than memorizing. An AAB doesn't contain a ready-to-install APK at all. It contains all of your app's compiled code and resources, plus enough metadata for Google to assemble a slimmed down, device specific APK on the fly, split by CPU architecture, screen density, and language, and to serve dynamic feature modules only when a user actually needs them.
That's also why Play App Signing is enforced the moment you upload a bundle. Since Google is now the one generating and signing the final APK a device receives, it has to be the one holding the app signing key.
./gradlew bundleRelease # produces app-release.aab, the required upload artifact
./gradlew assembleRelease # produces a monolithic APK, the legacy path
APK signing has gone through four schemes, and each one closed a real gap left by the last, which is a favorite interview thread because it tests whether you understand the mechanism rather than just the version number.
v1, JAR signing, only covers individual files inside the archive, so a v1-only APK is vulnerable to attacks that add or swap in unsigned entries around the parts that are actually signed. v2, introduced with Android 7, signs the whole APK as one contiguous block instead, closing that gap and speeding up verification at install time. v3, introduced with Android 9, adds something neither predecessor had: a proof-of-rotation key lineage, a signed chain of certificates that says this new key is authorized to replace that old key. That lets an app change its signing key over time while devices that still trust the old key accept updates signed by the new one. v4, from Android 11, layers incremental streaming installs on top of v3's signature; it doesn't add any rotation capability of its own.
Play App Signing leans on v3's lineage internally whenever it needs to change the key it uses to re-sign your APKs.
apksigner rotate --out lineage.bin --old-signer --ks old-release.jks --new-signer --ks new-release.jks
Once you have a signed bundle, you don't push it straight to every user. Play Console organizes exposure into tracks, in increasing order of reach: internal testing, which supports up to a hundred testers and processes in minutes, closed testing, an invited alpha list, open testing, a public beta anyone can opt into, and finally production. You promote the same build up through these tracks rather than rebuilding it each time, which matters because it means the artifact your production users get is byte for byte what your internal testers already verified.
Even once you reach the production track, you rarely go to a hundred percent of users immediately. A staged, or phased, rollout ships a new version to a percentage of users first, say five percent, then twenty, then fifty, while you watch crash rate and ANRs in Play Console's vitals dashboard. If something regresses, you halt the rollout before it reaches everyone else.
Here's the part that trips people up in interviews: halting a rollout only stops further distribution, it does nothing for the users who already updated. And Play never lets you push a lower version code to downgrade a device once it has a newer one installed, there is no server side revert button. So the real playbook after a bad staged rollout is to halt it, fix the bug, and ship a new release with a higher version code. The affected users get the fix exactly the same way everyone else gets updates, by installing a newer build.
android {
defaultConfig {
versionCode = 44 // was 43 for the release that got halted
versionName = "2.1.1"
}
}
Every release you upload is keyed by an integer called the version code, and Play enforces one hard rule on it across your app's entire history: it must strictly increase with every single upload, forever. Reusing a version code, or accidentally lowering it because you branched off an older release for a hotfix, gets the upload rejected outright with a version code already used error. This is completely independent of the version name, the human readable string like 2.1.0 that users actually see; Play doesn't look at that string to decide whether an upload is valid.
android {
defaultConfig {
// BAD: reusing or lowering the code, Play rejects with "Version code already used"
// versionCode 42
// GOOD: must strictly exceed the highest previously uploaded versionCode
versionCode 43
versionName "2.1.0"
}
}
When you ship a release build with R8 minification turned on, class and method names in your compiled code get shrunk down to short, meaningless identifiers to reduce app size and make reverse engineering harder. That's great for your APK size and terrible for reading a crash report, unless Play Console has a way to translate those identifiers back.
That translation lives in a file called mapping.txt, generated by R8 alongside every minified build. Upload your release as an AAB and Play Console automatically pulls the matching mapping file out of the bundle and uses it to deobfuscate stack traces in your crash and ANR vitals. Upload a standalone APK instead and there's no bundle for Play to extract it from, so you have to attach mapping.txt yourself under the App bundle explorer in Play Console, or every crash report you look at stays a wall of meaningless obfuscated names.
android {
buildTypes {
release {
isMinifyEnabled = true
proguardFiles(
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
}
}
}
You don't have to upload a build to Play just to see exactly what a real device would receive. bundletool, Google's own command line tool, turns a signed AAB into the same device specific APK set that Play's servers would generate. The build-apks command produces a signed .apks file containing every possible split, and install-apks then figures out which splits match a connected device and pushes only those.
This is the fastest way to catch a splitting or dynamic feature bug before it ever reaches a Play track, because you're testing the actual delivery mechanism locally instead of trusting that it will work once it's live.
bundletool build-apks --bundle=app-release.aab --output=app.apks --ks=release.jks --ks-key-alias=mykey --ks-pass=pass:secret
bundletool install-apks --apks=app.apks
A CI runner needs your actual release keystore and its passwords to produce a signed build, but a .jks file and its passwords must never be committed to source control or printed anywhere in a build log. The standard pattern is to base64 encode the keystore file and store both it and the passwords as encrypted secrets in your CI provider, GitHub Actions secrets or Bitrise secret environment variables for example, and only decode the keystore back into a real file inside the build job, right before it's needed.
- name: Decode keystore
run: echo "${{ secrets.KEYSTORE_BASE64 }}" | base64 --decode > release.jks
- name: Build release AAB
run: ./gradlew bundleRelease
env:
KEY_ALIAS: ${{ secrets.KEY_ALIAS }}
KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}
STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}
Done this way, the secret never appears anywhere in the repository, and most CI providers automatically mask any secret value that happens to show up in log output.
Most of the wall clock time in a clean Android CI build isn't spent compiling your code, it's spent downloading dependencies and rebuilding modules that haven't actually changed since the last run. Caching fixes both problems at once: point your CI provider at Gradle's dependency and build output directories, ~/.gradle/caches and ~/.gradle/wrapper, keyed by a hash of your Gradle files.
Change a build.gradle.kts file or a dependency version and that hash changes, so the cache key misses and you get a clean rebuild. Leave your dependencies untouched between commits and every later run restores the cache instead of hitting the network at all. It has nothing to do with signing or secrets, it's purely a build speed lever, but it's one of the easiest ones to pull on a slow pipeline.
- name: Cache Gradle dependencies
uses: actions/cache@v4
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}
restore-keys: ${{ runner.os }}-gradle-
When you add a CI service account to Play Console so a pipeline can upload releases automatically, the permissions you grant it matter as much as how you protect its key. The right answer is least privilege: scope it to release permissions for only the specific app, and only the specific tracks, it actually publishes to. Nothing more.
The reasoning is about blast radius. If that service account's JSON key ever leaks out of a compromised CI runner, an attacker with least-privilege access can push a bad release to one app's track at worst. An attacker with an admin grant, or with the human owner's actual login shared to the runner, could touch billing, other apps, or account level settings, which turns a bad day into a catastrophic one.
Fastlane automates a release pipeline end to end, and the unit of automation is the lane. A lane is a named workflow defined in a file called the Fastfile, and you run it with a single command, either on your laptop while testing locally or from inside a CI job, and it behaves identically either way. A lane typically chains several actions together, like building with Gradle and then uploading with supply.
lane :beta do
gradle(task: "bundle", build_type: "Release") # builds the AAB
supply(track: "beta") # uploads to the Play beta track
end
Run it from a terminal with fastlane beta, or trigger the exact same lane from a GitHub Actions workflow. The whole point of a lane is that the sequence of steps only has to be written once.
The action that actually gets a build in front of Play is supply, and what it authenticates with is worth knowing cold for an interview, because it's a common trick question. supply doesn't use your signing keystore, and it doesn't use your personal Google account login either. It authenticates to the Play Developer API using a Google Cloud service account, specifically a JSON key file for that service account, which has to be created in Google Cloud and then explicitly granted access inside Play Console before supply can use it.
lane :deploy do
supply(
track: "internal",
aab: "app/build/outputs/bundle/release/app-release.aab",
json_key: "service-account.json" # Play Developer API auth, not a signing key
)
end
Keep that distinction sharp: the app signing key controls what a device trusts, and the service account key controls what your automation is allowed to publish. They protect completely different things and neither one can substitute for the other.