Skip to content

Fix instrumentation tests execution #1105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: trunk
Choose a base branch
from
Draft

Fix instrumentation tests execution #1105

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
17ccd9e
Use non-deprecated wrapper-validation action
wzieba Jun 12, 2025
332cd28
Use Pixel 9a for connected tests
wzieba Jun 12, 2025
c2952fe
Use model id instead of name
wzieba Jun 12, 2025
1842919
Grant only necessary permission and use UiAutomation for it
wzieba Jun 13, 2025
2a93d62
Run UI tests on Pixel 5
wzieba Jun 13, 2025
242aaa1
Fix `EditLinkPage#updateURL` action by introducing safety mechanisms:…
wzieba Jun 13, 2025
e2ea258
Remove hints from text views of the test app
wzieba Jun 13, 2025
4746d9b
Fix formatting
wzieba Jun 13, 2025
a05f084
Print stack trace of RuntimeException
wzieba Jun 13, 2025
93ae87b
Merge branch 'trunk' into fix_ci
wzieba Jun 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .buildkite/commands/connected-tests.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@ gcloud firebase test android run \
--type instrumentation \
--app app/build/outputs/apk/debug/app-debug.apk \
--test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--device model=Nexus5X,version=26,locale=en,orientation=portrait \
--device model=redfin,version=30 \
--verbosity info
2 changes: 1 addition & 1 deletion .github/workflows/gradle-wrapper-validation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: gradle/wrapper-validation-action@v1
- uses: gradle/actions/wrapper-validation@v3
17 changes: 11 additions & 6 deletions app/src/androidTest/kotlin/org/wordpress/aztec/demo/BaseTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,23 @@

package org.wordpress.aztec.demo

import android.Manifest.permission.CAMERA
import android.util.Log
import androidx.test.rule.GrantPermissionRule
import androidx.test.platform.app.InstrumentationRegistry
import androidx.test.runner.AndroidJUnit4
import org.junit.Rule
import org.junit.Before
import org.junit.runner.RunWith

@RunWith(AndroidJUnit4::class)
abstract class BaseTest {
@Rule
@JvmField
val grantPermissionRule: GrantPermissionRule = GrantPermissionRule.grant(android.Manifest.permission.CAMERA,
android.Manifest.permission.WRITE_EXTERNAL_STORAGE)

@Before
fun grantPermissions() {
val (uiAutomation, packageName) = with(InstrumentationRegistry.getInstrumentation()) {
Pair(uiAutomation, targetContext.packageName)
}
uiAutomation.grantRuntimePermission(packageName, CAMERA)
}

companion object {
fun label(label: String) {
Expand Down
17 changes: 16 additions & 1 deletion app/src/androidTest/kotlin/org/wordpress/aztec/demo/pages/EditLinkPage.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,17 @@ class EditLinkPage : BasePage() {
}

fun updateURL(url: String): EditLinkPage {
urlField.perform(replaceText(url), ViewActions.closeSoftKeyboard())
threadSleep(2000L) // Wait for the page to load
try {
urlField.perform(click())
urlField.perform(replaceText(url), ViewActions.closeSoftKeyboard())
} catch (e: RuntimeException) {
// If the URL field is not visible, it might be because the keyboard is open.
// Close the keyboard and try again.
urlField.perform(ViewActions.closeSoftKeyboard())
urlField.perform(replaceText(url), ViewActions.closeSoftKeyboard())
e.printStackTrace()
}
Comment on lines +36 to +46
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the core change: a few of additional safety checks and delays to make UI tests not flaky on CI.

label("Entered url")

return this
Expand Down Expand Up @@ -80,4 +90,9 @@ class EditLinkPage : BasePage() {
cancelButton.perform(click())
label("Canceled")
}

fun threadSleep(millis: Long): EditLinkPage {
Thread.sleep(millis)
return this
}
}
11 changes: 7 additions & 4 deletions app/src/androidTest/kotlin/org/wordpress/aztec/demo/tests/LinkHistoryTests.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,16 +204,19 @@ class LinkHistoryTests : BaseHistoryTest() {
*/
private fun addLinkAndVerify(editorPage: EditorPage, link: String, expected: String, name: String? = null) {
editorPage.makeLink()
EditLinkPage().updateURL(link)
editorPage.threadSleep(throttleTime)

val editLinkPage = EditLinkPage()
editLinkPage.updateURL(link)
name?.let {
EditLinkPage().updateName(it)
editLinkPage.updateName(it)
}
EditLinkPage().ok()
editLinkPage.ok()

editorPage
.toggleHtml()
.verifyHTML(expected)
.toggleHtml()
.threadSleep(throttleTime)
}
}
}
2 changes: 0 additions & 2 deletions app/src/main/res/layout/activity_main.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
android:layout_width="match_parent"
android:layout_height="match_parent"
android:gravity="top|start"
android:hint="@string/edit_hint"
android:inputType="textCapSentences|textMultiLine"
android:paddingEnd="16dp"
android:paddingLeft="16dp"
Expand All @@ -46,7 +45,6 @@
android:layout_width="match_parent"
android:layout_height="match_parent"
android:gravity="top|start"
android:hint="@string/source_hint"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The android:hint looked quite strange on the device when running tests: almost like a normal test. After removing them, tests started to pass. I'm not entirely sure why (as they were flaky and the failure was not reproducible locally), so here's my best guess.

I think that's fair as it only affects the test app.

android:inputType="textNoSuggestions|textMultiLine"
android:paddingEnd="16dp"
android:paddingLeft="16dp"
Expand Down