Skip to content

Conversation

@jrainville
Copy link
Member

@jrainville jrainville commented Aug 5, 2025

Cherry-picks the local backup commits so that we can ship a 2.34 patch on the old mobile with it. This will enable old mobile users to use that backup file to upgrade to the new app.

The only missing part is the RPC test I wrote because it used some new test files and I didn't want to have to backport all the function test files that were committed in between.

In the end, we don't really care about importing on the old mobile, just the loop to export, so we can test that manually.

related mobile PR

@status-im-auto
Copy link
Member

status-im-auto commented Aug 5, 2025

Jenkins Builds

Click to see older builds (81)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 9940feb #1 2025-08-05 19:14:54 ~2 min android 📦aar
✔️ 9940feb #1 2025-08-05 19:15:53 ~3 min ios 📦zip
✖️ 9940feb #1 2025-08-05 19:16:08 ~3 min tests 📄log
✔️ 9940feb #1 2025-08-05 19:16:17 ~3 min linux 📦zip
9940feb #1 2025-08-05 19:17:25 ~4 min windows 📄log
✔️ 9940feb #1 2025-08-05 19:17:31 ~4 min macos 📦zip
✔️ 9940feb #1 2025-08-05 19:18:32 ~5 min macos 📦zip
✖️ 9940feb #1 2025-08-05 19:23:21 ~10 min tests-rpc 📄log
✔️ a152135 #2 2025-08-05 19:24:13 ~2 min android 📦aar
a152135 #2 2025-08-05 19:24:52 ~2 min windows 📄log
✔️ a152135 #2 2025-08-05 19:25:17 ~3 min ios 📦zip
✔️ a152135 #2 2025-08-05 19:26:18 ~4 min linux 📦zip
✔️ a152135 #2 2025-08-05 19:26:52 ~4 min macos 📦zip
✖️ a152135 #2 2025-08-05 19:27:41 ~5 min tests 📄log
✔️ a152135 #2 2025-08-05 19:28:04 ~5 min macos 📦zip
✖️ a152135 #2 2025-08-05 19:32:40 ~8 min tests-rpc 📄log
✔️ 6d14459 #3 2025-08-05 19:37:32 ~2 min android 📦aar
6d14459 #3 2025-08-05 19:38:11 ~2 min windows 📄log
✔️ 6d14459 #3 2025-08-05 19:38:34 ~3 min ios 📦zip
✔️ 6d14459 #3 2025-08-05 19:39:30 ~3 min linux 📦zip
✖️ 6d14459 #3 2025-08-05 19:40:06 ~4 min tests 📄log
✔️ 6d14459 #3 2025-08-05 19:40:12 ~4 min macos 📦zip
✔️ 6d14459 #3 2025-08-05 19:41:05 ~5 min macos 📦zip
✖️ 6d14459 #3 2025-08-05 19:44:54 ~9 min tests-rpc 📄log
✔️ ebabb25 #4 2025-08-05 19:49:57 ~2 min android 📦aar
ebabb25 #4 2025-08-05 19:50:29 ~2 min windows 📄log
✔️ ebabb25 #4 2025-08-05 19:50:52 ~3 min ios 📦zip
✔️ ebabb25 #4 2025-08-05 19:52:00 ~4 min linux 📦zip
✔️ ebabb25 #4 2025-08-05 19:52:27 ~4 min macos 📦zip
✔️ ebabb25 #4 2025-08-05 19:53:30 ~5 min macos 📦zip
✖️ ebabb25 #4 2025-08-05 19:56:00 ~8 min tests-rpc 📄log
✔️ ebabb25 #4 2025-08-05 20:13:45 ~25 min tests 📄log
✔️ e9d6b6b #5 2025-08-06 19:31:23 ~2 min android 📦aar
e9d6b6b #5 2025-08-06 19:31:51 ~2 min windows 📄log
✔️ e9d6b6b #5 2025-08-06 19:32:16 ~3 min ios 📦zip
✔️ e9d6b6b #5 2025-08-06 19:33:36 ~4 min linux 📦zip
✔️ e9d6b6b #5 2025-08-06 19:33:51 ~4 min macos 📦zip
✔️ e9d6b6b #5 2025-08-06 19:34:52 ~5 min macos 📦zip
✖️ e9d6b6b #5 2025-08-06 19:37:00 ~7 min tests-rpc 📄log
✔️ e9d6b6b #5 2025-08-06 19:56:11 ~26 min tests 📄log
✔️ 137eee3 #6 2025-08-06 20:58:29 ~2 min android 📦aar
137eee3 #6 2025-08-06 20:58:59 ~2 min windows 📄log
✔️ 137eee3 #6 2025-08-06 20:59:37 ~3 min ios 📦zip
✔️ 137eee3 #6 2025-08-06 21:00:28 ~4 min linux 📦zip
✔️ 137eee3 #6 2025-08-06 21:01:08 ~4 min macos 📦zip
✔️ 137eee3 #6 2025-08-06 21:02:02 ~5 min macos 📦zip
✖️ 137eee3 #6 2025-08-06 21:03:52 ~7 min tests-rpc 📄log
✔️ 137eee3 #6 2025-08-06 21:21:40 ~25 min tests 📄log
435f422 #7 2025-10-01 17:20:29 ~33 sec windows 📄log
435f422 #7 2025-10-01 17:22:06 ~2 min macos 📄log
435f422 #7 2025-10-01 17:27:55 ~8 min linux 📄log
✖️ 435f422 #7 2025-10-01 17:29:22 ~9 min tests 📄log
✖️ 435f422 #7 2025-10-01 17:38:33 ~18 min tests-rpc 📄log
513c3de #8 2025-10-01 18:11:45 ~33 sec windows 📄log
✔️ 513c3de #8 2025-10-01 18:14:55 ~3 min macos 📦zip
✖️ 513c3de #8 2025-10-01 18:21:12 ~10 min tests 📄log
✔️ 513c3de #8 2025-10-01 18:21:39 ~10 min linux 📦zip
✖️ 513c3de #8 2025-10-01 18:30:28 ~19 min tests-rpc 📄log
e2ea8b3 #9 2025-10-01 18:52:59 ~31 sec windows 📄log
✔️ e2ea8b3 #9 2025-10-01 18:55:56 ~3 min linux 📦zip
✔️ e2ea8b3 #9 2025-10-01 18:56:28 ~4 min macos 📦zip
4c068ab #10 2025-10-01 18:56:28 ~30 sec windows 📄log
✔️ 4c068ab #10 2025-10-01 19:00:31 ~3 min macos 📦zip
✔️ 4c068ab #10 2025-10-01 19:00:44 ~4 min linux 📦zip
✖️ 4c068ab #10 2025-10-01 19:07:56 ~11 min tests-rpc 📄log
✖️ 4c068ab #10 2025-10-01 19:10:00 ~13 min tests 📄log
13db971 #11 2025-10-01 19:40:49 ~31 sec windows 📄log
✔️ 13db971 #11 2025-10-01 19:44:14 ~4 min macos 📦zip
✔️ 13db971 #11 2025-10-01 19:44:33 ~4 min linux 📦zip
✖️ 13db971 #11 2025-10-01 19:52:08 ~11 min tests-rpc 📄log
✖️ 13db971 #11 2025-10-01 19:54:42 ~14 min tests 📄log
2f47fa8 #12 2025-10-02 15:21:42 ~31 sec windows 📄log
✔️ 2f47fa8 #12 2025-10-02 15:25:57 ~4 min macos 📦zip
✔️ 2f47fa8 #12 2025-10-02 16:19:37 ~58 min linux 📦zip
✖️ 2f47fa8 #12 2025-10-02 16:26:39 ~1 hr 5 min tests-rpc 📄log
✖️ 2f47fa8 #12 2025-10-02 16:28:43 ~1 hr 7 min tests 📄log
d1ff352 #13 2025-10-02 16:56:45 ~33 sec windows 📄log
✖️ d1ff352 #13 2025-10-02 16:58:18 ~2 min tests 📄log
✔️ d1ff352 #13 2025-10-02 17:00:13 ~4 min macos 📦zip
✔️ d1ff352 #13 2025-10-02 17:01:03 ~4 min linux 📦zip
✖️ d1ff352 #13 2025-10-02 17:04:03 ~7 min tests-rpc 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
b7c853d #14 2025-10-02 18:04:28 ~32 sec windows 📄log
✖️ b7c853d #14 2025-10-02 18:06:01 ~2 min tests 📄log
✔️ b7c853d #14 2025-10-02 18:07:29 ~3 min linux 📦zip
✔️ b7c853d #14 2025-10-02 18:07:47 ~3 min macos 📦zip
✖️ b7c853d #14 2025-10-02 18:11:53 ~7 min tests-rpc 📄log
✖️ b7c853d #15 2025-10-09 21:12:43 ~8 min tests-rpc 📄log
42ddea9 #15 2025-10-14 00:08:16 ~31 sec windows 📄log
✔️ 42ddea9 #15 2025-10-14 00:11:44 ~4 min macos 📦zip
✔️ 42ddea9 #15 2025-10-14 00:11:54 ~4 min linux 📦zip
✖️ 42ddea9 #16 2025-10-14 00:15:37 ~7 min tests-rpc 📄log
✔️ 42ddea9 #15 2025-10-14 00:36:31 ~28 min tests 📄log

@codecov
Copy link

codecov bot commented Aug 5, 2025

Codecov Report

❌ Patch coverage is 50.61881% with 399 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release/10.29.x@5985b16). Learn more about missing BASE report.

Files with missing lines Patch % Lines
services/wallet/service.go 0.00% 104 Missing ⚠️
protocol/message_persistence.go 78.00% 34 Missing and 19 partials ⚠️
services/accounts/service.go 8.88% 41 Missing ⚠️
node/get_status_node.go 39.28% 27 Missing and 7 partials ⚠️
protocol/messenger_pairing_and_syncing.go 9.37% 27 Missing and 2 partials ⚠️
node/backup/controller.go 62.85% 15 Missing and 11 partials ⚠️
mobile/status.go 0.00% 22 Missing ⚠️
node/backup/core.go 62.71% 11 Missing and 11 partials ⚠️
protocol/messenger_backup_handler.go 56.41% 10 Missing and 7 partials ⚠️
protocol/messenger_local_backup.go 73.58% 7 Missing and 7 partials ⚠️
... and 9 more
Additional details and impacted files
@@                Coverage Diff                 @@
##             release/10.29.x    #6804   +/-   ##
==================================================
  Coverage                   ?   60.28%           
==================================================
  Files                      ?      854           
  Lines                      ?   106109           
  Branches                   ?        0           
==================================================
  Hits                       ?    63971           
  Misses                     ?    34470           
  Partials                   ?     7668           
Flag Coverage Δ
functional 25.57% <9.28%> (?)
unit 58.24% <50.61%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
account/accounts.go 61.90% <ø> (ø)
multiaccounts/settings/columns.go 88.88% <ø> (ø)
multiaccounts/settings/structs.go 92.85% <ø> (ø)
protocol/communities/manager.go 65.68% <100.00%> (ø)
protocol/messenger.go 64.52% <100.00%> (ø)
protocol/messenger_handler.go 59.72% <100.00%> (ø)
protocol/protobuf/service.go 66.66% <ø> (ø)
server/pairing/config.go 70.37% <ø> (ø)
server/pairing/raw_message_handler.go 62.76% <100.00%> (ø)
server/pairing/server.go 69.66% <100.00%> (ø)
... and 21 more

@jrainville
Copy link
Member Author

I don't think you have to actually review this, but feel free to do it if you're a masochist lol

@jrainville
Copy link
Member Author

Alright, I finally managed to make it work. I had to hardcode the documents path for Android, but it works.

image

@github-actions
Copy link

github-actions bot commented Oct 1, 2025

We require commits to follow the Conventional Commits, but with _ for non-breaking changes.
Please fix these commit messages:

fix: linter
Revert "hardcode for testing"
fix test
fix(pairing): fix pairing messages using wrong chat ID for 1-1
hardcode for testing
fix path
chore: fix issues with cherry-picks
test(backup): improve tests by using a map instead of a loop
fix(backup): add missing clock on backup messages
feat(pairing): add config to let users enable message syncing
feat(pairing): enable syncing messages during local pairing
fix(settings): fix wrongly named reactFieldName for messages backup
feat(backup): implement messages backup poc
feat(backup): set backup path to user's config folder if not set
fix(backup): backup import fails if the community was left

Fixes status-im/status-desktop#18248 status-im/status-desktop#18107 status-im/status-desktop#18120 status-im/status-desktop#18122 status-im/status-desktop#18141 status-im/status-desktop#18121

Implements all the backend for the local user data backups.

Uses the scaffold @osmaczko created to make it more extendable. Instead of using the old Messenger code for backups, we now use a new controller that let's other modules register to be backed up. This includes the messenger for chats, contacts and communities, the accounts service for settings and the wallet service for watch-only accounts.

I also created new Protobufs for those backups, so that we no longer user the super generic Waku Backup protobuf.

Finally, I added some integration tests and a functional test that does the whole flow.

A lot of cleanups can still be done to reduce duplication, but they can be done in another commit/issue, since the internal of the services can be modified without the backup process being affected, since the protobufs are in place already.
This is a light deprecation of Waku backups
This is a fix for the backup and also just a performance improvement. Basically, when we have a SyncCommunity message, we leave the community if the state of said community is not pending or joined (meaning we left it). However, in the case of backup messages, we backup left communities too, but when trying to "leave" those communities, it threw an error about unknown chat, since we never spectated that community in that imported profile.

The solution is just to early return when we don't have anything to leave. We gracefully ignore the error in the Sync function and pass the error when actively trying to leave (you can't leave twice).

Added tests and also cleaned up some warnings.
This should make it work on mobile as well plus it sets a default to the setting
It is the name that users are used to see, so it will be less confusing
Fixes status-im/status-desktop#18529

Implements a rough but functional proof of concept for message backup and import.

It is guarded by a new setting so that you need to opt-in to have it, thus not breaking PFS by default.

It works for text messages, contacts requests, images, emojis, transaction commands, pins and stickers.
Fixes status-im/status-desktop#18895

Adds the code to sync messages when doing a local pairing.

The code is currently flagged out and the test commented. We will later add the param to let users accept syncing the messages and use that param as the condition.
Fixes status-im/status-desktop#18894

Adds a param for users to enable message syncing during local pairing.

Updates the test to use that param
@jrainville jrainville force-pushed the chore/cherry-pick-local-backup branch from e2ea8b3 to 4c068ab Compare October 1, 2025 18:55
There was an issue where 1-1 chat messages didn't seem to be synced and it was because we were using the ChatID instead of the LocalChatID, meaning that messages from other people had our own chat ID instead of having their chat ID.

Also upgrades the integration and functional test to validate the behaviour.
@jrainville
Copy link
Member Author

@vkjr This PR is ready to be used.

I hardcoded the settings to true in this commit: 2f47fa8

With it, I tested, on Android, that the backup file is created and that when imported in Desktop, I do have the data plus the messages.

I also tested local pairing, and with the hardcodes, I was also able to get the messages.

I have reverted the hardcoded values back to the normal behaviour in this commit: b7c853d

You can revert it to test for yourself.

I have not tested IOS, so there might be something buggy there, especially the default path for the backup file.
I think letting the user select the path is the way to go.

Since this PR is on an old branch than is still on Go 1.22, I don't think the CI will ever pass, so I can just force merge if you want or @alaibe or @igor-sirotin can do it.

@vkjr
Copy link
Contributor

vkjr commented Oct 9, 2025

@jrainville, this PR works well for mobile, please merge it whenever you have time 🙏

@vkjr
Copy link
Contributor

vkjr commented Oct 13, 2025

@igor-sirotin, would you mind to merge it?

@igor-sirotin
Copy link
Collaborator

hmmm @vkjr I see tests are not even starting because the linter stage is failing.
Can you please fix what linter complains about, to make sure that tests are passing? (or at least we'll see which are failing and decide if it's fine)

I might take a look soon as well 👌

@igor-sirotin
Copy link
Collaborator

@vkjr I've fixed the linter issues. Will check tests results tomorrow and merge

@igor-sirotin
Copy link
Collaborator

Tests passed now, merging

@igor-sirotin igor-sirotin merged commit 79b8fe9 into release/10.29.x Oct 14, 2025
11 of 14 checks passed
@igor-sirotin igor-sirotin deleted the chore/cherry-pick-local-backup branch October 14, 2025 13:24
@vkjr
Copy link
Contributor

vkjr commented Oct 14, 2025

Thank!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BackupBackPort] Cherry-pick status-go commits

5 participants