message: Consider unsubscribed channels in reconcileMessages #1912

chrisbobbe · 2025-10-10T22:38:55Z

This fixes the "fourth buggy behavior" in #1798:
#1798 (comment)

Fixes-partly: #1798

chrisbobbe · 2025-10-10T23:50:15Z

(Oops, I'll need to rethink this a bit following some discussion with Greg in the office.)

gnprice

Thanks! At #1798 (comment) you asked for an initial look at this logic, so comments on that below.

gnprice · 2025-10-16T00:44:43Z

lib/model/message.dart

    // If the message is one we already know about (from a fetch),
    // clobber it with the one from the event system.
    // See [fetchedMessages] for reasoning.


(Is this existing comment meant to refer to reconcileMessages? I think fetchedMessages was a previous name for that.)

gnprice · 2025-10-16T00:46:29Z

lib/model/message.dart

+  /// (We have seen a few such events, actually --
+  /// maybe because the channel was recently subscribed? --
+  /// but not consistently, and we're not supposed to rely on them.)


Suggested change

/// (We have seen a few such events, actually --

/// maybe because the channel was recently subscribed? --

/// but not consistently, and we're not supposed to rely on them.)

/// (We have seen a few such events, actually --

/// maybe because the channel was recently unsubscribed? --

/// but not consistently, and we're not supposed to rely on them.)

?

gnprice · 2025-10-16T00:48:33Z

lib/model/message.dart

+    // [1] With one exception: if the version we have was in an unsubscribed
+    //     channel when we got it or sometime since, we take the fetched version


I like this formulation of which messages go in _maybeStaleChannelMessages. It'd be good to put it in that field's doc.

gnprice · 2025-10-16T00:55:16Z

lib/model/message.dart

+        // Side effect: update our "potentially stale" knowledge based on
+        // whether the channel is subscribed right now.
+        && (subscriptions[message.streamId] != null
+            ? _maybeStaleChannelMessages.remove(message.id)
+            : !_maybeStaleChannelMessages.add(message.id));


It's a bit confusing that we need this side effect on the maybe-stale set here. I think this logic could benefit from some unpacking.

In particular, it seems like there are really four cases here:

currently subscribed vs. not;

for each of those, currently in the maybe-stale set or not.

Thanks to Greg for pointing this out: zulip#1912 (comment)

chrisbobbe · 2025-10-16T21:04:40Z

Thanks for taking a look! Revision pushed.

gnprice

Thanks! I think that's helpful. A couple of comments below on the new version.

gnprice · 2025-10-17T00:14:00Z

lib/model/message.dart

+  Message _stripMatchFields(Message message) {
+    message.matchContent = null;
+    message.matchTopic = null;
+    return message;
  }


The factoring out of this step could happen in a prep commit. That would simplify the complexity of the main commit a bit — it'd be moving around where an expression like _stripMatchFields(incoming) appears, and adding a case where something else gets used instead, but the expression itself would remain the same.

(Or maybe the main commit would change from "message" to "incoming", but anyway that's a rather smaller change.)

gnprice · 2025-10-17T00:17:20Z

lib/model/message.dart

+  Message _reconcileWhenPresent(Message current, Message incoming) {
+    bool currentIsMaybeStale = false;
+    if (incoming is StreamMessage) {
+      if (subscriptions[incoming.streamId] != null) {
+        // The incoming message won't grow stale; it's in a subscribed channel.
+        // Remove it from _maybeStaleChannelMessages if it was there.
+        currentIsMaybeStale = _maybeStaleChannelMessages.remove(incoming.id);
+      } else {
+        assert(_maybeStaleChannelMessages.contains(incoming.id));
+        currentIsMaybeStale = true;
+      }
+    }
+
+    return currentIsMaybeStale
+      ? _stripMatchFields(incoming)
+      : current;


I think this logic would benefit from having the explanation in the long comment above worked into it.

Probably that mostly means just moving that comment from reconcileMessages into this method. After all, it starts with:

What to do when some of the just-fetched messages are already known?

so it's entirely about this "present" case.

Beyond that, I think once the comment is here in this method, the "normally" vs. "with one exception" cases can be worked a bit into the corresponding cases in this code's control flow.

That makes a lot of sense, thanks!!

gnprice · 2025-10-17T00:18:50Z

lib/model/message.dart

+    }
+  }
+
+  Message _reconcileWhenPresent(Message current, Message incoming) {


nit: maybe give these names that are a bit more semantic: like "reconcile known message" vs. "reconcile unknown message" (or "new" or "previously unknown"?)

gnprice · 2025-10-17T00:20:35Z

lib/model/message.dart

+      : current;
+  }
+
+  Message _reconcileWhenAbsent(Message incoming) {


nit: maybe put the absent/unknown case first — it happens first chronologically, after all (except in the case of a message we learn about from an event), and it's also simpler, so I think conceptually prior

(can do the same at the call site too, thanks to Dart accepting named arguments before positional)

Thanks to Greg for pointing this out: zulip#1912 (comment)

It's way more common to be using the app with subscribed channels than unsubscribed channels, so we might as well test with subscribed channels except where we specifically want to check behavior for unsubscribed channels.

Like we did in the compose-box tests in the previous commit.

…ulip#1798)

This fixes the "fourth buggy behavior" in zulip#1798: zulip#1798 (comment) Fixes-partly: zulip#1798

chrisbobbe · 2025-10-17T21:13:25Z

Thanks for the review! Revision pushed.

gnprice · 2025-10-18T03:02:45Z

Thanks! I think that revision is helpful.

Over to @rajveermalviya for normal maintainer-review.

rajveermalviya

Thanks @chrisbobbe! LGTM, and tests great.
Moving over to Greg's review.

chrisbobbe · 2025-10-23T15:33:50Z

Thanks for the review!

gnprice

Thanks! Comments below after a first full review.

gnprice · 2025-10-30T19:42:26Z

lib/model/message.dart

+
+      messages[i] = this.messages.update(message.id,


nit:

Suggested change

messages[i] = this.messages.update(message.id,

messages[i] = this.messages.update(message.id,

gnprice · 2025-10-30T19:46:02Z

lib/model/message.dart

+    // This message is one we already know about. This is common and normal:
+    // in particular it happens when one message list overlaps another,
+    // e.g. a stream and a topic within it.


nit: keep the semantic line-breaks the existing comment has:

Suggested change

// This message is one we already know about. This is common and normal:

// in particular it happens when one message list overlaps another,

// e.g. a stream and a topic within it.

// This message is one we already know about.

// This is common and normal: in particular it happens when one message list

// overlaps another, e.g. a stream and a topic within it.

Separately, I think saying "just-fetched" here is helpful for connecting to the later paragraphs, and setting context of what "this" message is:

Suggested change

// This message is one we already know about. This is common and normal:

// in particular it happens when one message list overlaps another,

// e.g. a stream and a topic within it.

// This just-fetched message is one we already know about.

// This is common and normal: in particular it happens when one message list

// overlaps another, e.g. a stream and a topic within it.

gnprice · 2025-10-30T19:58:52Z

lib/model/message.dart

+    if (channelIds.length > 1) {
+      assert(channelIds is! Set);
+      channelIds = Set.from(channelIds); // optimization
+    }


At length 2, for a set of ints, I think List.contains is going to be considerably faster than Set.contains.

The former just has to take the "needle" int in question and compare it against two other ints in turn. The latter has to take a hash of the needle, look to see if there's an entry at the resulting spot in the hash table, then compare the needle against what it finds there.

Similarly for a handful of ints. At a guess, the breakpoint I'd use for this optimization would be 8 rather than 1.

(If this were a likely hot path so the optimization was important then naturally we'd want to measure. But it's definitely not worth that level of effort.)

Oh I see, at one call site channelIds is a List.map iterable rather than a List itself.

I think that doesn't change much. The map callback is just looking up a field on an object; so basically it just adds one load from memory through a pointer. Maybe that tips my guess toward 4 as the breakpoint rather than 8 (I was already waffling a bit between those).

gnprice · 2025-10-30T20:03:16Z

lib/model/message.dart

-    // See [fetchedMessages] for reasoning.
-    messages[event.message.id] = event.message;
+    // See [reconcileMessages] for reasoning.
+    messages[event.message.id] = message;


nit:

Suggested change

messages[event.message.id] = message;

messages[message.id] = message;

gnprice · 2025-10-30T20:08:40Z

test/model/message_test.dart

+    if (isChannelSubscribed) {
+      subscription = eg.subscription(stream);


Should set subscription to null in the alternative case — otherwise the previous test's value for it leaks into this test

gnprice · 2025-10-30T20:11:00Z

test/model/message_test.dart

-      check(messages).single.identicalTo(message);
-      check(store.messages).deepEquals({1: message});
+    group('fetched message with ID already in store.messages', () {
+      late Message messageCopy;


It looks like we can avoid a shared variable here — instead make it local within each test body, by just inserting Message before the first assignment.

gnprice · 2025-10-30T20:11:43Z

test/model/message_test.dart

+      Message copyStoredMessage({String? displayRecipient}) {
+        final message = store.messages.values.single;
+
+        Map<String, dynamic> json = message.toJson();


nit:

Suggested change

Map<String, dynamic> json = message.toJson();

final json = message.toJson();

gnprice · 2025-10-30T20:22:36Z

test/model/message_test.dart

+          messageCopy = copyStoredMessage();
+          store.reconcileMessages([messageCopy]);
+          // The channel became subscribed,
+          // but the message's data hasn't been refreshed, so clobber…
+          checkStoredMessageIdenticalTo(messageCopy);
+
+          store.reconcileMessages([copyStoredMessage()]);
+          // …Now it's been refreshed, by reconcileMessages, so don't clobber.
+          checkStoredMessageIdenticalTo(messageCopy);


This test case is a bit long and fiddly to read. I think it could be simplified with a pair of helpers along these lines:

Suggested change

messageCopy = copyStoredMessage();

store.reconcileMessages([messageCopy]);

// The channel became subscribed,

// but the message's data hasn't been refreshed, so clobber…

checkStoredMessageIdenticalTo(messageCopy);

store.reconcileMessages([copyStoredMessage()]);

// …Now it's been refreshed, by reconcileMessages, so don't clobber.

checkStoredMessageIdenticalTo(messageCopy);

// The channel became subscribed,

// but the message's data hasn't been refreshed, so clobber…

checkClobber();

// …Now it's been refreshed, by reconcileMessages, so don't clobber.

checkNoClobber();

which should make it easier to scan and see all the different scenarios it's going through.

chrisbobbe requested a review from rajveermalviya October 10, 2025 22:38

chrisbobbe assigned rajveermalviya Oct 10, 2025

chrisbobbe added the maintainer review PR ready for review by Zulip maintainers label Oct 10, 2025

chrisbobbe marked this pull request as draft October 10, 2025 23:49

chrisbobbe force-pushed the pr-unsubscribed-channels-fourth-buggy-behavior branch from 3612e86 to 4fa0a1a Compare October 15, 2025 23:49

chrisbobbe marked this pull request as ready for review October 15, 2025 23:49

chrisbobbe mentioned this pull request Oct 15, 2025

In unsubscribed channel, handle that message-send/edit events are not expected #1798

Open

gnprice reviewed Oct 16, 2025

View reviewed changes

chrisbobbe added a commit to chrisbobbe/zulip-flutter that referenced this pull request Oct 16, 2025

message [nfc]: Fix a reference in a comment

73b74ca

Thanks to Greg for pointing this out: zulip#1912 (comment)

chrisbobbe force-pushed the pr-unsubscribed-channels-fourth-buggy-behavior branch from 4fa0a1a to 6ed2244 Compare October 16, 2025 21:04

chrisbobbe force-pushed the pr-unsubscribed-channels-fourth-buggy-behavior branch 3 times, most recently from e8f8054 to a9b4906 Compare October 16, 2025 21:20

gnprice reviewed Oct 17, 2025

View reviewed changes

chrisbobbe added 7 commits October 17, 2025 13:24

message [nfc]: Fix a reference in a comment

bb598a8

Thanks to Greg for pointing this out: zulip#1912 (comment)

msglist test: Switch to using subscribed channels, where boring

56254e8

Like we did in the compose-box tests in the previous commit.

message test [nfc]: Add conditionIdentical helper, for reconcileMessages

4d86c3f

message [nfc]: Pull out _stripMatchFields helper for reconcileMessages

21ac9c6

message [nfc]: Pull out some helpers for reconcileMessages; add TODO(z…

8593e1a

…ulip#1798)

message: Consider unsubscribed/unknown channels in reconcileMessages

6f75b2b

This fixes the "fourth buggy behavior" in zulip#1798: zulip#1798 (comment) Fixes-partly: zulip#1798

chrisbobbe force-pushed the pr-unsubscribed-channels-fourth-buggy-behavior branch from a9b4906 to 6f75b2b Compare October 17, 2025 21:13

rajveermalviya approved these changes Oct 23, 2025

View reviewed changes

rajveermalviya added integration review Added by maintainers when PR may be ready for integration and removed maintainer review PR ready for review by Zulip maintainers labels Oct 23, 2025

rajveermalviya assigned gnprice Oct 23, 2025

rajveermalviya removed their assignment Oct 23, 2025

rajveermalviya requested a review from gnprice October 23, 2025 12:56

gnprice reviewed Oct 30, 2025

View reviewed changes

		// [1] With one exception: if the version we have was in an unsubscribed
		// channel when we got it or sometime since, we take the fetched version


	messages[i] = this.messages.update(message.id,
	messages[i] = this.messages.update(message.id,

	messages[event.message.id] = message;
	messages[message.id] = message;

		if (isChannelSubscribed) {
		subscription = eg.subscription(stream);

	Map<String, dynamic> json = message.toJson();
	final json = message.toJson();

message: Consider unsubscribed channels in reconcileMessages #1912

Are you sure you want to change the base?

message: Consider unsubscribed channels in reconcileMessages #1912

Uh oh!

Conversation

chrisbobbe commented Oct 10, 2025

Uh oh!

chrisbobbe commented Oct 10, 2025

Uh oh!

gnprice left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chrisbobbe commented Oct 16, 2025

Uh oh!

gnprice left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chrisbobbe Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chrisbobbe commented Oct 17, 2025

Uh oh!

gnprice commented Oct 18, 2025

Uh oh!

rajveermalviya left a comment

Choose a reason for hiding this comment

Uh oh!

chrisbobbe commented Oct 23, 2025

Uh oh!

gnprice left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chrisbobbe Oct 17, 2025 •

edited

Loading