Formatter: Keep right-hanging comments aligned

[warsaw]

ruff format follows black's style for right-hanging comments (i.e. inline comments), but that leads to poorly readable results, especially when a series of consecutive lines uses comment alignment by column. Blue fixes this readability problem.

Let's start with this file:

def foo():
    x = 1                                         # this comment
    why = 2                                       # aligns with this comment
    zebra = 3                                     # and this one

As you can see, every # on each of these lines appears at the same column. How does ruff reformat the line?

def foo():
    x = 1  # this comment
    why = 2  # aligns with this comment
    zebra = 3  # and this one

This is the way black does it, i.e. by jamming all comment starts to two space characters between the last code character and the #. This destroys the alignment.

How does blue format the file? Nothing changes! Blue preserves the number of spaces between the last code character and the # with the assumption that the spacing is deliberate and done for readability.

ruff format should follow blue's rule here.

[warsaw]

Now that the quote handling (#7310) has been fixed in 0.0.292, this is the last formatting incompatibility that I've been able to find between ruff and blue. Once this ticket is resolved I can officially start promoting ruff as a replacement for blue. Great work y'all and thank you!

[zanieb]

I have also been annoyed with this when using Black. I'm not sure if there's a downside to supporting this. Ideally it would only apply if they start out aligned. Additionally, it'd be cool if it shrunk to the minimum compatible distance for alignment — but that's a little magical.

[charliermarsh]

Enforcing this conditionally seems really hard... and enforcing this globally by default would be a big deviation from Black. I suspect it would need to be a global setting, if we choose to support it.

[JelleZijlstra]

For what it's worth, I have consistently rejected this sort of feature in Black, as I don't think it leads to a consistent, stable formatting style. Obviously, what style Ruff chooses is up to you.

Barry's proposed rule is that the number of spaces before the hash gets preserved. But what if someone comes along and decides that zebras are worth more:

def foo():
    x = 1                                         # this comment
    why = 2                                       # aligns with this comment
    zebra = 31                                     # and this one

Now either the code gets misaligned (ugly), or they have to manually fiddle with the spaces, which defeats the point of an autoformatter.

Now, possibly the formatter could be made smart enough to detect that the comments are misaligned, and fix the spacing on the zebra line to get them aligned again.

But now what if we decide that zebras are worth a lot more:

def foo():
    x = 1                                         # this comment
    why = 2                                       # aligns with this comment
    zebra = 300000000000000000000000000000000000000  # and this one

Then, to get the lines aligned again, the formatter would have to change the x and why line, creating noise in the diff.

[warsaw]

There's a little bit of missing context that might be helpful. In python-mode you can set a comment column, and then hit M-; (I'm not sure if that's a default keybinding, but it's set to comment-dwim for me) and Emacs will do it's best to ensure that hanging comment starts at that column. Of course in @JelleZijlstra 's second example, it will put a minimum number of spaces there but they won't align.

So @JelleZijlstra 's first example, it's relatively easy, albeit manual, to re-indent everything to an aligning column. Emacs only looks at the current, line so it doesn't actively try to keep things lined up, but the effect is usually that they are. The preservation of whitespace rule that blue implements works well with this approach.

It's never going to be perfect, but I'd argue that blue's rule is better than the step-wise always-2-spaces rule that black (and currently ruff) implements because that leads to difficult to read lines. It's worse with black though because you don't even have the option of manually aligning those hanging comments -- black will just crush them.

[zanieb]

Hm... a comment-column setting is an interesting approach. Overall this sounds pretty difficult to implement and I agree that manually fiddling with the alignment would be a disappointing outcome. I appreciate all of the extra context!

[konstin]

I'd prefer not to support this, i think comments directly after the statements or alternatively leading own line comments (where they are column aligned) are fine enough to read. For the the case of not modifying whitespace i agree with @JelleZijlstra that would defeat the point of an autoformatter while for the case of column aligning i don't think the possible improvement warrants introducing a column layout when we don't use one elsewhere¹.

This is also bad from a point of locality, currently the formatting of one statement does not influence the formatting of other statements (we see those comments as trailing on the statement).

---

¹With column layout, i mean e.g.

class Foo:
    x     = 1                  # this comment
    why   = 22                 # aligns with this comment
    zebra = 333                # and this one

[warsaw]

Hm... a comment-column setting is an interesting approach

Thinking about this overnight, this could be the best way forward. It would be a local-to-the-line only setting -- no need for global parsing or alignment. It also, um, aligns more with the Emacs behavior. ruff would do its best to preserve comment-column but in the "big zebra" case above, it would just push the hanging comment out to the right. That's fine because the user would still have a lot of flexibility in getting the format they want.

Straw man proposal:

• comment-column = 60 - best effort to align the hanging comment # on column 60, the Emacs behavior
• comment-column = +2 - a relative setting, i.e. the current black/ruff default
• comment-column = false - preserve existing whitespace, i.e. the blue behavior

[effigies]

The top two reasons I use blue and not black are single quotes and not messing with comment spacing. I don't mind bumping the spaces to 2, but removing additional spaces is a real pain. If I'm using more, there's pretty much always a reason.

Here's an example that even blue currently damages (blue fixed the single-line expression case). These are pairs of field names and numpy dtype codes, followed by comments:

 header_dtd = [
-    ('sizeof_hdr', 'i4'),      # 0; must be 348
-    ('data_type', 'S10'),      # 4; unused
-    ('db_name', 'S18'),        # 14; unused
-    ('extents', 'i4'),         # 32; unused
-    ('session_error', 'i2'),   # 36; unused
-    ('regular', 'S1'),         # 38; unused
-    ('dim_info', 'u1'),        # 39; MRI slice ordering code
-    ('dim', 'i2', (8,)),       # 40; data array dimensions
+    ('sizeof_hdr', 'i4'),  # 0; must be 348
+    ('data_type', 'S10'),  # 4; unused
+    ('db_name', 'S18'),  # 14; unused
+    ('extents', 'i4'),  # 32; unused
+    ('session_error', 'i2'),  # 36; unused
+    ('regular', 'S1'),  # 38; unused
+    ('dim_info', 'u1'),  # 39; MRI slice ordering code
+    ('dim', 'i2', (8,)),  # 40; data array dimensions
 ]

I think it's hard to argue that this change makes things more readable. I understand that preserving an aligned column is difficult, and I really don't care if an autoformatter chooses not to try. But if I go through the trouble of making an aligned column of comments, it's frustrating for an autoformatter to destroy it and give no option to just leave it alone. The PR I opened for blue about this (grantjenks/blue#83) still enforces at least two spaces, but otherwise leaves inline comment spacing alone.

[dhirschfeld]

Destroying carefully crafted comment spacing can make code harder to parse/understand which in turn can make it easier for bugs to creep in.

Some people don't use comment spacing to make their code easier to visually parse and for them keeping the option of enforcing 2-spaces is fine. For those who do use spacing it would be nice to have the option to at least turn off any messing with comment spacing at all. I'd much rather manually align my comments (or not) than have an auto-formatter make my code harder to understand and maintain.

The ability to easily parse the code and therefore more easily spot bugs is also at the core of Jarrod's request to not mess with numpy spacing (which I'm also interested in, for the same reasons).

I'd be fine with switches such as --no-align-comments, --no-align-numpy to globally turn off autoformatting for those use cases.