gh-101828: Fix jisx0213 codecs removing null characters

[StanFromIreland]

• Issue: shift_jisx0213 truncates null-terminator #101828

[corona10]

@StanFromIreland Thank you for the investigation. Let me take a look at this weekend :)

[corona10]

LGTM!
This patch prevents NULL bytes from being consumed as part of character pair encoding in the string. While this fixes the data loss bug, it does change existing behavior, so backporting needs discussion.

cc @methane

[corona10]

I will wait Naoki san for a week and plan to merge this PR.

[methane]

euc_jis_2004 has same logic. would you update it too?

>>> "\u00e6abc".encode('euc_jis_2004')
b'\xa9\xdcabc'
>>> "\u00e6\0abc".encode('euc_jis_2004')
b'\xa9\xdcabc'

[StanFromIreland]

euc_jis_2004 has same logic. would you update it too?

Is done.

[methane]

iso2022_jp_3 and iso2022_jp_2004 have same issue.
Would you add this patch?

diff --git a/Modules/cjkcodecs/_codecs_iso2022.c b/Modules/cjkcodecs/_codecs_iso2022.c
index ef6faeb7127..83afdd0a1ee 100644
--- a/Modules/cjkcodecs/_codecs_iso2022.c
+++ b/Modules/cjkcodecs/_codecs_iso2022.c
@@ -802,10 +802,12 @@ jisx0213_encoder(const MultibyteCodec *codec, const Py_UCS4 *data,
         return coded;

     case 2: /* second character of unicode pair */
-        coded = find_pairencmap((ucs2_t)data[0], (ucs2_t)data[1],
-                                jisx0213_pair_encmap, JISX0213_ENCPAIRS);
-        if (coded != DBCINV)
-            return coded;
+        if (data[1] != 0) { /* Don't consume null char as part of pair */
+            coded = find_pairencmap((ucs2_t)data[0], (ucs2_t)data[1],
+                                    jisx0213_pair_encmap, JISX0213_ENCPAIRS);
+            if (coded != DBCINV)
+                return coded;
+        }
         _Py_FALLTHROUGH;