VS2022 Test op/magic.t, uni/gv.t, ext/POSIX/t/time.t failure in perl 5.37.4 on Windows 8.1

[twata1]

Description

Hello,

I'am building Perl 5.37.4 on Windows 8.1 64 bit using Visual Studio 2022 Community Edition (Microsoft(R) C/C++ Optimizing Compiler Version 19.33.31630 for x64).

The compilation itself succeeds, although there are some warnings, such as C4244 and C4267.
Several tests fail when running nmake test.

C:\VCPerl64\perl-5.37.4\win32>nmake

(snip)

        if exist ..\..\lib\auto\threads\shared\shared.dll.manifest mt -nologo -manifest ..\..\lib\auto\threads\shared\shared.dll.manifest -outputresource:..\..\lib\auto\threads\shared\shared.dll;2
        if exist ..\..\lib\auto\threads\shared\shared.dll.manifest del ..\..\lib\auto\threads\shared\shared.dll.manifest        "C:\VCPerl64\perl-5.37.4\miniperl.exe" "-I..\..\lib" -MExtUtils::Command -e chmod -- 755 ..\..\lib\auto\threads\shared\shared.dll
Everything is up to date. 'nmake test' to run test suite.

C:\VCPerl64\perl-5.37.4\win32>nmake test

(snip)

Test Summary Report
-------------------
op/magic.t                                                         (Wstat: 0 Tests: 208 Failed: 4)
  Failed tests:  190, 193, 197, 199
uni/gv.t                                                           (Wstat: 0 Tests: 206 Failed: 1)
  Failed test:  78
../dist/Net-Ping/t/450_service.t                                   (Wstat: 0 Tests: 26 Failed: 0)
  TODO passed:   9
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
../ext/POSIX/t/time.t                                              (Wstat: 512 (exited 2) Tests: 19 Failed: 2)
  Failed tests:  9-10
  Non-zero exit status: 2
Files=2730, Tests=1119663, 4274 wallclock secs (94.50 usr + 13.05 sys = 107.55 CPU)
Result: FAIL
NMAKE : fatal error U1077: '.\perl.exe' : return code '0x7'
Stop.

C:\VCPerl64\perl-5.37.4\win32>cd ..\t

C:\VCPerl64\perl-5.37.4\t>.\perl.exe harness op\magic.t
op/magic.t .. 182/208 # Failed test 190 - ENV store takes utf8-encoded key in se
tenv at op/magic.t line 75
#      got ''
# expected /(?^:^(?:X\-Day\ \a\|\?=)?widekey$)/
Use of uninitialized value in substitution (s///) at op/magic.t line 73.
# Failed test 193 - ENV store encodes high utf8 in SV at op/magic.t line 75
#      got ''
# expected /(?^:^(?:foo=)?X\-Day\ \a\|\?$)/
Use of uninitialized value in substitution (s///) at op/magic.t line 73.
Use of uninitialized value in substitution (s///) at op/magic.t line 73.
# Failed test 197 - at op/magic.t line 75
#      got ''
# expected /(?^:^(?:__NoNeSuCh=)?foo$)/
Use of uninitialized value in substitution (s///) at op/magic.t line 73.
# Failed test 199 - at op/magic.t line 75
#      got ''
# expected /(?^:^(?:__NoNeLoCaL=)?foo$)/
Use of uninitialized value in substitution (s///) at op/magic.t line 73.
op/magic.t .. Failed 4/208 subtests
        (less 13 skipped subtests: 191 okay)

Test Summary Report
-------------------
op/magic.t (Wstat: 0 Tests: 208 Failed: 4)
  Failed tests:  190, 193, 197, 199
Files=1, Tests=208,  8 wallclock secs ( 0.06 usr +  0.00 sys =  0.06 CPU)
Result: FAIL

C:\VCPerl64\perl-5.37.4\t>.\perl.exe harness uni\gv.t
uni/gv.t .. 1/206 Unrecognized character \x{3101}; marked by <-- HERE after less \$╚║::<-- HERE near column 24 at -e line 8.
# Failed test 78 - DESTROY was called at uni/gv.t line 319
#      got ''
# expected /(?^u:^Farewell ᴹ=SCALAR)/
uni/gv.t .. Failed 1/206 subtests

Test Summary Report
-------------------
uni/gv.t (Wstat: 0 Tests: 206 Failed: 1)
  Failed test:  78
Files=1, Tests=206,  0 wallclock secs ( 0.05 usr +  0.00 sys =  0.05 CPU)
Result: FAIL

C:\VCPerl64\perl-5.37.4\t>.\perl.exe harness ..\ext\POSIX\t\time.t
../ext/POSIX/t/time.t .. 1/19
#   Failed test 'strftime() can handle unicode chars in the format string'
#   at t/time.t line 70.
Wide character in print at ../../lib/Test2/Formatter/TAP.pm line 125.
#          got: '1970年01朁Ed日'
#     expected: '1970年01月16日'

#   Failed test 'strftime() can handle unicode chars in the format string'
#   at t/time.t line 73.
Wide character in print at ../../lib/Test2/Formatter/TAP.pm line 125.
#          got: '1970年01朁Ed日'
#     expected: '1970年01月16日'
# Looks like you failed 2 tests of 19.
../ext/POSIX/t/time.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/19 subtests
        (less 2 skipped subtests: 15 okay)

Test Summary Report
-------------------
../ext/POSIX/t/time.t (Wstat: 512 (exited 2) Tests: 19 Failed: 2)
  Failed tests:  9-10
  Non-zero exit status: 2
Files=1, Tests=19,  0 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)
Result: FAIL

C:\VCPerl64\perl-5.37.4\t>

Steps to Reproduce

Download perl-5.37.4.tar.gz from metacpan(https://metacpan.org/release/ETHER/perl-5.37.4).
Extract from perl-5.37.4.tar.gz to C:\VCPerl64.
Open C:\VCPerl64\perl-5.37.4\win32\Makefile in your editor and edit as follows.

# Visual C++ 2022 (aka Visual C++ 14.3) (full version or Community Edition)
CCTYPE		= MSVC143

INST_TOP	= $(INST_DRV)\VCPerl64

INST_VER	= \5.37.4

Launch "x64 Native Tools Command Prompt for VS 2022" from the Start menu.
(Note: I run mode con cp select=437 on "x64 Native Tools Command Prompt for VS 2022" for English output.
I have already confirmed that code page 1252 or 932 fails in the same place.)
On that Command Prompt, run the following.

cd C:\VCPerl64\perl-5.37.4\win32
nmake

After nmake is finished , run nmake test on the same x64 Native Tools Command Prompt for VS 2022.

Expected behavior

No tests failure.

Perl configuration

C:\VCPerl64\perl-5.37.4>perl -V
Summary of my perl5 (revision 5 version 37 subversion 4) configuration:
   
  Platform:
    osname=MSWin32
    osvers=6.3.9600
    archname=MSWin32-x64-multi-thread
    uname=''
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cl'
    ccflags ='-nologo -GF -W3 -MD -DWIN32 -D_CONSOLE -DNO_STRICT -DWIN64 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS  -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS'
    optimize='-O1 -Zi -GL -fp:precise'
    cppflags='-DWIN32'
    ccversion='19.33.31630'
    gccversion=''
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=undef
    longlongsize=8
    d_longdbl=define
    longdblsize=8
    longdblkind=0
    ivtype='__int64'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='__int64'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='link'
    ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg  -libpath:"c:\VCPerl64\5.37.4\lib\CORE"  -machine:AMD64 -subsystem:console,"5.02"'
    libpth="C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\\lib\x64"
    libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
    perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
    libc=ucrt.lib
    so=dll
    useshrplib=true
    libperl=perl537.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg  -libpath:"c:\VCPerl64\5.37.4\lib\CORE"  -machine:AMD64 -subsystem:console,"5.02"'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    HAVE_INTERP_INTERN
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_IMPLICIT_SYS
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_THREAD_SAFE_LOCALE
  Built under MSWin32
  Compiled at Oct  8 2022 21:31:26
  @INC:
    C:/VCPerl64/perl-5.37.4/lib

C:\VCPerl64\perl-5.37.4>

Thank you,

[hakonhagland]

I tested this on Windows 11, using code page 850 (OEM Multilingual Latin 1; Western European (DOS)), and all tests passed here:

../lib/Tie/Handle/stdhandle_from_handle.t ............................ ok
../lib/Tie/Hash.t .................................................... ok
../lib/Tie/Scalar.t .................................................. ok
../lib/Tie/SubstrHash.t .............................................. ok
../lib/Time/gmtime.t ................................................. ok
../lib/Time/localtime.t .............................................. ok
../lib/Unicode/testnorm.t ............................................ skipped: Lengthy Tests Disabled; to enable set environmentvariable $ENV{PERL_DEBUG_FULL_TEST} to a true value
../lib/Unicode/UCD.t ................................................. ok
../lib/User/grent.t .................................................. skipped: no getgrgid
../lib/User/pwent.t .................................................. skipped: no getpwuid
../lib/utf8.t ........................................................ ok
../lib/vars.t ........................................................ ok
../lib/vars_carp.t ................................................... ok
../lib/vmsish.t ...................................................... ok
../lib/warnings.t .................................................... ok
All tests successful.

Test Summary Report
-------------------
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2730, Tests=1184944, 1659 wallclock secs (44.03 usr + 11.05 sys = 55.08 CPU)
Result: PASS
        cd ..\win32

[sisyphus]

I, too, get an identical "All tests successful" report on Windows 11 with VS2022.
However, on Windows 7 using VS2019, although all tests passed with 5.37.3, I get some (different) tests failing with 5.37.4.
So it seems that there's something about 5.37.4 that doesn't go well with older Windows systems - but they're totally different failures to what @twata is seeing, and I don't see a connection.
Here's what I have from the failing build of 5.37.4 on the Windows7/VS2019 build:

Test Summary Report
-------------------
../cpan/IPC-Cmd/t/01_IPC-Cmd.t                                     (Wstat: 256 (exited 1) Tests: 20 Failed: 0)
  Non-zero exit status: 1
  Parse errors: No plan found in TAP output
../dist/Net-Ping/t/450_service.t                                   (Wstat: 0 Tests: 26 Failed: 0)
  TODO passed:   9
../dist/Storable/t/code.t                                          (Wstat: 256 (exited 1) Tests: 35 Failed: 0)
  Non-zero exit status: 1
  Parse errors: Bad plan.  You planned 63 tests but ran 35.
../ext/IPC-Open3/t/IPC-Open3.t                                     (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2730, Tests=1119264, 2625 wallclock secs (67.92 usr +  6.57 sys = 74.49 CPU)
Result: FAIL
NMAKE : fatal error U1077: '.\perl.exe' : return code '0x2'
Stop.

The 01_IPC-Cmd.t failure seems to be a "me only" thing that routinely occurs on that box when I build with mingw-w64 compilers.
It hangs and I have to kill it.
I'm not surprised that the same issue should arise with my VS build - the strange thing (IMO) is that it didn't happen with my VS build of 5.37.3.
When I run that test script outside of the Test Harness it runs fine and all tests pass.

The ../dist/Storable/t/code.t failure is totally new - and, in my experience, unique to that Windows7/VS2019 build of perl-5.37.4.
That test script also runs fine outside of the Test Harness.

Seems like @twata's issues might be specific to Windows 8.1 and/or the environment in which the build is being done.

[twata1]

@hakonhagland @sisyphus
Thank you for your response.

I have looked into it since then. It seems to me that it depends on the system locale.
After changing the OS system locale as shown below, the tests now pass.
(However, Other windows application I am using will become Mojibake, so I will change the system locale back.)

1. Open "Control Panel".
2. Click "Clock and Region".
3. Click "Region".
4. Click "Administrative" tab.
5. Click "Change system locale..." button.
6. Change from "Japanese (Japan)" to "English (United States)" using Current system locale drop-down list.
7. Click "OK" button and restart PC.

Test results after performing the above steps.

**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.3.5
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\Program Files\Microsoft Visual Studio\2022\Community>mode con

Status for device CON:
----------------------
    Lines:          300
    Columns:        120
    Keyboard rate:  31
    Keyboard delay: 1
    Code page:      437


C:\Program Files\Microsoft Visual Studio\2022\Community>cd C:\VCPerl64\perl-5.37.4\t

C:\VCPerl64\perl-5.37.4\t>.\perl.exe harness op\magic.t
op/magic.t .. ok
All tests successful.
Files=1, Tests=208,  4 wallclock secs ( 0.03 usr +  0.00 sys =  0.03 CPU)
Result: PASS

C:\VCPerl64\perl-5.37.4\t>.\perl.exe harness uni\gv.t
uni/gv.t .. ok
All tests successful.
Files=1, Tests=206,  0 wallclock secs ( 0.02 usr +  0.00 sys =  0.02 CPU)
Result: PASS

C:\VCPerl64\perl-5.37.4\t>.\perl.exe harness ..\ext\POSIX\t\time.t
../ext/POSIX/t/time.t .. ok
All tests successful.
Files=1, Tests=19,  0 wallclock secs ( 0.00 usr +  0.01 sys =  0.01 CPU)
Result: PASS

C:\VCPerl64\perl-5.37.4\t>

[tonycoz]

I tried setting my system language to Japanese on Windows 1110, but couldn't reproduce this.

[twata1]

Probably it is a issue specific to the Windows 8.1 Japanese edition.
Or I may have missed something.

[twata1]

Detailed information on my English and Japanese settings on Windows 8.1 is below.
Please note that whichever language you try, the keyboard layout may change for that language.

Control Panel -> Clock, Language, and Region - Add a language

Control Panel -> Clock, Language, and Region - Change date, time, or number formats -> Formats tab

Control Panel -> Clock, Language, and Region - Change date, time, or number formats -> Copy settings... button of Administrative tab

Control Panel -> Clock, Language, and Region - Change date, time, or number formats -> Change system locale... button of Administrative tab

[tonycoz]

Probably it is a issue specific to the Windows 8.1 Japanese edition.

I suspect it's a problem with conversion between the default code page (presumably 932) and wide characters, at least in most of the cases.

op/magic.t: each of these accesses the environment, perl uses SetEnvironmentVariableA() under the hood, and in this case we're supplying UTF-8 encoded names, which may not be compatible with the MBCS defined for code page 932, and SetEnvironmentVariableA() fails.

This would be a complex fix.

op/gv.t: the test I looked at uses runperl(), which supplies the program on the command-line, so the supplied UTF-8 encoded text may fail to be converted to wide characters as above.

This could be fixed by using fresh_perl*() which writes the program to a file, which avoids testing the Win32 command-line conversion in addition to what we're trying to test.

ext/POSIX/t/time.t: in this case we're selecting the "C" locale for both LC_CTYPE and LC_TIME, so it should be safe. but the difference between the expected and actual output looks like it might be an encoding round-trip issue. The conversion in ucrt ends up calling MultiByteToWideChar() to convert to wide characters, if the implementation assumed that cp932 was safe to use for the C locale (it's safe for ASCII), the high-bit characters may have failed to round-trip, I believe more than one cp932 encoding of the same unicode character exists in some cases, which can result in changes when the text round-trips like here.

I think we'd need to write our own strftime() which wraps wcsftime(), though I'm not sure what we're testing here is reasonable. @khwilliamson would know better.

[khwilliamson]

time.t is wrong. It is setting the locale to C, and then throwing non-ASCII at it. The C locale knows only about ASCII. And so, is under no obligation to handle code points above 127. The file already requires loc_tools.pl. That utilities file contains find_utf8_ctype_locale() which will return a locale on the system, if any exist, that is a valid UTF-8 locale. The .t file should be revised to use that, and if no locale is returned to skip the tests. Such a locale should properly handle the internal UTF-8 flag, and may fix this issue, but no promises from me. If it doesn't, I will investigate further.

[khwilliamson]

(The C locale does not set the UTF-8 flag for strftime)

[tonycoz]

(The C locale does not set the UTF-8 flag for strftime)

The POSIX::strftime() implementation marks the result as UTF-8 if the format string is marked so. This strikes me as a way to get badly encoded UTF-8 marked SVs, if the current locale isn't UTF-8.

The test seems to be intended to test that this marking occurs, even for non-UTF-8 locales, so using a UTF-8 locale seems like would allow the test to pass even with it "shouldn't"*.

* assuming the behaviour is sane, which it doesn't appear to be for the general case

[khwilliamson]

I hadn't realized that about the format string's UTF-8ness.

This behavior is wrong. I see that this was commented on in p5p more than a decade ago. But until recently there wasn't the infrastructure in our locale handling to adequately deal with that. But now there is

There are actually several cases to consider. I'm confident I know how it should work except for the case where the format is UTF-8, the locale isn't, and the returned string is non-ASCII valid UTF-8. I think it would be best to fail setting errno to EINVAL.

[tonycoz]

I expect POSIX::strftime should be upgrading the format for UTF-8 locales before calling my_strftime8().