[API Proposal]: Overriding the default behavior in case of unhandled exceptions and fatal errors.

[VSadov]

Re: The previous proposal and a discussion that led to this proposal - (#42275)

Background and motivation

The current default behavior in a case of unhandled exception is termination of a process.
The current default behavior in a case of a fatal error is print exception to console and invoke Watson/CrashDump.

While satisfactory to the majority of uses, the scheme is not flexible enough for some classes of scenarios.

Scenarios like Designers, REPLs or game scripting that host user provided code are not able to handle unhandled exceptions thrown by the user provided code. Unhandled exceptions on finalizer thread, threadpool threads or user created threads will take down the whole process. This is not desirable experience for these types of scenarios.

In addition, there are customers that have existing infrastructure for postmortem analysis of failures and inclusion of .NET components requires interfacing with or overriding the way the fatal errors are handled.

API Proposal

API for process-wide handling of unhandled exception

namespace System.Runtime.ExceptionServices
{
    public delegate bool UnhandledExceptionHandler(System.Exception exception);

    public static class ExceptionHandling
    {
        /// <summary>
        /// Sets a handler for unhandled exceptions.
        /// </summary>
        /// <exception cref="ArgumentNullException">If handler is null</exception>
        /// <exception cref="InvalidOperationException">If a handler is already set</exception>
        public static void SetUnhandledExceptionHandler(UnhandledExceptionHandler handler);
    }
}

The semantics of unhandled exception handler follows the model of imaginary handler like the following inserted in places where the exception will not lead to process termination regardless of what handler() returns.

try { UserCode(); } catch (Exception ex) when handler(ex){};

In particular:

• only exceptions that can be caught and ignored will cause the handler to be invoked. (i.e. stack overflow will not)
• an unhandled exception thrown in a handler will not invoke the handler, but will be treated as returning false.
• when an exception is handled via a handler in a user-started thread, the thread will still exit (but not escalate to process termination)
• when an exception is handled in a task-like scenario on an infrastructure thread (thread pool, finalizer queue...), the execution of tasks will continue.
  (Whether the infrastructure thread continues or restarted is unspecified, but the process should be able to proceed)
• a reverse pinvoke will not install the try/catch like above.
• main() will not install the try/catch like above

API Proposal for custom handling of fatal errors

Managed API to set up the handler.

namespace System.Runtime.ExceptionServices
{
    public static class ExceptionHandling
    {
        /// <summary>
        /// .NET runtime is going to call `fatalErrorHandler` set by this method before its own
        /// fatal error handling (creating .NET runtime-specific crash dump, etc.).
        /// </summary>
        /// <exception cref="ArgumentNullException">If fatalErrorHandler is null</exception>
        /// <exception cref="InvalidOperationException">If a handler is already set</exception>
        public static void SetFatalErrorHandler(delegate* unmanaged<int, void*, int> fatalErrorHandler);
    }
}

The shape of the FatalErrorHandler, if implemented in c++
(the default calling convention for the given platform is used)

// expected signature of the handler
FatalErrorHandlerResult FatalErrorHandler(int32_t hresult, struct FatalErrorInfo* data);

With FatalErrorHandlerResult and FatalErrorInfo defined in "FatalErrorHandling.h" under src/native/public:

enum FatalErrorHandlerResult : int32_t
{
    RunDefaultHandler = 0,
    SkipDefaultHandler = 1,
};

#if defined(_MSC_VER) && defined(_M_IX86)
#define DOTNET_CALLCONV __stdcall
#else
#define DOTNET_CALLCONV
#endif

struct FatalErrorInfo
{
    size_t size;    // size of the FatalErrorInfo instance
    void*  address; // code location correlated with the failure (i.e. location where FailFast was called)

    // exception/signal information, if available
    void* info;     // Cast to PEXCEPTION_RECORD on Windows or siginfo_t* on non-Windows.
    void* context;  // Cast to PCONTEXT on Windows or ucontext_t* on non-Windows.

    // An entry point for logging additional information about the crash.
    // As runtime finds information suitable for logging, it will invoke pfnLogAction and pass the information in logString.
    // The callback may be called multiple times.
    // Combined, the logString will contain the same parts as in the console output of the default crash handler.
    // The errorLog string will have UTF-8 encoding.
    void (DOTNET_CALLCONV *pfnGetFatalErrorLog)(
           FatalErrorInfo* errorData, 
           void (DOTNET_CALLCONV *pfnLogAction)(char8_t* logString, void *userContext), 
           void* userContext);

    // More information can be exposed for querying in the future by adding
    // entry points with similar pattern as in pfnGetFatalErrorLog
};

API Usage

Setting up a handler for unhandled exceptions:

using System.Runtime.ExceptionServices;

ExceptionHandling.SetUnhandledExceptionHandler(
    (ex) =>
    {
        if (DesignMode)
        {
            DisplayException(ex);
            // the exception is now "handled"
            return true;
        }
        return false;
    }
);

Setting up a handler for fatal errors:

Setting up the handler for the process (C# code in the actual app):

internal unsafe class Program
{
    [DllImport("myCustomCrashHandler.dll")]
    public static extern delegate* unmanaged<int, void*, int> GetFatalErrorHandler();

    static void Main(string[] args)
    {
        ExceptionHandling.SetFatalErrorHandler(GetFatalErrorHandler());

        RunMyProgram();
    }
}

The handler. (c++ in myCustomCrashHandler.dll)

#include "FatalErrorHandling.h"

static FatalErrorHandlerResult FatalErrorHandler(int32_t hresult, struct FatalErrorInfo* data)
{
    // this is a special handler that analyzes OOM crashes
    if (hresult != COR_E_OUTOFMEMORY)
    {
        return FatalErrorHandlerResult::RunDefaultHandler;
    }

    DoSomeCustomProcessingOfOOM(data);

    // retain the additional error data
    data->pfnGetFatalErrorLog(data, &LogErrorMessage, NULL);

    // no need for a huge crash dump after an OOM.
    return FatalErrorHandlerResult::SkipDefaultHandler;
}

static void LogErrorMessage(char8_t* logString, void *userContext)
{
    AppendToBlob(logString);
}

extern "C" DLL_EXPORT void* GetFatalErrorHandler()
{
    return &FatalErrorHandler;
}

Alternative Designs

Unmanaged hosting API that enables this behavior. (CoreCLR has undocumented and poorly tested configuration option for this today. #39587. This option is going to be replaced by this API.)

Extending AppDomain.CurrentDomain.UnhandledException API and make IsTerminating property writeable to allow "handling".
Upon scanning the existing use of this API it was found that IsTerminating is often used as a fact - whether an exception is terminal or not. Changing the behavior to mean "configurable" will be a breaking change to those uses.

Risks

This APIs can be abused to ignore unhandled exceptions or fatal errors in scenarios where it is not warranted.

[AaronRobinsonMSFT]

FatalErrorHandlerResult FatalErrorHandler(int32_t hresult, struct FatalErrorInfo* data);

Defines int32_t, but the C# function pointer in SetFatalErrorHandler() defined uint. I assume both should be signed.

[janvorli]

I wonder if having the messages etc. as 16 bit strings is the right way. Our coreclr_xxx hosting APIs use 8 bit (UTF-8) strings so that users on Unix don't have to convert them using some external library. 16 bit strings are very unusual on Unix and on Windows, it is trivial to do the conversion if needed thanks to the Windows APIs.

[VSadov]

We had to choose between char_t and char16_t. See: #42275 (comment)

The key observations were -

• unlike hosting, where strings originate from the native host and thus in the platform-specific form, these originate from the runtime and they are in UTF-16 form.
• the process might be in somewhat precarious state, so we'd like to minimize the work that is done "inline".
  For example the strings could be uninteresting to the given handler anyways, or could be stored as-is and converted later.

[janvorli]

In case the native handler on Unix would want to do anything sensible with these strings, even just writing them to a text log file, I think it would most likely need to convert them. In the runtime, we can easily convert them into stack allocated buffers without disturbing the process state. I still think it would be better if whatever string enters or leaves the runtime from the native side was 8 bit. Similar to the hosting APIs, it would add a little overhead for the handler on Windows, but much less than the overhead that would otherwise be needed on Unix.
I am not sure I understand the comment in the other issue on a need to have some PAL definitions if we went with 8 bit strings.

[AaronRobinsonMSFT]

I am not sure I understand the comment in the other issue on a need to have some PAL definitions if we went with 8 bit strings.

The approach in question was to have a char_t that represented wchar_t on Windows and char on non-Windows. If we wanted to make it always UTF-8, I could get behind that. That seems rather annoying, but I do get it. My desire is to avoid any sort of PAL types that users would need to think about. Casting doesn't concern me, but encoding types in signatures or data structures would be a red flag to me.

[VSadov]

How complex is UTF-16 to UTF-8 conversion? Is there IO or allocations?

[AaronRobinsonMSFT]

How complex is UTF-16 to UTF-8 conversion? Is there IO or allocations?

There will be allocations for sure. The IO is nil or very limited if done from managed code if I recall. The native side trans-coding is also new and I believe avoids IO. It isn't the worst idea if it is just for logging. The tricky bit is if the callback comes back into managed though - that is my biggest concern.