BinaryFormatter deprecation issue while using PythonEngine.Shutdown()

Environment

Pythonnet version: 3.0.4
Python version: 3.11
Operating System: Windows 10
.NET Runtime: .NET8/netstandard 2.0

Details

Hi, I am facing a known issue with PythonEngine.Shutdown() that should have been solved. I saw looking around here that many others have had a problem while using the Shutdown function since it caused an exception due to the deprecation of BinaryFormatter in .NET8.
Here the exception I am getting:

System.NotSupportedException: 'BinaryFormatter serialization and deserialization are disabled within this application. See https://aka.ms/binaryformatter for more information.'

I am aware of the workarounds already provided such as the management of "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization" parameter, if I manually manage this set the issue is not presenting anymore, but it seems to me that is more of a patch than a fix.

Since .NET9 will totally get rid of BinaryFormatter, and since I read in other threads that this issue should have been fixed, I was wondering if I was missing something or if the only way to manage this is by manually managing the serialization parameter.

Here follows the code I am using (in a project that targets netstandard2.0, while it is called by a .NET8 project).

using Python.Runtime;
using System;
using System.IO;

namespace ExecutePython
{
    public class Class1
    {
        string filePath;
        
        public Class1(string FilePath) 
        { 
            filePath = FilePath;
            Initialize();
        }

        public void Initialize()
        {
            string pythonDll = @"C:\Users\MyUser\AppData\Local\Programs\Python\Python311\python311.dll";
            Environment.SetEnvironmentVariable("PYTHONNET_PYDLL", pythonDll);
            PythonEngine.Initialize();
        }
        public void ExecutePython()
        {
            string pyCode = File.ReadAllText(filePath);

            using (Py.GIL())
            {
                try
                {
                    PythonEngine.Exec(pyCode);
                }
                catch (Exception ex)
                {
                    //log error
                }
            }
            
            try
            {
                //AppContext.SetSwitch("System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization", true);
                PythonEngine.Shutdown();  
                //AppContext.SetSwitch("System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization", false);
            }
            catch (Exception e)
            { }
        }
    }
}

The code that is executing correctly in my python script is a sample code, in my case it just pops up a window to see that the script is executing correctly

@lostmsu

Ah, I see. The issue is that BinaryFormatter's constructor still works, so my feeble attempt to catch an exception there will not work.

@lostmsu Suggestions?

Should we just switch to the NoopFormatter by default?
Should we default to "non-recoverable" in Shutdown and provide a new method to "shutdown with stashing"?

@filmor

@filmor can't we detect that we are running on .NET Framework/Mono and use BinaryFormatter, otherwise use noo?

True, that's probably the quick-fix. On the other hand, since this is such an uncommon scenario, wouldn't it make sense to adjust the defaults to also get "quicker" shutdown unless one explicitly chooses otherwise?

I'm OK in the future, but right now it would be a breaking change for anyone who happens to use it with Unity.

So, to clarify, is this blocking us from moving to .net 9?

I can also confirm that this issue happens using dotnet 9. The workaround to surround the shutdown method in a try catch statement works so far. Not using the shtutdown method causes that the process endlessly hangs, so it's required even if it throws the exception above.

It would be great if this issue could be fixed. It's not a show-stopper (if you aware of this issue), but it's definitely not elegant.

Fixed BinaryFormatter problem: pythonnet/pythonnet#2469

It's possible to create your own 'NullFormatter' as a replacement, I've tested this using NET 9.0 and the most recent NuGet release of Python.NET (3.0.5) and it didn't cause any issues yet. This is probably not a good idea, so use on your own risk. Out of all possible solutions, I felt like this is the 'cleanest' one. 😅

#pragma warning disable SYSLIB0011
public class NullFormatter : IFormatter
#pragma warning restore SYSLIB0011
{
    public SerializationBinder Binder { get; set; }
    public StreamingContext Context { get; set; }
    #pragma warning disable SYSLIB0050
    public ISurrogateSelector SurrogateSelector { get; set; }
    #pragma warning restore SYSLIB0050

    public object Deserialize(Stream serializationStream)
    {
        return null;
    }

    public void Serialize(Stream serializationStream, object graph)
    {
        // ...
    }
}

IFormatter is unavailable on .NET 8.0 with compile fail.
We can use Harmony to bypass RuntimeData.Stash for prevent error on PythonEngine.Shutdown(), then we can ensure runtime safe exit.

static void ApplyHarmonyPatchToStashForShutdown()
{
    var harmony = new Harmony("com.example.pythonnet.patch.runtimedata.stash");

    var runtimeDataType = typeof(RuntimeData);

    var stashMethod = runtimeDataType.GetMethod("Stash", BindingFlags.Static | BindingFlags.NonPublic)!;
    if (stashMethod == null) throw new Exception("Could not find RuntimeData.Stash method");

    static bool StashReplace()
    {
        Console.WriteLine("Skipping RuntimeData.Stash!");
        return false;  // return false for skip original method
    }

    var prefix = ((Func<bool>)StashReplace).Method;

    harmony.Patch(stashMethod, prefix: new HarmonyMethod(prefix));
    Console.WriteLine("Harmony patch applied to RuntimeData.Stash");
}

Non-breaking workaround for BinaryFormatter deprecation, issue python…

Another workaround:

typeof(Runtime).GetField("ProcessIsTerminating", BindingFlags.Static | BindingFlags.NonPublic)!.SetValue(null, true);

PythonEngine.Shutdown();

The ProcessIsTerminating field is only used to decide whether to run the GC and stash during shutdown, so having it be true seems fine.

This was fixed in #2639

Navigation Menu

Repository navigation

BinaryFormatter deprecation issue while using PythonEngine.Shutdown() #2469

Environment

Details

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BinaryFormatter deprecation issue while using PythonEngine.Shutdown() #2469

Description

Environment

Details

Activity

filmor commented on Oct 1, 2024

lostmsu commented on Oct 1, 2024

filmor commented on Oct 1, 2024

lostmsu commented on Oct 1, 2024

christopher-coy commented on Feb 14, 2025

davidbitterlich commented on Apr 1, 2025

aba-hb commented on Apr 25, 2025

coddoge commented on Sep 15, 2025

carlreinke commented on Apr 1, 2026

filmor commented on May 16, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions