Skip to content

[Bug] Python 3.13 - retry policy on Event Hub trigger not working using v2 programming model #1823

@MihailV1989

Description

@MihailV1989

Expected Behavior

Functions in Python 3.13 decorated with @app.retry(...) run and upon exceptions re-run correctly.

Actual Behavior

Functions in Python 3.13 decorated with @app.retry(...) cannot start.

Steps to Reproduce

  1. Create a new Function Project in Python 3.13 using VS Code in Win 11, the v2 programming model and choose to use a "EventHub trigger" template
  2. In tasks.json add "--verbose" to the task's command like this "command": "host start --verbose", so you can see additional logs
  3. Configure the @app.event_hub_message_trigger() properly, in my case I was using an Microsoft Entra identity
  4. run the function to see that it is working
  5. add @app.retry() decorator to the function and configure at least the strategy and max_retry_count args
  6. run the function again to see that it is not working anymore
  7. now add "PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION": "python" to local.setting.json
  8. run again function app and you'll see a proper traceback pointing to a problem with build_variable_interval_retry() or build_fixed_delay_retry() in loader.py from the azure functions runtime
  9. optionally you can try to run the Function App in a Debian devcontainer, there I also got the same problem

Relevant code being tried

import azure.functions as func
import logging
import os


EVENTHUB__fullyQualifiedNamespace = os.environ["EVENTHUB__fullyQualifiedNamespace"]
REQUEST_EVENT_HUB_NAME = os.environ["REQUEST_EVENT_HUB_NAME"]
REQUEST_CONSUMER_GROUP_NAME = os.environ[
    "REQUEST_CONSUMER_GROUP_NAME"
]

app = func.FunctionApp()

@app.retry(
    strategy="exponential_backoff",
    max_retry_count="3",
    # manually setting the retry intervals will also not work
    # minimum_interval="00:00:02",
    # maximum_interval="00:01:00",
)
@app.event_hub_message_trigger(
    arg_name="azeventhub",
    event_hub_name=REQUEST_EVENT_HUB_NAME,
    connection="EVENTHUB",
    consumer_group=REQUEST_CONSUMER_GROUP_NAME,
) 
def eventhub_trigger(azeventhub: func.EventHubEvent):
    body = azeventhub.get_body().decode("utf-8")
    logging.info(f'Python EventHub trigger processed an event: {body}')

Relevant log output

# The log output can be different. I'm posting here only the (imho) important lines of the logs from the different configurations I've tried.

## 1. Without setting the "PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION": "python" in local.settings.json it is this:

Found Python version 3.13.11 (py).
Selected out-of-process host.

Core Tools Version:       4.6.0+ab90faafcab539d63cd3d0ce5faf1bca4395fccc (64-bit)
Function Runtime Version: 4.1045.200.25556

Skipping 'AzureWebJobsStorage' from local settings as it's already defined in current environment variables.
[2026-02-10T13:13:13.910Z] Applying platform release channel configuration LATEST. Bundle version 4.30.0 will be used
[2026-02-10T13:13:13.911Z] Found a matching extension bundle at C:\Users\DEVALMIH\.azure-functions-core-tools\Functions\ExtensionBundles\Microsoft.Azure.Functions.ExtensionBundle\4.30.0     
[2026-02-10T13:13:13.939Z] Loading functions metadata
[2026-02-10T13:13:13.942Z] Worker indexing is enabled
[2026-02-10T13:13:13.944Z] Fetching metadata for workerRuntime: python
[2026-02-10T13:13:13.945Z] Reading functions metadata (Worker)
[2026-02-10T13:13:14.886Z]  INFO: Starting proxy worker. Worker ID: e7fbc598-8e33-407c-8a55-8a65201468ff, Request ID: bd4fcd34-a0a5-46a0-84f1-b20d4362998b, Host Address: 127.0.0.1:55356     
[2026-02-10T13:13:15.064Z]  INFO: Successfully opened gRPC channel to 127.0.0.1:55356 
[2026-02-10T13:13:15.066Z]  INFO: Detaching console logging.
[2026-02-10T13:13:17.122Z] Switched to gRPC logging.
[2026-02-10T13:13:17.136Z] Received WorkerInitRequest, python version 3.13.11 (tags/v3.13.11:6278944, Dec  5 2025, 16:26:58) [MSC v.1944 64 bit (AMD64)], worker version 4.40.1, request ID bd4fcd34-a0a5-46a0-84f1-b20d4362998b.
[2026-02-10T13:13:17.145Z] Finished prioritize_customer_dependencies: worker_dependencies_path: C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64, customer_dependencies_path: C:\Users\***\PycharmProjects\delete_me2\.venv\Lib\site-packages, working_directory: C:\Users\***\PycharmProjects\delete_me2, Placeholder: False, sys.path: ['C:\\Users\\***\\PycharmProjects\\delete_me2\\.venv\\Lib\\site-packages', 'C:\\Program Files\\Microsoft\\Azure Functions Core Tools\\workers\\python\\3.13/WINDOWS/X64', 'C:\\Users\\***\\AppData\\Local\\Programs\\Python\\Python313\\python313.zip', 'C:\\Users\\***\\AppData\\Local\\Programs\\Python\\Python313\\DLLs', 'C:\\Users\\***\\AppData\\Local\\Programs\\Python\\Python313\\Lib', 'C:\\Users\\***\\AppData\\Local\\Programs\\Python\\Python313', 'C:\\Users\\***\\PycharmProjects\\delete_me2\\.venv', 'C:\\Program Files\\Microsoft\\Azure Functions Core Tools\\workers\\python\\3.13\\WINDOWS\\X64', 'C:\\Users\\***\\PycharmProjects\\delete_me2']
[2026-02-10T13:13:17.182Z] Using library: <module 'azure_functions_runtime' from 'C:\\Program Files\\Microsoft\\Azure Functions Core Tools\\workers\\python\\3.13/WINDOWS/X64\\azure_functions_runtime\\__init__.py'>, library version: 1.1.0b1
[2026-02-10T13:13:17.829Z] Language Worker Process exited. Pid=13012.
[2026-02-10T13:13:17.830Z] py exited with code -1073741819 (0xC0000005). .



## 2. If I set "PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION": "python" the log output is the same until the last two lines. After that I get:


[2026-02-10T13:16:18.084Z] Initializing worker process failed
[2026-02-10T13:16:18.085Z] Result: Failure
Type:
Exception: TypeError: strptime() argument 0 must be str, not <class 'NoneType'>
Stack:   File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 99, in worker_init_request
    load_function_metadata(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 410, in load_function_metadata
    _metadata_result = (index_functions(function_path, function_app_directory)) \
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 420, in index_functions
    process_indexed_function(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 140, in process_indexed_function
    retry_protos = build_retry_protos(protos, indexed_function)
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 61, in build_retry_protos
    return build_variable_interval_retry(protos, retry, max_retry_count,
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 104, in build_variable_interval_retry
    seconds=convert_to_seconds(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 32, in convert_to_seconds
    x = time.strptime(timestr, '%H:%M:%S')
  File "C:\Users\***\AppData\Local\Programs\Python\Python313\Lib\_strptime.py", line 783, in _strptime_time
    tt = _strptime(data_string, format)[0]
  File "C:\Users\***\AppData\Local\Programs\Python\Python313\Lib\_strptime.py", line 524, in _strptime
    raise TypeError(msg.format(index, type(arg)))
.
[2026-02-10T13:16:18.089Z] Failed to start language worker process for runtime: python. workerId:f7d730d4-880b-4c07-ac48-6780737dadd9
[2026-02-10T13:16:18.090Z] Removing errored webhost language worker channel for runtime: python workerId:f7d730d4-880b-4c07-ac48-6780737dadd9
[2026-02-10T13:16:18.091Z] System.Private.CoreLib: Result: Failure
Type:
Exception: TypeError: strptime() argument 0 must be str, not <class 'NoneType'>
Stack:   File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 99, in worker_init_request
    load_function_metadata(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 410, in load_function_metadata
    _metadata_result = (index_functions(function_path, function_app_directory)) \
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 420, in index_functions
    process_indexed_function(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 140, in process_indexed_function
    retry_protos = build_retry_protos(protos, indexed_function)
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 61, in build_retry_protos
    return build_variable_interval_retry(protos, retry, max_retry_count,
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 104, in build_variable_interval_retry
    seconds=convert_to_seconds(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 32, in convert_to_seconds
    x = time.strptime(timestr, '%H:%M:%S')
  File "C:\Users\***\AppData\Local\Programs\Python\Python313\Lib\_strptime.py", line 783, in _strptime_time
    tt = _strptime(data_string, format)[0]
  File "C:\Users\***\AppData\Local\Programs\Python\Python313\Lib\_strptime.py", line 524, in _strptime
    raise TypeError(msg.format(index, type(arg)))
.
[2026-02-10T13:16:18.100Z] 0 functions loaded

# [here there were a lot of lines showing different configurations]

[2026-02-10T13:16:18.501Z] Starting JobHost
[2026-02-10T13:16:18.504Z] Starting Host (HostId=delt01353-157237864, InstanceId=2304af69-73f2-4daa-b481-8b1f3d1948d6, Version=4.1045.200.25556, ProcessId=47672, AppDomainId=1, InDebugMode=False, InDiagnosticMode=False, FunctionsExtensionVersion=(null))
[2026-02-10T13:16:18.508Z] Loading functions metadata
[2026-02-10T13:16:18.509Z] Worker indexing is enabled
[2026-02-10T13:16:18.510Z] Fetching metadata for workerRuntime: python
[2026-02-10T13:16:18.511Z] Reading functions metadata (Worker)
[2026-02-10T13:16:19.315Z] Traceback (most recent call last):
[2026-02-10T13:16:19.317Z]   File "<frozen runpy>", line 198, in _run_module_as_main
[2026-02-10T13:16:19.318Z]   File "<frozen runpy>", line 88, in _run_code
[2026-02-10T13:16:19.319Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy\__main__.py", line 71, in <module>
[2026-02-10T13:16:19.319Z]     cli.main()
[2026-02-10T13:16:19.320Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\cli.py", line 508, in main
[2026-02-10T13:16:19.321Z]     run()
[2026-02-10T13:16:19.322Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\cli.py", line 342, in run_file
[2026-02-10T13:16:19.324Z]     start_debugging(target)
[2026-02-10T13:16:19.325Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\cli.py", line 328, in start_debugging    
[2026-02-10T13:16:19.325Z]     debugpy.listen(options.address)
[2026-02-10T13:16:19.327Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\public_api.py", line 47, in wrapper
[2026-02-10T13:16:19.328Z]     return wrapped(*args, **kwargs)
[2026-02-10T13:16:19.329Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\api.py", line 133, in debug
[2026-02-10T13:16:19.331Z]     log.reraise_exception("{0}() failed:", func.__name__, level="info")
[2026-02-10T13:16:19.332Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\api.py", line 131, in debug
[2026-02-10T13:16:19.333Z]     return func(address, settrace_kwargs, **kwargs)
[2026-02-10T13:16:19.334Z]   File "c:\Users\***\.vscode\extensions\ms-python.debugpy-2025.18.0-win32-x64\bundled\libs\debugpy/..\debugpy\server\api.py", line 260, in listen
[2026-02-10T13:16:19.336Z]     raise RuntimeError(str(endpoints["error"]))
[2026-02-10T13:16:19.337Z] RuntimeError: Can't listen for client connections: [WinError 10048]
[2026-02-10T13:16:19.359Z] Language Worker Process exited. Pid=52912.
[2026-02-10T13:16:19.360Z] py exited with code 1 (0x1).     log.reraise_exception("{0}() failed:", func.__name__, level="info"),    raise RuntimeError(str(endpoints["error"])),RuntimeError: Can't listen for client connections: [WinError 10048]
[2026-02-10T13:16:19.369Z] Exceeded language worker restart retry count for runtime:python. Shutting down and proactively recycling the Functions Host to recover
[2026-02-10T13:16:23.490Z] Host lock lease acquired by instance ID '0000000000000000000000009AE3DE10'.
[2026-02-10T13:17:18.521Z] Starting worker process failed
[2026-02-10T13:17:18.523Z] Microsoft.Azure.WebJobs.Script.Grpc: The operation has timed out.
[2026-02-10T13:17:18.525Z] Failed to start language worker process for runtime: python. workerId:5c73827d-09ea-4e86-b595-c15261bbb78a
[2026-02-10T13:17:18.531Z] Reading functions metadata (Custom)
[2026-02-10T13:17:18.548Z] 0 functions found (Custom)


## 3. With minimum_interval and maximum_interval set on the @app.retry() the stack trace is a bit different:


[2026-02-10T14:54:51.715Z] System.Private.CoreLib: Result: Failure
Type:
Exception: AttributeError: Fail to convert to Duration. Expected a timedelta like object got Duration: 'Duration' object has no attribute 'days'
Stack:   File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 99, in worker_init_request
    load_function_metadata(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 410, in load_function_metadata
    _metadata_result = (index_functions(function_path, function_app_directory)) \
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\handle_event.py", line 420, in index_functions
    process_indexed_function(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 140, in process_indexed_function
    retry_protos = build_retry_protos(protos, indexed_function)
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 61, in build_retry_protos
    return build_variable_interval_retry(protos, retry, max_retry_count,
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\azure_functions_runtime\loader.py", line 111, in build_variable_interval_retry
    return protos.RpcRetryOptions(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\google\protobuf\internal\python_message.py", line 560, in init
    field_copy._internal_assign(field_value)
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\google\protobuf\internal\well_known_types.py", line 449, in _internal_assign
    self.FromTimedelta(td)
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.13/WINDOWS/X64\google\protobuf\internal\well_known_types.py", line 443, in FromTimedelta
    raise AttributeError(
.

requirements.txt file

azure-functions

Where are you facing this problem?

Local - Core Tools in Windows 11 and in a Debian devcontainer

Function app name

No response

Additional Information

I've searched but didn't found any information if the retry policies are depricated in Python 3.13. So the aren't right? Because there is an issue that it states they were removed temporary from azure-functions 1.17.0.

I've also tried to use Copilot to trobleshoot my problem and I was told that the problem is incompatibility with the protobuf package. Namely, only in a devcontainer I get the following additional log:

Attempt to remove module cache for google._upb but failed with 'google'. Using the original module cache.

Because of this error I was suggested to try to use the Python native implementation of protobuf package by setting PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to python.

I also ran the app with azure_functions_runtime downgraded to 1.0.0 (last stable version) by adding it to the requirements.txt, but the problem was still there.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions