feat: add Litestar framework support and related middleware

[pythonhubdev]

Summary of change

(A few sentences about this PR)

Related issues

• Support LiteStar #310

Test Plan

(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work. Bonus points for screenshots and videos!)

Documentation changes

(If relevant, please create a PR in our docs repo, or create a checklist here highlighting the necessary changes)

Checklist for important updates

• Changelog has been updated
• coreDriverInterfaceSupported.json file has been updated (if needed)
  • Along with the associated array in supertokens_python/constants.py
• frontendDriverInterfaceSupported.json file has been updated (if needed)
• Changes to the version if needed
  • In setup.py
  • In supertokens_python/constants.py
• Had installed and ran the pre-commit hook
• Issue this PR against the latest non released version branch.
  • To know which one it is, run find the latest released tag (git tag) in the format vX.Y.Z, and then find the latest branch (git branch --all) whose X.Y is greater than the latest released tag.
  • If no such branch exists, then create one from the latest released branch.
• If have added a new web framework, update the supertokens_python/utils.py file to include that in the FRAMEWORKS variable
• If added a new recipe that has a User type with extra info, then be sure to change the User type in supertokens_python/types.py
• Make sure that syncio / asyncio functions are consistent.
• If access token structure has changed
  • Modified test in tests/sessions/test_access_token_version.py to account for any new claims that are optional or omitted by the core

Remaining TODOs for this PR

• Add framework/litestar/__init__.py
• Add litestar to
  • setup.py
  • Makefile
• Support litestar in website tests
  • Setup tests/frontendIntegration/litestar
  • Ensure supertokens-website tests run with litestar
• Support litestar in auth-react tests
  • Setup tests/auth-react/litestar
  • Ensure supertokens-auth-react tests run with litestar

[pythonhubdev]

Hi @rishabhpoddar @namsnath I have made some changes to support litestar but the tests are failing need some help here since I couldn't debug where the issue is exactly coming from can any one guide me through what am I doing wrong?

[promptless]

📝 Documentation updates detected!

New suggestion: Add Litestar framework support documentation

[namsnath]

Hey @pythonhubdev ,
Thanks for the contribution!

Which test suite has the failures? Any specific tests?

I can take a look at this in detail sometime next week.

[pythonhubdev]

Hi @namsnath thanks for the response so except the top tests all other tests are failing. Mainly the tests that are calling the internal API of supertokens are failing but they are passing in FastAPI I don't know if I have to mount / attach the app or something so mostly it's 404 a little guidance help me I can fix it and the other issues are 500 Internal Server Error I don't know exactly why it's failing.

[pythonhubdev]

Hi @namsnath I have updated the test and modified the Litestar middleware to use ASGIMiddleware as well but the routes related to supertoken i.e all the routes following /auth are NOT FOUND. Can you help me in fixing this even if you guide me where I have to look into I can take it from there. Thanks

[namsnath]

Hey @pythonhubdev ,

I see that you're using the domain as testserver.local. This is probably the cause of the 404s you're seeing in tests.
This is probably not the issue. I will look into this and get back.

When you fix this, please also make sure the code works in the lowest supported version of Python (3.8). You seem to be running Python 3.10+.

I'll add a few more review comments later today and look at the framework in detail sometime later.

[pythonhubdev]

Hi @namsnath the major issue is I was not able to access those routes even manually not alone in testing I think the app is not getting attached or something is getting missed out due to some reason I can't access the /auth/dasboard route even. Can you guide me a little more please?

[namsnath]

@pythonhubdev ,
I looked into this and found a few interesting issues.

• The requests to SuperTokens routes are not defined in the route_handlers, and result in a 404.
• Litestar handles 404/405s at the app level, and thus these do not flow down to the middleware and are handled directly through the exception_handlers map.
• exception_handlers can not be async functions, so it does not seem possible to call the ST middleware through these and handle the request.

I could get the requests routed correctly with these changes in the example app:

# supertokens_python/framework/litestar/litestar_request.py
class LitestarRequest(BaseRequest):
    ....
    def get_path(self) -> str:
        return self.request.scope["raw_path"].decode("utf-8")

# main.py

# TODO: Move to `litestar_middleware.py`
# If using the import from `litestar_middleware` does not work, convert to a factory function
@asgi("/auth", is_mount=True, copy_scope=True)
async def supertokens_app(scope: Scope, receive: Receive, send: Send) -> None:
    # <code from `LitestarMiddleware.handle` without `next_app` handling>
    # Additional changes:
    asgi_response = result.response.to_asgi_response(app=None, request=request) # Removed `await`, Set `app=None`

app = Litestar(
    route_handlers=[supertokens_app, ...],
    # middleware=[],
    exception_handlers={...},
)

A few things to keep in mind for this PR:

• Code needs to work on Python 3.8
• Please set up hooks by running make set-up-hooks
• Try to make sure types are correctly mentioned, and avoid Any wherever possible

I'll add a few items to the TODO section of the PR as well.

Ref:

• https://docs.litestar.dev/2/usage/exceptions.html#exception-handling-layers
• https://docs.litestar.dev/2/usage/routing/overview.html#mounting-asgi-apps
• Middleware are not executed on Starlite error litestar-org/litestar#175
• Enhancement: allow async exception handlers litestar-org/litestar#2262

[pythonhubdev]

@namsnath fixed the PR comments working on the ASGI handled you mentioned some are failing and facing some issues too will fix and update.

[Realsid]

Hey any progress on this ? Does this need to be picked up ?

[pythonhubdev]

Hi, I may require some help in this facing various issues and things are failing

[pythonhubdev]

@namsnath can you please help me out in fixing this got some time I want complete this PR this week. I am getting into issues:

/auth is throwing 404 and for some reason the middleware is not being called at all even after I use ASGIMiddleware. Can you help me here so that I can get this done?

[pythonhubdev]

@namsnath @Realsid I have pushed some changes please check it out there are some obvious issues which I need help with

1. In the test_listestar.py file currently the middleware is passed like TestClient(get_middleware()(app)) 4 tests are failing but this is not the ideal way to call a middleware but in this implementation I get the below error:

>       assert res.status_code == 401
E       assert 500 == 401
E        +  where 500 = <Response [500 Internal Server Error]>.status_code

2. In the ideal way where I pass the get_middleware function into Litestar app like below:

app = Litestar(
    middleware=[get_middleware()]
)

Around 16 tests are failing and mostly it's 404 not found. I don't know what exactly is the issue here but any help here would be great.

[namsnath]

Hi @pythonhubdev ,

Around 16 tests are failing and mostly it's 404 not found. I don't know what exactly is the issue here but any help here would be great.

Please look at #591 (comment) for an explanation of what's happening with the 404s.

I would be interested to find out if there is a way to circumvent the exception handling and allow our middleware to work as expected.

[pythonhubdev]

Yeah I tried out different setups but facing the same issue for one or the other routes. I will try to ask regarding this in the Litestar community for better understanding.

[pythonhubdev]

Hi @namsnath cleanup the functions and added plugin instead of middleware to mount supertokens as asgi app to Litestar instead of middleware as per guidance from the Litestar team. Can you please review the code? And also can you help me how can I run the tests from supertokens-auth-react and supertokens-website in my local?

[Realsid]

@pythonhubdev , a side note,could you please describe why you switched from middleware to implementing supertokens functionality as asgi app rather than middleware or you could point to that discussion. Thank you.

[pythonhubdev]

Yeah the discussion was on discord: https://discord.com/channels/919193495116337154/1433816355177496596

[pythonhubdev]

Hi @namsnath cleanup the functions and added plugin instead of middleware to mount supertokens as asgi app to Litestar instead of middleware as per guidance from the Litestar team. Can you please review the code? And also can you help me how can I run the tests from supertokens-auth-react and supertokens-website in my local?

@namsnath can you help me with this please?

[namsnath]

Hi @pythonhubdev ,

You can rebase your branch off the latest version branch. This has a bunch of CI improvements. Adding the run-tests label to the PR will allow you to run all the tests.

[pythonhubdev]

Thanks @namsnath

[pythonhubdev]

@namsnath I have rebased with branch 0.30 and pushed. Please check. Also I am not able to add / update the label.

[namsnath]

Please use 0.31 @pythonhubdev

[namsnath]

Please use 0.31 @pythonhubdev

Sorry, 0.30 is correct
That branch has a few CI fixes that don't seem to be present in your branch. Specifically, there is a change to use supertokens/actions/get-versions-from-repo in the workflows instead of supertokens/get-versions-action, which will fix the failures you see in the runs right now. You will need to make this same change in the litestar workflows as well.

Please also make sure the lowest supported Python version (3.8) works with your changes. Currently, I see a new Django version being used, which is not compatible with this.

[pythonhubdev]

@namsnath will fix these referring to the latest version branch.

[pythonhubdev]

@namsnath I have reverted all packages to the same version as in branch 0.30 and also I have changed to supertokens/actions/get-versions-from-repo in one place where it was using the previous action. Can you please check if these changes are fine?

[namsnath]

Please look at the other changes required for the changed action as well.
There will be changes required for the params and how the outputs are used.

Approving a workflow run to see if the rest of the tests run fine.

[pythonhubdev]

@namsnath I have fixed the linting issues and also I have updated the workflow similar to other workflows as well. Can you please check?

[namsnath]

Looks like there is a dependency on litestar outside of the frameworks module. Please fix this. You can find the logs as downloadable artifacts in the failing workflow runs.
Please also remove the circleCI configs/changes. These are not required.

Trace from a website tests run:

  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/tests/frontendIntegration/django3x/mysite/asgi.py", line 18, in <module>
    from polls.views import config
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/tests/frontendIntegration/django3x/polls/views.py", line 25, in <module>
    from supertokens_python import (
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/supertokens_python/__init__.py", line 22, in <module>
    from . import supertokens
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/supertokens_python/supertokens.py", line 40, in <module>
    from .normalised_url_domain import NormalisedURLDomain
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/supertokens_python/normalised_url_domain.py", line 21, in <module>
    from .utils import is_an_ip_address
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/supertokens_python/utils.py", line 48, in <module>
    from .framework.litestar.framework import LitestarFramework
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/supertokens_python/framework/litestar/__init__.py", line 15, in <module>
    from .litestar_exception_handlers import (
  File "/home/runner/work/supertokens-python/supertokens-python/supertokens-python/supertokens_python/framework/litestar/litestar_exception_handlers.py", line 17, in <module>
    from litestar import Request, Response
ModuleNotFoundError: No module named 'litestar'

[pythonhubdev]

Hi I was using a relative import that was causing the issue I guess I have fixed it. Can you please check the latest commit if the changes are fine? @namsnath