✨(oidc) add support for generic OIDC IdPs and OIDC clients as users

CHANGELOG.md

@@ -15,6 +15,13 @@ and this project adheres to
 ### Fixed
 
 - Fix type of `statement.result.score.scaled` from `int` to `Decimal`
+- Fix type of OIDC ID tokens
+- Fix error with OIDC scopes unrelated to Ralph
+
+### Changed
+
+- OIDC: Add query to `/userinfo` endpoint when receiving a token to support more OIDC IdPs
+- OIDC: Add token introspection to support querying from OIDC clients (Client Credentials flow)
 
 ## [5.0.1] - 2024-07-11
 

docs/tutorials/lrs/authentication/oidc.md

@@ -2,14 +2,20 @@
 
 Ralph LRS also supports OpenID Connect on top of OAuth 2.0 for authentication and authorization.
 
-To enable OpenID Connect authentication mode, we should change the `RALPH_RUNSERVER_AUTH_BACKENDS` environment variable to `oidc` and we should define the `RALPH_RUNSERVER_AUTH_OIDC_ISSUER_URI` environment variable with the identity provider's Issuer Identifier URI as follows:
+To enable OpenID Connect authentication mode, we should change the `RALPH_RUNSERVER_AUTH_BACKENDS` environment variable to `oidc` and we should define the environment variables as follows:
+
+- `RALPH_RUNSERVER_AUTH_OIDC_ISSUER_URI` the identity provider's Issuer Identifier URI
+  This address must be accessible to the LRS on startup as it will perform OpenID Connect Discovery to retrieve public keys and other information about the OpenID Connect environment.
+- `RALPH_RUNSERVER_AUTH_OIDC_CLIENT_ID` the OIDC client id issued by the identity provider for this instance
+- `RALPH_RUNSERVER_AUTH_OIDC_CLIENT_SECRET` the OIDC client secret issued by the identity provider for this instance
 
 ```bash
 RALPH_RUNSERVER_AUTH_BACKENDS=oidc
 RALPH_RUNSERVER_AUTH_OIDC_ISSUER_URI=http://{provider_host}:{provider_port}/auth/realms/{realm_name}
+RALPH_RUNSERVER_AUTH_OIDC_CLIENT_ID=some_client_id
+RALPH_RUNSERVER_AUTH_OIDC_CLIENT_SECRET=some_client_secret
 ```
 
-This address must be accessible to the LRS on startup as it will perform OpenID Connect Discovery to retrieve public keys and other information about the OpenID Connect environment.
 
 It is also strongly recommended to set the optional `RALPH_RUNSERVER_AUTH_OIDC_AUDIENCE` environment variable to the origin address of Ralph LRS itself (e.g. "http://localhost:8100") to enable verification that a given token was issued specifically for that Ralph LRS.
 
@@ -74,7 +80,7 @@ services:
 networks:
   ralph:
     external: true
-    
+
 ```
 
 Again, we need to create the `.ralph` directory:
@@ -102,7 +108,7 @@ Now that both Keycloak and Ralph LRS server are up and running, we should be abl
     ```
 
     ```bash
-    {"access_token":"<access token content>","expires_in":300,"refresh_expires_in":1800,"refresh_token":"<refresh token content>","token_type":"Bearer","not-before-policy":0,"session_state":"0889b3a5-d742-45fb-98b3-20e967960e74","scope":"email profile"} 
+    {"access_token":"<access token content>","expires_in":300,"refresh_expires_in":1800,"refresh_token":"<refresh token content>","token_type":"Bearer","not-before-policy":0,"session_state":"0889b3a5-d742-45fb-98b3-20e967960e74","scope":"email profile"}
     ```
 === "HTTPie"
 
@@ -134,12 +140,12 @@ Now that both Keycloak and Ralph LRS server are up and running, we should be abl
 With this access token, we can now make a request to the Ralph LRS server:
 
 === "curl"
-    
+
     ```bash
     curl -H 'Authorization: Bearer <access token content>' \
     http://localhost:8100/whoami
     ```
-    
+
     ```bash
     {"agent":{"openid":"http://localhost:8080/auth/realms/fun-mooc/b6e85bd0-ce6e-4b24-9f0e-6e18d8744e54"},"scopes":["email","profile"]}
     ```

src/ralph/api/auth/oidc.py

@@ -1,8 +1,9 @@
 """OpenID Connect authentication tool for the Ralph API."""
 
+import base64
 import logging
 from functools import lru_cache
-from typing import Dict, Optional
+from typing import Dict, Literal, Optional, Union, get_args
 
 import requests
 from fastapi import Depends, HTTPException, status
@@ -12,7 +13,7 @@
 from pydantic import AnyUrl, BaseModel, ConfigDict
 from typing_extensions import Annotated
 
-from ralph.api.auth.user import AuthenticatedUser, UserScopes
+from ralph.api.auth.user import AuthenticatedUser, Scope, UserScopes
 from ralph.conf import settings
 
 OPENID_CONFIGURATION_PATH = "/.well-known/openid-configuration"
@@ -26,29 +27,61 @@
 logger = logging.getLogger(__name__)
 
 
-class IDToken(BaseModel):
-    """Pydantic model representing the core of an OpenID Connect ID Token.
+class UserInfo(BaseModel):
+    """Pydantic model representing the UserInfo response of the OIDC IdP.
 
-    ID Tokens are polymorphic and may have many attributes not defined in the
+    They are common to both the ID token and the UserInfo endpoint.
+    We do not use it for authentication, so may claims are ignored.
+    They are polymorphic and may have many attributes not defined in the
     specification. This model ignores all additional fields.
 
     Attributes:
-        iss (str): Issuer Identifier for the Issuer of the response.
         sub (str): Subject Identifier.
-        aud (str): Audience(s) that this ID Token is intended for.
+        scope (str): Scope(s) for resource authorization.
+        target (str): Target for storing the statements (custom claim).
+    """
+
+    sub: str
+    scope: Optional[str] = None
+    target: Optional[str] = None
+
+    model_config = ConfigDict(extra="ignore")
+
+
+class TokenInfo(BaseModel):
+    """Pydantic model representing the Introspection response of the OIDC IdP.
+
+    Based on the RFC 7662 section 2.2 definition of token /introspect response
+    This model does not use all fields defined in the RFC,
+    and we force some optional fields to be present.
+
+    The 'active' field is assumed to be true and is not included in this model.
+
+    Attributes:
+        client_id (str): ID of the client that owns this token
+        username (str): Name of the user referred to by this this token, if any
+        iss (str): Issuer Identifier for the Issuer of the response.
+        sub (str): Subject Identifier, if any
+                  No sub means that this is a purely 'client' token
+        aud (str or list of str): Audience(s) that this ID Token is intended for.
         exp (int): Expiration time on or after which the ID Token MUST NOT be
                    accepted for processing.
         iat (int): Time at which the JWT was issued.
         scope (str): Scope(s) for resource authorization.
-        target (str): Target for storing the statements.
+        target (str): Target for storing the statements (custom claim).
+
     """
 
+    client_id: str
+    username: Optional[str] = None
+    token_type: Optional[str] = None
     iss: str
-    sub: str
-    aud: Optional[str] = None
-    exp: int
-    iat: int
+    sub: Optional[str] = None
+    aud: Optional[Union[list[str], str]] = None
+    exp: float
+    iat: float
     scope: Optional[str] = None
+
     target: Optional[str] = None
 
     model_config = ConfigDict(extra="ignore")
@@ -72,6 +105,129 @@ def discover_provider(base_url: AnyUrl) -> Dict:
         ) from exc
 
 
+def get_user_info(provider_config: dict, access_token: str) -> UserInfo:
+    """Get the user's info from the IdP using the /userinfo OIDC endpoint."""
+    user_info, is_encoded = get_user_info_data(
+        provider_config["userinfo_endpoint"], access_token
+    )
+    if not is_encoded:
+        # nothing to do
+        return UserInfo.model_validate(user_info)
+    return decode_user_info(user_info, provider_config=provider_config)
+
+
+@lru_cache()
+def get_user_info_data(
+    userinfo_endpoint: AnyUrl, access_token: str
+) -> Union[tuple[dict, Literal[False]], tuple[str, Literal[True]]]:
+    """Get the user's info from the IdP using the /userinfo OIDC endpoint.
+
+    The data may be unencoded (Content-Type: 'application/json',
+    in which case it is a json dictionary.
+    If it is encoded (Content-Type: 'application/jwt', it is a JWT
+    see: https://openid.net/specs/openid-connect-core-1_0.html#UserInfo
+
+    Returns the data and whether that data is encoded (a JWT string) or plain (a dict)
+    """
+    try:
+        response = requests.get(
+            f"{userinfo_endpoint}",
+            headers={"Authorization": f"Bearer {access_token}"},
+            timeout=5,
+        )
+        response.raise_for_status()
+        content_type = response.headers["Content-Type"]
+        return (response.json(), content_type.lower() == "application/jwt")
+    except requests.exceptions.RequestException as exc:
+        logger.error("Unable to get the user's ID token: %s", exc)
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="Could not validate credentials",
+            headers={"WWW-Authenticate": "Bearer"},
+        ) from exc
+
+
+def decode_user_info(encoded_user_info: str, provider_config: dict) -> UserInfo:
+    """Decode and verify an OpenID Connect ID token."""
+    key = get_public_keys(provider_config["jwks_uri"])
+    algorithms = provider_config["id_token_signing_alg_values_supported"]
+    audience = settings.RUNSERVER_AUTH_OIDC_AUDIENCE
+    options = {
+        "verify_signature": True,
+        "verify_aud": bool(audience),
+        "verify_exp": True,
+    }
+    try:
+        decoded_token = jwt.decode(
+            token=encoded_user_info,
+            key=key,
+            algorithms=algorithms,
+            options=options,
+            audience=audience,
+        )
+    except (ExpiredSignatureError, JWTError, JWTClaimsError) as exc:
+        logger.error("Unable to decode the ID token: %s", exc)
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="Could not validate credentials",
+            headers={"WWW-Authenticate": "Bearer"},
+        ) from exc
+
+    return UserInfo.model_validate(decoded_token)
+
+
+def encode_client_secret_basic_token(client_id: str, client_secret: str) -> str:
+    """Encode client id and client secret inside an opaque token.
+
+    Should be sent as Authorization: Basic {token}
+
+    This corresponds to the `client_secret_basic`
+    token endpoint authentication method.
+    """
+    return base64.b64encode(
+        client_id.encode("utf-8") + b":" + client_secret.encode("utf-8")
+    ).decode("utf-8")
+
+
+@lru_cache()
+def get_token_info(
+    introspection_endpoint: AnyUrl, token: str, client_id: str, client_secret: str
+) -> UserInfo:
+    """Get info on given token from the IdP using /introspection OIDC endpoint."""
+    token_info = None
+    try:
+        response = requests.post(
+            f"{introspection_endpoint}",
+            headers={
+                "Authorization": f"Basic {encode_client_secret_basic_token(
+                    client_id=client_id,
+                    client_secret=client_secret
+                )}"
+            },
+            data={
+                "token": f"{token}",
+            },
+            timeout=5,
+        )
+        response.raise_for_status()
+        token_info = response.json()
+    except requests.exceptions.RequestException as exc:
+        logger.error("Unable to get token info: %s", exc)
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="Could not validate credentials",
+            headers={"WWW-Authenticate": "Bearer"},
+        ) from exc
+    if not token_info["active"]:
+        logger.error("Inactive or invalid token info.")
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail="Could not validate credentials",
+            headers={"WWW-Authenticate": "Bearer"},
+        )
+    return TokenInfo.model_validate(token_info)
+
+
 @lru_cache()
 def get_public_keys(jwks_uri: AnyUrl) -> Dict:
     """Retrieve the public keys used by the provider server for signing."""
@@ -94,6 +250,19 @@ def get_public_keys(jwks_uri: AnyUrl) -> Dict:
         ) from exc
 
 
+def get_user_scopes(oidc_scopes: Optional[str]) -> UserScopes:
+    """Extract Ralph's custom OAuth2 scopes from the global scope list.
+
+    Ignore incompatible scopes.
+    """
+    compatible_scopes = (
+        [scope for scope in oidc_scopes.split(" ") if scope in get_args(Scope)]
+        if oidc_scopes
+        else []
+    )
+    return UserScopes(compatible_scopes)
+
+
 def get_oidc_user(
     auth_header: Annotated[Optional[HTTPBearer], Depends(oauth2_scheme)],
 ) -> AuthenticatedUser:
@@ -117,37 +286,38 @@ def get_oidc_user(
         )
         return None
 
-    id_token = auth_header.split(" ")[-1]
+    access_token = auth_header.split(" ")[-1]
     provider_config = discover_provider(settings.RUNSERVER_AUTH_OIDC_ISSUER_URI)
-    key = get_public_keys(provider_config["jwks_uri"])
-    algorithms = provider_config["id_token_signing_alg_values_supported"]
-    audience = settings.RUNSERVER_AUTH_OIDC_AUDIENCE
-    options = {
-        "verify_signature": True,
-        "verify_aud": bool(audience),
-        "verify_exp": True,
-    }
-    try:
-        decoded_token = jwt.decode(
-            token=id_token,
-            key=key,
-            algorithms=algorithms,
-            options=options,
-            audience=audience,
-        )
-    except (ExpiredSignatureError, JWTError, JWTClaimsError) as exc:
-        logger.error("Unable to decode the ID token: %s", exc)
-        raise HTTPException(
-            status_code=status.HTTP_401_UNAUTHORIZED,
-            detail="Could not validate credentials",
-            headers={"WWW-Authenticate": "Bearer"},
-        ) from exc
-
-    id_token = IDToken.model_validate(decoded_token)
 
-    user = AuthenticatedUser(
-        agent={"openid": f"{id_token.iss}/{id_token.sub}"},
-        scopes=UserScopes(id_token.scope.split(" ") if id_token.scope else []),
-        target=id_token.target,
+    token_info = get_token_info(
+        provider_config["introspection_endpoint"],
+        token=access_token,
+        client_id=settings.RUNSERVER_AUTH_OIDC_CLIENT_ID,
+        client_secret=settings.RUNSERVER_AUTH_OIDC_CLIENT_SECRET,
     )
-    return user
+    if token_info.sub:
+        # This is a real user, we can retrieve their user info
+        user_info = get_user_info(provider_config, access_token=access_token)
+        if user_info.sub != token_info.sub:
+            logger.error(
+                ("Inconsistent token subject: %s != %s"), user_info.sub, token_info.sub
+            )
+            raise HTTPException(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                detail="Could not validate credentials",
+                headers={"WWW-Authenticate": "Bearer"},
+            )
+
+        return AuthenticatedUser(
+            agent={"openid": f"{token_info.iss}/user/{user_info.sub}"},
+            scopes=get_user_scopes(user_info.scope),
+            target=user_info.target,
+        )
+    else:
+        # this is an application token, we don't have a user to get
+        # so we use the client_id to indentify it instead
+        return AuthenticatedUser(
+            agent={"openid": f"{token_info.iss}/application/{token_info.client_id}"},
+            scopes=get_user_scopes(token_info.scope),
+            target=token_info.target,
+        )