CapabilityInspector and null "__type" responses

[wizzfizz2097]

Product

Strawberry Shake

Version

15.1.11

Link to minimal reproduction

https://gist.github.com/wizzfizz2097/eb3b415af07bc0c47ad03430ccfc6dc5#file-introspection-bash

Steps to reproduce

1. Start a new project
2. Add StrawberryShake tooling and client package
3. Attempt to perform introspection init or download on a public schema: https://partner.thetradedesk.com/v3/graphql-schema

What is expected?

To create the client without an exception, or to provide a helpful message regarding the failure.

What is actually happening?

I believe some of the queries within HotChocolate.Utilities.Introspection.CapabilityInspector are not handling a property in the response that exists, but has a null value. For example the following response when running the query from inspect_directive_type.graphql
{ "data": { "__type": null } }

Relevant log output

dotnet : Unhandled exception. System.InvalidOperationException: The requested operation requires an element of type 'Object', but the target element has type 'Null'.
At line:1 char:1
+ dotnet graphql init https://partner.thetradedesk.com/v3/graphql-schem ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Unhandled excep...as type 'Null'.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
   at System.Text.Json.ThrowHelper.ThrowJsonElementWrongTypeException(JsonTokenType expectedType, JsonTokenType actualType)
   at System.Text.Json.JsonDocument.TryGetNamedPropertyValue(Int32 index, ReadOnlySpan`1 propertyName, JsonElement& value)
   at System.Text.Json.JsonElement.TryGetProperty(String propertyName, JsonElement& value)
   at HotChocolate.Utilities.Introspection.CapabilityInspector.InspectDirectiveTypeAsync() in /home/runner/work/graphql-platform/graphql-platform/src/HotChocolate/Utilities/src/Utilities.Introspection/CapabilityInspector.cs:line 132
   at HotChocolate.Utilities.Introspection.CapabilityInspector.InspectAsync(GraphQLHttpClient client, IntrospectionOptions options, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/HotChocolate/Utilities/src/Utilities.Introspection/CapabilityInspector.cs:line 33
   at HotChocolate.Utilities.Introspection.IntrospectionClient.IntrospectServerInternalAsync(GraphQLHttpClient client, IntrospectionOptions options, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/HotChocolate/Utilities/src/Utilities.Introspection/IntrospectionClient.cs:line 124
   at HotChocolate.Utilities.Introspection.IntrospectionClient.IntrospectServerInternalAsync(HttpClient client, IntrospectionOptions options, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/HotChocolate/Utilities/src/Utilities.Introspection/IntrospectionClient.cs:line 73
   at StrawberryShake.Tools.IntrospectionHelper.DownloadSchemaAsync(HttpClient client, IFileSystem fileSystem, IActivity activity, String fileName, Int32 typeDepth, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/StrawberryShake/Tooling/src/dotnet-graphql/IntrospectionHelper.cs:line 20
   at StrawberryShake.Tools.InitCommandHandler.DownloadSchemaAsync(InitCommandContext context, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/StrawberryShake/Tooling/src/dotnet-graphql/InitCommandHandler.cs:line 105
   at StrawberryShake.Tools.InitCommandHandler.ExecuteInternalAsync(InitCommandContext context, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/StrawberryShake/Tooling/src/dotnet-graphql/InitCommandHandler.cs:line 82
   at StrawberryShake.Tools.InitCommandHandler.ExecuteAsync(InitCommandArguments arguments, CancellationToken cancellationToken) in /home/runner/work/graphql-platform/graphql-platform/src/StrawberryShake/Tooling/src/dotnet-graphql/InitCommandHandler.cs:line 68
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at StrawberryShake.Tools.Program.<Main>(String[] args)

Additional context

No response

[michaelstaib]

{
  "operationName": "IntrospectionQuery",
  "query": "query IntrospectionQuery {\n  __type(name: \"__InputValue\") {\n    fields {\n      name\n    }\n  }\n}"
}

This request yields null although this is a spec introspection type.
https://spec.graphql.org/September2025/#sec-The-__InputValue-Type

The capabilities check inspects the system types to know what kind of introspection query it can build.

[michaelstaib]

We investigated a bit more and the execution behavior of the GraphQL server is very inconsistent and not spec compliant.

For instance:

query IntrospectionQuery {
  __type(name: "__Schema") {
    fields {
      name
    }
  }
}

Responds with:

{
  "data": {
    "__type": null
  }
}

But changing the query slightly and also asking for the name of the schema will return the actual type:

query IntrospectionQuery {
  __type(name: "__Schema") {
    name
    fields {
      name
    }
  }
}

Responds with:

{
  "data": {
    "__type": {
      "name": "__Schema",
      "fields": [
        {
          "name": "description"
        },
        {
          "name": "types"
        },
        {
          "name": "queryType"
        },
        {
          "name": "mutationType"
        },
        {
          "name": "subscriptionType"
        },
        {
          "name": "directives"
        }
      ]
    }
  }
}

The fields you request should not influence if __type returns something of not. The server is at fault here. Nevertheless, we will implement a fallback that will assume no capabilities and basically not allow more modern GraphQL features if a server does fail the queries.

[wizzfizz2097]

Thankyou @michaelstaib for confirming what I also thought was happening. As a quick-fix I experimented with the following change:
wizzfizz2097@b890a9b
But this only revealed further issues on this particular server which are far outside the scope of this bug report.