Incorrectly nesting fetch requests

[ChuckJonas]

I have two output fetch requests that happen one after another.

1. A call to lamdba layer to get secrets
2. A call to an external API

For some reason, the external API call is getting nested under the localhost segment

I've made sure the close the secrets segment and event tried flushing it. It seems like it's probably happening outside my custom segments

import * as AWSXRay from "aws-xray-sdk";

type Secret = {
  SecretString: string;
};

export async function getSecrets(secretId: string): Promise<string> {
  const rootSegment = AWSXRay.getSegment();
  const subsegment = rootSegment?.addNewSubsegment("Secrets Manager");

  try {
    subsegment?.addAnnotation("secretId", secretId);


    const uri = `http://localhost:2773/secretsmanager/get?secretId=${secretId}`;
    const res = await fetch(uri, {
      headers: {
        "X-Aws-Parameters-Secrets-Token": process.env.AWS_SESSION_TOKEN!,
      },
    });

    subsegment?.addMetadata("request", {
      url: uri,
      method: "GET",
    });

    const data = (await res.json()) as Secret;
    return data["SecretString"];
  } catch (error) {
    subsegment?.addError(error as Error);
    throw error;
  } finally {
    subsegment?.close();
    subsegment?.flush();
  }
}

// xray init, called at top of lambda
import * as AWSXRay from "aws-xray-sdk";

import { captureFetchGlobal } from "aws-xray-sdk-fetch";

export const initXRay = () => {
  AWSXRay.captureHTTPsGlobal(require("http"));
  AWSXRay.captureHTTPsGlobal(require("https"));
  captureFetchGlobal();
};

Raw Trace

    {
            "Id": "c5a8d4141cdf256a",
            "Document": {
                "id": "c5a8d4141cdf256a",
                "name": "dev-eg-practifi-braze",
                "start_time": 1744472658.073189,
                "trace_id": "1-67fa8a51-6a8f27832663e1347fae739c",
                "end_time": 1744472660.633492,
                "parent_id": "33ca25824e061177",
                "annotations": {
                    "aws:responseLatency": 2540.15,
                    "aws:runtimeOverhead": 19.412,
                    "aws:extensionOverhead": 0.004,
                    "aws:responseDuration": 0.081
                },
                "origin": "AWS::Lambda::Function",
                "subsegments": [
                    {
                        "id": "a102671b59c8c6de",
                        "name": "Overhead",
                        "start_time": 1744472660.612,
                        "end_time": 1744472660.632004
                    },
                    {
                        "id": "143fee48aa516ab3",
                        "name": "localhost",
                        "start_time": 1744472658.151,
                        "end_time": 1744472659.152,
                        "http": {
                            "request": {
                                "url": "http://localhost:2773/secretsmanager/",
                                "method": "GET"
                            },
                            "response": {
                                "status": 200,
                                "content_length": 323
                            }
                        },
                        "namespace": "remote",
                        "subsegments": [
                            {
                                "id": "a55e83fa4b9460fa",
                                "name": "rest.iad-05.braze.com",
                                "start_time": 1744472659.234,
                                "end_time": 1744472660.507,
                                "http": {
                                    "request": {
                                        "url": "https://rest.iad-05.braze.com/users/track/sync",
                                        "method": "POST"
                                    },
                                    "response": {
                                        "status": 201
                                    }
                                },
                                "namespace": "remote"
                            },
                            {
                                "id": "5180154238c3762f",
                                "name": "Braze API",
                                "start_time": 1744472659.233,
                                "end_time": 1744472660.572,
                                "annotations": {
                                    "method": "track user"
                                }
                            }
                        ]
                    },
                    {
                        "id": "507c230d4d8340de",
                        "name": "Init",
                        "start_time": 1744472657.479804,
                        "end_time": 1744472658.068975
                    },
                    {
                        "id": "ae0087abd50d0eae",
                        "name": "Secrets Manager",
                        "start_time": 1744472658.112,
                        "end_time": 1744472659.231,
                        "annotations": {
                            "secretId": "arn:aws:secretsmanager"
                        }
                    }
                ]
            }
        },

[ChuckJonas]

if I had to guess, it seems like aws-xray-sdk-fetch isn't properly closing the localhost segment

[ChuckJonas]

I swapped my code to use node-fetch and updated my init to:

AWSXRay.captureHTTPsGlobal(require("http"));
AWSXRay.captureHTTPsGlobal(require("https"));

And now everything displays as expected:

[jj22ee]

https://github.com/aws/aws-xray-sdk-node/tree/master/packages/core#capture-subsegments-within-chained-native-promise-using-automatic-mode

If you are using promises/await, you may need to use AWSXRay.capturePromise(); to avoid the nesting issue you are facing. Can you try to reproduce the origin issue after using AWSXRay.capturePromise();

[ChuckJonas]

@jj22ee no dice.

Added AWSXRay.capturePromise(); and switched back to native fetch, and now the segments are nested again.

I don't see any noticeable difference in the raw trace with or without AWSXRay.capturePromise(); so maybe it's not registering properly? (I don't entirely understand what I should expect to see)

[HankieCodes]

@ChuckJonas is this with nodejs22.x ?

I'm seeing the same thing with my code, and other custom subsegments I create when I'm in async / await syntax. Haven't tried the "traditional" Promise syntax yet.

My guess is that one of the newer NodeJS versions broke the AWSXRay.capturePromise() monkeypatch

[ChuckJonas]

@hnryjms ya, we're also on 22.x.