Bulk request agents on the same runner

[stonkmate-lkh]

Hi there!

First of thanks for the community calls, great to see the people behind the scenes.

I have a question about how to bulk request a root agent within the same runner. Usually, I’ve had use cases where I read a CSV (or similar) and then make a request for each row in the file. Using REST endpoints, this was pretty straightforward; however, I’m unsure if this is supported in ADK, and if so, how you would approach it.

The reason I’m asking is that ADK seems tailored to chat applications with sessions. In this case, the use doesn’t require any context handling or content persistence, but I’d still like to use the framework to combine tools and other LLMs.

Currently, I’m doing it like this: I create one runner and have a process_row_task function that creates a unique session for each request to avoid context conflicts. Then it runs runner.run_async and returns the results.

Is there a smarter or better way to do this?

    # Limit concurrent in-flight generations
    concurrency_limit = 50
    semaphore = asyncio.Semaphore(concurrency_limit)
    runner = InMemoryRunner(agent=root_agent, app_name="pdp_writer")

    for csv_path in csv_paths:
        tasks: list[asyncio.Task] = []
        df_products = pd.read_csv(csv_path)
        for idx, row in df_products.iterrows():
            tasks.append(
                asyncio.create_task(
                    process_row_task(
                        str(csv_path), idx, row.to_dict(), user_id, semaphore, runner
                    )
                )
            )
        results = await asyncio.gather(*tasks)
        for result in results:
            if result["final_text"]:
                df_products.at[result["row_index"], "LongDescription"] = result["final_text"]
    return df_products

[LukasKH]

Could it make sense to just have it hosted seperately instead?

So have Cloud Run run the agents, tools etc and then have a simple Cloud function pull in csv data, then concurrently call the cloud run function so you basically call the agent like a rest endpoint?