diff --git a/README.md b/README.md index 64951b9..ab7e810 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,31 @@ # Multi-Agent System -A sample application demonstrating how to build a multi-agent system using Akka and an LLM model. +A sample application demonstrating how to build a multi-agent system using Akka and an AI model. An Autonomous Agent coordinator delegates dynamically to specialized worker agents, with the model deciding which workers to consult for each request. ## Overview -This project illustrates an Agentic workflow for a multi-agent system using Akka. The system: +This project illustrates a multi-agent system built around the Autonomous Agent component. The system: -- Processes user queries and determines the appropriate agents to handle the request -- Plans the execution order of the selected agents -- Executes the plan by interacting with the agents in sequence -- Summarizes the results from all agents into a final response +- Receives an activity-suggestion request for a user +- Personalizes the request with any stored user preferences +- Hands the request to an Autonomous Agent coordinator that delegates dynamically to a weather agent and an activity agent +- Returns a typed answer once the coordinator's task completes +- Evaluates each completed task with a custom LLM-as-judge and the built-in toxicity evaluator; verdicts are logged and surfaced through metrics and traces ### Akka components This sample leverages specific Akka components: -- **Workflow**: Manages the user query process, handling the sequential steps of agent selection, plan creation, execution, and summarization. -- **EventSourced Entity**: Maintains the session memory, storing the sequence of interactions between the user and the system. -- **HTTP Endpoint**: Serves the application endpoints for interacting with the multi-agent system (`/activities`). +- **Autonomous Agent (`ActivityCoordinator`)**: Accepts a `SuggestActivities` task and declares `Delegation` to the worker agents. The runtime drives its decision loop until the task completes. +- **Agent (`WeatherAgent`, `ActivityAgent`)**: Plain request-based agents that the coordinator delegates to. Each exposes a `query` method whose parameter type is serialized into a tool schema for the coordinator's model. `WeatherAgent.query` takes a `String`; `ActivityAgent.query` takes an `AgentRequest` record so it can look up the user's preferences with the userId. +- **Agent (`EvaluatorAgent`)**: An LLM-as-judge agent that evaluates the coordinator's answer against the original (preference-aware) request. +- **EventSourced Entity (`PreferencesEntity`)**: Holds the user's preferences. +- **Consumer (`EvaluationConsumer`)**: Subscribes to the runtime's task entity events. On task completion it runs `EvaluatorAgent` and the built-in `ToxicityEvaluator`, logging the verdicts. +- **HTTP Endpoint (`ActivityEndpoint`)**: Exposes the `/activities` and `/preferences` routes. ### Other -- **LLM model**: The system uses an LLM model to assist in agent selection, plan creation, and summarization. The LLM is integrated with tools specific to each agent's domain of expertise. +- **AI model**: The coordinator uses an AI model to choose which workers to consult, what to ask them, and how to synthesize the final answer. The evaluator uses a model to judge each answer. ## Example flow @@ -29,42 +33,37 @@ This sample leverages specific Akka components: sequenceDiagram participant User participant HTTPEndpoint as HTTP Endpoint - participant Workflow as Akka Workflow - participant Selector as Agent Selector - participant Planner as Execution Planner + participant Task as Task Entity + participant Coordinator as ActivityCoordinator participant WeatherAgent as Weather Agent participant ActivityAgent as Activity Agent - participant Summarizer as Summarizer + participant EvalConsumer as EvaluationConsumer - User->>HTTPEndpoint: "I do not work today. I am in Madrid. What should I do? Beware of the weather" - HTTPEndpoint->>Workflow: Create new workflow instance - Note over Workflow: Initialize multi-agent query process - HTTPEndpoint-->>User: Return activity id for follow up + User->>HTTPEndpoint: POST /activities/alice "I am in Madrid..." + HTTPEndpoint->>Coordinator: runSingleTask(SUGGEST_ACTIVITIES.instructions(message)) + Coordinator-->>HTTPEndpoint: taskId + HTTPEndpoint-->>User: 201 Created (taskId in body + Location) - Workflow->>Selector: Select appropriate agents - Selector-->>Workflow: Return selected agents (e.g., WeatherAgent, ActivityAgent) + Note over Coordinator: Model decides which workers to delegate to - Workflow->>Planner: Create execution plan - Planner-->>Workflow: Return ordered plan (e.g., WeatherAgent -> ActivityAgent) + Coordinator->>WeatherAgent: delegate "What is the weather in Madrid?" + WeatherAgent-->>Coordinator: "Rainy in Madrid" - loop Execute plan - Workflow->>WeatherAgent: "What is the weather in Madrid?" - WeatherAgent-->>Workflow: "Rainy in Madrid" + Coordinator->>ActivityAgent: delegate "Suggest activities for a rainy day in Madrid" + ActivityAgent-->>Coordinator: "Visit the Prado Museum or enjoy local cafes" - Workflow->>ActivityAgent: "Suggest activities for a day in Madrid" - ActivityAgent-->>Workflow: "Visit the Prado Museum or enjoy local cafes" - end + Note over Coordinator: Synthesize final answer + Coordinator->>Task: complete with result - Workflow->>Summarizer: Summarize responses - Summarizer-->>Workflow: "The weather in Madrid is rainy today, so you might want to explore indoor attractions..." + Task-->>EvalConsumer: TaskCompleted event + EvalConsumer->>EvalConsumer: run EvaluatorAgent + ToxicityEvaluator (log verdicts) - User->>HTTPEndpoint: Send query by session id - HTTPEndpoint->>Workflow: Request result from workflow - Workflow-->>HTTPEndpoint: Return result - HTTPEndpoint-->>User: Deliver response + User->>HTTPEndpoint: GET /activities/alice/{taskId} + HTTPEndpoint->>Task: get snapshot + HTTPEndpoint-->>User: typed answer ``` -Note that the agents selected depend on the user's query and the available agents. Thus, the flow varies arbitrarily depending on the Planner agent reasoning and the steps selected to respond to the query. Also, another important aspect is that all agents share the same context and thus, when answering a question can take into consideration the context of the previous interactions with other agents. +The set of workers the coordinator consults depends on the user's query and the worker descriptions. Different requests will lead the model to consult only one worker, both, or even loop back for follow-ups. ## Running the application @@ -118,24 +117,18 @@ mvn compile exec:java With the application running, you can test the system using the following endpoints: -* Start a new session: +* Start a new task: ```shell -curl -i -XPOST --location "http://localhost:9000/activities/alice/1" \ +curl -i -XPOST --location "http://localhost:9000/activities/alice" \ --header "Content-Type: application/json" \ --data '{"message": "I do not work tomorrow. I am in Madrid. What should I do? Beware of the weather"}' ``` -The system will process the query, select the appropriate agents, and return a response. +The endpoint personalizes the request with any stored preferences and calls `runSingleTask` on the coordinator. The response body and `Location` header carry the task id. -* Retrieve the response for a specific session: +* Retrieve the response for a specific task: ```shell -curl -i -XGET --location "http://localhost:9000/activities/alice/1" -``` - -You can also retrieve for all previous suggestions for a user: - -```shell -curl -i -XGET --location "http://localhost:9000/activities/alice" +curl -i -XGET --location "http://localhost:9000/activities/alice/{taskId}" ``` Preferences can be added with: @@ -149,6 +142,8 @@ curl -i localhost:9000/preferences/alice \ }' ``` +Preferences are read by the endpoint on each new request and inlined into the task instructions, so subsequent suggestions take them into account. The `EvaluationConsumer` runs on every completed task; inspect the service logs to see its verdicts. + ## Deployment You can use the [Akka Console](https://console.akka.io) to create a project and deploy this service. diff --git a/pom.xml b/pom.xml index 27ccb8d..37abbff 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ io.akka akka-javasdk-parent - 3.5.19 + 3.6.0 io.akka.ai diff --git a/src/main/java/demo/multiagent/api/ActivityEndpoint.java b/src/main/java/demo/multiagent/api/ActivityEndpoint.java index d169b27..76da501 100644 --- a/src/main/java/demo/multiagent/api/ActivityEndpoint.java +++ b/src/main/java/demo/multiagent/api/ActivityEndpoint.java @@ -7,13 +7,9 @@ import akka.javasdk.annotations.http.Post; import akka.javasdk.client.ComponentClient; import akka.javasdk.http.HttpResponses; -import com.fasterxml.jackson.annotation.JsonSubTypes; -import com.fasterxml.jackson.annotation.JsonTypeInfo; -import demo.multiagent.application.ActivityView; -import demo.multiagent.application.AgentTeamWorkflow; -import demo.multiagent.application.AgentTeamWorkflow.AgentTeamNotification; +import demo.multiagent.application.ActivityCoordinator; +import demo.multiagent.application.ActivityTasks; import demo.multiagent.application.PreferencesEntity; -import java.util.List; import java.util.UUID; // Opened up for access from the public internet to make the service easy to try out. @@ -27,57 +23,33 @@ public record Request(String message) {} public record AddPreference(String preference) {} - public record ActivitiesList(List suggestions) { - static ActivitiesList fromView(ActivityView.ActivityEntries entries) { - return new ActivitiesList( - entries.entries().stream().map(Suggestion::fromView).toList() - ); - } - } - - public record Suggestion(String userQuestion, String answer) { - static Suggestion fromView(ActivityView.ActivityEntry entry) { - return new Suggestion(entry.userQuestion(), entry.finalAnswer()); - } - } - private final ComponentClient componentClient; public ActivityEndpoint(ComponentClient componentClient) { this.componentClient = componentClient; } - @Post("/activities/{userId}/{sessionId}") - public HttpResponse suggestActivities(String userId, String sessionId, Request request) { - var res = componentClient - .forWorkflow(sessionId) - .method(AgentTeamWorkflow::start) - .invoke(new AgentTeamWorkflow.Request(userId, request.message())); - - return HttpResponses.created(res, "/activities/" + userId + "/" + sessionId); - } + @Post("/activities/{userId}") + public HttpResponse suggestActivities(String userId, Request request) { + var instructions = "User: " + userId + "\n\n" + request.message(); // <1> - @Get("/activities/{userId}/{sessionId}") - public HttpResponse getAnswer(String userId, String sessionId) { - var res = componentClient - .forWorkflow(sessionId) - .method(AgentTeamWorkflow::getAnswer) - .invoke(); + var taskId = componentClient + .forAutonomousAgent(ActivityCoordinator.class, UUID.randomUUID().toString()) + .runSingleTask(ActivityTasks.SUGGEST_ACTIVITIES.instructions(instructions)); // <2> - if (res.isEmpty()) return HttpResponses.notFound( - "Answer for '" + sessionId + "' not available (yet)" - ); - else return HttpResponses.ok(res); + return HttpResponses.created(taskId, "/activities/" + userId + "/" + taskId); } - @Get("/activities/{userId}") - public ActivitiesList listActivities(String userId) { - var viewResult = componentClient - .forView() - .method(ActivityView::getActivities) - .invoke(userId); + @Get("/activities/{userId}/{taskId}") + public HttpResponse getAnswer(String userId, String taskId) { + var snapshot = componentClient.forTask(taskId).get(ActivityTasks.SUGGEST_ACTIVITIES); // <3> - return ActivitiesList.fromView(viewResult); + return snapshot + .result() + .map(HttpResponses::ok) + .orElseGet( + () -> HttpResponses.notFound("Answer for '" + taskId + "' not available (yet)") + ); } @Post("/preferences/{userId}") @@ -89,44 +61,4 @@ public HttpResponse addPreference(String userId, AddPreference request) { return HttpResponses.created(); } - - @JsonTypeInfo(use = JsonTypeInfo.Id.NAME) - @JsonSubTypes( - { - @JsonSubTypes.Type(value = UpdateEvent.StatusUpdate.class, name = "S"), - @JsonSubTypes.Type(value = UpdateEvent.LlmResponseStart.class, name = "LS"), - @JsonSubTypes.Type(value = UpdateEvent.LlmResponseDelta.class, name = "LD"), - @JsonSubTypes.Type(value = UpdateEvent.LlmResponseEnd.class, name = "LE"), - } - ) - public sealed interface UpdateEvent { - record StatusUpdate(String msg) implements UpdateEvent {} - - record LlmResponseStart() implements UpdateEvent {} - - record LlmResponseDelta(String response) implements UpdateEvent {} - - record LlmResponseEnd() implements UpdateEvent {} - } - - @Get("/updates/{sessionId}") - public HttpResponse updates(String sessionId) { - return HttpResponses.serverSentEvents( - componentClient - .forWorkflow(sessionId) - .notificationStream(AgentTeamWorkflow::updates) - .source() - .map(notification -> - switch (notification) { - case AgentTeamNotification.LlmResponseDelta llmResponseDelta -> new UpdateEvent.LlmResponseDelta( - llmResponseDelta.response() - ); - case AgentTeamNotification.LlmResponseEnd __ -> new UpdateEvent.LlmResponseEnd(); - case AgentTeamNotification.LlmResponseStart __ -> new UpdateEvent.LlmResponseStart(); - case AgentTeamNotification.StatusUpdate statusUpdate -> new UpdateEvent.StatusUpdate( - statusUpdate.msg() - ); - }) - ); - } } diff --git a/src/main/java/demo/multiagent/api/UiEndpoint.java b/src/main/java/demo/multiagent/api/UiEndpoint.java deleted file mode 100644 index 62102ec..0000000 --- a/src/main/java/demo/multiagent/api/UiEndpoint.java +++ /dev/null @@ -1,21 +0,0 @@ -package demo.multiagent.api; - -import akka.http.javadsl.model.HttpResponse; -import akka.javasdk.annotations.Acl; -import akka.javasdk.annotations.http.Get; -import akka.javasdk.annotations.http.HttpEndpoint; -import akka.javasdk.http.HttpResponses; - -/** - * This Http endpoint returns the static UI page located under - * src/main/resources/static-resources/ - */ -@HttpEndpoint -@Acl(allow = @Acl.Matcher(principal = Acl.Principal.ALL)) -public class UiEndpoint { - - @Get("/") - public HttpResponse index() { - return HttpResponses.staticResource("index.html"); // <1> - } -} diff --git a/src/main/java/demo/multiagent/application/ActivityAgent.java b/src/main/java/demo/multiagent/application/ActivityAgent.java index 2eabff6..95478dc 100644 --- a/src/main/java/demo/multiagent/application/ActivityAgent.java +++ b/src/main/java/demo/multiagent/application/ActivityAgent.java @@ -1,25 +1,24 @@ package demo.multiagent.application; import akka.javasdk.agent.Agent; -import akka.javasdk.annotations.AgentRole; import akka.javasdk.annotations.Component; import akka.javasdk.client.ComponentClient; import demo.multiagent.domain.AgentRequest; import java.util.stream.Collectors; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; @Component( id = "activity-agent", name = "Activity Agent", description = """ - An agent that suggests activities in the real world. Like for example, - a team building activity, sports, an indoor or outdoor game, - board games, a city trip, etc. + An agent that suggests activities in the real world. Like for example, \ + a team building activity, sports, an indoor or outdoor game, \ + board games, a city trip, etc.\ """ ) -@AgentRole("worker") public class ActivityAgent extends Agent { - private static final String SYSTEM_MESSAGE = """ You are an activity agent. Your job is to suggest activities in the real world. @@ -32,6 +31,8 @@ public class ActivityAgent extends Agent { Start the error response with ERROR. """.stripIndent(); + private static final Logger logger = LoggerFactory.getLogger(ActivityAgent.class); + private final ComponentClient componentClient; public ActivityAgent(ComponentClient componentClient) { @@ -39,6 +40,7 @@ public ActivityAgent(ComponentClient componentClient) { } public Effect query(AgentRequest request) { + logger.info("Invoked for user [{}] with: {}", request.userId(), request.message()); var allPreferences = componentClient .forEventSourcedEntity(request.userId()) .method(PreferencesEntity::getPreferences) diff --git a/src/main/java/demo/multiagent/application/ActivityCoordinator.java b/src/main/java/demo/multiagent/application/ActivityCoordinator.java new file mode 100644 index 0000000..81e1928 --- /dev/null +++ b/src/main/java/demo/multiagent/application/ActivityCoordinator.java @@ -0,0 +1,32 @@ +package demo.multiagent.application; + +import akka.javasdk.agent.autonomous.AgentDefinition; +import akka.javasdk.agent.autonomous.AutonomousAgent; +import akka.javasdk.agent.autonomous.capability.Delegation; +import akka.javasdk.agent.autonomous.capability.TaskAcceptance; +import akka.javasdk.annotations.Component; + +@Component( + id = "activity-coordinator", + description = """ + Coordinates worker agents to suggest real-world activities for a user. \ + Decides whether to consult the weather agent, the activity agent, or both, \ + and synthesizes their results into a single suggestion.\ + """ +) +public class ActivityCoordinator extends AutonomousAgent { // <1> + + @Override + public AgentDefinition definition() { + return define() + .instructions( + """ + When delegating to the activity agent, include the userId from the task header \ + (the "User: " line) in the request so the agent can fetch the user's \ + preferences.\ + """ + ) // <2> + .capability(TaskAcceptance.of(ActivityTasks.SUGGEST_ACTIVITIES).maxIterationsPerTask(5)) // <3> + .capability(Delegation.to(WeatherAgent.class, ActivityAgent.class)); // <4> + } +} diff --git a/src/main/java/demo/multiagent/application/ActivityTasks.java b/src/main/java/demo/multiagent/application/ActivityTasks.java new file mode 100644 index 0000000..7adb763 --- /dev/null +++ b/src/main/java/demo/multiagent/application/ActivityTasks.java @@ -0,0 +1,16 @@ +package demo.multiagent.application; + +import akka.javasdk.agent.task.Task; + +public class ActivityTasks { + + public static final Task SUGGEST_ACTIVITIES = Task.name( + "SuggestActivities" + ).description( + """ + Suggest real-world activities for a user, taking weather and any stated preferences \ + into account. The task instructions begin with a "User: " line followed by \ + the user's question.\ + """ + ); +} diff --git a/src/main/java/demo/multiagent/application/ActivityView.java b/src/main/java/demo/multiagent/application/ActivityView.java deleted file mode 100644 index b076b5f..0000000 --- a/src/main/java/demo/multiagent/application/ActivityView.java +++ /dev/null @@ -1,59 +0,0 @@ -package demo.multiagent.application; - -import akka.javasdk.annotations.Component; -import akka.javasdk.annotations.Consume; -import akka.javasdk.annotations.DeleteHandler; -import akka.javasdk.annotations.Query; -import akka.javasdk.view.TableUpdater; -import akka.javasdk.view.View; -import java.util.List; -import java.util.Optional; - -@Component(id = "activity-view") -public class ActivityView extends View { - - public record ActivityEntries(List entries) {} - - public record ActivityEntry( - String userId, - String sessionId, - String userQuestion, - String finalAnswer - ) { - public boolean hasFinalAnswer() { - return finalAnswer != null && !finalAnswer.isEmpty(); - } - } - - @Query("SELECT * AS entries FROM activities WHERE userId = :userId") - public QueryEffect getActivities(String userId) { - return queryResult(); - } - - @Consume.FromWorkflow(AgentTeamWorkflow.class) - public static class Updater extends TableUpdater { - - public Effect onStateChange(AgentTeamWorkflow.State state) { - var sessionId = updateContext().eventSubject().get(); // the workflow id - var currentAnswer = rowState() == null ? "" : rowState().finalAnswer; - if (currentAnswer.equals(state.finalAnswer())) { // avoid updating the state if no relevant changes - return effects().ignore(); - } else { - return effects() - .updateRow( - new ActivityEntry( - state.userId(), - sessionId, - state.userQuery(), - state.finalAnswer() - ) - ); - } - } - - @DeleteHandler - public Effect onDelete() { - return effects().deleteRow(); - } - } -} diff --git a/src/main/java/demo/multiagent/application/AgentTeamEvaluatorConsumer.java b/src/main/java/demo/multiagent/application/AgentTeamEvaluatorConsumer.java deleted file mode 100644 index 8378230..0000000 --- a/src/main/java/demo/multiagent/application/AgentTeamEvaluatorConsumer.java +++ /dev/null @@ -1,75 +0,0 @@ -package demo.multiagent.application; - -import akka.javasdk.agent.evaluator.SummarizationEvaluator; -import akka.javasdk.agent.evaluator.ToxicityEvaluator; -import akka.javasdk.annotations.Component; -import akka.javasdk.annotations.Consume; -import akka.javasdk.client.ComponentClient; -import akka.javasdk.consumer.Consumer; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -@Component(id = "agent-team-eval-consumer") -@Consume.FromWorkflow(AgentTeamWorkflow.class) -public class AgentTeamEvaluatorConsumer extends Consumer { // <1> - - private static final Logger logger = LoggerFactory.getLogger( - AgentTeamEvaluatorConsumer.class - ); - - private final ComponentClient componentClient; - - public AgentTeamEvaluatorConsumer(ComponentClient componentClient) { - this.componentClient = componentClient; - } - - public Effect onStateChanged(AgentTeamWorkflow.State state) { // <2> - if (state.status() == AgentTeamWorkflow.Status.COMPLETED) { - evalToxicity(state); - evalSummarization(state); - } - return effects().done(); - } - - private void evalToxicity(AgentTeamWorkflow.State state) { - var result = componentClient - .forAgent() - .inSession(sessionId()) - .method(ToxicityEvaluator::evaluate) // <3> - .invoke(state.finalAnswer()); - if (result.passed()) { - logger.debug("Eval toxicity passed, session [{}]", sessionId()); // <4> - } else { - logger.warn( - "Eval toxicity failed, session [{}], explanation: {}", - sessionId(), - result.explanation() - ); - } - } - - private void evalSummarization(AgentTeamWorkflow.State state) { - var agentsAnswers = String.join("\n\n", state.agentResponses().values()); - - var result = componentClient - .forAgent() - .inSession(sessionId()) - .method(SummarizationEvaluator::evaluate) - .invoke( - new SummarizationEvaluator.EvaluationRequest(agentsAnswers, state.finalAnswer()) - ); - if (result.passed()) { - logger.debug("Eval summarization passed, session [{}]", sessionId()); - } else { - logger.warn( - "Eval summarization failed, session [{}], explanation: {}", - sessionId(), - result.explanation() - ); - } - } - - private String sessionId() { - return messageContext().eventSubject().get(); - } -} diff --git a/src/main/java/demo/multiagent/application/AgentTeamWorkflow.java b/src/main/java/demo/multiagent/application/AgentTeamWorkflow.java deleted file mode 100644 index 46ba0d6..0000000 --- a/src/main/java/demo/multiagent/application/AgentTeamWorkflow.java +++ /dev/null @@ -1,309 +0,0 @@ -package demo.multiagent.application; - -import static demo.multiagent.application.AgentTeamWorkflow.Status.COMPLETED; -import static demo.multiagent.application.AgentTeamWorkflow.Status.FAILED; -import static demo.multiagent.application.AgentTeamWorkflow.Status.STARTED; -import static java.time.Duration.ofMillis; -import static java.time.Duration.ofSeconds; - -import akka.Done; -import akka.javasdk.NotificationPublisher; -import akka.javasdk.annotations.Component; -import akka.javasdk.annotations.StepName; -import akka.javasdk.client.ComponentClient; -import akka.javasdk.client.DynamicMethodRef; -import akka.javasdk.workflow.Workflow; -import akka.javasdk.workflow.Workflow.RecoverStrategy; -import akka.stream.Materializer; -import com.fasterxml.jackson.annotation.JsonSubTypes; -import com.fasterxml.jackson.annotation.JsonTypeInfo; -import demo.multiagent.domain.AgentRequest; -import demo.multiagent.domain.AgentSelection; -import demo.multiagent.domain.Plan; -import demo.multiagent.domain.PlanStep; -import java.util.HashMap; -import java.util.Map; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -@Component(id = "agent-team") -public class AgentTeamWorkflow extends Workflow { // <1> - - public record Request(String userId, String message) {} - - enum Status { - STARTED, - COMPLETED, - FAILED, - } - - public record State( - String userId, - String userQuery, - Plan plan, - String finalAnswer, - Map agentResponses, - Status status - ) { - public static State init(String userId, String query) { - return new State(userId, query, new Plan(), "", new HashMap<>(), STARTED); - } - - public State withFinalAnswer(String answer) { - return new State(userId, userQuery, plan, answer, agentResponses, status); - } - - public State addAgentResponse(String response) { - // when we add a response, we always do it for the agent at the head of the plan queue - // therefore we remove it from the queue and proceed - var agentId = plan.steps().removeFirst().agentId(); - agentResponses.put(agentId, response); - return this; - } - - public PlanStep nextStepPlan() { - return plan.steps().getFirst(); - } - - public boolean hasMoreSteps() { - return !plan.steps().isEmpty(); - } - - public State withPlan(Plan plan) { - return new State(userId, userQuery, plan, finalAnswer, agentResponses, STARTED); - } - - public State complete() { - return new State(userId, userQuery, plan, finalAnswer, agentResponses, COMPLETED); - } - - public State failed() { - return new State(userId, userQuery, plan, finalAnswer, agentResponses, FAILED); - } - } - - private static final Logger logger = LoggerFactory.getLogger(AgentTeamWorkflow.class); - - private final ComponentClient componentClient; - private final NotificationPublisher notificationPublisher; - private final Materializer materializer; - - public AgentTeamWorkflow( - ComponentClient componentClient, - NotificationPublisher notificationPublisher, - Materializer materializer - ) { - this.componentClient = componentClient; - this.notificationPublisher = notificationPublisher; - this.materializer = materializer; - } - - @JsonTypeInfo(use = JsonTypeInfo.Id.NAME) - @JsonSubTypes( - { - @JsonSubTypes.Type(value = AgentTeamNotification.StatusUpdate.class, name = "S"), - @JsonSubTypes.Type(value = AgentTeamNotification.LlmResponseStart.class, name = "LS"), - @JsonSubTypes.Type(value = AgentTeamNotification.LlmResponseDelta.class, name = "LD"), - @JsonSubTypes.Type(value = AgentTeamNotification.LlmResponseEnd.class, name = "LE"), - } - ) - public sealed interface AgentTeamNotification { - record StatusUpdate(String msg) implements AgentTeamNotification {} - - record LlmResponseStart() implements AgentTeamNotification {} - - record LlmResponseDelta(String response) implements AgentTeamNotification {} - - record LlmResponseEnd() implements AgentTeamNotification {} - } - - @Override - public WorkflowSettings settings() { - return WorkflowSettings.builder() - .defaultStepTimeout(ofSeconds(30)) - .defaultStepRecovery( - RecoverStrategy.maxRetries(1).failoverTo(AgentTeamWorkflow::interruptStep) - ) - .stepRecovery( - AgentTeamWorkflow::selectAgentsStep, - RecoverStrategy.maxRetries(1).failoverTo(AgentTeamWorkflow::summarizeStep) - ) - .build(); - } - - public Effect start(Request request) { - if (currentState() == null) { - return effects() - .updateState(State.init(request.userId(), request.message())) - .transitionTo(AgentTeamWorkflow::selectAgentsStep) // <3> - .thenReply(Done.getInstance()); - } else { - return effects() - .error("Workflow '" + commandContext().workflowId() + "' already started"); - } - } - - - public Effect runAgain() { - if (currentState() != null) { - return effects() - .updateState(State.init(currentState().userId(), currentState().userQuery())) - .transitionTo(AgentTeamWorkflow::selectAgentsStep) // <3> - .thenReply(Done.getInstance()); - } else { - return effects() - .error("Workflow '" + commandContext().workflowId() + "' has not been started"); - } - } - - - public ReadOnlyEffect getAnswer() { - if (currentState() == null) { - return effects().error("Workflow '" + commandContext().workflowId() + "' not started"); - } else { - return effects().reply(currentState().finalAnswer()); - } - } - - @StepName("select-agents") - private StepEffect selectAgentsStep() { // <2> - var selection = componentClient - .forAgent() - .inSession(sessionId()) - .method(SelectorAgent::selectAgents) - .invoke(currentState().userQuery); // <4> - - logger.info("Selected agents: {}", selection.agents()); - notificationPublisher.publish( - new AgentTeamNotification.StatusUpdate("Agents selected: " + selection.agents()) - ); - if (selection.agents().isEmpty()) { - var newState = currentState() - .withFinalAnswer("Couldn't find any agent(s) able to respond to the original query.") - .failed(); - return stepEffects().updateState(newState).thenEnd(); // terminate workflow - } else { - return stepEffects() - .thenTransitionTo(AgentTeamWorkflow::createPlanStep) - .withInput(selection); // <5> - } - } - - @StepName("create-plan") - private StepEffect createPlanStep(AgentSelection agentSelection) { // <2> - logger.info( - "Calling planner with: '{}' / {}", - currentState().userQuery, - agentSelection.agents() - ); - - var plan = componentClient - .forAgent() - .inSession(sessionId()) - .method(PlannerAgent::createPlan) - .invoke(new PlannerAgent.Request(currentState().userQuery, agentSelection)); // <6> - - logger.info("Execution plan: {}", plan); - notificationPublisher.publish( - new AgentTeamNotification.StatusUpdate( - "Execution plan formed. Number of steps: " + plan.steps().size() - ) - ); - return stepEffects() - .updateState(currentState().withPlan(plan)) - .thenTransitionTo(AgentTeamWorkflow::executePlanStep); // <7> - } - - @StepName("execute-plan") - private StepEffect executePlanStep() { // <2> - var stepPlan = currentState().nextStepPlan(); // <8> - logger.info( - "Executing plan step (agent:{}), asking {}", - stepPlan.agentId(), - stepPlan.query() - ); - notificationPublisher.publish( - new AgentTeamNotification.StatusUpdate("Calling: " + stepPlan.agentId()) - ); - var agentResponse = callAgent(stepPlan.agentId(), stepPlan.query()); // <9> - if (agentResponse.startsWith("ERROR")) { - throw new RuntimeException( - "Agent '" + stepPlan.agentId() + "' responded with error: " + agentResponse - ); - } else { - logger.info("Response from [agent:{}]: '{}'", stepPlan.agentId(), agentResponse); - var newState = currentState().addAgentResponse(agentResponse); - - if (newState.hasMoreSteps()) { - logger.info("Still {} steps to execute.", newState.plan().steps().size()); - return stepEffects() - .updateState(newState) - .thenTransitionTo(AgentTeamWorkflow::executePlanStep); // <10> - } else { - logger.info("No further steps to execute."); - return stepEffects() - .updateState(newState) - .thenTransitionTo(AgentTeamWorkflow::summarizeStep); - } - } - } - - private String callAgent(String agentId, String query) { - // We know the id of the agent to call, but not the agent class. - // Could be WeatherAgent or ActivityAgent. - // We can still invoke the agent based on its id, given that we know that it - // takes an AgentRequest parameter and returns String. - var request = new AgentRequest(currentState().userId(), query); - DynamicMethodRef call = componentClient - .forAgent() - .inSession(sessionId()) - .dynamicCall(agentId); // <9> - return call.invoke(request); - } - - - @StepName("summarize") - private StepEffect summarizeStep() { // <2> - var agentsAnswers = currentState().agentResponses.values(); - - var tokenSource = componentClient - .forAgent() - .inSession(sessionId()) - .tokenStream(SummarizerAgent::summarize) - .source(new SummarizerAgent.Request(currentState().userQuery, agentsAnswers)); - - notificationPublisher.publish(new AgentTeamNotification.LlmResponseStart()); - var finalAnswer = notificationPublisher.publishTokenStream( - tokenSource, - 10, - ofMillis(200), - AgentTeamNotification.LlmResponseDelta::new, - materializer - ); - - notificationPublisher.publish(new AgentTeamNotification.LlmResponseEnd()); - notificationPublisher.publish( - new AgentTeamNotification.StatusUpdate("All steps completed!") - ); - - return stepEffects() - .updateState(currentState().withFinalAnswer(finalAnswer).complete()) - .thenPause(); - } - - - @StepName("interrupt") - private StepEffect interruptStep() { - logger.info("Interrupting workflow"); - - return stepEffects().updateState(currentState().failed()).thenEnd(); - } - - public NotificationPublisher.NotificationStream updates() { - return notificationPublisher.stream(); - } - - private String sessionId() { - return commandContext().workflowId(); - } -} diff --git a/src/main/java/demo/multiagent/application/EvaluationConsumer.java b/src/main/java/demo/multiagent/application/EvaluationConsumer.java new file mode 100644 index 0000000..faec33c --- /dev/null +++ b/src/main/java/demo/multiagent/application/EvaluationConsumer.java @@ -0,0 +1,72 @@ +package demo.multiagent.application; + +import akka.javasdk.agent.evaluator.ToxicityEvaluator; +import akka.javasdk.agent.task.TaskEntity; +import akka.javasdk.agent.task.TaskEvent; +import akka.javasdk.annotations.Component; +import akka.javasdk.annotations.Consume; +import akka.javasdk.client.ComponentClient; +import akka.javasdk.consumer.Consumer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +@Component(id = "evaluation-consumer") +@Consume.FromEventSourcedEntity(TaskEntity.class) // <1> +public class EvaluationConsumer extends Consumer { + + private static final Logger logger = LoggerFactory.getLogger(EvaluationConsumer.class); + + private final ComponentClient componentClient; + + public EvaluationConsumer(ComponentClient componentClient) { + this.componentClient = componentClient; + } + + public Effect onEvent(TaskEvent event) { + if ( + event instanceof TaskEvent.TaskCompleted completed && + ActivityTasks.SUGGEST_ACTIVITIES.name().equals(completed.name()) + ) { // <2> + var taskId = completed.taskId(); + var snapshot = componentClient.forTask(taskId).get(ActivityTasks.SUGGEST_ACTIVITIES); // <3> + + // Custom LLM-as-judge: compare answer against the original (preference-aware) request + var judgement = componentClient + .forAgent() + .inSession(taskId) + .method(EvaluatorAgent::evaluate) + .invoke( + new EvaluatorAgent.EvaluationRequest( + snapshot.instructions(), + snapshot.result().orElse("") + ) + ); // <4> + if (judgement.passed()) { + logger.debug("LLM judge passed for task [{}]", taskId); + } else { + logger.warn( + "LLM judge failed for task [{}], explanation: {}", + taskId, + judgement.explanation() + ); + } + + // Built-in toxicity evaluator on the final answer + var toxicity = componentClient + .forAgent() + .inSession(taskId) + .method(ToxicityEvaluator::evaluate) + .invoke(snapshot.result().orElse("")); // <5> + if (toxicity.passed()) { + logger.debug("Toxicity check passed for task [{}]", taskId); + } else { + logger.warn( + "Toxicity check failed for task [{}], explanation: {}", + taskId, + toxicity.explanation() + ); + } + } + return effects().done(); + } +} diff --git a/src/main/java/demo/multiagent/application/EvaluatorAgent.java b/src/main/java/demo/multiagent/application/EvaluatorAgent.java index ff367c1..c735e80 100644 --- a/src/main/java/demo/multiagent/application/EvaluatorAgent.java +++ b/src/main/java/demo/multiagent/application/EvaluatorAgent.java @@ -4,27 +4,20 @@ import akka.javasdk.agent.Agent; import akka.javasdk.agent.EvaluationResult; import akka.javasdk.annotations.Component; -import akka.javasdk.client.ComponentClient; -import java.util.List; import java.util.Locale; -import java.util.stream.Collectors; @Component( id = "evaluator-agent", name = "Evaluator Agent", description = """ - An agent that acts as an LLM judge to evaluate the quality of AI responses. - It assesses whether the final answer is appropriate for the original question - and checks for any deviations from user preferences. + An agent that acts as an LLM judge to evaluate the quality of AI responses. \ + It assesses whether the final answer appropriately addresses the original request \ + and respects any user preferences carried in the request.\ """ ) public class EvaluatorAgent extends Agent { - public record EvaluationRequest( - String userId, - String originalRequest, - String finalAnswer - ) {} + public record EvaluationRequest(String originalRequest, String finalAnswer) {} public record Result(String explanation, String label) implements EvaluationResult { public boolean passed() { @@ -42,27 +35,26 @@ public boolean passed() { } } - private static final String SYSTEM_MESSAGE = // <1> + private static final String SYSTEM_MESSAGE = """ You are an evaluator agent that acts as an LLM judge. Your job is to evaluate the quality and appropriateness of AI-generated responses. - Your evaluation should focus on: + The original request may include user preferences. Your evaluation should focus on: 1. Whether the final answer appropriately addresses the original question - 2. Whether the answer respects and aligns with the user's stated preferences + 2. Whether the answer respects and aligns with any stated user preferences 3. The overall quality, relevance, and helpfulness of the response - 4. Any potential deviations or inconsistencies with user preferences A response is "Incorrect" if it meets ANY of the following failure conditions: - poor response with significant issues or minor preference violations - unacceptable response that fails to address the question or violates preferences A response is "Correct" if it: - - fully addresses the question and respects all preferences + - fully addresses the question and respects all stated preferences - good response with minor issues but respects preferences IMPORTANT: - - Any violations of user preferences should result in an incorrect evaluation since + - Any violations of user preferences should result in an Incorrect evaluation since respecting user preferences is the most important criteria Your response must be a single JSON object with the following fields: @@ -70,53 +62,22 @@ public boolean passed() { - "label": A string, either "Correct" or "Incorrect". """.stripIndent(); - private final ComponentClient componentClient; - - public EvaluatorAgent(ComponentClient componentClient) { - this.componentClient = componentClient; - } - public Effect evaluate(EvaluationRequest request) { - var allPreferences = componentClient - .forEventSourcedEntity(request.userId()) - .method(PreferencesEntity::getPreferences) - .invoke(); // <2> + var prompt = + """ + ORIGINAL REQUEST: + %s - String evaluationPrompt = buildEvaluationPrompt( - request.originalRequest(), - request.finalAnswer(), - allPreferences.entries() - ); + FINAL ANSWER TO EVALUATE: + %s + + Please evaluate the final answer against the original request. + """.formatted(request.originalRequest(), request.finalAnswer()); return effects() .systemMessage(SYSTEM_MESSAGE) - .userMessage(evaluationPrompt) - .responseConformsTo(Result.class) // <3> + .userMessage(prompt) + .responseConformsTo(Result.class) .thenReply(); } - - private String buildEvaluationPrompt( - String originalRequest, - String finalAnswer, - List preferences - ) { - StringBuilder prompt = new StringBuilder(); - - prompt.append("ORIGINAL REQUEST:\n").append(originalRequest).append("\n\n"); - - prompt.append("FINAL ANSWER TO EVALUATE:\n").append(finalAnswer).append("\n\n"); - - if (!preferences.isEmpty()) { - prompt - .append("USER PREFERENCES:\n") - .append(preferences.stream().collect(Collectors.joining("\n", "- ", ""))) - .append("\n\n"); - } - - prompt - .append("Please evaluate the final answer against the original request") - .append(preferences.isEmpty() ? "." : " and user preferences."); - - return prompt.toString(); - } } diff --git a/src/main/java/demo/multiagent/application/PlannerAgent.java b/src/main/java/demo/multiagent/application/PlannerAgent.java deleted file mode 100644 index 053e7eb..0000000 --- a/src/main/java/demo/multiagent/application/PlannerAgent.java +++ /dev/null @@ -1,82 +0,0 @@ -package demo.multiagent.application; - -import akka.javasdk.JsonSupport; -import akka.javasdk.agent.Agent; -import akka.javasdk.agent.AgentRegistry; -import akka.javasdk.annotations.Component; -import demo.multiagent.domain.AgentSelection; -import demo.multiagent.domain.Plan; -import demo.multiagent.domain.PlanStep; -import java.util.List; - -@Component( - id = "planner-agent", - name = "Planner", - description = """ - An agent that analyzes the user request and available agents to plan the tasks - to produce a suitable answer. - """ -) -public class PlannerAgent extends Agent { - - public record Request(String message, AgentSelection agentSelection) {} - - private final AgentRegistry agentsRegistry; - - public PlannerAgent(AgentRegistry agentsRegistry) { - this.agentsRegistry = agentsRegistry; - } - - private String buildSystemMessage(AgentSelection agentSelection) { - var agents = agentSelection.agents().stream().map(agentsRegistry::agentInfo).toList(); // <1> - return """ - Your job is to analyse the user request and the list of agents and devise the - best order in which the agents should be called in order to produce a - suitable answer to the user. - - You can find the list of exiting agents below (in JSON format): - %s - - Note that each agent has a description of its capabilities. - Given the user request, you must define the right ordering. - - Moreover, you must generate a concise request to be sent to each agent. - This agent request is of course based on the user original request, - but is tailored to the specific agent. Each individual agent should not - receive requests or any text that is not related with its domain of expertise. - - Your response should follow a strict json schema as defined bellow. - { - "steps": [ - { - "agentId": "", - "query: "", - } - ] - } - - The '' should be filled with the agent id. - The '' should contain the agent tailored message. - The order of the items inside the "steps" array should be the order of execution. - - Do not include any explanations or text outside of the JSON structure. - - """.stripIndent() - // note: here we are not using the full list of agents, but a pre-selection - .formatted(JsonSupport.encodeToString(agents)); // <2> - } - - public Effect createPlan(Request request) { - if (request.agentSelection.agents().size() == 1) { - // no need to call an LLM to make a plan where selection has a single agent - var step = new PlanStep(request.agentSelection.agents().getFirst(), request.message()); - return effects().reply(new Plan(List.of(step))); - } else { - return effects() - .systemMessage(buildSystemMessage(request.agentSelection)) - .userMessage(request.message()) - .responseConformsTo(Plan.class) - .thenReply(); - } - } -} diff --git a/src/main/java/demo/multiagent/application/PreferencesConsumer.java b/src/main/java/demo/multiagent/application/PreferencesConsumer.java deleted file mode 100644 index cbffe8a..0000000 --- a/src/main/java/demo/multiagent/application/PreferencesConsumer.java +++ /dev/null @@ -1,75 +0,0 @@ -package demo.multiagent.application; - -import akka.javasdk.annotations.Component; -import akka.javasdk.annotations.Consume; -import akka.javasdk.client.ComponentClient; -import akka.javasdk.consumer.Consumer; -import demo.multiagent.domain.PreferencesEvent; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -@Component(id = "preferences-consumer") -@Consume.FromEventSourcedEntity(PreferencesEntity.class) // <1> -public class PreferencesConsumer extends Consumer { // <2> - - private static final Logger logger = LoggerFactory.getLogger(PreferencesConsumer.class); - - private final ComponentClient componentClient; - - public PreferencesConsumer(ComponentClient componentClient) { - this.componentClient = componentClient; - } - - public Effect onPreferenceAdded(PreferencesEvent.PreferenceAdded event) { - var userId = messageContext().eventSubject().get(); // the entity id - logger.info("Preference added for user {}: {}", userId, event.preference()); - - // Get all activities (sessions) for this user from the ActivityView - var activities = componentClient - .forView() - .method(ActivityView::getActivities) - .invoke(userId); // <3> - - // Call EvaluatorAgent for each session - for (var activity : activities.entries()) { - if (activity.hasFinalAnswer()) { - var evaluationRequest = new EvaluatorAgent.EvaluationRequest( - userId, - activity.userQuestion(), - activity.finalAnswer() - ); - - var evaluationResult = componentClient - .forAgent() - .inSession(activity.sessionId()) - .method(EvaluatorAgent::evaluate) - .invoke(evaluationRequest); // <4> - - logger.info( - "Evaluation completed for session {}: label={}, explanation='{}'", - activity.sessionId(), - evaluationResult.label(), - evaluationResult.explanation() - ); - - if (!evaluationResult.passed()) { - // run the workflow again to generate a better answer - - componentClient - .forWorkflow(activity.sessionId()) - .method(AgentTeamWorkflow::runAgain) // <5> - .invoke(); - - logger.info( - "Started workflow {} for user {} to re-answer question: '{}'", - activity.sessionId(), - userId, - activity.userQuestion() - ); - } - } - } - - return effects().done(); - } -} diff --git a/src/main/java/demo/multiagent/application/SelectorAgent.java b/src/main/java/demo/multiagent/application/SelectorAgent.java deleted file mode 100644 index dfbd4cb..0000000 --- a/src/main/java/demo/multiagent/application/SelectorAgent.java +++ /dev/null @@ -1,64 +0,0 @@ -package demo.multiagent.application; - -import akka.javasdk.JsonSupport; -import akka.javasdk.agent.Agent; -import akka.javasdk.agent.AgentRegistry; -import akka.javasdk.annotations.Component; -import demo.multiagent.domain.AgentSelection; - -@Component( - id = "selector-agent", - name = "Selector Agent", - description = """ - An agent that analyses the user request and selects useful agents for - answering the request. - """ -) -public class SelectorAgent extends Agent { - - private final String systemMessage; - - public SelectorAgent(AgentRegistry agentsRegistry) { // <1> - var agents = agentsRegistry.agentsWithRole("worker"); // <2> - - this.systemMessage = """ - Your job is to analyse the user request and select the agents that should be - used to answer the user. In order to do that, you will receive a list of - available agents. Each agent has an id, a name and a description of its capabilities. - - For example, a user may be asking to book a trip. If you see that there is a - weather agent, a city trip agent and a hotel booking agent, you should select - those agents to complete the task. Note that this is just an example. The list - of available agents may vary, so you need to use reasoning to dissect the original - user request and using the list of available agents, - decide which agents must be selected. - - You don't need to come up with an execution order. Your task is to - analyze user's request and select the agents. - - Your response should follow a strict json schema as defined bellow. - It should contain a single field 'agents'. The field agents must be array of strings - containing the agent's IDs. If none of the existing agents are suitable for executing - the task, you return an empty array. - - { - "agents": [] - } - - Do not include any explanations or text outside of the JSON structure. - - You can find the list of existing agents below (in JSON format): - Also important, use the agent id to identify the agents. - %s - """.stripIndent() - .formatted(JsonSupport.encodeToString(agents)); // <3> - } - - public Effect selectAgents(String message) { - return effects() - .systemMessage(systemMessage) - .userMessage(message) - .responseConformsTo(AgentSelection.class) - .thenReply(); - } -} diff --git a/src/main/java/demo/multiagent/application/SummarizerAgent.java b/src/main/java/demo/multiagent/application/SummarizerAgent.java deleted file mode 100644 index c8d6afe..0000000 --- a/src/main/java/demo/multiagent/application/SummarizerAgent.java +++ /dev/null @@ -1,43 +0,0 @@ -package demo.multiagent.application; - -import akka.javasdk.agent.Agent; -import akka.javasdk.annotations.Component; -import java.util.Collection; -import java.util.stream.Collectors; - -@Component( - id = "summarizer-agent", - name = "Summarizer", - description = "An agent that creates a summary from responses provided by other agents" -) -public class SummarizerAgent extends Agent { - - public record Request(String originalQuery, Collection agentsResponses) {} - - private String buildSystemMessage(String userQuery) { - return """ - You will receive the original query and a message generate by different other agents. - - Your task is to build a new message using the message provided by the other agents. - You are not allowed to add any new information, you should only re-phrase it to make - them part of coherent message. - - The message to summarize will be provided between single quotes. - - ORIGINAL USER QUERY: - %s - """.formatted(userQuery); - } - - public StreamEffect summarize(Request request) { - var allResponses = request.agentsResponses - .stream() - .filter(response -> !response.startsWith("ERROR")) - .collect(Collectors.joining("\n\n")); - - return streamEffects() - .systemMessage(buildSystemMessage(request.originalQuery)) - .userMessage("Summarize the following: \n" + allResponses) - .thenReply(); - } -} diff --git a/src/main/java/demo/multiagent/application/WeatherAgent.java b/src/main/java/demo/multiagent/application/WeatherAgent.java index 9694e1a..ff6141e 100644 --- a/src/main/java/demo/multiagent/application/WeatherAgent.java +++ b/src/main/java/demo/multiagent/application/WeatherAgent.java @@ -1,22 +1,21 @@ package demo.multiagent.application; import akka.javasdk.agent.Agent; -import akka.javasdk.annotations.AgentRole; import akka.javasdk.annotations.Component; import akka.javasdk.annotations.FunctionTool; -import demo.multiagent.domain.AgentRequest; import java.time.LocalDateTime; import java.time.format.DateTimeFormatter; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; @Component( id = "weather-agent", name = "Weather Agent", description = """ - An agent that provides weather information. It can provide current weather, - forecasts, and other related information. + An agent that provides weather information. It can provide current weather, \ + forecasts, and other related information.\ """ ) -@AgentRole("worker") public class WeatherAgent extends Agent { @@ -36,17 +35,20 @@ public class WeatherAgent extends Agent { Start the error response with ERROR. """.stripIndent(); + private static final Logger logger = LoggerFactory.getLogger(WeatherAgent.class); + private final WeatherService weatherService; public WeatherAgent(WeatherService weatherService) { this.weatherService = weatherService; // <1> } - public Effect query(AgentRequest request) { + public Effect query(String request) { + logger.info("Invoked with: {}", request); return effects() .systemMessage(SYSTEM_MESSAGE) .tools(weatherService) // <2> - .userMessage(request.message()) + .userMessage(request) .thenReply(); } diff --git a/src/main/java/demo/multiagent/domain/AgentSelection.java b/src/main/java/demo/multiagent/domain/AgentSelection.java deleted file mode 100644 index 27222a4..0000000 --- a/src/main/java/demo/multiagent/domain/AgentSelection.java +++ /dev/null @@ -1,5 +0,0 @@ -package demo.multiagent.domain; - -import java.util.List; - -public record AgentSelection(List agents) {} diff --git a/src/main/java/demo/multiagent/domain/Plan.java b/src/main/java/demo/multiagent/domain/Plan.java deleted file mode 100644 index 0fe562f..0000000 --- a/src/main/java/demo/multiagent/domain/Plan.java +++ /dev/null @@ -1,16 +0,0 @@ -package demo.multiagent.domain; - -import java.util.ArrayList; -import java.util.List; - -/** - * Represents a plan consisting of multiple steps to be executed by different agents. - */ -public record Plan(List steps) { - /** - * Creates an empty plan with no steps. - */ - public Plan() { - this(new ArrayList<>()); - } -} diff --git a/src/main/java/demo/multiagent/domain/PlanStep.java b/src/main/java/demo/multiagent/domain/PlanStep.java deleted file mode 100644 index bf11fac..0000000 --- a/src/main/java/demo/multiagent/domain/PlanStep.java +++ /dev/null @@ -1,7 +0,0 @@ -package demo.multiagent.domain; - -/** - * Represents a single step within a Plan. - * Each step is assigned to a specific agent and contains a command description. - */ -public record PlanStep(String agentId, String query) {} diff --git a/src/main/resources/application.conf b/src/main/resources/application.conf index 3e57da4..085d6ae 100644 --- a/src/main/resources/application.conf +++ b/src/main/resources/application.conf @@ -8,17 +8,20 @@ akka.javasdk { openai { model-name = "gpt-4o-mini" + #model-name = "gpt-4.1" # Environment variable override for the API key api-key = ${?OPENAI_API_KEY} } googleai-gemini { - model-name = "gemini-2.5-flash" + model-name = "gemini-3-flash-preview" + #model-name = "gemini-3.1-pro-preview" api-key = ${?GOOGLE_AI_GEMINI_API_KEY} } anthropic { - model-name = "claude-opus-4-6" + model-name = "claude-haiku-4-5" + #model-name = "claude-opus-4-6" api-key = ${?ANTHROPIC_API_KEY} max-tokens = 5000 } diff --git a/src/main/resources/static-resources/index.html b/src/main/resources/static-resources/index.html deleted file mode 100644 index 0990977..0000000 --- a/src/main/resources/static-resources/index.html +++ /dev/null @@ -1,781 +0,0 @@ - - - - - - Activity Recommendation - - - - - - - - - - -
- - - - \ No newline at end of file diff --git a/src/test/java/demo/multiagent/api/ActivityEndpointIntegrationTest.java b/src/test/java/demo/multiagent/api/ActivityEndpointIntegrationTest.java index 5036692..b5d3a3a 100644 --- a/src/test/java/demo/multiagent/api/ActivityEndpointIntegrationTest.java +++ b/src/test/java/demo/multiagent/api/ActivityEndpointIntegrationTest.java @@ -1,92 +1,74 @@ package demo.multiagent.api; +import static akka.javasdk.testkit.TestModelProvider.AutonomousAgentTools.completeTask; +import static akka.javasdk.testkit.TestModelProvider.AutonomousAgentTools.delegateTo; import static org.assertj.core.api.Assertions.assertThat; import akka.http.javadsl.model.StatusCodes; -import akka.javasdk.JsonSupport; -import akka.javasdk.agent.evaluator.SummarizationEvaluator; import akka.javasdk.agent.evaluator.ToxicityEvaluator; import akka.javasdk.testkit.TestKit; import akka.javasdk.testkit.TestKitSupport; import akka.javasdk.testkit.TestModelProvider; import demo.multiagent.application.ActivityAgent; +import demo.multiagent.application.ActivityCoordinator; import demo.multiagent.application.EvaluatorAgent; -import demo.multiagent.application.PlannerAgent; -import demo.multiagent.application.SelectorAgent; -import demo.multiagent.application.SummarizerAgent; import demo.multiagent.application.WeatherAgent; -import demo.multiagent.domain.AgentSelection; -import demo.multiagent.domain.Plan; -import demo.multiagent.domain.PlanStep; import java.util.List; -import java.util.UUID; import java.util.concurrent.TimeUnit; import org.awaitility.Awaitility; import org.junit.jupiter.api.Test; public class ActivityEndpointIntegrationTest extends TestKitSupport { - private final TestModelProvider selectorModel = new TestModelProvider(); - private final TestModelProvider plannerModel = new TestModelProvider(); + private final TestModelProvider coordinatorModel = new TestModelProvider(); private final TestModelProvider activitiesModel = new TestModelProvider(); private final TestModelProvider weatherModel = new TestModelProvider(); - private final TestModelProvider summaryModel = new TestModelProvider(); private final TestModelProvider evaluatorModel = new TestModelProvider(); private final TestModelProvider toxicityEvalModel = new TestModelProvider(); - private final TestModelProvider summarizationEvalModel = new TestModelProvider(); @Override protected TestKit.Settings testKitSettings() { return TestKit.Settings.DEFAULT.withAdditionalConfig( "akka.javasdk.agent.openai.api-key = n/a" ) - .withModelProvider(SelectorAgent.class, selectorModel) - .withModelProvider(PlannerAgent.class, plannerModel) + .withModelProvider(ActivityCoordinator.class, coordinatorModel) .withModelProvider(ActivityAgent.class, activitiesModel) .withModelProvider(WeatherAgent.class, weatherModel) - .withModelProvider(SummarizerAgent.class, summaryModel) .withModelProvider(EvaluatorAgent.class, evaluatorModel) - .withModelProvider(ToxicityEvaluator.class, toxicityEvalModel) - .withModelProvider(SummarizationEvaluator.class, summarizationEvalModel); + .withModelProvider(ToxicityEvaluator.class, toxicityEvalModel); } @Test - public void shouldHandleFullActivitySuggestionWorkflowWithPreferenceUpdate() { - var sessionId = UUID.randomUUID().toString(); + public void shouldRunCoordinatorAndReturnAnswer() { var userId = "alice"; var query = "I am in Stockholm. What should I do? Beware of the weather"; - // Setup initial AI model responses - setupInitialModelResponses(); + setupModelResponses(); - //debug id for correlating tracing information - String debugId = "12345"; - - // 1. Call suggestActivities endpoint var suggestResponse = httpClient - .POST("/activities/" + userId + "/" + sessionId) + .POST("/activities/" + userId) .withRequestBody(new ActivityEndpoint.Request(query)) - .addHeader("akka-debug-id", debugId) + .responseBodyAs(String.class) .invoke(); assertThat(suggestResponse.status()).isEqualTo(StatusCodes.CREATED); - // Extract sessionId from Location header + var taskId = suggestResponse.body(); + assertThat(taskId).isNotBlank(); + var locationHeader = suggestResponse .httpResponse() .getHeader("Location") .orElseThrow() .value(); - var locationSessionId = extractSessionIdFromLocation(locationHeader, userId); - assertThat(locationSessionId).isEqualTo(sessionId); + assertThat(locationHeader).isEqualTo("/activities/" + userId + "/" + taskId); - // 2. Retrieve answer using the sessionId Awaitility.await() .ignoreExceptions() - .atMost(15, TimeUnit.SECONDS) + .atMost(20, TimeUnit.SECONDS) .untilAsserted(() -> { var answerResponse = httpClient - .GET("/activities/" + userId + "/" + sessionId) + .GET("/activities/" + userId + "/" + taskId) .responseBodyAs(String.class) .invoke(); assertThat(answerResponse.status()).isEqualTo(StatusCodes.OK); @@ -97,125 +79,43 @@ public void shouldHandleFullActivitySuggestionWorkflowWithPreferenceUpdate() { assertThat(answer).contains("sunny"); assertThat(answer).contains("bike tour"); }); - - // 3. Retrieve via listActivities - Awaitility.await() - .ignoreExceptions() - .atMost(10, TimeUnit.SECONDS) - .untilAsserted(() -> { - var listResponse = httpClient - .GET("/activities/" + userId) - .responseBodyAs(ActivityEndpoint.ActivitiesList.class) - .invoke(); - assertThat(listResponse.status()).isEqualTo(StatusCodes.OK); - - var activitiesList = listResponse.body(); - assertThat(activitiesList.suggestions()).hasSize(1); - - var suggestion = activitiesList.suggestions().getFirst(); - assertThat(suggestion.userQuestion()).isEqualTo(query); - assertThat(suggestion.answer()).contains("bike tour"); - - var steps = telemetryReader.getWorkflowSteps(debugId); - assertThat(steps).containsOnly( - "select-agents", - "create-plan", - "execute-plan", - "execute-plan", - "summarize" - ); - - var agents = telemetryReader.getAgents(debugId); - assertThat(agents).containsOnly( - "selector-agent", - "planner-agent", - "weather-agent", - "activity-agent", - "summarizer-agent", - "toxicity-evaluator", - "summarization-evaluator" - ); - }); - - // 4. Add preference that invalidates previous suggestion - setupUpdatedModelResponsesForPreference(); - - var nextDebugId = "67890"; - var preferenceResponse = httpClient - .POST("/preferences/" + userId) - .addHeader("akka-debug-id", nextDebugId) - .withRequestBody( - new ActivityEndpoint.AddPreference( - "I hate outdoor activities and prefer indoor museums" - ) - ) - .invoke(); - - assertThat(preferenceResponse.status()).isEqualTo(StatusCodes.CREATED); - - // Wait for preference to be processed and new suggestion generated - Awaitility.await() - .ignoreExceptions() - .atMost(20, TimeUnit.SECONDS) - .untilAsserted(() -> { - var updatedAnswerResponse = httpClient - .GET("/activities/" + userId + "/" + sessionId) - .responseBodyAs(String.class) - .invoke(); - assertThat(updatedAnswerResponse.status()).isEqualTo(StatusCodes.OK); - - var updatedAnswer = updatedAnswerResponse.body(); - // 5. Verify the new suggestion reflects the preference - assertThat(updatedAnswer).contains("Vasa Museum"); - assertThat(updatedAnswer).contains("indoor"); - assertThat(updatedAnswer).doesNotContain("bike tour"); - - assertThat(telemetryReader.getAgents(nextDebugId)).containsOnly( - "evaluator-agent", - "selector-agent", - "planner-agent", - "weather-agent", - "activity-agent", - "summarizer-agent", - "toxicity-evaluator", - "summarization-evaluator" - ); - }); } - private void setupInitialModelResponses() { - var selection = new AgentSelection(List.of("activity-agent", "weather-agent")); - selectorModel.fixedResponse(JsonSupport.encodeToString(selection)); - - var weatherQuery = "What is the current weather in Stockholm?"; - var activityQuery = - "Suggest activities to do in Stockholm considering the current weather."; - var plan = new Plan( - List.of( - new PlanStep("weather-agent", weatherQuery), - new PlanStep("activity-agent", activityQuery) - ) - ); - plannerModel.fixedResponse(JsonSupport.encodeToString(plan)); - - weatherModel - .whenMessage(req -> req.equals(weatherQuery)) - .reply("The weather in Stockholm is sunny."); + private void setupModelResponses() { + // Coordinator turn 1: the initial task instructions mention the location. + coordinatorModel + .whenMessage(msg -> msg.contains("Stockholm")) + .reply( + List.of( + delegateTo( + WeatherAgent.class, + "{\"request\":\"What is the current weather in Stockholm?\"}" + ), + delegateTo( + ActivityAgent.class, + "{\"request\":{\"userId\":\"alice\",\"message\":\"Suggest activities to do in Stockholm considering the current weather.\"}}" + ) + ) + ); - activitiesModel - .whenMessage(req -> req.equals(activityQuery)) + // Coordinator turn 2+: runtime sends "Continue working ..." or "Reminder ..." prompts + // after the workers complete. Synthesise the final answer on any such follow-up. + coordinatorModel + .whenMessage(msg -> msg.contains("Continue working") || msg.contains("Reminder")) .reply( - "You can take a bike tour around Djurgården Park, " + - "visit the Vasa Museum, explore Gamla Stan (Old Town)..." + completeTask( + "The weather in Stockholm is sunny, so you can enjoy outdoor activities like " + + "a bike tour around Djurgården Park, visiting the Vasa Museum, exploring Gamla Stan." + ) ); - summaryModel.fixedResponse( - "The weather in Stockholm is sunny, so you can enjoy " + - "outdoor activities like a bike tour around Djurgården Park, " + - "visiting the Vasa Museum, exploring Gamla Stan (Old Town)" + weatherModel.fixedResponse("The weather in Stockholm is sunny."); + + activitiesModel.fixedResponse( + "You can take a bike tour around Djurgården Park, " + + "visit the Vasa Museum, explore Gamla Stan (Old Town)..." ); - // Initial evaluator response (no preference conflict) evaluatorModel.fixedResponse( """ { @@ -232,52 +132,5 @@ private void setupInitialModelResponses() { } """.stripIndent() ); - - summarizationEvalModel.fixedResponse( - """ - { - "label" : "good" - } - """.stripIndent() - ); - } - - private void setupUpdatedModelResponsesForPreference() { - // Evaluator detects preference conflict and triggers new suggestion - evaluatorModel - .whenMessage(req -> req.contains("hate outdoor activities")) - .reply( - """ - { - "label": "Incorrect", - "feedback": "The previous suggestion conflicts with user preferences for indoor activities. Outdoor bike tours are not suitable." - } - """ - ); - - // Updated activity suggestion based on preference - activitiesModel - .whenMessage(req -> req.contains("hate outdoor activities")) - .reply( - "Based on your preference for indoor activities, I recommend visiting the " + - "Vasa Museum, the ABBA Museum, or exploring the Royal Palace indoor exhibitions." - ); - - // Updated summary reflecting preference - summaryModel - .whenMessage(req -> req.contains("preference for indoor activities")) - .reply( - "Given your preference for indoor activities, Stockholm offers excellent museums " + - "like the Vasa Museum and ABBA Museum, perfect for a cultural indoor experience." - ); - } - - private String extractSessionIdFromLocation(String locationHeader, String userId) { - // Location header format: /activities/{userId}/{sessionId} - var prefix = "/activities/" + userId + "/"; - if (locationHeader.startsWith(prefix)) { - return locationHeader.substring(prefix.length()); - } - throw new IllegalArgumentException("Invalid location header format: " + locationHeader); } } diff --git a/src/test/java/demo/multiagent/application/ActivityViewTest.java b/src/test/java/demo/multiagent/application/ActivityViewTest.java deleted file mode 100644 index b6b725f..0000000 --- a/src/test/java/demo/multiagent/application/ActivityViewTest.java +++ /dev/null @@ -1,165 +0,0 @@ -package demo.multiagent.application; - -import static org.assertj.core.api.Assertions.assertThat; - -import akka.javasdk.testkit.TestKit; -import akka.javasdk.testkit.TestKitSupport; -import com.typesafe.config.ConfigFactory; -import demo.multiagent.application.ActivityView.ActivityEntries; -import demo.multiagent.application.ActivityView.ActivityEntry; -import java.util.Set; -import java.util.concurrent.TimeUnit; -import org.awaitility.Awaitility; -import org.junit.jupiter.api.Test; - -public class ActivityViewTest extends TestKitSupport { - - @Override - protected TestKit.Settings testKitSettings() { - // Bootstrap will check if key exists when running integation tests. - // We don't need a real one though. - return TestKit.Settings.DEFAULT.withAdditionalConfig( - ConfigFactory.parseString("akka.javasdk.agent.openai.api-key=fake-key") - ) - .withWorkflowIncomingMessages(AgentTeamWorkflow.class) - .withDisabledComponents(Set.of(AgentTeamEvaluatorConsumer.class)); - } - - @Test - public void shouldHandleMultipleSessionsForSameUser() { - var workflowMessages = testKit.getWorkflowIncomingMessages(AgentTeamWorkflow.class); - - var userId = "alice"; - - // First session - var sessionId1 = "session-1"; - var userQuery1 = "What should I do in Stockholm?"; - var finalAnswer1 = "Visit the Vasa Museum and explore Gamla Stan."; - var workflowState1 = new AgentTeamWorkflow.State( - userId, - userQuery1, - null, - finalAnswer1, - null, - AgentTeamWorkflow.Status.COMPLETED - ); - - // Second session - var sessionId2 = "session-2"; - var userQuery2 = "What's the weather like?"; - var finalAnswer2 = "It's sunny today."; - var workflowState2 = new AgentTeamWorkflow.State( - userId, - userQuery2, - null, - finalAnswer2, - null, - AgentTeamWorkflow.Status.COMPLETED - ); - - // Publish both workflow state changes - workflowMessages.publish(workflowState1, sessionId1); - workflowMessages.publish(workflowState2, sessionId2); - - // Wait for view to be updated and verify both activities - Awaitility.await() - .ignoreExceptions() - .atMost(10, TimeUnit.SECONDS) - .untilAsserted(() -> { - ActivityEntries activities = componentClient - .forView() - .method(ActivityView::getActivities) - .invoke(userId); - - assertThat(activities.entries()).hasSize(2); - - // Verify both entries are present (order may vary) - assertThat(activities.entries()) - .extracting(ActivityEntry::sessionId) - .containsExactlyInAnyOrder(sessionId1, sessionId2); - - assertThat(activities.entries()) - .extracting(ActivityEntry::userId) - .containsOnly(userId); - - // Verify specific content of each activity - assertThat(activities.entries()) - .extracting(ActivityEntry::userQuestion) - .containsExactlyInAnyOrder(userQuery1, userQuery2); - - assertThat(activities.entries()) - .extracting(ActivityEntry::finalAnswer) - .containsExactlyInAnyOrder(finalAnswer1, finalAnswer2); - }); - } - - @Test - public void shouldReturnEmptyForNonExistentUser() { - ActivityEntries activities = componentClient - .forView() - .method(ActivityView::getActivities) - .invoke("non-existent-user"); - - assertThat(activities.entries()).isEmpty(); - } - - @Test - public void shouldFilterActivitiesByUserId() { - var workflowMessages = testKit.getWorkflowIncomingMessages(AgentTeamWorkflow.class); - - // Activity for user bob - var aliceState = new AgentTeamWorkflow.State( - "bob", - "Bob's query", - null, - "Bob's answer", - null, - AgentTeamWorkflow.Status.COMPLETED - ); - - // Activity for user charlie - var charlieState = new AgentTeamWorkflow.State( - "charlie", - "Charlie's query", - null, - "Charlie's answer", - null, - AgentTeamWorkflow.Status.COMPLETED - ); - - workflowMessages.publish(aliceState, "bob-session"); - workflowMessages.publish(charlieState, "charlie-session"); - - // Verify bob only sees her activities - Awaitility.await() - .ignoreExceptions() - .atMost(10, TimeUnit.SECONDS) - .untilAsserted(() -> { - ActivityEntries aliceActivities = componentClient - .forView() - .method(ActivityView::getActivities) - .invoke("bob"); - - assertThat(aliceActivities.entries()).hasSize(1); - assertThat(aliceActivities.entries().get(0).userId()).isEqualTo("bob"); - assertThat(aliceActivities.entries().get(0).userQuestion()).isEqualTo("Bob's query"); - }); - - // Verify charlie only sees his activities - Awaitility.await() - .ignoreExceptions() - .atMost(10, TimeUnit.SECONDS) - .untilAsserted(() -> { - ActivityEntries charlieActivities = componentClient - .forView() - .method(ActivityView::getActivities) - .invoke("charlie"); - - assertThat(charlieActivities.entries()).hasSize(1); - assertThat(charlieActivities.entries().get(0).userId()).isEqualTo("charlie"); - assertThat(charlieActivities.entries().get(0).userQuestion()).isEqualTo( - "Charlie's query" - ); - }); - } -} diff --git a/src/test/java/demo/multiagent/application/AgentTeamWorkflowTest.java b/src/test/java/demo/multiagent/application/AgentTeamWorkflowTest.java deleted file mode 100644 index 9431a99..0000000 --- a/src/test/java/demo/multiagent/application/AgentTeamWorkflowTest.java +++ /dev/null @@ -1,117 +0,0 @@ -package demo.multiagent.application; - -import static java.util.concurrent.TimeUnit.SECONDS; -import static org.assertj.core.api.Assertions.assertThat; - -import akka.javasdk.JsonSupport; -import akka.javasdk.agent.evaluator.SummarizationEvaluator; -import akka.javasdk.agent.evaluator.ToxicityEvaluator; -import akka.javasdk.testkit.TestKit; -import akka.javasdk.testkit.TestKitSupport; -import akka.javasdk.testkit.TestModelProvider; -import demo.multiagent.domain.AgentSelection; -import demo.multiagent.domain.Plan; -import demo.multiagent.domain.PlanStep; -import java.util.List; -import java.util.UUID; -import org.awaitility.Awaitility; -import org.junit.jupiter.api.Test; - -public class AgentTeamWorkflowTest extends TestKitSupport { // <1> - - private final TestModelProvider selectorModel = new TestModelProvider(); // <2> - private final TestModelProvider plannerModel = new TestModelProvider(); - private final TestModelProvider activitiesModel = new TestModelProvider(); - private final TestModelProvider weatherModel = new TestModelProvider(); - private final TestModelProvider summaryModel = new TestModelProvider(); - private final TestModelProvider toxicityEvalModel = new TestModelProvider(); - private final TestModelProvider summarizationEvalModel = new TestModelProvider(); - - @Override - protected TestKit.Settings testKitSettings() { - return TestKit.Settings.DEFAULT.withAdditionalConfig( - "akka.javasdk.agent.openai.api-key = n/a" - ) - .withModelProvider(SelectorAgent.class, selectorModel) // <3> - .withModelProvider(PlannerAgent.class, plannerModel) - .withModelProvider(ActivityAgent.class, activitiesModel) - .withModelProvider(WeatherAgent.class, weatherModel) - .withModelProvider(SummarizerAgent.class, summaryModel) - .withModelProvider(ToxicityEvaluator.class, toxicityEvalModel) - .withModelProvider(SummarizationEvaluator.class, summarizationEvalModel); - } - - @Test - public void test() { - var selection = new AgentSelection(List.of("activity-agent", "weather-agent")); - selectorModel.fixedResponse(JsonSupport.encodeToString(selection)); // <4> - - var weatherQuery = "What is the current weather in Stockholm?"; - var activityQuery = - "Suggest activities to do in Stockholm considering the current weather."; - var plan = new Plan( - List.of( - new PlanStep("weather-agent", weatherQuery), - new PlanStep("activity-agent", activityQuery) - ) - ); - plannerModel.fixedResponse(JsonSupport.encodeToString(plan)); - - weatherModel - .whenMessage(req -> req.equals(weatherQuery)) // <5> - .reply("The weather in Stockholm is sunny."); - - activitiesModel - .whenMessage(req -> req.equals(activityQuery)) - .reply( - "You can take a bike tour around Djurgården Park, " + - "visit the Vasa Museum, explore Gamla Stan (Old Town)..." - ); - - summaryModel.fixedResponse( - "The weather in Stockholm is sunny, so you can enjoy " + - "outdoor activities like a bike tour around Djurgården Park, " + - "visiting the Vasa Museum, exploring Gamla Stan (Old Town)" - ); - - toxicityEvalModel.fixedResponse( - """ - { - "label" : "non-toxic" - } - """.stripIndent() - ); - - summarizationEvalModel.fixedResponse( - """ - { - "label" : "good" - } - """.stripIndent() - ); - - var query = "I am in Stockholm. What should I do? Beware of the weather"; - - var sessionId = UUID.randomUUID().toString(); - var request = new AgentTeamWorkflow.Request("alice", query); - - componentClient - .forWorkflow(sessionId) - .method(AgentTeamWorkflow::start) // <6> - .invoke(request); - - Awaitility.await() - .ignoreExceptions() - .atMost(10, SECONDS) - .untilAsserted(() -> { - var answer = componentClient - .forWorkflow(sessionId) - .method(AgentTeamWorkflow::getAnswer) - .invoke(); - assertThat(answer).isNotBlank(); - assertThat(answer).contains("Stockholm"); - assertThat(answer).contains("sunny"); - assertThat(answer).contains("bike tour"); - }); - } -} diff --git a/src/test/java/demo/multiagent/application/WeatherAgentIntegrationTest.java b/src/test/java/demo/multiagent/application/WeatherAgentIntegrationTest.java new file mode 100644 index 0000000..2c07c5f --- /dev/null +++ b/src/test/java/demo/multiagent/application/WeatherAgentIntegrationTest.java @@ -0,0 +1,66 @@ +package demo.multiagent.application; + +import static org.assertj.core.api.Assertions.assertThat; + +import akka.javasdk.testkit.TestKit; +import akka.javasdk.testkit.TestKitSupport; +import akka.javasdk.testkit.TestModelProvider; +import akka.javasdk.testkit.TestModelProvider.AiResponse; +import akka.javasdk.testkit.TestModelProvider.ToolInvocationRequest; +import java.util.UUID; +import org.junit.jupiter.api.Test; + +public class WeatherAgentIntegrationTest extends TestKitSupport { // <1> + + private final TestModelProvider weatherModel = new TestModelProvider(); // <2> + + @Override + protected TestKit.Settings testKitSettings() { + return TestKit.Settings.DEFAULT.withAdditionalConfig( + "akka.javasdk.agent.openai.api-key = n/a" + ).withModelProvider(WeatherAgent.class, weatherModel); // <3> + } + + @Test + public void replyWithFixedResponse() { + weatherModel.fixedResponse("The weather in Madrid is sunny, 25°C."); // <4> + + var reply = componentClient + .forAgent() + .inSession(UUID.randomUUID().toString()) + .method(WeatherAgent::query) + .invoke("What is the weather in Madrid?"); + + assertThat(reply).contains("sunny"); + } + + + // The runtime prefixes a tool name with the simple name of the registered tool's class; + // here the WeatherAgent receives a FakeWeatherService instance (wired by the bootstrap + // when WEATHER_API_KEY is unset), so the tool name the model sees is "FakeWeatherService_getWeather". + private static final String GET_WEATHER_TOOL = "FakeWeatherService_getWeather"; + + @Test + public void invokeWeatherTool() { + // Turn 1: the mocked model asks the runtime to invoke the getWeather tool. + weatherModel + .whenMessage(msg -> msg.contains("Stockholm")) + .reply(new ToolInvocationRequest(GET_WEATHER_TOOL, "{\"location\":\"Stockholm\"}")); // <5> + + // Turn 2: the mocked model receives the tool result and produces the final answer. + // FakeWeatherService returns "It's always sunny in ." + weatherModel + .whenToolResult(tr -> tr.name().equals(GET_WEATHER_TOOL)) + .thenReply(tr -> new AiResponse("Forecast: " + tr.content())); // <6> + + var reply = componentClient + .forAgent() + .inSession(UUID.randomUUID().toString()) + .method(WeatherAgent::query) + .invoke("What is the weather in Stockholm?"); + + assertThat(reply).startsWith("Forecast:"); + assertThat(reply).contains("sunny"); + assertThat(reply).contains("Stockholm"); + } +}