diff --git a/README.md b/README.md index 2cfbecc..74f3f63 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # RubyLLM::Test -This gem provides testing utilities for RubyLLM, a Ruby library for working with large language models (LLMs). It enables calls to LLM's to be stubbed so that the surrounding application logic can be tested without making actual calls to the LLM. This is particularly useful for testing code that interacts with LLMs, as it allows developers to simulate responses from the LLM without incurring the cost, latency, or randomness of real API calls. +This gem provides testing utilities for RubyLLM, a Ruby library for working with large language models (LLMs). It enables calls to LLMs to be stubbed so that the surrounding application logic can be tested without making actual calls to the LLM. This is particularly useful for testing code that interacts with LLMs, as it allows developers to simulate responses from the LLM without incurring the cost, latency, or randomness of real API calls. ```ruby RubyLLM::Test.stub_response("Outlook good") @@ -13,7 +13,7 @@ assert_equal "Outlook good", response.content ## Installation -Add this line to your application's Gemfile: +Add this line to your application's Gemfile in the test group: ```ruby gem 'ruby_llm-test' @@ -109,6 +109,23 @@ RubyLLM::Test.with_responses('Hello, world!') do end ``` +### Testing Arguments + +You can verify arguments passed to the LLM by checking the requests received by the test provider with methods `requests` and `last_request`. + +```ruby +RubyLLM::Test.stub_response('Hello, world!') +chat = RubyLLM.chat(model: 'gpt-5-nano') +chat.with_tools(GreeterTool) +chat.ask('Hello?') +request = RubyLLM::Test.last_request + +assert_equal 'gpt-5-nano', request.model +assert_includes request.tool_classes, GreeterTool +``` + +Any parameters passed to the Provider's `complete` method are available on the request object. The `tool_classes` method is a helper that returns the classes of any tools included in the request. + ## Development After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. diff --git a/lib/ruby_llm/test/complete_parameters.rb b/lib/ruby_llm/test/complete_parameters.rb index 9565920..1efb524 100644 --- a/lib/ruby_llm/test/complete_parameters.rb +++ b/lib/ruby_llm/test/complete_parameters.rb @@ -51,6 +51,10 @@ def respond_to_missing?(name, include_private = false) key?(name) || super end + def tool_classes + (kwargs[:tools] || {}).values.map(&:class) + end + private def positional_name_to_value diff --git a/test/system/tools_test.rb b/test/system/tools_test.rb new file mode 100644 index 0000000..af31cf4 --- /dev/null +++ b/test/system/tools_test.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +require "test_helper" + +class ToolsTest < Minitest::Test + class ToolOne < RubyLLM::Tool + description "This is the first tool" + end + + class ToolTwo < RubyLLM::Tool + description "This is the second tool, it has a constructor" + + def initialize(foo:) + super() + @foo = foo + end + end + + def setup + RubyLLM::Test.reset + RubyLLM::Test.stub_response("stubbed response") + end + + def test_tools_can_be_inspected + tool_two_instance = ToolTwo.new(foo: "bar") + + chat = RubyLLM::Chat.new(model: "gpt-4-turbo") + chat.with_tools(ToolOne, tool_two_instance) + chat.ask("What is the meaning of life?") + + tools = RubyLLM::Test.last_request.tools + + assert_kind_of ToolOne, tools[ToolOne.new.name.to_sym] + assert_equal tool_two_instance, tools[tool_two_instance.name.to_sym] + end + + def test_tool_classes_can_be_inspected + chat = RubyLLM::Chat.new(model: "gpt-4-turbo") + chat.with_tools(ToolOne, ToolTwo.new(foo: "bar")) + chat.ask("What is the meaning of life?") + + tool_classes = RubyLLM::Test.last_request.tool_classes + + assert_includes tool_classes, ToolOne + assert_includes tool_classes, ToolTwo + end +end