Conversation
don't strip all whitespace, just the ones around the edges that make things look weird
this is important because the tool card is re-rendered frequently when streaming in a result
these are wasted tokens if using coding assistants to edit the main js
There was a problem hiding this comment.
So sharp. Interleaving that output is so nice. This looks great!
Models' "default" Amount Of Code Written when calling this tool still feels too long to me. This leads to issues like that below, where the model hallucinates column names in data is hasn't seen yet because it didn't stop to examine the output of glimpse(forested). We've prompted PA/Databot/side::kick() in the same way in their analogous tools for this reason.
If you still disagree, okay with me that maybe this is a matter of preference best resolved in our btw.mds. :)
|
Is there any way could suppress package startup messages by default in the tool UI? |
@simonpcouch ah that's a great point; thanks for showing me the example. I didn't take those lines initially because there are some shinychat limitations we need to fix around how the tool UI works when you're streaming in results. So I wasn't sure if I wanted to create a situation where the model tries to run code in too small of a chunk and ends up making that problem worse. I think I'll go back and add those lines in though after seeing your example. I'll look into suppressing package startup messages too! |
Closes #118
Summary
This PR adds
btw_tool_run_r(), a tool that executes R code in the global environment and returns the results to the LLM. I've marked the tool Experimental for now.What it captures
The tool captures and returns:
print(),cat(), etc.message()warning()stop()When an error occurs, all output up to the error is returned.
The tool makes use of recent changes in ellmer v0.4.0 to allow tools to return lists of
Content, includeContentImagetypes from plots. Each of the above output types are given btw-local content types which are also used to customize their display in shinychat.Security
This tool is disabled by default. It executes arbitrary code in the global environment without sandboxing or review. We recommend:
To enable the tool
The tool can be enable via an R option (in a session, an
.Rprofileor inbtw.md):Or equivalently via environment variable:
When this option is set,
btw_tools()will include thebtw_tool_run_r()tool, otherwise it is excluded frombtw_tools().In
btw_tools(), you can also explicitly include the"run","run_r"or"btw_tool_run_r"tool intools:Or in
btw.md:Dependencies
This feature adds a few additional suggested dependencies.
evaluatefor running and evaluating the LLM-written codefansiis available, we use it to translate ANSI colors to HTMLraggis available, we use it as the plot rendering device. The plot device can be customized by providing a function via the R optionbtw.run_r.graphics_device.