after making this post a while ago, i tried out these three techniques for providing tool-result data to the LLM

• append to assistant message
• send as user response
• send model-specific tool-response type

Findings

turns out - the assistant-message appending works great for larger LLMs, but not so well for smol ones.

meanwhile the user-side method works better than expected!

i didnt spend too much time with the model-specific tool role stuff, since i want my tooling to remain model-agnostic.

i will probably switch to the user-side method now for gopilot, leaving behind the assistant-only approach

Tool call formatting improvements

Turns out - my initial tool calling formatting was SUPER token-inefficient - who knew...

So I went from this formatting

okay lemme look that up online
{"tool_name": "web_search", "args": {"query": "how to make milk rice"}}
just put milk and rice in a bowl and mix em

to this, MUCH simpler format

okay lemme look that up online
Tool: web_search("how to make milk rice")
Result: just put milk and rice in a bowl and mix em

which is like - just.... WAY better!!!!

• tokens reduced from 43 down to 24 (cost savings)
• way easier to read
• relies on models code-writing ability
• allows for specific assignment like in json: Tool: web_search(query="my query here")

i hope this is useful to someone out there.

if so, maybe share where you are applying it and tell us about your experience! <3