Every task, your coding agent starts blind. Before it changes anything, it re-explores the repo: grep a term, open a file, follow an import, back out, try again. It is rebuilding a picture of a codebase it mapped an hour ago and threw away.
That rediscovery burns most of a run's tool calls, tokens, and latency, and it is pure overhead"
The author of this article brings up a very interesting problem -- that, at least as far as using LLM's as coders/coding assistants go, eventually context runs out and context related to the underlying codebase does too.
Historically (well, in the past couple of years!), a bunch of solutions have been proposed to address this problem (i.e., take abstracts/subsets/maps of code, write them to different databases and persistent storage methods, bring them back in when the LLM requires it, etc., etc.)...
But there's no really good solution to this problem (although, arguably Graft goes a lot farther than past tools and should be commended for that!) because the problem seems to lie in separate parts, across several problem domains:
1) LLM context window size -- limited. Anything that future LLM's do to make context windows larger will help ameliorate this problem.
2) Lack of a good way to represent a codebase to an LLM for training other than text.
In other words, first we need some kind of way to map codebases into Tensors rather than text (i.e., a higher-level "map" of the code) then train future LLM's on those code-specific Tensors.
3) Arguably, programming languages themselves share some of the blame...
Programming languages have historically been written so that an arbitrary corpus of text represents and can be interpreted and/or compiled into a computer program.
That is, while tools for mapping codebases exist, tools for directly training LLM's on those specific created "code maps" as Tensors, do not, do not seem to, or at least I'm currently unaware of any! :-)
(Anyway, just thinking aloud!)
Graft looks good, and looks like it has made some serious inroads to solving the problem...
They seem to understand the problem exceptionally well!
I'm glad Claude is so recognisable, it lets me bounce right off the empty calorie language very efficiently.
Maybe this thing is great, but it cannot be determined with this presentation.
>"The problem:
Every task, your coding agent starts blind. Before it changes anything, it re-explores the repo: grep a term, open a file, follow an import, back out, try again. It is rebuilding a picture of a codebase it mapped an hour ago and threw away.
That rediscovery burns most of a run's tool calls, tokens, and latency, and it is pure overhead"
The author of this article brings up a very interesting problem -- that, at least as far as using LLM's as coders/coding assistants go, eventually context runs out and context related to the underlying codebase does too.
Historically (well, in the past couple of years!), a bunch of solutions have been proposed to address this problem (i.e., take abstracts/subsets/maps of code, write them to different databases and persistent storage methods, bring them back in when the LLM requires it, etc., etc.)...
But there's no really good solution to this problem (although, arguably Graft goes a lot farther than past tools and should be commended for that!) because the problem seems to lie in separate parts, across several problem domains:
1) LLM context window size -- limited. Anything that future LLM's do to make context windows larger will help ameliorate this problem.
2) Lack of a good way to represent a codebase to an LLM for training other than text.
In other words, first we need some kind of way to map codebases into Tensors rather than text (i.e., a higher-level "map" of the code) then train future LLM's on those code-specific Tensors.
3) Arguably, programming languages themselves share some of the blame...
Programming languages have historically been written so that an arbitrary corpus of text represents and can be interpreted and/or compiled into a computer program.
That is, while tools for mapping codebases exist, tools for directly training LLM's on those specific created "code maps" as Tensors, do not, do not seem to, or at least I'm currently unaware of any! :-)
(Anyway, just thinking aloud!)
Graft looks good, and looks like it has made some serious inroads to solving the problem...
They seem to understand the problem exceptionally well!
Is this still cheaper when stale?
we use claude hooks. so that, post edits it auto updates and before using the graft tools also we check if it's already in sync.
Hooks also solve for the issue of the LLMs not calling our CLI tools instead of Grep.