fix(agent): prefer Ollama Modelfile num_ctx over GGUF training max
_query_local_context_length was checking model_info.context_length (the GGUF training max) before num_ctx (the Modelfile runtime override), inverse to query_ollama_num_ctx. The two helpers therefore disagreed on the same model: hermes-brain:qwen3-14b-ctx32k # Modelfile: num_ctx 32768 underlying qwen3:14b GGUF # qwen3.context_length: 40960 query_ollama_num_ctx correctly returned 32768 (the value Ollama will actually allocate KV cache for). _query_local_context_length returned 40960, which let ContextCompressor grow conversations past 32768 before triggering compression — at which point Ollama silently truncated the prefix, corrupting context. Swap the order so num_ctx is checked first, matching query_ollama_num_ctx. Adds a parametrized test that seeds both values and asserts num_ctx wins. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -70,6 +70,44 @@ class TestQueryLocalContextLengthOllama:
|
||||
|
||||
assert result == 32768
|
||||
|
||||
def test_ollama_num_ctx_wins_over_model_info(self):
|
||||
"""When both num_ctx (Modelfile) and model_info (GGUF) are present,
|
||||
num_ctx wins because it's the *runtime* context Ollama actually
|
||||
allocates KV cache for. The GGUF model_info.context_length is the
|
||||
training max — using it would let Hermes grow conversations past
|
||||
the runtime limit and Ollama would silently truncate.
|
||||
|
||||
Concrete example: hermes-brain:qwen3-14b-ctx32k is a Modelfile
|
||||
derived from qwen3:14b with `num_ctx 32768`, but the underlying
|
||||
GGUF reports `qwen3.context_length: 40960` (training max). If
|
||||
Hermes used 40960 it would let the conversation grow past 32768
|
||||
before compressing, and Ollama would truncate the prefix.
|
||||
"""
|
||||
from agent.model_metadata import _query_local_context_length
|
||||
|
||||
show_resp = self._make_resp(200, {
|
||||
"model_info": {"qwen3.context_length": 40960},
|
||||
"parameters": "num_ctx 32768\ntemperature 0.6\n",
|
||||
})
|
||||
models_resp = self._make_resp(404, {})
|
||||
|
||||
client_mock = MagicMock()
|
||||
client_mock.__enter__ = lambda s: client_mock
|
||||
client_mock.__exit__ = MagicMock(return_value=False)
|
||||
client_mock.post.return_value = show_resp
|
||||
client_mock.get.return_value = models_resp
|
||||
|
||||
with patch("agent.model_metadata.detect_local_server_type", return_value="ollama"), \
|
||||
patch("httpx.Client", return_value=client_mock):
|
||||
result = _query_local_context_length(
|
||||
"hermes-brain:qwen3-14b-ctx32k", "http://100.77.243.5:11434/v1"
|
||||
)
|
||||
|
||||
assert result == 32768, (
|
||||
f"Expected num_ctx (32768) to win over model_info (40960), got {result}. "
|
||||
"If Hermes uses the GGUF training max, conversations will silently truncate."
|
||||
)
|
||||
|
||||
def test_ollama_show_404_falls_through(self):
|
||||
"""When /api/show returns 404, falls through to /v1/models/{model}."""
|
||||
from agent.model_metadata import _query_local_context_length
|
||||
|
||||
Reference in New Issue
Block a user