small change

[shubhammalhotra28]

Greptile Summary

Enhanced the llamacpp_vtable_get_info function to populate the context_length field with the actual context size from the loaded model. Previously, this field was always set to 0.

Changes:

• Added #include <nlohmann/json.hpp> header for JSON parsing
• When model is loaded (is_ready), retrieves model info JSON via rac_llm_llamacpp_get_model_info
• Parses JSON and extracts context_size field to populate out_info->context_length
• Includes proper error handling with try-catch block, defaulting to 0 on parse errors
• Correctly frees the allocated JSON string after parsing

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The change is well-implemented with proper error handling (try-catch), null checks, and memory management (freeing json_str). The logic safely falls back to the default value of 0 on any error, maintaining backward compatibility. The implementation follows the existing pattern used in the JNI layer and correctly uses the already-available rac_llm_llamacpp_get_model_info API.
• No files require special attention

Important Files Changed

Filename	Overview
src/backends/llamacpp/rac_backend_llamacpp_register.cpp	Added JSON parsing to populate context_length from model info when model is loaded

Sequence Diagram

sequenceDiagram
    participant Client
    participant VTable as llamacpp_vtable_get_info
    participant Backend as rac_llm_llamacpp
    participant Model as LlamaCppTextGeneration
    participant JSON as nlohmann::json

    Client->>VTable: get_info(impl, out_info)
    VTable->>VTable: Check null pointer
    VTable->>Backend: is_model_loaded(impl)
    Backend-->>VTable: is_ready status
    VTable->>VTable: Set is_ready, supports_streaming
    VTable->>VTable: Set default context_length = 0
    
    alt Model is loaded
        VTable->>Backend: get_model_info(impl, &json_str)
        Backend->>Model: get_model_info()
        Model-->>Backend: nlohmann::json object
        Backend->>Backend: dump() to string
        Backend->>Backend: strdup(json_str)
        Backend-->>VTable: RAC_SUCCESS + json_str
        
        VTable->>JSON: parse(json_str)
        
        alt JSON parse successful
            JSON-->>VTable: parsed json object
            VTable->>VTable: Check contains("context_size")
            VTable->>VTable: Check is_number()
            VTable->>VTable: Extract int32_t value
            VTable->>VTable: Set out_info->context_length
        else JSON parse error
            VTable->>VTable: Catch exception
            VTable->>VTable: Keep context_length = 0
        end
        
        VTable->>VTable: free(json_str)
    end
    
    VTable-->>Client: RAC_SUCCESS

Loading

[shubhammalhotra28]

@coderabbitai please review