Clients.FFLiteLLMClient

Synchronous LiteLLM-backed AI client implementing FFAIClientBase contract.

Delegates all shared logic to BaseLiteLLMClient and provides only the synchronous completion() call and clone() factory.

Synchronous LiteLLM-backed AI client implementing FFAIClientBase contract.

Delegates all shared logic to BaseLiteLLMClient and provides only the synchronous completion() call and clone() factory.

class FFLiteLLMClient(model_string, config=None, *, api_key=None, api_base=None, api_version=None, system_instructions=None, temperature=None, max_tokens=None, fallbacks=None, retry_config=None, **kwargs)[source]

Bases: BaseLiteLLMClient, FFAIClientBase

LiteLLM-backed AI client implementing FFAIClientBase.

This client wraps LiteLLM’s completion() function while maintaining the FFAIClientBase contract for compatibility with FFAI wrapper.

Key features: - Internal conversation history management - Clone pattern for parallel execution - Model string routing (e.g., “azure/mistral-small-2503”) - Retry and fallback support

Parameters:

  • model_string (str) – LiteLLM model identifier (e.g., “openai/gpt-4”, “azure/my-deployment”)

  • config (dict[str, Any] | None) – Optional configuration dictionary

  • api_key (str | None) – API key (overrides env var)

  • api_base (str | None) – API base URL (overrides env var)

  • system_instructions (str) – System prompt

  • temperature (float) – Sampling temperature (0-2)

  • max_tokens (int) – Maximum tokens to generate

  • fallbacks (list[str] | None) – List of fallback model strings

  • retry_config (dict[str, Any] | None) – Retry configuration

  • api_version (str | None)

  • kwargs (Any)

Example

>>> client = FFLiteLLMClient(model_string="azure/mistral-small-2503")
>>> response = client.generate_response("Hello!")
>>>
>>> # With fallbacks
>>> client = FFLiteLLMClient(
...     model_string="anthropic/claude-3-opus",
...     fallbacks=["openai/gpt-4", "azure/gpt-4"]
... )
generate_response(prompt, model=None, system_instructions=None, temperature=None, max_tokens=None, **kwargs)[source]

Generate a response from the AI model with retry and fallback logic.

Retries are handled by retry_utils.get_configured_retry_decorator on the inner _call_primary method. If the primary model (and all its retries) fail, fallback models are tried once each.

Parameters:

  • prompt (str) – The user prompt

  • model (str | None) – Override model (appends to provider prefix)

  • system_instructions (str | None) – Override system instructions

  • temperature (float | None) – Override temperature

  • max_tokens (int | None) – Override max tokens

  • **kwargs (Any) – Additional LiteLLM parameters

Returns:

The generated response text

Raises:
Return type:

str

clone()[source]

Create a fresh clone of this client with empty history.

Used for thread-safe parallel execution where each thread needs an isolated client instance with the same configuration.

Returns:

New FFLiteLLMClient with same config, empty history.

Return type:

FFLiteLLMClient

Classes

class FFLiteLLMClient(model_string, config=None, *, api_key=None, api_base=None, api_version=None, system_instructions=None, temperature=None, max_tokens=None, fallbacks=None, retry_config=None, **kwargs)[source]

Bases: BaseLiteLLMClient, FFAIClientBase

LiteLLM-backed AI client implementing FFAIClientBase.

This client wraps LiteLLM’s completion() function while maintaining the FFAIClientBase contract for compatibility with FFAI wrapper.

Key features: - Internal conversation history management - Clone pattern for parallel execution - Model string routing (e.g., “azure/mistral-small-2503”) - Retry and fallback support

Parameters:

  • model_string (str) – LiteLLM model identifier (e.g., “openai/gpt-4”, “azure/my-deployment”)

  • config (dict[str, Any] | None) – Optional configuration dictionary

  • api_key (str | None) – API key (overrides env var)

  • api_base (str | None) – API base URL (overrides env var)

  • system_instructions (str) – System prompt

  • temperature (float) – Sampling temperature (0-2)

  • max_tokens (int) – Maximum tokens to generate

  • fallbacks (list[str] | None) – List of fallback model strings

  • retry_config (dict[str, Any] | None) – Retry configuration

  • api_version (str | None)

  • kwargs (Any)

Example

>>> client = FFLiteLLMClient(model_string="azure/mistral-small-2503")
>>> response = client.generate_response("Hello!")
>>>
>>> # With fallbacks
>>> client = FFLiteLLMClient(
...     model_string="anthropic/claude-3-opus",
...     fallbacks=["openai/gpt-4", "azure/gpt-4"]
... )
generate_response(prompt, model=None, system_instructions=None, temperature=None, max_tokens=None, **kwargs)[source]

Generate a response from the AI model with retry and fallback logic.

Retries are handled by retry_utils.get_configured_retry_decorator on the inner _call_primary method. If the primary model (and all its retries) fail, fallback models are tried once each.

Parameters:

  • prompt (str) – The user prompt

  • model (str | None) – Override model (appends to provider prefix)

  • system_instructions (str | None) – Override system instructions

  • temperature (float | None) – Override temperature

  • max_tokens (int | None) – Override max tokens

  • **kwargs (Any) – Additional LiteLLM parameters

Returns:

The generated response text

Raises:
Return type:

str

clone()[source]

Create a fresh clone of this client with empty history.

Used for thread-safe parallel execution where each thread needs an isolated client instance with the same configuration.

Returns:

New FFLiteLLMClient with same config, empty history.

Return type:

FFLiteLLMClient

add_tool_result(tool_call_id, content)

Append a tool result message to the conversation history.

Parameters:

  • tool_call_id (str) – Provider-specific ID of the tool call being answered.

  • content (str) – The tool’s return value as a string.

Return type:

None

clear_conversation()

Remove all messages from the conversation history.

Return type:

None

configure_retry(retry_config=None)

Configure retry behavior for this client.

Parameters:

retry_config (dict[str, Any] | None) – Optional retry configuration. If None, uses global config.

Return type:

None

get_conversation_history()

Return a shallow copy of the conversation history.

Returns:

List of message dictionaries.

Return type:

list[dict[str, Any]]

static get_default_retry_config()

Get default retry configuration from global config.

Returns:

Dictionary with retry configuration parameters.

Return type:

dict[str, Any]

property last_cost_usd: float

Estimated cost in USD from the most recent generate_response() call.

property last_duration_ms: float | None

Wall-clock duration in ms of the most recent generate_response() call.

property last_usage: TokenUsage | None

Token usage from the most recent generate_response() call.

retry_config: dict[str, Any] | None = None
set_conversation_history(history)

Replace the conversation history with a new list of messages.

Parameters:

history (list[dict[str, Any]]) – List of message dictionaries to set.

Return type:

None

model: str
system_instructions: str
conversation_history: list[dict[str, Any]]
api_key: str | None
api_base: str | None
api_version: str | None
temperature: float
max_tokens: int