evofabric.core.clients

Chat Clients

class evofabric.core.clients.ChatClientBase

Base class for obtaining responses from large model Chat Mode

async create_on_stream(self, messages: Sequence[StateMessage], **kwargs) → AsyncGenerator[ChatStreamChunk, LLMChatResponse]

Stream the response from the large model.

Parameters:

• messages (Sequence[StateMessage]) – Represents a sequence of multi-round historical dialogue messages.

• **kwargs – Configuration parameters that need to be set during inference of other models

Returns:

An asynchronous generator that, during the streaming process, continuously returns ChatStreamChunk objects recording streaming messages, and finally returns the large model’s response LLMChatResponse

Return type:

AsyncGenerator[ChatStreamChunk, LLMChatResponse]

async create(self, messages: Sequence[StateMessage], **kwargs) → LLMChatResponse

Non-streaming acquisition of large model response

Parameters:

• messages (Sequence[StateMessage]) – Represents a sequence of multi-round historical dialogue messages.

• **kwargs – Configuration parameters that need to be set during inference of other models

Returns:

Response Result of the Large Model LLMChatResponse

class evofabric.core.clients.OpenAIChatClient(ChatClientBase)

Chat client implementation based on the OpenAI interface, inheriting from ChatClientBase.

Parameters:

• model (str) – The OpenAI model name to be used, such as “gpt-3.5-turbo”, “gpt-4”, etc.

• stream (bool) – Whether to default to requesting the model in a streaming manner; when True, call create_on_stream(); when False, call create().

• client_kwargs (Dict) – Additional keyword arguments for initializing the openai.AsyncOpenAI client, such as base_url, api_key, timeout, etc.

• http_client_kwargs (Dict) – Keyword arguments used to initialize the underlying httpx.AsyncClient, such as proxy, limits, verify, etc.

• inference_kwargs (Dict) – The inference parameters passed with each call to chat.completions.create(), such as temperature, top_p, max_tokens, etc.

• stream_parser (Callable) – An asynchronous callable object for parsing OpenAI’s streaming data packets in chunks, which must satisfy the protocol AsyncIterator[str] -> ChatStreamChunk.

async create_on_stream(self, messages: Sequence[StateMessage], **kwargs) → AsyncGenerator[ChatStreamChunk, LLMChatResponse]

Stream the response from the large model.

Parameters:

• messages (Sequence[StateMessage]) – Represents a sequence of multi-round historical dialogue messages.

• kwargs – Configuration parameters that need to be set for inference with other models (will override the same-named parameters in the class attribute inference_kwargs)

Returns:

An asynchronous generator that, during the streaming process, continuously returns ChatStreamChunk objects recording streaming messages, and finally returns the large model’s response LLMChatResponse

Return type:

AsyncGenerator[ChatStreamChunk, LLMChatResponse]

async create(self, messages: Sequence[StateMessage], **kwargs) → LLMChatResponse

Non-streaming acquisition of the large model’s response.

Parameters:

• messages (Sequence[StateMessage]) – Represents a sequence of multi-round historical dialogue messages.

• **kwargs – Configuration parameters that need to be set for inference with other models (will override the same-named parameters in the class attribute inference_kwargs)

Returns:

Response Result of the Large Model LLMChatResponse

Return type:

LLMChatResponse

class evofabric.core.clients.PanguClient(OpenAIChatClient)

Chat client implementation based on the PanGu large model interface, inheriting from OpenAIChatClient.

Parameters:

• model (str) – Model name to be used.

• stream (bool) – Whether to default to requesting the model in a streaming manner; when True, call create_on_stream(); when False, call create().

• client_kwargs (Dict) – Additional keyword arguments for initializing the openai.AsyncOpenAI client, such as base_url, api_key, timeout, etc.

• http_client_kwargs (Dict) – Keyword arguments used to initialize the underlying httpx.AsyncClient, such as proxy, limits, verify, etc.

• inference_kwargs (Dict) – The inference parameters passed with each call to chat.completions.create(), such as temperature, top_p, max_tokens, etc.

• stream_parser (Callable) – An asynchronous callable object used to parse the streaming data packets returned by PanGu in chunks, which must satisfy the protocol AsyncIterator[str] -> ChatStreamChunk.

• enable_think (bool) – Whether to enable ‘Thinking’ mode.

async create_on_stream(self, messages: Sequence[StateMessage], **kwargs) → AsyncGenerator[ChatStreamChunk, LLMChatResponse]

Stream the response from the large model.

Parameters:

• messages (Sequence[StateMessage]) – Represents a sequence of multi-round historical dialogue messages.

• kwargs – Configuration parameters that need to be set for inference with other models (will override the same-named parameters in the class attribute inference_kwargs)

Returns:

An asynchronous generator that, during the streaming process, continuously returns ChatStreamChunk objects recording streaming messages, and finally returns the large model’s response LLMChatResponse

Return type:

AsyncGenerator[ChatStreamChunk, LLMChatResponse]

async create(self, messages: Sequence[StateMessage], **kwargs) → LLMChatResponse

Non-streaming acquisition of the large model’s response.

Parameters:

• messages (Sequence[StateMessage]) – Represents a sequence of multi-round historical dialogue messages.

• **kwargs – Configuration parameters that need to be set for inference with other models (will override the same-named parameters in the class attribute inference_kwargs)

Returns:

Response Result of the Large Model LLMChatResponse

Return type:

LLMChatResponse

Embedding Clients

class evofabric.core.clients.EmbedClientBase

Base client class for interacting with any backend embedding model. Compatible with LangChain OpenAI embedding format, directly usable in LangChain, ChromaDB, and other ecosystem tools.

embed_query(self, text: str) → list[float]

Synchronously generate embedding vectors for single-segment text.

Parameters:

text (str) – Text string to be embedded

Returns:

A floating-point vector of length embedding_dim.

Return type:

list[float]

embed_documents(self, texts: list[str], **kwargs) → list[list[float]]

Synchronously generate a list of embedding vectors for multiple text segments.

Parameters:

• texts (list[str]) – Text string list.

• kwargs (Any) – Additional inference parameters, such as chunk_size, retry, etc.

Returns:

A list of vectors corresponding to the order of texts, each vector’s length is embedding_dim.

Return type:

list[list[float]]

async aembed_query(self, text: str, **kwargs) → list[float]

Asynchronous generation of embedding vectors for single text segment.

Parameters:

• text (str) – Text string to be embedded

• kwargs (Any) – Additional inference parameters.

Returns:

A floating-point vector of length embedding_dim.

Return type:

list[float]

async aembed_documents(self, texts: list[str], **kwargs) → AsyncGenerator[list[list[float]], None]

Asynchronously generate a list of embedding vectors for multi-segment text.

Parameters:

• texts (list[str]) – Text string list.

• kwargs (Any) – Additional inference parameters, such as chunk_size, retry, etc.

Returns:

A list of vectors corresponding to the order of texts, each vector’s length is embedding_dim.

Return type:

list[list[float]]

class evofabric.core.clients.OpenAIEmbedClient(EmbedClientBase)

Embedding model client based on OpenAI interface specifications, supporting synchronous and asynchronous batch embeddings, compatible with Ollama and other OpenAI-Format backends.

Parameters:

• base_url (Optional[str]) – Request endpoint URL, when the default is an empty string, use the official address.

• api_key (Optional[str]) – Service Access Key. When the default is an empty string, attempt to read environment variables or local configuration.

• model (str) – The name of the embedding model to be called, for example, text-embedding-3-small.

• dimensions (Optional[int]) – Specify the dimension of the returned vector; effective when the model supports dimensionality reduction, leave blank to use the model’s default dimension.

• max_retries (Optional[int]) – Maximum number of retries on request failure, default 2.

• request_timeout (Optional[Union[float, tuple, Any]]) – Maximum wait time per request (seconds), supports floating-point numbers or (connect, read) tuples.

model_post_init(self, context: Any, /) → None

After instantiation, automatically initialize the underlying OpenAI client.

embed_documents(self, texts: List[str], **kwargs) → List[List[float]]

Batch generate embedding vectors for multiple text segments.

Parameters:

• texts (List[str]) – List of text to be embedded.

• kwargs (Any) – Additional inference parameters, such as dimensions, user, etc., will be transparently passed to the underlying API.

Returns:

Vector matrix corresponding to the input order, with each row’s dimension determined by the model or the dimensions field.

Return type:

List[List[float]]

embed_query(self, text: str, **kwargs) → List[float]

Generates an embedding vector for a single text segment, internally calls embed_documents and returns the first result.

Parameters:

• text (str) – Text to be embedded.

• kwargs (Any) – Additional inference parameters, same as embed_documents.

Returns:

A floating-point vector of length dimensions (or model default).

Return type:

List[float]

async aembed_query(self, text: str, **kwargs) → list[float]

Asynchronous generation of embedding vectors for single text segment.

Parameters:

• text (str) – Text string to be embedded

• kwargs (Any) – Additional inference parameters.

Returns:

A floating-point vector of length embedding_dim.

Return type:

list[float]

async aembed_documents(self, texts: list[str], **kwargs) → AsyncGenerator[list[list[float]], None]

Asynchronously generate a list of embedding vectors for multi-segment text.

Parameters:

• texts (list[str]) – Text string list.

• kwargs (Any) – Additional inference parameters, such as chunk_size, retry, etc.

Returns:

A list of vectors corresponding to the order of texts, each vector’s length is embedding_dim.

Return type:

list[list[float]]

class evofabric.core.clients.SentenceTransformerEmbed(EmbedClientBase)

A lightweight embedding client based on a local Sentence-Transformer model, capable of generating high-quality vectors without external APIs, suitable for offline, private, and edge deployment scenarios.

Parameters:

• model (str) – Local Sentence-Transformer model name or HuggingFace Hub ID, for example all-MiniLM-L6-v2.

• device (str) – Model runtime device, default is "cpu"; can specify "cuda", "mps", etc. to enable GPU acceleration.

model_post_init(self, context: Any, /) → None

After instantiation, the local model is automatically loaded, and the device and model name are injected via field values.

embed_documents(self, texts: List[str], **kwargs) → List[List[float]]

Batch generate embedding vectors for multiple text segments.

Parameters:

• texts (List[str]) – List of text to be embedded.

• kwargs (Any) – Additional inference parameters, such as batch_size, normalize_embeddings, etc., will be passed through to the underlying model.

Returns:

Vector matrix corresponding to the input order, each row’s dimension determined by the model.

Return type:

List[List[float]]

embed_query(self, text: str, **kwargs) → List[float]

Generates an embedding vector for a single text segment, internally calls embed_documents and returns the first result.

Parameters:

• text (str) – Text to be embedded.

• kwargs (Any) – Additional inference parameters, same as embed_documents.

Returns:

A floating-point vector of length equal to the model output dimension.

Return type:

List[float]

async aembed_query(self, text: str, **kwargs) → list[float]

Asynchronous generation of embedding vectors for single text segment.

Parameters:

• text (str) – Text string to be embedded

• kwargs (Any) – Additional inference parameters.

Returns:

A floating-point vector of length embedding_dim.

Return type:

list[float]

async aembed_documents(self, texts: list[str], **kwargs) → AsyncGenerator[list[list[float]], None]

Asynchronously generate a list of embedding vectors for multi-segment text.

Parameters:

• texts (list[str]) – Text string list.

• kwargs (Any) – Additional inference parameters, such as chunk_size, retry, etc.

Returns:

A list of vectors corresponding to the order of texts, each vector’s length is embedding_dim.

Return type:

list[list[float]]

Rerank Clients

class evofabric.core.clients.RerankClientBase(BaseComponent)

A base class client that interacts with any backend re-ranking model to re-rank “query-text” pairs based on relevance and return the sorted index sequence.

async rank(self, query: str, texts: List[str], **kwargs) → List[int]

Asynchronously re-rank multiple texts based on their relevance to the query, returning the original index list sorted in descending order of relevance.

Parameters:

• query (str) – query string.

• texts (List[str]) – List of texts to be reordered.

• kwargs (Any) – Other inference parameters, such as top_n, truncate, temperature, etc., will be passed through to the underlying model.

Returns:

A list of original text indices sorted in descending order of relevance, with the length defaulting to len(texts) or top_n (if specified).

Return type:

List[int]

class evofabric.core.clients.FlagRerankModel(RerankClientBase)

Implementation of a local re-ranking model based on FlagEmbedding, which can perform relevance scoring and re-ranking on ‘query-text’ pairs without external APIs, is suitable for private deployment and offline scenarios.

Parameters:

• model (str) – Local FlagRerank model name or HuggingFace Hub ID, for example BAAI/bge-reranker-base.

• top_n (int) – Return the top N most relevant indexes, default 1.

• device (str) – Model runtime device, default is "cpu"; can specify "cuda", "cuda:0", etc., to enable GPU acceleration.

model_post_init(self, context: Any, /) → None

After instantiation is completed, automatically load the local re-ranking model, with the device and model name injected by field values.

async rank(self, query: str, texts: List[str], **kwargs) → List[int]

Asynchronously re-rank multiple texts by relevance to the query, returning the original index list sorted in descending order of relevance.

Parameters:

• query (str) – query string.

• texts (List[str]) – List of texts to be reordered.

• kwargs (Any) – Additional inference parameters, such as truncate, batch_size, etc., will be passed through to the underlying model.

Returns:

An index list with length not exceeding top_n, sorted in descending order of relevance.

Return type:

List[int]