agentlab.llm.huggingface_utils
Classes
|
Custom LLM Chatbot that can interface with HuggingFace models with support for multiple samples. |
- class agentlab.llm.huggingface_utils.HFBaseChatModel(model_name, base_model_name, n_retry_server)
Bases:
AbstractChatModelCustom LLM Chatbot that can interface with HuggingFace models with support for multiple samples.
This class allows for the creation of a custom chatbot using models hosted on HuggingFace Hub or a local checkpoint. It provides flexibility in defining the temperature for response sampling and the maximum number of new tokens in the response.
- llm
The HuggingFaceHub model instance.
- Type:
Any
- prompt_template
Template for the prompt to be used for the model’s input sequence.
- Type:
Any
- tokenizer
The tokenizer to use for the model.
- Type:
Any
- llm: Any = FieldInfo(annotation=NoneType, required=True, description='The HuggingFaceHub model instance')
- n_retry_server: int = FieldInfo(annotation=NoneType, required=False, default=4, description='The number of times to retry the server if it fails to respond')
- prompt_template: PromptTemplate | None = FieldInfo(annotation=NoneType, required=False, default=None, description="Template for the prompt to be used for the model's input sequence")