39
ManyStrongWords
u/ManyStrongWords
15913
How do I use custom LLM templates with the API? For example, if I want to use a gguf version of Meta-Llama-3-8B-Instruct model. How do I specify the chat template and format the api calls for it to work?
12
10
I am having issues using they text gen python api, Curl works, silly tavern works but the api using langchain doesn't. I get the error "ERROR: response: <Response [404]>". To the best of my knowledge this should work. I'm running flags of --api --verbose --listen, and the openai extension. Does anyone have any ideas what I am doing wrong?
model_url = "http://localhost:5000"
from langchain.chains import LLMChain
from langchain.globals import set_debug
from langchain_community.llms import TextGen
from langchain_core.prompts import PromptTemplate
set_debug(True)
template = """Question: {question}
Answer: Let's think step by step."""
33
Does anyone have an English card translation made for the Directors' Cut Expansion fan expansion? http://bangcardgame.blogspot.com/2011/08/directors-cut-expansion.html
10
Splitgate referral code: 3N2H71 Get 50 coin for using it
10
