Prompt format settings
Hello! I'm not sure if this is in the right subreddit, but I'm trying to get coding models to work but I get some weird behavior. Often I just get "0.00 tokens/s, 0 tokens, context 40, seed...". I think it might be because of the input format. On HF, I see these kind of prompt templates
For CodeLlama-13B-Instruct-GPTQ
Prompt template: CodeLlama
[INST] Write code to solve the following coding problem that obeys the constraints and passes the example test cases. Please wrap your code answer using ```:
{prompt}
[/INST]
For Phind-CodeLlama-34B-v2-GPTQ
Prompt template: Phind
### System Prompt
{system_message}
### User Message
{prompt}
### Assistant
But I have no clue how to work with those templates and there is now presets for those models.
Thanks in advance
Posted Aug 31, 2023, 10:36 PM UTC
10