r/Oobabooga•
phyphys

Prompt format settings

Question

Hello! I'm not sure if this is in the right subreddit, but I'm trying to get coding models to work but I get some weird behavior. Often I just get "0.00 tokens/s, 0 tokens, context 40, seed...". I think it might be because of the input format. On HF, I see these kind of prompt templates

For CodeLlama-13B-Instruct-GPTQ

Prompt template: CodeLlama
[INST] Write code to solve the following coding problem that obeys the constraints and passes the example test cases. Please wrap your code answer using ```:
{prompt}
[/INST]

For Phind-CodeLlama-34B-v2-GPTQ

Prompt template: Phind
### System Prompt
{system_message}

### User Message
{prompt}

### Assistant

But I have no clue how to work with those templates and there is now presets for those models.

Thanks in advance

Posted Aug 31, 2023, 10:36 PM UTC

10

No comments yet

Nobody commented on this post, or the comments weren't captured.