github/fauxpilot
1
0
Fork 0
mirror of https://github.com/fauxpilot/fauxpilot.git synced 2025-07-30 19:40:59 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add python backend support

Browse source 
- Modify dockerfile to include bitsandbytes, transformers and latest version of pytorch
- Minor modifications in utils/codegen.py so that same client works with FT and Py-backend
- Minor modifications in launch.sh (no need to name models by GPU)
- Add installation script for adding a new python model (with super simple config_template)
- Modify setup.sh so that it aworks with both FT and Python backend models

Signed-off-by: Parth Thakkar <thakkarparth007@gmail.com>
This commit is contained in:
Parth Thakkar 2022-10-16 22:05:00 -05:00
parent 9b2bc84670
commit 01f1cbb629
9 changed files with 487 additions and 73 deletions
Download patch file Download diff file Expand all files Collapse all files

2
copilot_proxy/models.py
View file

@ -4,7 +4,7 @@ from pydantic import BaseModel
class OpenAIinput(BaseModel):
model: str
model: str = "fastertransformer|py-model"
prompt: Optional[str]
suffix: Optional[str]
max_tokens: Optional[int] = 16