github/fauxpilot
1
0
Fork 0
mirror of https://github.com/fauxpilot/fauxpilot.git synced 2025-08-21 05:44:09 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'main' of github.com:moyix/fauxpilot

Browse source
This commit is contained in:
Brendan Dolan-Gavitt 2022-08-04 17:21:18 -04:00
commit 759727664b
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -9,6 +9,7 @@ You'll need:
* Docker
* `docker-compose` >= 1.28
* An NVIDIA GPU with enough VRAM to run the model you want.
* [`nvidia-docker`](https://github.com/NVIDIA/nvidia-docker)
Note that the VRAM requirements listed by `setup.sh` are *total* -- if you have multiple GPUs, you can split the model across them. So, if you have two NVIDIA RTX 3080 GPUs, you *should* be able to run the 6B model by putting half on each GPU.
@ -210,4 +211,4 @@ Perhaps more excitingly, you can configure the official [VSCode Copilot plugin](
And you should be able to use Copilot with your own locally hosted suggestions! Of course, probably a lot of stuff is subtly broken. In particular, the probabilities returned by the server are partly fake. Fixing this would require changing FasterTransformer so that it can return log-probabilities for the top k tokens rather that just the chosen token.
Have fun!
Have fun!