* add config files, update docs, add codegen2 to setup, small tweaks to converter scripts - JH RJ
* Update documentation/server.md
---------
Co-authored-by: James Hoekzema <james.hoekzema@wwt.com>
Co-authored-by: Fred de Gier <freddegier@me.com>
Resolved#157.
The setup.sh generates the "integer expression expected" issue.
This commit is to fix this issue.
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
Signed-off-by: Geunsik Lim <geunsik.lim@samsung.com>
CC (Compute Capability) of Nvidia GPU supports INT8 from version 9.0.
Therefore, this patch is to change the default SM (Streaming Machine) type to INT8
so that stable operation is compatible with versions between CC 6.0 and CC 9.0.
* Prerequisites: An NVIDIA GPU with Compute Capability >= 6.0
https://github.com/fauxpilot/fauxpilot#prerequisites
* Architecture of Compute Capability (CC)9.0
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#id100
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
This commit is to handle both "docker-compose" and "docker compose"
environment while running the python_backend function in the setup.sh.
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
Signed-off-by: Geunsik Lim <geunsik.lim@samsuang.com>
- Modify dockerfile to include bitsandbytes, transformers and latest version of pytorch
- Minor modifications in utils/codegen.py so that same client works with FT and Py-backend
- Minor modifications in launch.sh (no need to name models by GPU)
- Add installation script for adding a new python model (with super simple config_template)
- Modify setup.sh so that it aworks with both FT and Python backend models
Signed-off-by: Parth Thakkar <thakkarparth007@gmail.com>
This commit fixes an incorrect return value in setup.sh. In this bash program.
A score of zero (0) indicates that everything went smoothly. Anything else indicates
a problem. A value of 1 indicates that some type of error has occurred.
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
Signed-off-by: Geunsik Lim <geunsik.lim@samsung.com>
This commit is trivial. It is to append the shebang in the setup.sh file.
It is not always the case that /bin/bash is the path to the Bash shell.
It is only a different path in extremely rare instances, so it is typically irrelevant.
This commit is to specify the Bash shell (i.e., bash) using the environment variable.
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
I added the 1 and 2 GPU versions of the converted models to HuggingFace
so that in the most common case we don't have to do the (very RAM
intensive) conversion process at all. Hopefully HuggingFace won't mind
us hosting them there!
This should fix#7.
