- Modify dockerfile to include bitsandbytes, transformers and latest version of pytorch
- Minor modifications in utils/codegen.py so that same client works with FT and Py-backend
- Minor modifications in launch.sh (no need to name models by GPU)
- Add installation script for adding a new python model (with super simple config_template)
- Modify setup.sh so that it aworks with both FT and Python backend models
Signed-off-by: Parth Thakkar <thakkarparth007@gmail.com>
This commit fixes an incorrect return value in setup.sh. In this bash program.
A score of zero (0) indicates that everything went smoothly. Anything else indicates
a problem. A value of 1 indicates that some type of error has occurred.
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
Signed-off-by: Geunsik Lim <geunsik.lim@samsung.com>
This commit is trivial. It is to append the shebang in the setup.sh file.
It is not always the case that /bin/bash is the path to the Bash shell.
It is only a different path in extremely rare instances, so it is typically irrelevant.
This commit is to specify the Bash shell (i.e., bash) using the environment variable.
Signed-off-by: Geunsik Lim <leemgs@gmail.com>
I added the 1 and 2 GPU versions of the converted models to HuggingFace
so that in the most common case we don't have to do the (very RAM
intensive) conversion process at all. Hopefully HuggingFace won't mind
us hosting them there!
This should fix#7.
