oobabooga 7618f3fe8c Add -gptq-preload for 4-bit offloading (#460) ...

This works in a 4GB card now:

```
python server.py --model llama-7b-hf --gptq-bits 4 --gptq-pre-layer 20
```

2023-03-20 16:30:56 -03:00

..

callbacks.py

Sort the imports

2023-03-17 11:42:25 -03:00

chat.py

Use markdown in the "HTML" tab

2023-03-17 16:06:11 -03:00

deepspeed_parameters.py

Fix deepspeed (oops)

2023-02-02 10:39:37 -03:00

extensions.py

Exception handling (#454)

2023-03-20 13:36:52 -03:00

GPTQ_loader.py

Add -gptq-preload for 4-bit offloading (#460)

2023-03-20 16:30:56 -03:00

html_generator.py

Use markdown in the "HTML" tab

2023-03-17 16:06:11 -03:00

LoRA.py

Make custom LoRAs work by default #385

2023-03-19 12:11:35 -03:00

models.py

--no-cache and --gpu-memory in MiB for fine VRAM control

2023-03-19 19:21:41 -03:00

RWKV.py

Use str(Path) instead of os.path.abspath(Path)

2023-03-13 00:08:01 -03:00

shared.py

Add -gptq-preload for 4-bit offloading (#460)

2023-03-20 16:30:56 -03:00

text_generation.py

Exception handling (#454)

2023-03-20 13:36:52 -03:00

ui.py

Further reorganize the UI

2023-03-15 13:24:54 -03:00