Run Mixtral 8x7b on Google Colab Free
A clever trick allows offloading some layers
Published in
5 min readDec 31, 2023
Hello, wonderful people! 2023 is almost over. But it seems like the development in LLMs has no breather.
Today, we will see how Mixtral 8x7B could be run on Google Colab.
Show me how
Google Colab comes with the following confirmation. It has a T4 instance with 12.7 GB memory and 16GB of VRAM. The disk size does not matter, really, but as you can see, you start with 80GB of effective disk space.
