GPU ML on Intel Arc
Intel is releasing discrete GPUs and that's a good thing for consumers. I think a lot of the progress in the Deep Learning space is thanks to cheap and plentiful compute available to researchers without the need for access to a supercomputer. The boom in computer vision research after AlexNet and all. Then I found out that Intel wanted to support deep learning on their GPU as well. I always disliked the layers of great open-source software we have build on the proprietary CUDA language. Both AMD's ROCM and Intel's OneAPI are open-source. Then I had to pick: seeing Intel creating cheaper GPUs with 16GB memory, and AMD still not officially supporting consumer GPUs (they seem to only support the consumer variants similar to their professional Instinct line). Long story short, I bought an Intel Arc 770 to try out their software stack and record my experiences.
Installation
Intel is new at this so I am expecting some teething issues. Their own dgpu-documentation's installation steps only work for Ubuntu 22.04, which is fine as a first Linux OS to support. But this means older packages than rolling release OSes like ArchLinux or Intel's very own ClearLinux. Moreover, the steps will tell you to install an older Linux kernel before you can install the specific gpgpu drivers. This means before the installation is complete no video support. I had to run all the steps in recovery mode. Compare this to starting up a new install of ClearLinux with a very up to date kernel version with pretty good normal (non-gpgpu) video support out of the box. Sadly, I couldn't install the drivers there.
These were the driver installation steps I followed.
The next step was installing the base oneAPI toolkit. I added the sources to APT and ran the sudo apt install intel-basekit succesfully.
I had some problems with running sudo apt install intel-aikit for the AI Analytics tookit next. Somehow some packages included could not be found on the server. Luckily there is a conda method for installing that toolkit as well and I know conda still from cuda installs some time in the past. So with a miniconda install in the hand I continue.
Another failed attempt was doing the described conda installation with pytorch. I got intel with MKL-backend and I got pytorch with xpu support but it couln't find the GPU.
Finally what worked for me was:
conda create -n gputest -c intel intelpython3_full=3.9
conda activate gputest
python -m pip install torch==1.13.0a0 torchvision==0.14.1a0 intel_extension_for_pytorch==1.13.10+xpu -f https://developer.intel.com/ipex-whl-stable-xpu-idp
source /opt/intel/oneapi/setvars.sh # Doesn't work in fish :(
python -c "import torch; import intel_extension_for_pytorch as ipex; print(torch.__version__); print(ipex.__version__); [print(f'[{i}]: {torch.xpu.get_device_properties(i)}') for i in range(torch.xpu.device_count())];"
The instructions of the GPU version of ipex (intel extension for pytorch) said that the setvars script needs to run and only 3.9 of the intel distribution of python is supported.
Running some samples
There is a given sample from Intel which ran fine on both the cpu and by adding .to("xpu") on the Arc 770 GPU. I noticed that the iGPU was also detected, but sending the data to xpu:1 returns a "Double type is not supported on this platform" error. I'm curious what is supported on that platform since there are still shader units in the iGPU which could be made to multiply matrices.
Final notes
I hope that we will get to a point where training neural networks does not require running special older linux kernels or downloading proprietary drivers. Perhaps a Tinygrad with OpenCL support is currently our best bet for long term support for deep learning on any hardware. Maybe we can use MLIR between the training code and the hardware. I'm still hoping that Intel shows that by writing better Software and drivers they can become an important player for hobbyist researchers. And for the record, I'm also hoping AMD will step up and support their consumer GPU devices. This is something Nvidia does very well and there is still tons of ways to use old nvidia GPUs if you'd like to.