Categories
Programming

How to Setup a Dell Precision M6500 for GPGPU Development with FC13

Disclaimer

This post will hopefully save someone a few hours of trial-and-error. It is a result of three separate attempts in a 12-hour period to get things up and running myself. I believe that my notes are complete, but please remember, you are getting the information for free so YMMV! If it doesn’t get you a 100% solution hopefully it gets you 90%. Just so you are aware up front–this process as documented is for a 32-bit Fedora Core 13 installation with the latest NVIDIA drivers (v260.19.12 at the time of writing.) The graphics card on my M6500 is the NVIDIA Quadro FX 2800M. I expect the process would not be much different any other CUDA-capable NVIDIA card. And one last point, if you are afraid of, or enable to use, the command line then this is not the post for you.

Installing FC13

I started with the FC13 LiveCD since I needed to verify that the certain peripheral drivers worked for out-of-the-box. I am not a big fan of rebuilding a kernel unless it is absolutely necessary and like to start with as complete a solution as I can. After verifying the drivers needed were present I did an install-to-disk from the LiveCD. This was done via the icon on the LiveCD user desktop.

Once the install is complete, eject the CD and reboot the system. When the system returns, complete the install setting the root password, creating a user (for the purpose of this post that name of that user will be ‘me’,) etc. Once the setup tasks are complete I reboot again for good measure. I then login as ‘me’.

I find it useful to add myself to the sudoers list to allow sudo access without requiring a password. Do this from a terminal (Applications | System Tools | Terminal):

[me@m6500 ~]$ su -
Password:********
[root@m6500]$ cat >> /etc/sudoers
me ALL=(ALL) NOPASSWD: ALL

[root@m6500]$ exit
logout
[me@m6500 ~]$

This simplifies things moving forward since much of the following requires root privs and would require numerous password entries to complete. This configuration allows sudo usage without a password, moving things along a little quicker. It also means that I can stay logged in as ‘me’ to accomplish everything.

The next thing I like to do is disable the firewall, which is enabled in the default install. Using System | Administration | Firewall allows the firewall to be disabled (after entering the root password.)

One last step may be required before we get to updating the default installation with yum and it depends on your network configuration. If you have a proxy server in place you need to let yum know about it. I prefer to do this in the /etc/yum.conf file:

[me@m6500 ~]$ sudo cat >> /etc/yum.conf
proxy=http://:

[me@m6500 ~]$

Of course, set and to values consistent with your network configuration.

The system is now ready to invoke yum to update the default install. This is done with the following command:

[me@m6500 ~]$ sudo yum update
.
.
.
[me@m6500 ~]$ 

In my instance, there were over 450 updates to be applied and this process takes quite a while. Be patient. In the mean time, we can do some parallel processing and download the driver, toolkit and SDK sample code from NVIDIA. These are the links I used for the 3.2 RC version of things (full paths are included in case you want to work from a hardcopy of this post):

NVIDIA Downloads Page – http://developer.nvidia.com/object/cuda_3_2_toolkit_rc.html

Driver – http://developer.download.nvidia.com/compute/cuda/3_2/drivers/devdriver_3.2_linux_32_260.19.12.run

Toolkit – http://www.nvidia.com/object/thankyou.html?url=/compute/cuda/3_2/toolkit/cudatoolkit_3.2.9_linux_32_fedora13.run

SDK Samples – http://developer.download.nvidia.com/compute/cuda/3_2/sdk/gpucomputingsdk_3.2_linux.run

You should ensure you are getting the latest (unless you are trying to replicate my install) from NVIDIA here:

NVIDIA – http://developer.nvidia.com/page/home.html

I saved all the downloads to my home folder (~me or /home/me in my case.) Once the yum update is finished and all your files are finished downloading it is time to once again reboot. The installed updates include a kernel update so a reboot is required prior to driver installation so we do not apply the driver to the current kernel but instead to the freshly updated kernel.

Since some X drivers are going to be installed, it is now time to shutdown the X-server and login at the command line. You can do this either in the above reboot by editing the boot command line and entering a ‘3’ at the end (indicating you want to boot to runlevel 3,) or once the GUI boot is complete, login, start a terminal and enter the command:

[me@m6500 ~]$ sudo init 3
.
.
.
m6500 login: me
Password: ********
[me@m6500 ~]$ 

Login to your user account (‘me’ in my case) once you see the command line login prompt. We can now move on to completing the install.

Installing the NVIDIA Pieces

FC13 ships with a default open-source driver for NVIDIA cards called nouveau. Unfortunately, installing over this driver is more complicated than simply following the default driver installation instructions from NVIDIA. Fortunately for you, others before me (reference links: http://fedorasolved.org/video-solutions/nvidia-yum-kmod, and http://forums.fedoraforum.org/showthread.php?t=204752) have done the hard work and I am passing on their knowledge in a more complete form (as it fit my purposes, at least.)

Based on the references above, I created a few scripts (also located in ~me.) The first, do-nvidia.sh, adds RPMFusion repositories to the yum.conf, installs a few NVIDIA packages from RPMFusion, rebuilds the initrd image, and reconfigures grub to override the default nouveau driver.

do-nvidia.sh

#!/bin/bash

# add RPMFusion repositories
rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm
rpm -Uvh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

# install nvidia from RPMFusion
yum install kmod-nvidia xorg-x11-drv-nvidia-libs.i686

# blacklist nouveau driver from initrd in grub.conf
sed -i ‘/root=/s|$| rdblacklist=nouveau vmalloc=256M|’ /boot/grub/grub.conf

# regen initrd
mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r)-nouveau.img
dracut /boot/initramfs-$(uname -r).img $(uname -r)

This script must be invoked with sudo:

[me@m6500 ~]$ sudo ./do-nvidia.sh
.
.
.
[me@m6500 ~]$

The second script, do-installs.sh, installs development packages required for the NVIDIA GPU Computing SDK (and a couple of GUI config utilities I find useful):

do-installs.sh

#!/bin/bash

# install development stuff
yum install kernel-source kernel-devel
yum install gcc gcc-c++
yum install mesa-libGLU-devel
yum install libXi-devel
yum install libXmu-devel
yum install freeglut
ln -s /usr/lib/libglut.so.3 /usr/lib/libglut.so

# install misc
yum install samba
yum install system-config-samba
yum install system-config-network
yum install system-config-services

This script is also invoked with sudo:

[me@m6500 ~]$ sudo ./do-installs.sh
.
.
.
[me@m6500 ~]$

Next we move on to installing the CUDA Toolkit. I chose to use the default install paths for everything, and this and any future posts will reflect this. So the toolkit, by default, gets installed in /usr/local/cuda:

[me@m6500 ~]$ sudo ./cudatoolkit_3.2.9_linux_32_fedora13.run
.
.
.
[me@m6500 ~]$

Once the toolkit is installed we can move on to the GPU Computing SDK. This can be a local install in a single user directory, so as invoked below using default paths, it installs to /home/me/NVIDIA_GPU_Computing_SDK:

[me@m6500 ~]$ cd && pwd
/home/me
[me@m6500 ~]$ ./gpucomputingsdk_3.2_linux.run
.
.
.
[me@m6500 ~]$

In the reference material I found, there were indications that one did not have to run the devdriver install from NVIDIA. My experience was that I was missing libGL and found reference to the fact it is built by the driver install. I ran the install, YMMV:

[me@m6500 ~]$ sudo ./devdriver_3.2_linux_32_260.19.12.run
.
.
.
[me@m6500 ~]$

At this point the install should be complete. Invoking the SDK build, it should now succeed:

[me@m6500 ~]$ cd ~/NVIDIA_GPU_Computing_SDK/C
[me@m6500 ~]$ make && bin/linux/release/deviceQuery
.
.
.
Finished building all
bin/linux/release/deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

There is 1 device supporting CUDA

Device 0: "Quadro FX 2800M"
  CUDA Driver Version:                           3.20
  CUDA Runtime Version:                          3.20
  CUDA Capability Major/Minor version number:    1.1
  Total amount of global memory:                 1073020928 bytes
  Multiprocessors x Cores/MP = Cores:            12 (MP) x 8 (Cores/MP) = 96 (Cores)
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       16384 bytes
  Total number of registers available per block: 8192
  Warp size:                                     32
  Maximum number of threads per block:           512
  Maximum sizes of each dimension of a block:    512 x 512 x 64
  Maximum sizes of each dimension of a grid:     65535 x 65535 x 1
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             256 bytes
  Clock rate:                                    1.50 GHz
  Concurrent copy and execution:                 Yes
  Run time limit on kernels:                     Yes
  Integrated:                                    No
  Support host page-locked memory mapping:       Yes
  Compute mode:                                  Default (multiple host threads can use this device 
simultaneously)
  Concurrent kernel execution:                   No
  Device has ECC support enabled:                No
  Device is using TCC driver mode:               No

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 3.20, CUDA Runtime Version = 3.20, NumDevs 
= 1, Device = Quadro FX 2800M


PASSED

Press  to Quit...
-----------------------------------------------------------

[me@m6500 ~]$

Success!!! (Hopefully your installation is successful as well.

This post is long enough. I will follow it shortly with a post describing how to setup your own Makefile-based CUDA projects based on the NVIDIA GPU Computing SDK.

So long for now.

A Quick Follow-up

After looking closer at the boot logs I noticed I had an error:

Checking for module nvidia.ko: ESC[60G[ESC[0;31mFAILEDESC[0;39m]^M
nvidia.ko for kernel 2.6.34.7-61.fc13.i686 was not found.ESC[60G[ESC[0;33mWARNINGESC[0;39m]^M
The nvidia driver will not be enabled until one is found.ESC[60G[ESC[0;33mWARNINGESC[0;39m]^M
*** glibc detected *** /usr/bin/python: free(): invalid pointer: 0x00902822 ***
======= Backtrace: =========
/lib/libc.so.6[0xbc9fe1]
/usr/lib/python2.6/site-packages/ixf86configmodule.so(+0xdfe7)[0x8fcfe7]
/usr/lib/python2.6/site-packages/ixf86configmodule.so(xf86freeConfig+0x8c)[0x8f97fc]
.
.
.

I did two things two correct these errors. First I fixed the path in /etc/init.d/nvidia, adding:

elif test -e “${modpath}/kernel/drivers/video/${modname}”;then
module=”${modpath}/kernel/drivers/video/${modname}”

at line 28. This is where my nvidia.ko module landed–once this was added the “module not found” message disappeared, but I still had the invalid pointer error.

The second thing I did was a result of reading this (http://fedoraforum.org/forum/showthread.php?t=248592). Following the steps in post #4 of this thread:

[me@m6500 ~]$ sudo rm -f /etc/X11/xorg.conf
[me@m6500 ~]$ sudo nvidia-config-display disable
[me@m6500 ~]$ sudo nvidia-config-display enable

cleared up the remaining errors. The system now boots with no errors.