Docker OS GPU for Apt Installation Recipe

These are the steps to install OmniSci as a Docker container on an Ubuntu machine running with NVIDIA Kepler or Pascal series GPU cards.

The installation phases are:

Preparation
Installation
Activation

Preparation

Prepare your host by installing NVIDIA drivers, Docker, and NVIDIA runtime.

Install NVIDIA Drivers

To install NVIDIA drivers, open a terminal window on the host. Run apt update and apt upgrade to ensure that you are using the latest operating system software.

sudo apt update
sudo apt upgrade

Use apt-get to install required utilities.

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

Reboot your system to activate all of your changes.

sudo reboot

Install CUDA Drivers

Verify that the gcc compiler is installed with the following command.

gcc --version

If no version information returns, run the following command.

sudo apt install gcc

OmniSci requires only the CUDA drivers and not the entire CUDA package. To install the drivers:

Go to https://developer.nvidia.com/cuda-downloads.
Select the target platform by selecting the operating system (Linux), architecture (based on your environment), distribution (Ubuntu), version (based on your environment), and installer type (OmniSci recommends deb (network)).
In Download Installer..., right-click the Download button and copy the link location of the Base Installer. You will use the URL to that link in the next step. Do not use the installation instructions on the CUDA site.
Use one of the following methods to download the installer from the command line, using the download link you copied (in this example, https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb):
- curl:
```
sudo curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
```
- wget:
```
sudo wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
```
  If wget is not installed in your environment, use sudo apt install wget to install it.
Install the CUDA drivers, using the filename you just downloaded (in this example, cuda-repo-ubuntu1804_10.0.130-1_amd64.deb):
```
sudo dpkg -i <file_name>
```
If you do not have the public CUDA GPG key installed, run the installation command provided by NVIDIA in the terminal window; for example:
```
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
```
Update the local repository cache:
```
sudo apt update
```

Install the CUDA Toolkit and GPU drivers:

sudo apt install cuda-drivers linux-image-extra-virtual

Reboot your system to ensure that all changes are active:
```
sudo reboot
```

Install Docker

Remove any existing Docker installs and the legacy NVIDIA docker runtime.

sudo docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge nvidia-docker

Remove Docker.

sudo apt-get remove docker docker-engine docker.io containerd runc

Update with apt-get.

sudo apt-get update

Use curl to download the latest Docker version.

sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add Docker to your Apt repository.

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

Update your repository.

sudo apt-get update

Install Docker, the command line interface, and the container runtime.

sudo apt-get install docker-ce docker-ce-cli containerd.io

Checkpoint

Verify your Docker installation.

sudo docker run hello-world

Install NVIDIA Docker runtime

Use curl to add a gpg key.

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
  sudo apt-key add -
  distribution=$(. /etc/os-release;echo $ID$VERSION_ID)

Update your sources list.

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list

Update apt-get and install nvidia-docker2.

sudo apt-get update
sudo apt-get install -y nvidia-docker2

Restart the Docker daemon

sudo pkill -SIGHUP dockerd

Checkpoint

Verify docker and NVIDIA runtime work together.

sudo docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi

Standard NVIDIA-SMI output shows the GPUs in your instance. NVIDIA SMI

Installation

Download OmniSci from DockerHub and Start OmniSci in Docker.

docker run --runtime=nvidia \
--name omnisci \
-v $HOME/omnisci-docker-storage:/omnisci-storage \
-p 6273-6280:6273-6280 \
omnisci/core-os-cuda:latest

Note

You receive an error on startup.

config file does not exist, ignoring --config /omnisci-storage/omnisci.conf

This is expected behavior, as OmniSci does not ship with a default omnisci.conf file.

To use OmniSCi with the capabilities enabled by omnisci.conf, create the file as described in Configuration Parameters. and place it in $HOME/omnisci-docker-storage.

For more information on CUDA driver installation, see the CUDA Installation Guide.

For more information on Docker installation, see the Docker Installation Guide.

Command Line Access

You can access the command line in the Docker image to perform configuration and run OmniSci utilities.

You need to know the container-id to access the command line. Use the command below to list the running containers.

docker container ls

You see output similar to the following.

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                            NAMES
9e01e520c30c        omnisci/omnisci-ee-cpu    "/bin/sh -c '/omnisci..."   3 days ago          Up 3 days           0.0.0.0:6273-6280->6273-6280/tcp                 confident_neumann

Once you have your container ID, you can access the command line using the Docker exec command. For example, here is the command to start a Bash session in the Docker instance listed above. The -it switch makes the session interactive.

docker exec -it 9e01e520c30c bash

You can end the Bash session with the exit command.

Note

On Ubuntu, nvidia-docker has a dependency on nvidia-modprobe. If you receive the message: Error: Could not load UVM kernel module. Is nvidia-modprobe installed? you can install nvidia-modprobe using the following commands.

sudo apt-add-repository multiverse
sudo apt update
sudo apt install nvidia-modprobe

Activation

To verify that all systems are go, load some sample data and perform an omnisql query.

OmniSci ships with two sample datasets of airline flight information collected in 2008, and one dataset of New York City census information collected in 2015. To install the sample data, run the following command.

docker exec -it <container-id> ./insert_sample_data

Where <container-id> is the container in which OmniSci is running.

When prompted, choose whether to insert dataset 1 (7,000,000 rows), dataset 2 (10,000 rows), or dataset 3 (683,000 rows). The examples below use dataset 2.

Enter dataset number to download, or 'q' to quit:
#     Dataset                   Rows    Table Name             File Name
1)    Flights (2008)            7M      flights_2008_7M        flights_2008_7M.tar.gz
2)    Flights (2008)            10k     flights_2008_10k       flights_2008_10k.tar.gz
3)    NYC Tree Census (2015)    683k    nyc_trees_2015_683k    nyc_trees_2015_683k.tar.gz

Connect to OmniSci Core by entering the following command (default password is HyperInteractive):

docker exec -it <container-id> /omnisci/bin/omnisql
password: ••••••••••••••••

Enter a SQL query such as the following:

omnisql> SELECT origin_city AS "Origin", dest_city AS "Destination", AVG(airtime) AS
"Average Airtime" FROM flights_2008_10k WHERE distance < 175 GROUP BY origin_city,
dest_city;

The results should be similar to the results below.

Origin|Destination|Average Airtime
Austin|Houston|33.055556
Norfolk|Baltimore|36.071429
Ft. Myers|Orlando|28.666667
Orlando|Ft. Myers|32.583333
Houston|Austin|29.611111
Baltimore|Norfolk|31.714286

Main Content