Skip to content

System Setup

This is a work in progress document

Keep in mind that we can't run Pillar as a standalone app, rather we need to create an app based on it.

Setting up the enviroment

Before getting started with Pillar, some system requirements need to be satisfied. The system requirements are the following:

  • Python 3.6
  • MongoDB
  • Redis
  • RabbitMQ
  • gulp (NPM package)
  • a /data directory writable by the user

Step by step setup

A suggested way to get them up and running is:

Install Docker CE on Ubuntu or macOS. If running Docker on Linux as non-root user, add the user to the docker group.

Install Python 3.6 and 3.6-dev

sudo apt-get install python3.6{,-dev} python3-pip
brew install python3.6

Install virtualenvwrapper

sudo pip3 install virtualenvwrapper

Follow the configuration steps for virtualenvwrapper, which usually means setup your ~/.bashrc file with something similar to the following:

export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export WORKON_HOME=$HOME/.venvs
export PROJECT_HOME=$HOME/Developer
source /usr/local/bin/virtualenvwrapper.sh

Install NodeJS and NPM

sudo apt-get install nodejs npm
brew install node

Create writable /data directory and chown it to the user

sudo mkdir /data
sudo chown $USER:$(id -g -n) /data

Set up various Docker containers with the requirements

docker run -d -v /data/storage/db:/data/db -p 27017:27017 --name mongo mongo
docker run -d -p 6379:6379 --name redis redis
docker run -d -p 5672:5672 --name rabbit rabbitmq

Append the following names to the line in /etc/hosts that contains 127.0.0.1, after localhost: rabbit mongo redis

Optional: Video Encoding

History lesson

Originally, video encoding was performed in a subprocess of the webserver. This was not the best solution. We moved to a dedicated encoding solution, which can fetch the uploaded files from GCS, encode them, send them back to GCS and notify our server about it. In the future we will perform our own encoding on Flamenco.

In order to use Zencoder, we need GCLOUD_APP_CREDENTIALS in our config.

Setup video encoding for development by installing the Zencoder notification fetcher:

gem install zencoder-fetcher

Add zencoderfetcher to your /etc/hosts.

And add to the configuration:

ZENCODER_API_KEY
ZENCODER_NOTIFICATIONS_SECRET

Run the fetcher with

zencoder_fetcher <API KEY> --url http://cloud.local:5001/api/encoding/zencoder/notifications -l

Ensure that your config is correct for:

BIN_FFPROBE
BIN_FFMPEG

Application Setup

Follow the application guide on how to get the system setup. For example, check out the Development setup section in the Blender Cloud README.md file.