Concierge

The counter-part of Portier, which will execute the required processes to make what's configured in Portier happen.

It is implemented as a very simple Python application. It will periodically ask Portier for "tasks" to execute. If there are any new tasks, it will claim them and then execute them through an executor. If any tasks are not required anymore by Portier, it will stop them.

Tasks are essentially background processes, that need to do some continuous work. They are not something that does work and finishes at some point, but rather something that needs to be running all the time. This could be an ffmpeg process that re-transmits an existing video stream as long as it is available.

Executors

Concierge needs an executor to execute the required processes. The current structure allows to switch between different executors, but the only one implemented currently is the docker executor. It will use the Docker API to start and stop containers. For every required task type, there's a corresponding Docker image, which will be started with the required parameters. Docker in this case makes our life easier, as it handles running the process for us and schedule it across multiple machines in a Docker swarm mode cluster.

Task types

Tasks have different types, which determine what exactly they are doing. Concierge is mostly indifferent to the type of task, but it does need to know what parameters for the executor to use, and how many resources to allocate for the task. These definitions are stored in the defaults.yaml, but can be extended through environment variables as defined by dynaconf.

To implement the task types there are currently individual Docker images, which are started by the docker executor. The build files for these images can be found in workerimages, and are available as built images from our container registry.

Implementing new task types is as simple as creating a new Docker image, and adding the required parameters to the defaults.yaml file. The configuration that Portier attaches to a Task will be passed to the Docker container through the CONCIERGE_CONFIGURATION environment variable, so the worker image can use them to configure itself.

Running

Currently only the docker executor is implemented, so running Concierge makes most sense in a Docker environment. The following command will start Concierge and let it use the underlying Docker daemon to start and stop containers.

$ docker run -d --name concierge --restart unless-stopped -e CONCIERGE_PORTIER_ENDPOINT=http://portier:8080 -e CONCIERGE_IDENTITY=12345 -v /var/run/docker.sock:/var/run/docker.sock git.chaoswest.tv/cwtv/concierge:latest