Developer information

This chapter describes key areas useful for developers.

Environment variables

Several environment variables fundamentally control the deployment and development environment. These include:

• TANGO_HOST

• TANGO_STATION_CONTROL

• TANGO_SKIP_BUILD

Firstly, TANGO_HOST should point to the tango database server including its port. An example would be 10.14.0.205:10000. If TANGO_HOST is not set instead tango.service.consul:10000 is used.

Finally TANGO_STATION_CONTROL can be used to control if device containers should build software from source (developer mode). Or if the software should be built into the lofar-device-base docker image directly. If TANGO_STATION_CONTROL is set the makefile will build a wheel package which will be installed into the docker image.

If instead a particular wheel package needs to be installed TANGO_SKIP_BUILD can be set as well. Be sure the wheel package is placed in the tangostationcontrol/dist/ directory.

In the future the actual value of the TANGO_STATION_CONTROL variable might be used to control various types of different behavior.

Docker

Docker containers are build using make in the docker directory. Key commands are:

• make <container> to build the image for the container,

Since the Python code is taken from the host when the container starts, restarting is enough to use the code you have in your local git repo. Rebuilding is unnecessary. Docker networking ————————-

The Docker containers started use a consul based virtual network to communicate among each other. This means that:

• Containers address each other by a service name as defined in the job file (f.e. tango.service.consul for the TANGO_HOST),

• localhost can only be used within the containers to access other containers, if sidecar proxy is used.

• Most ports are dynamically allocated. It will be mapped to the right port within the container.

CORBA

Tango devices use CORBA, which require all servers to be able to reach each other directly. Each CORBA device opens a port and advertises its address to the CORBA broker. The broker then forwards this address to any interested clients. A device within a docker container cannot know under which name it can be reached, however, and any port opened needs to be exposed explicitly in the docker-compose file for the device. To solve all this, we assign a unique port to each device, and explictly tell CORBA to use that port, and what the hostname is under which others can reach it. Each device thus has these lines in their compose file:

ports:
  - "5701:5701" # unique port for this DS
entrypoint:
  # configure CORBA to _listen_ on 0:port, but tell others we're _reachable_ through ${HOSTNAME}:port, since CORBA
  # can't know about our Docker port forwarding
  - python3 -u /opt/lofar/tango/devices/devices/sdp/sdp.py STAT -v -ORBendPoint giop:tcp:0:5701 -ORBendPointPublish giop:tcp:${HOSTNAME}:5701

Specifying the wrong $HOSTNAME or port can make your device unreachable, even if it is running. Note that $HOSTNAME is advertised as is, that is, it is resolved to an IP address by any client that wants to connect. This means the $HOSTNAME needs to be correct for both the other containers, and external clients.

The docker-compose/Makefile tries to set a good default for $HOSTNAME, but you can override it by exporting the environment variable yourself (and run make restart <container> to effectuate the change).

For more information, see:

• https://huihoo.org/ace_tao/ACE-5.2+TAO-1.2/TAO/docs/ORBEndpoint.html

• http://omniorb.sourceforge.net/omni42/omniNames.html

• https://sourceforge.net/p/omniorb/svn/HEAD/tree/trunk/omniORB/src/lib/omniORB/orbcore/tcp/tcpEndpoint.cc

Logging

Overview of the data flow between docker services to facilitate logging

The Logstash pipeline collects the logs from the containers, as well as any external processes that send theirs. The following interfaces are available for this purpose:

Interface	Port	Note
Syslog	1514/udp	Recommended over TCP, as the Logstash pipeline might be down.
Syslog	1514/tcp
JSON	5959/tcp	From python, recommended is the LogStash Async module.
Beats	5044/tcp	Use FileBeat to watch logs locally, and forward them to Loki.

We recommend making sure the contents of your log lines are parsed correctly, especially if logs are routed to the Syslog input. These configurations are stored in docker-compose/logstash/loki.conf.

Log from Python

The common.lofar_logging module provides an easy way to log to Loki through Logstash from a Python Tango device.

Log from Docker

Not all Docker containers run our Python programs, and can forward the logs themselves. For those, we use the syslog log driver in Docker. Extend the docker compose files with:

Logs forwarded in this way are provided with the container name, their timestamp, and a log level guessed by Docker. It is thus wise to parse the message content further in Logstash (see above).

Services