Developer information ========================= This chapter describes key areas useful for developers. Environment variables --------------------- Several environment variables fundamentally control the deployment and development environment. These include: - `TANGO_HOST` - `TANGO_STATION_CONTROL` - `TANGO_SKIP_BUILD` Firstly, `TANGO_HOST` should point to the tango database server including its port. An example would be `10.14.0.205:10000`. If `TANGO_HOST` is not set instead `tango.service.consul:10000` is used. Finally `TANGO_STATION_CONTROL` can be used to control if device containers should build software from source (developer mode). Or if the software should be built into the `lofar-device-base` docker image directly. If `TANGO_STATION_CONTROL` is set the makefile will build a wheel package which will be installed into the docker image. If instead a particular wheel package needs to be installed `TANGO_SKIP_BUILD` can be set as well. Be sure the wheel package is placed in the `tangostationcontrol/dist/` directory. In the future the actual value of the `TANGO_STATION_CONTROL` variable might be used to control various types of different behavior. Docker ------------------------- Docker containers are build using ``make`` in the ``docker`` directory. Key commands are: - ``make `` to build the image for the container, Since the *Python code is taken from the host when the container starts*, restarting is enough to use the code you have in your local git repo. Rebuilding is unnecessary. Docker networking ------------------------- The Docker containers started use a consul based *virtual network* to communicate among each other. This means that: - Containers address each other by a service name as defined in the job file (f.e. ``tango.service.consul`` for the TANGO_HOST), - ``localhost`` can only be used within the containers to access other containers, if sidecar proxy is used. - Most ports are dynamically allocated. It will be mapped to the right port within the container. .. _corba: CORBA ```````````````````` Tango devices use CORBA, which require all servers to be able to reach each other directly. Each CORBA device opens a port and advertises its address to the CORBA broker. The broker then forwards this address to any interested clients. A device within a docker container cannot know under which name it can be reached, however, and any port opened needs to be exposed explicitly in the docker-compose file for the device. To solve all this, we *assign a unique port to each device*, and explictly tell CORBA to use that port, and what the hostname is under which others can reach it. Each device thus has these lines in their compose file:: ports: - "5701:5701" # unique port for this DS entrypoint: # configure CORBA to _listen_ on 0:port, but tell others we're _reachable_ through ${HOSTNAME}:port, since CORBA # can't know about our Docker port forwarding - python3 -u /opt/lofar/tango/devices/devices/sdp/sdp.py STAT -v -ORBendPoint giop:tcp:0:5701 -ORBendPointPublish giop:tcp:${HOSTNAME}:5701 Specifying the wrong ``$HOSTNAME`` or port can make your device unreachable, even if it is running. Note that ``$HOSTNAME`` is advertised as is, that is, it is resolved to an IP address by any client that wants to connect. This means the ``$HOSTNAME`` needs to be correct for both the other containers, and external clients. The ``docker-compose/Makefile`` tries to set a good default for ``$HOSTNAME``, but you can override it by exporting the environment variable yourself (and run ``make restart `` to effectuate the change). For more information, see: - https://huihoo.org/ace_tao/ACE-5.2+TAO-1.2/TAO/docs/ORBEndpoint.html - http://omniorb.sourceforge.net/omni42/omniNames.html - https://sourceforge.net/p/omniorb/svn/HEAD/tree/trunk/omniORB/src/lib/omniORB/orbcore/tcp/tcpEndpoint.cc Logging ------------------------- Overview of the data flow between docker services to facilitate logging .. image:: static/images/logging-data-flow.png :target: _images/logging-data-flow.png The Logstash pipeline collects the logs from the containers, as well as any external processes that send theirs. The following interfaces are available for this purpose: +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | Interface | Port | Note | +=============+============+=============================================================================================================+ | Syslog | 1514/udp | Recommended over TCP, as the Logstash pipeline might be down. | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | Syslog | 1514/tcp | | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | JSON | 5959/tcp | From python, recommended is the `LogStash Async `_ module. | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | Beats | 5044/tcp | Use `FileBeat `_ to watch logs locally, and forward them to Loki. | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ We recommend making sure the contents of your log lines are parsed correctly, especially if logs are routed to the *Syslog* input. These configurations are stored in ``docker-compose/logstash/loki.conf``. Log from Python ````````````````` The ``common.lofar_logging`` module provides an easy way to log to Loki through Logstash from a Python Tango device. Log from Docker ````````````````` Not all Docker containers run our Python programs, and can forward the logs themselves. For those, we use the ``syslog`` log driver in Docker. Extend the ``docker compose`` files with: Logs forwarded in this way are provided with the container name, their timestamp, and a log level guessed by Docker. It is thus wise to parse the message content further in Logstash (see above). Services ------------------------- .. image:: static/images/station-services.png :target: _images/station-services.png