Developer information ========================= This chapter describes key areas useful for developers. Docker compose ------------------------- The docker setup is managed using ``make`` in the ``docker-compose`` directory. Key commands are: - ``make status`` to check which containers are running, - ``make build `` to rebuild the image for the container, - ``make build-nocache `` to rebuild the image for the container from scratch, - ``make restart `` to restart a specific container, for example to effectuate a code change. - ``make clean`` to remove all images and containers, and the ``tangodb`` volume. To do a deeper clean, we need to remove all volumes and rebuild all containers from scratch:: make clean docker volume prune docker build-nocache Since the *Python code is taken from the host when the container starts*, restarting is enough to use the code you have in your local git repo. Rebuilding is unnecessary. Docker networking ------------------------- The Docker containers started use a *virtual network* to communicate among each other. This means that: - Containers address each other by a host name equal to the container name (f.e. ``elk`` for the elk stack, and ``databaseds`` for the TANGO_HOST), - ``localhost`` cannot be used within the containers to access ports of other containers. - ``host.docker.internal`` resolves to the actual host running the containers, - All ports used by external parties need to be exposed explicitly in the docker-compose files. The container must open the same port as is thus exposed, or the port will not be reachable. The networks are defined in ``docker-compose/networks.yml``: .. literalinclude:: ../../../docker-compose/networks.yml The ``$NETWORK_MODE`` defaults to ``tangonet`` in the ``docker-compose/Makefile``. .. _corba: CORBA ```````````````````` Tango devices use CORBA, which require all servers to be able to reach each other directly. Each CORBA device opens a port and advertises its address to the CORBA broker. The broker then forwards this address to any interested clients. A device within a docker container cannot know under which name it can be reached, however, and any port opened needs to be exposed explicitly in the docker-compose file for the device. To solve all this, we *assign a unique port to each device*, and explictly tell CORBA to use that port, and what the hostname is under which others can reach it. Each device thus has these lines in their compose file:: ports: - "5701:5701" # unique port for this DS entrypoint: # configure CORBA to _listen_ on 0:port, but tell others we're _reachable_ through ${HOSTNAME}:port, since CORBA # can't know about our Docker port forwarding - python3 -u /opt/lofar/tango/devices/devices/sdp/sdp.py STAT -v -ORBendPoint giop:tcp:0:5701 -ORBendPointPublish giop:tcp:${HOSTNAME}:5701 Specifying the wrong ``$HOSTNAME`` or port can make your device unreachable, even if it is running. Note that ``$HOSTNAME`` is advertised as is, that is, it is resolved to an IP address by any client that wants to connect. This means the ``$HOSTNAME`` needs to be correct for both the other containers, and external clients. The ``docker-compose/Makefile`` tries to set a good default for ``$HOSTNAME``, but you can override it by exporting the environment variable yourself (and run ``make restart `` to effectuate the change). For more information, see: - https://huihoo.org/ace_tao/ACE-5.2+TAO-1.2/TAO/docs/ORBEndpoint.html - http://omniorb.sourceforge.net/omni42/omniNames.html - https://sourceforge.net/p/omniorb/svn/HEAD/tree/trunk/omniORB/src/lib/omniORB/orbcore/tcp/tcpEndpoint.cc Logging ------------------------- The ELK stack collects the logs from the containers, as well as any external processes that send theirs. It is the *Logstash* part of ELK that is responsible for this. The following interfaces are available for this purpose: +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | Interface | Port | Note | +=============+============+=============================================================================================================+ | Syslog | 1514/udp | Recommended over TCP, as the ELK stack might be down. | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | Syslog | 1514/tcp | | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | JSON | 5959/tcp | From python, recommended is the `LogStash Async `_ module. | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ | Beats | 5044/tcp | Use `FileBeat `_ to watch logs locally, and forward them to ELK. | +-------------+------------+-------------------------------------------------------------------------------------------------------------+ We recommend making sure the contents of your log lines are parsed correctly, especially if logs are routed to the *Syslog* input. These configurations are stored in ``docker-compose/elk/logstash/conf.d``. An example: .. literalinclude:: ../../../docker-compose/elk/logstash/conf.d/22-parse-tango-rest.conf Log from Python ````````````````` The ``common.lofar_logging`` module provides an easy way to log to the ELK stack from a Python Tango device. Log from Docker ````````````````` Not all Docker containers run our Python programs, and can forward the logs themselves. For those, we use the ``syslog`` log driver in Docker. Extend the ``docker compose`` files with: .. literalinclude:: ../../../docker-compose/rest.yml :start-at: logging: :end-before: restart: Logs forwarded in this way are provided with the container name, their timestamp, and a log level guessed by Docker. It is thus wise to parse the message content further in Logstash (see above).