All you need to know about the IT solution to build MAX IV Laboratory.

. . .

icon_hardware

Control Hardware

The standard and supported hardware.

icon_network

Network

The standard network equipments.

icon_system

System

The server side of MAX IV

icon_KITS

Control Software

Or how thousand of devices can produce experimental data

icon_data_management

Data Management

The gold mining applied to the Big Data

icon_scientific_SW

Scientific Software

Reduce the entropy to the understandable question

Network and System


The network and the IT infrastructure are the IT foundation of MAX IV.

Juniper Technology

The network at MAX IV rely on the Juniper equipment. The network is essentially based on TCP/IP v4 with a physical separation between Machine, Beamlines and offices. Each of them is divided by a set of virtual networks, usually corresponding to a subsystem, to minimise the influence of broadcast equipment. The layer 2 is based on Ethernet with 2 levels of switches in order to have a maximum network distance of 4.

VMWare Technology

The MAX IV infrastructure take advantage of the virtualisation. Almost all of the accelerator and beamlines servers are virtual machines running by a VMWare hypervisor EsXi cluster of a redundant 36 cores and 384 GB memory. This infrastructure is direct benefit of the choice to have Ethernet as a recommended interface for all the equipment.

Linux

The standard OS is CentOS 7 as many providers support the RedHat drivers.

Ansible

The configuration under control. Ansible Website is used for the configuration management of all the Control System computers and the web services, from the OS settings to the specific role (workstation, database, IOC, Web server) in the MAX IV ecosystem. A Continuous Delivery process has been set up with the help of the RPM system, Jenkins a continuous integration server and Ansible. Little bonus: it’s open source.

Electronics


IcePap

The Motorisation is essential to integrate Mechanics system to the IT. The standard motion equipment ICEPAP is provided by the KITS group. Around hundreds of axes have been installed to mainly control the phase shifters, some diagnostic equipment like screens, lenses, scrapers and the diagnostic beamlines. All the beamline optics are steered through this controller except one specific axis. The EPU and InVacuum undulators are also steered via ICEPAP.

General Purpose Input Output

The experimentation at MAX IV requires to acquire different signals synchronously. The first standard Input Output Controller (IOC) have been installed in the SPECIES beamline to accurately read the encoder positions. The IOC is based on Adlink IPC, ADC, DAC and National Instrument for the counter card. This equipment is intended to be used as a General Purpose Input Output acquisition system on the experimental station of the beamlines

MRF Timing system

Give the “Sync” of the Synchrotron. The IT group provides the entire timing system[9] for the facility, based on Micro Research Finland (MRF). A special development of the Beam Position Monitor “Libera Brilliance Plus” done by ITech allows to use them as a compatible event receiver.

Electrometer

Monitor the photon beam. The number of XBPM per beamline is estimated up to 20 per beamline, controlled by a new Electrometer[11] developed in collaboration with Alba, based on the first version deployed at Alba. The development consists to compose 4 current amplifiers in the same chassis, controlled and synchronised through a backplane steered by an FPGA. This will allow synchronised measurement and the possibility to embed custom algorithm for a control loop to steer, for example, a piezo fine tuning based on current (Licence Open Hardware[12] and Open Source). In location where the signal/noise ratio is expected to allow it, a multiplexer will be used to reduce the number of amplifiers. This multiplexer is currently in development at MAX IV.

User Software


In the overall the initial specification of the MAX IV control system allowed the reuse of widgets, panels and applications coming from different European Facilities and especially the ALBA Synchrotron. The best example is the VACCA[17] application for the monitoring of the 3 GeV storage ring vacuum subsystem and also the beamlines.

Tango

On the back end side the Tango control system version 8[19] is used as a middleware. For each type of equipment KITS develops the Tango device, software process to establish the communication with the hardware. In some case the drivers software have been contracted within a framework agreement with the manufacturers (Scandinova RF modulator[20], Libera Single Pass e and Brilliance Plus from ITech[21]) who have developed a compatible Tango server.
Additionally MAX IV received a software API from different collaborators and manufacturers. Budker Institut have provided a complete suite of software to control and set up their pulsed power supply via a web gateway[18] and Solaris has developed the Tango Server integration. Danfysik[22] and Itest[23] have provided a C++ libraries for their respective power supplies. In total 90 different classes of devices, mainly written in python, compose the catalogue of supported hardware and services for the MAXIV CS. For example on the accelerator the control system represents:

  • 106 hosts
  • 1187 server instances
  • 7921 devices
  • 287378 control points
  • 132833 attributes
  • 83735 commands

The majority of the Tango servers run on virtual machines.

Python

Python is a natural language between scientist and IT.

Taurus

The User Interface proposed by KITS are mainly based on the Taurus SCADA framework that allow a rapid creation of the application without development (TaurusGUI). By this way several dedicated subsystem GUI have been developed for the commissioning of the Beamlines and the Accelerators. The composition and the layout of the panels can be easily changed by the Users and Operators.

Sardana

Sardana Controls is a software suite for Supervision, Control and Data Acquisition in scientific installations. It aims to reduce cost and time of design, development and support of the control and data acquisition systems. Sardana development was started at the ALBA synchrotron and today is supported by a larger community which includes several other laboratories and individuals (ALBA, DESY, MaxIV, Solaris, ESRF).

Synoptic

The Synoptic application[18], showing the logical representation of the system like a map and the State Grid application, showing the states of the control system by a reduced Subsystem by Location matrix. These applications make use of the web technologies.

Data Management


The user of MAX IV will access a High Performance Computing (HPC) to run the off line analysis on their data through a remote CentOS desktop solution, ThinLinc[27], running the application in seamless mode from a native OS node (no emulation). A high performance storage system GPFS/InfiniBand allows to record and analyse the experimental data.

HDF5

The HDF5 format and Nexus when applicable has been defined as the standard format for the data storage, and the control system component like Sardana[16] or the Fast Data Archiver[26] have proved their compatibility and performance.

Security

All the IT systems at MAX IV will enforce a security at 2 factors authentication, One Time Password foresee. The ability to connect through different ID providers like the Umbrella project is also part of the expectation. In this context MAX IV leverages the collaboration with the partners like the Lund University[28] and especially with LUNARC[29].

Agile and Lean Management


Scrum, Kanban, DevOps: The KITS project management is based on Lean[3] where the focus is to bring the user value incrementally and as soon as possible without technical overhead. The Control System (CS) has been delivered following the Kanban[4] and the Scrum[5] methods since 2012 [1], while adapting continuously the rules to follow the pace and the nature of a synchrotron project.

Scrum

Scrum is an iterative and incremental agile software development methodology for managing product development.
The product backlog is what will ultimately be delivered, ordered into the sequence in which it should be delivered. Product Backlogs are broken into manageable, executable chunks named sprints. Every certain amount of time the team initiates a new sprint and commits to deliver a certain number of user stories from the backlog, in accordance with their skills, abilities and resources. The project advances as the backlog becomes depleted.

Kanban

In Japanese, 看 (kan) can mean to look closely at, take care of, or keep watch. 板 (ban, or ita by itself) means a board. Therefore, Kanban roughly translated should be something like “a board to look at”.
The Kanban methodology is used to divide project development (any sort of project) into stages.
A kanban card is like an index card or post-it note that details every task (or user story) in a project that needs to be completed. The Kanban board is used to move each card from one state of completion to the next and in so doing, helps track progress.

REFERENCES

[1] J. Lidon-Simon et al., “Status of the MAX IV Laboratory Control System”, MOPPC109, ICALEPCS’13.
[2] M. Linberg et al., “Motion Control on the Max IV Soft X-Ray Beamlines With Tango and Sardana”, MOPGF065, ICALEPCS’15.
[3] The Lean Enterprise Institute, website: http://www.lean.org/WhatsLean/.
[4] Kanban from Wikipedia, website: https://en.wikipedia.org/wiki/Kanban
[5] Scrum, the agile software development framework, website: http://www.scrum.org
[6] CentOS, the Open Source Linux distribution, website: https://www.centos.org/.
[7] V. Hardion et al., “Configuration Management of the control system”, THPPC013, ICALEPCS’13.
[8] The ICEPAP motion controller from ESRF, website: http://www.esrf.eu/Instrumentation/DetectorsAndElectronics/icepap
[9] J. Jamroz et al, “Timing System at MAX IV”, THPPC103, ICALEPCS’13.
[10] J. Jamroz et al, “Timing System at MAX IV – Status and Development”, WEPGF120, ICALEPCS’15.
[11] O. Matilla et al, “Em# Platform: Towards a Hardware Interface Standardization Scheme”, WEPGF081, ICALEPCS’15.
[12] The CERN Open Hardware Licence from Wikipedia, https://en.wikipedia.org/wiki/CERN_Open_Hardware_License
[13] The ELOG Home Page, https://midas.psi.ch/elog/.
[14] V. Hardion et al., “Manage the MAX IV Laboratory Control System as an Open Source Project”, MOPPC086, ICALEPCS’13.
[15] A. Gotz et al., “The TANGO Controls Collaboration in 2015”, WEA3O01, ICALEPCS’15.
[16] Sardana control user environment, http://www.tangocontrols.org/Documents/tools/Sardana
[17] S. Rubio-Manrique et al., “Unifying All TANGO Control Services in a Customizable Graphical User Interface”, WEPGF148, ICALEPCS’15.
[18] J. Forsberg et al., “A Graphical Tool for Viewing and Interacting with a Control System”, WEM309, ICALEPCS’15, these proceedings.
[19] Scandinova, website: http://www.scandinovasystems.com/.
[20] I-Tech Libera Brilliance Plus, website: http://www.i-tech.si/accelerators-instrumentation/libera-brilliance-plus
[21] A. Panov et al., “TANGO Integration of a Specific Hardware through HTTP-server”, MOPGF111, ICALEPCS’15.
[22] Danfysik, website: http://www.danfysik.com
[23] Itest, website: http://www.bilt-system.com/.
[24] G. Portmann et al., “An Accelerator Control Middle Layer Using MATLAB”, FPAT077, ICALEPCS’05.
[25] L.S.N. Nadolski et al., “High Level Control Applications for SOLEIL Commissioning and Operation”, ROPA005, ICALEPCS’05.
[26] P. Bell et al., “100Hz Data Acquisition in the TANGO Control System at the Max IV Linac”, MOPGF049, ICALEPCS’15.
[27] ThinLic, website:, https://www.cendio.com/thinlinc/what-is-thinlinc
[28] Lund Universitet, website: http://www.lunduniversity.lu.se/
[29] LUNARC website, http://www.lunarc.lu.se/.
[30] ISpyB, website: http://www.esrf.eu/UsersAndScience/Experiments/MX/How_to_use_our_beamlines/ISPYB
[31] P.P. Goryl et al., “Status of the Solaris Control System – Collaborations and Technology”, MOPGF151, ICALEPCS’15.