Remote experiments at BioMAX
Remote experiments in BioMAX are currently under commission. Users who are interested in trying out remote access should contact beamline staff. This document describes the tools required for remote access and lists the experimental facilities currently supported under this mode. Click on each of the links below for information on each topic.
Because BioMAX is still under development and new user features are being implemented regularly, it is required require that users visit in person MAX IV at least once per year. Remote users should be familiar with the correct procedures to ship samples to and from BioMAX, with MXCuBE3, ISPyB and the computing environment at MAX IV, and tools for handling the data and for remote access. Users who are interested in remote experiments, especially if new to BioMAX, should communicate in advance with staff to make sure that they receive appropriate training on these subjects. Attending a BAG training session would also qualify as training for remote access.
Fully remote access implies that the users must be able to mount and dismount samples, collect, process and backup the data without being present at MAX IV. This requires using the sample exchanger to mount and dismount the samples, which must be mounted on spine pins of the standard length and loaded in UniPucks. Standard data collection at 100 K is readily supported, and all the beamline functions and collection modes available through MXCuBE3 are also accessible during remote experiments. Data must be transferred through the internet trough SFTP or Globus only: copying data to external disks is not supported for remote users. Room temperature and serial crystallography experiments are not included in the remote program for the foreseeable future and neither are experiments from hazardous samples or any experiment that requires special instrumentation or setup. Regardless of the experiment, remote access may be granted to eligible collaborators outside MAX IV as long as one or more members of the proposal are present at the beamline.
Support staff will load and unload the UniPucks into the sample exchanger dewar and transport the dewar to the outshipping area, but will not be participate in the experiment beyond normal support duties (eg, providing basic scientific support and solving beamline issues which affect data collection). Any extensive involvement of staff in tasks which would be normally be done by users (transfer samples shipped in ESRF pucks, manual sample mounting or dealing with sample issues) should be agreed before hand and treated as a collaboration.
MAX IV users can access a dedicated BioMAX remote desktop after establishing a VPN connection to the beamline network. Users who have already installed the Pulse Secure VPN client and the Cendio ThinLinc client to connect to the off-line cluster for data processing do not need to install any additional software. If this is not the case, follow these one time steps:
- Ensure that you enter a cell phone number in DUO where they can receive an SMS. This must be the primary phone number. When you connect to MAX IV, you will receive an otp (“one time password”) for two-step authentication.
- On a web browser, open http://vpn-white.maxiv.lu.se. Type your DUO username and password to log in. Then type in the otp code sent to your phone.
- Download the Pulse Secure VPN application and install it on your machine. For Windows, you simply need to execute a launcher script. For Mac and Linux, you must download the appropriate packages. In any case you will need administrative permission to install the software.
- Next, download the Thinlinc client and install it.
Once the software is installed and you have been notified by MAX IV that you have permission to connect to the beamline network, configure the Pulse Secure client:
- Open the Pulse Secure client and click on the + sign to add a new connection. Choose any name for the connection (e.g., “BioMAX”). The name of the server is vpn-blue.maxiv.lu.se. Ensure that there are no connections open to any other server.
Important: The VPN connection to the beamline network may not be open far in advance of your beamtime. If you have never used VPN to MAX IV before, we strongly recommend that you try connecting to the MAX IV office network on the vpn-white.maxiv.lu.se server and logging in to the offline data processing cluster front end (offline-fe1) well in advance of your experiment. Connection to the “white” network is available any time to all users listed in an active proposal or beamtime session. Consult the remote connection and the remote desktop documentation for more details.
- When you click on the Connect button on the beamline network, you will get a list of “Realms”. Scroll down to find remoteUsrBIOMAX and click on the name to select it. Check the Save settings button if you want to store the connection details for future connections.
- Enter your DUO user name and password. After authenticating, type the otp code you receive on your phone.
To connect to the Thinlinc client, follow the same instructions to connect to the offline MAX IV HPC, but use biomax-remote as the server name.
For remote experiments, ship the dewar containing UniPucks only to MAX IV as usual. It is advisable to ship the dewar to get to MAX IV one or two days before the experiment. Please ensure that you enclose the return paperwork in the dewar, including the ISPyB generated labels, the courier return labels and, for shipments outside the EU, the proforma documentation. The beamline staff will load the UniPucks in the dewar and let you know their position so that you can enter the information in ISPyB. Staff will also inform you of when the hutch is searched and you can start the experiment. They will also provide you with contact information in case you have questions during the experiment or run into problems.
To start the experiment, open the connection to the beamline network in Pulse Secure. Then, open Thinlinc, enter the beamline server name (biomax-remote) and your log in credentials. The remote desktop is configured to look like the workstations at BioMAX and is part of the same computing environment. Links to start MXCuBE3 and ALBULA (to view diffraction images) are available on the desktop.
To run MXCuBE3, we recommend using a screen of at least 1680×1050. Otherwise the GUI will not fit in the window, and it can become difficult to access all the buttons and links. The bottom right side of the GUI has a icon that links to video feeds from the BioMAX hutch. You can select between a wide angle view of the beamline, useful to monitor the sample mounting robot, or a closer view of the diffractometer. Because remote access enables participation in the experiment from collaborators at different locations, MXCuBE also supports features for multiuser access. Please see the MXCuBE remote access documentation.
The data will be stored in the usual location and you will be able to display the autoprocessing results on ISPyB either via the remote server or a browser on your computer. If you wish to process the data manually during the experiment, connect to the online HPC cluster from the biomax-remote server as described in the documentation. Note that you will not be able to open a connection to the offline HPC on the same machine as the remote beamline server as long as you have an open VPN connection to the beamline network.
If you lose the connection temporarily, you will be able to reconnect to the Thinlinc session and resume the experiment; connection outages do not affect ongoing data collections or beamline tasks. Also, you can conduct the experiments at different locations (eg, you can start at work and then continue from home). If you do not log out from the Thinlinc session, you will be able to reconnect to it from any location. However, MXCuBE3 does not allow the same user to be logged in twice, so if you try to log in twice (for example, both from the beamline and remotely) you will not be able to connect unless you force-quit the first MXCuBE instance from the log in window.
When you finish the experiment, please log out from MXCuBE and from the remote server Thinlic session and let the beamline support staff know (outside working hours please use email or send a message). Staff will unload the samples and, provided that all the required paperwork is present, take the dewar to the out shipping area.
Problems with the VPN connection
- Double check that you are logging in with the correct credentials; the login is case-sensitive. If at some point you save your credentials in the connection and then change the password, you will have to create a new connection to be able to log in again.
- If you do not get an otp, make sure that the phone in your DUO profile can receive messages, and that it is in the correct format (+ sign preceding the country code and then the number)
- If you can connect to the office network vpn-white, but not the beamline network vpn-blue, check that you are included in the current beamline session; note that the permission to access the beamline network is not updated inmediately, so if you have been added to the session just before or after the beamtime starts, it can take up to half an hour until you are actually enabled to open the VPN connection.
- Make sure that you do not have another VPN connection on the same computer.
- The PUlse Secure client checks for updates so at some point you will get a message about updating the application after you log in. Accepting the message will terminate your current session; it is OK to ignore this message and do the update later.
Problems with the Thinlinc server
- If the server name cannot be resolved, check that the VPN connection is still on and that you are trying to log in to the correct server (eg, if you are connected to the vpn-white server, you will not be able to log in to the biomax-remote server. Sometimes the problem can be solved by typing the entire domain name for the server. e.g. biomax-remote.maxiv.lu.se
- If you the connection fails to open an SSH tunnel to the server, you may have a corrupted old session running. Click on the “End existing session” checkbox.
- If the server has run out of licences, contact staff.
- The Thinlinc window is maximized by default. If this is a problem, use the F8 key to change this behaviour in the Thinlinc menu.
- If you close the Thinlinc window by mistake, simply log in again. You should be reconnected to the old session, as long as the “End existing session” checkbox is unchecked.
Problems during the experiment
There are some recurrent issues at MXCuBE commonly observed by users, documented here. As a remote user you may be more likely to fail to click on some menu buttons (eg, the Remote access menu, or the signout button), if you are running your remote session on a laptop or desktop with a small screen. Resizing the browser window should solve this.
The remote desktop has four working spaces by default (they are displayed, iconized, at the right bottom part of the Thinlinc window). This is useful because you can open different programs in different spaces (eg, you can run MXCuBE, Albula and ISPyB in different spaces to make more room for each GUI). If you drag a window off to a side you may move it to the adjacent space by accident. Double check this before trying to reopen or restart the window.
If the entire GUI (MXCuBE or other) becomes irresponsive, check other windows in the remote desktop and on your local computer before contacting staff. If your computer crashes or you lose the connection, you should be able to reconnect to the remote session again.