Transfer Data from MAX IV using Globus Connect
How to download your data from MAX IV.
Due to the large amount of data generated during experiments, copying datasets from MAX IV beamlines to external hard disks are not always practicle. We highly recommend using Globus Connect to retrieve experimental data from MAX IV
Globus works as a broker, managing your data transfer while you do something else. You give Globus temporary permissions to access location A and location B on your behalf. Provide Globus a list of files to transfer from A to B. Globus will then do it’s best to complete the task, trying to overcome bad network situations and computers that for one reason or the other are not always available.
Prerequisites
- A computer running macOS, Windows or Linux where you can install the Globus software or access to a registered Globus Endpoint
- Globus ID credentials (Either personal or organizational)
- MAX IV credentials, for users the DUO account.
- A high-speed internet connection to complete the transfer in reasonable time.
Limitations
- It is not possible to connect to the MAX IV endpoint from within the MAX IV network
1. Log in to Globus
- Go to the Globus website: https://app.globus.org/
- Select your university/organization from the drop down menu and click on the Continue button to access the log in page; or use one of the alternative options provided to sign in.
- It is important to understand that this first login is to gain access to Globus, from here you then connect to the two endpoints that send and receive data respectively. The access to MAX IV will be configured in the next step.
2. Setup MAX IV Data Transfer Service endpoint
- After successful log in to Globus, the Transfer Files screen will be loaded and both endpoints can be configured. First, enter the Collection end point (the MAX IV endpoint). Click in the input box and and type MAX IV Laboratory.
- Click continue. The next window asks for authentication using your DUO Login credentials.
- Once you log in to MAX IV successfully you will see the visitor MAX IV folder, which contains the data collected at each beamline. Note that you will only have access to data from proposals you are part of.
- If you already have an endpoint, personal or perhaps institutional, enter it as the second Collection in the right panel, depending on which panel layout you chosen, then continue with the download in section 4.
- If you need to setup a new personal endpoint follow the instructions in section 3.
3. Setup Personal endpoint
- To download data to a personal computer, one will have to define that computer as Personal Endpoint using Globus Connect software. Select File Manager in the toolbar, then click in the Collection Search field.
- A screen like the image below will appear, click on the Get Globus Connect Personal link.
- Next, download and install Globus Connect Personal software in your computer, using the link corresponding to your operating system. Follow the on-screen instructions keeping the default settings.
- You will be requested to sign in again (refer to the section 1 above), allow Globus the required access and provide a name for your new collection.
- Once you finish the setup, the local endpoint will be available under the Administrated by me tab, found after clicking the endpoint name on the right-hand side. This option enables searching local folders to download data.
4. Download MAX IV data
- To transfer the data to the final endpoint, go back to the File Manager by selecting the item in the left-side menu.
- On the left window pane, select the data you want to transfer, which is under the following path:
/data/visitors/beamline/proposal_number/collection_day - On the right pane, select your local destination folder and click on the blue button on the top to start the data download.
- To monitor the transfer progress, you can click the Activity button at the bottom of your Transfer Files page. When the transfer finishes, you will receive an email.