GridFTP

GridFTP is a high-performance data transfer protocol based on FTP and optimized for high-bandwidth wide-area networks. More information on GridFTP may be found on the Globus site.


  1. Installation

    2. To use GridFTP, it must be installed on both the client and the server. See GridFTP Installation.

  2. Choose Authentication Method
    3. DOEGrid Certificate SSH
    Use DOEGrid certificate for authentication. Use system’s ssh authentication method.
  3. Setup
    4. DOEGrid Certificate SSH
    You may use DOEGrid certificate registered with the center to authenticate.

    1. Obtain a DOEGrid Certificate
    2. Register DOEGrid Certificate
    		No additional setup necessary.
  4. Transfer Data
    5. DOEGrid Certificate SSH
    Once the above steps have been completed, you can transfer data using GridFTP. A valid proxy certificate must exist for each GridFTP transfer.

    1. Generate Proxy Certificate
    2. Transfer Data
    		Transfer Data






Installation


GridFTP Installation

To use GridFTP, it must be installed on both the client and the server. It is currently available on each NCCS system and can be added to your environment using the globus module.

module load globus

If your sited does not already have GridFTP available, it can be downloaded from Globus. Download and installation information can be found on the found on the Globus site.






SSH Authentication


Data Transfer using SSH Authentication


globus module

Before using the GridFTP tools you will need to add the globus build to your environment:
module load globus

Examples

  • From NCCS, transfer file1 in your NCCS work area to file2 on a remote system 


    • globus-url-copy -tcp-bs 12M -bs 12M -p 4 -v -vb file:/tmp/work/$USER/file1 sshftp://user@remote.system/remote/dir/file2


  • From NCCS, transfer file1 on a remote system to file2 in your NCCS work area 


    • globus-url-copy -tcp-bs 12M -bs 12M -p 4 -v -vb sshftp://remote.system/remote/dir/file1 file:/tmp/work/$USER/file2


  • From remote system, transfer file1 on a remote system to file2 in your NCCS work area 


    • globus-url-copy -tcp-bs 12M -bs 12M -p 4 -v -vb file:/remote/dir/file1 sshftp://userid@dtn01.ccs.ornl.gov/tmp/work/$USER/file2







DOEGrid Certificate Authentication


Obtain DOEGrid Certificate

Before using GridFTP each user must have a DOEGrid Certificate. The basic steps are listed below. Details may be found on the DOEGrid site.

  1. Import DOEGrid Chain of Certificate Authorities
  2. Request DOEGrid Certificate
  3. Import DOEGrid Certificate
  4. Export/Extract DOEGrid Certificate

    5. File permissions on userkey.pem and usercert.pem must be set as follows:

    usercert.pem | 644
    chmod 644 $HOME/.globus/usercert.pem
    userkey.pem | 600
    chmod 600 $HOME/.globus/userkey.pem




Register DOEGrid Certificate

Before using your DOEGrid Certificate on NCCS systems you must register your certificate with the NCCS. Once your DOEGrid Certificate has been extracted to $HOME/.globus/usercert.pem and $HOME/.globus/userkey.pem run the following:
> module load globus
> register_globus_creds

You will receive an email once your certificate has been successfully registered.




Generate Proxy Certificate

Prior to transferring data a new proxy certificate must be generated. Once the DOEGrid certificate has been registered a proxy certificate can be generated using the myproxy-init utility.

  1. Add globus tools to your environment

    2. > module load globus

  2. Generate a new certificate

    3. > myproxy-init -n

  3. Logon

    4. > myproxy-logon
    Note: Enter your PASSCODE at Enter MyProxy pass phrase: prompt.




Transfer Data using DOEGrid Certificate Authentication

Once a proxy certificate has been generated and you have loged on, data may be transferred using the globus-url-copy tool.

globus module

Before using the GridFTP tools you will need to add the globus build to your environment:
module load globus

Examples

  • From NCCS, transfer file1 in your NCCS home area to file2 on a remote system 


    • globus-url-copy -tcp-bs 12M -bs 12M -p 4 gsiftp://dtn01.ccs.ornl.gov/$HOME/file1 gsiftp:///remote.sys//tmp/shared/$USER/file2