Difference: ComputingT2Transfers (3 vs. 4)

Revision 42014-07-30 - samir

Line: 1 to 1
 

USCMS T2 Transfers

Line: 8 to 8
  The general picture on transfers over 20 Gbps and some of these configuration problems are mentioned in Samir's talk at the T2 meeting of 07/29.
Added:
>
>		So far, the showstopper was the uplink bandwidth for most sites. Since July 2014 this is starting to change.
 The ideal is that even 10 Gbps sites could participate, as it is possible that the currrent settings are not optimal for fast transfers. We could tune it until it saturates the 10 Gbps link and everyone would have exercised how to improve transfer rates in Debug.
Changed:
<
<

Current connectivity at sites

>
>

Plan for the exercise

As discussed in the meeting, we would like to use Caltech as the source site, as it managed to do 25/29 Gbps with its setup, being a good source for sites optimizing their configurations. Once everyone else optimizes their download configurations and we observe which rates we get to which sites, we could start rotating who is the source site, and see what are the maximum rates that we get from them. It is important that we have multiple sink sites, as even if there are limitations in sites, the others will add up to the total rate.

 
Changed:
<
<		So far, the showstopper was the uplink bandwidth for most sites. Since July 2014 this is starting to change, this is to track the current state from different sites
>
>		There are 3 major steps on this exercise, 2 of them will require coordination among sites :
 
Changed:
<
<
  • MIT - 10 Gbps
  • Caltech - 100 Gbps
  • Nebraska - Could do more then 10 Gbps currently, will have 100 Gbps on the T2 soon.
  • Purdue - 100 Gbps. Running tests with FNAL.
  • Florida - 100 Gbps
  • UCSD - 10 Gbps
  • SPRACE - 10 Gbps
  • Wisconsin - 10 Gbps
  • Vanderbilt - 10 Gbps
>
>
  • Tuning PhEDEx download configurations, so LoadTest settings will correspond better to reality
  • Observing how transfers behave at the FTS level, note if the Optimizer algorithm is a limiting factor or it actually helps to achieve the optimal setting of active transfers for the available bandwidth at a given moment.
  • The logical limitations would have been removed, sites can focus on setting their upload rates as they want and optimize their GridFTP setups, start observing what are the best rates they can get out of the storage.
When we are done with these, the transfer test framework through PhEDEx will be more responsive and we will actually be able to run more advanced tests on higher rates. For example coordinate pushing data from many sites to one.

Participation of sites

In order to contact only the interested sites, please fill out the table below :

Site Connectivity Participating Notes
T2_BR_SPRACE 10G N/A  
T2_US_Caltech 100G DONE  
T2_US_Florida 100G N/A  
T2_US_MIT 10G N/A  
T2_US_Nebraska 10G N/A Upgrading to 100G soon
T2_US_Purdue 100G N/A  
T2_US_UCSD 10G N/A  
T2_US_Wisconsin 10G N/A  
T2_US_Vanderbilt 10G N/A  
 

FTS Notes

Added:
>
>		Currently we have 3 official FTS servers :

  • cmsfts3.fnal.gov
  • fts3.cern.ch
  • lcgfts3.gridpp.rl.ac.uk
There is an official recommendation that is the most logic distribution of what you should use, however for this exercise people are encouraged to try other deployments and possibly different behaviors. For example it was observed 208 transfers in parallel in CERN's FTS, but not more than 50 at FNAL (yet).

In the long run, US Sites should use FNAL. But it might be worth to understand if other FTS servers have a different optimizer behavior and why.

 

PhEDEx Documentation

Changed:
<
<		We will be exercising mostly the Download agent, therefore the most useful documenation for us is this. However there is also this if you would like to read more.
>
>		We will be exercising mostly the Download agent, therefore the most useful documenation for us is this.

However there is also this if you would like to read more.

 

PhEDEx configurations

Changed:
<
<		Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed condimentum rhoncus ligula, et lobortis ipsum aliquet id. In fringilla felis id venenatis placerat. Vivamus ornare egestas mattis. Donec sapien leo, gravida vitae magna sed, dictum convallis metus. Nulla mattis iaculis diam a gravida. Sed ac sem eget nisi tristique convallis vel sit amet dolor.
>
>		One of the limitations is how much the download site PhEDEx agent submits to FTS. Caltech was asked in the meeting how they control that. In that case, we have 2 agents. One for general transfers and another exclusively for US Transfers. the -ignore and -accept flags will do the separation. Also, see that one can throttle the number of active transfers for each site as needed and set a default for the sites not specified. The relevant part for Config.Debug is :
 
Changed:
<
<		. . . -batch-files 500
>
>		### AGENT LABEL=download-debug-fts PROGRAM=Toolkit/Transfer/FileDownload DEFAULT=on -db ${PHEDEX_DBPARAM} -nodes ${PHEDEX_NODE} -delete ${PHEDEX_CONF}/FileDownloadDelete -validate ${PHEDEX_CONF}/FileDownloadVerify -ignore '%T2_US%' -verbose -backend FTS -batch-files 50
  -link-pending-files 200 -max-active-files 700 -link-active-files 'T1_CH_CERN_Buffer=50'
Line: 48 to 78
  -link-active-files 'T2_RU_RRC_KI=2' -link-active-files 'T1_FR_CCIN2P3_Buffer=100' -link-active-files 'T1_FR_CCIN2P3_Disk=100'
Changed:
<
<		. . .
>
>		 -link-active-files 'T1_IT_CNAF_Buffer=150' -link-active-files 'T1_TW_ASGC_Buffer=100' -link-active-files 'T1_UK_RAL_Buffer=50' -link-active-files 'T1_US_FNAL_Buffer=100' -link-active-files 'T2_DE_RWTH=10' -link-active-files 'T2_IT_Pisa=20' -default-link-active-files 100 -protocols srmv2 -mapfile ${PHEDEX_FTS_MAP}

### AGENT LABEL=download-debug-t2fts PROGRAM=Toolkit/Transfer/FileDownload DEFAULT=on -db ${PHEDEX_DBPARAM} -nodes ${PHEDEX_NODE} -delete ${PHEDEX_CONF}/FileDownloadDelete -validate ${PHEDEX_CONF}/FileDownloadVerify -accept '%T2_US%' -verbose -backend FTS -batch-files 20 -link-pending-files 300 -max-active-files 300 -protocols srmv2 -mapfile ${PHEDEX_FTS_MAP}

 

-- Main.samir - 2014-07-29

View topic | History: r9 < r8 < r7 < r6 | More topic actions...
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback