Caltech CMS Computing 2010

Notes from the review meeting on October 5th and the follow up discussion.

Resources

Presentation from Mike and Dorian at the Computing Review. It contains a short overview of the resources, services, and issues.

T3-review-Oct5_2010_v2.pdf: CMS @ Caltech Computing Review, October 5th 2010. M. Thomas / D. Kcira

Issues

Mike / Dorian

• T3 CPU is underutilized
  • Use T3 login nodes for running interactive jobs
    • t3-susy, t3-higgs: 8 CPUs each
  • Use the SGE batch system for non-interactive jobs
    • Priorities taken care by SGE. Simple default SGE priorities for now: resources and user based. Can be refined if needed: group based, different queues.
• Storage is in short supply
  • Need to get organized
  • Automatic notification for unused data in hadoop?
  • Twiki page with datasets and ntuples as well as code for running over them
  • Strategy for scaling the T3 storage with increase in data size
• Fixed times for routine maintenance
  • 1 hour weekly time slot
  • 8 hour monthly time slot

• Over the past 6-12 months I have been using mainly the LPC
• resources and CAF (for the ECAL calibration).
• My main issues with T3-SUSY is that the queue had been completely clogged when I tried to use it a few months ago. For T3-higgs, I think we also need clear instructions on how to use the batch queue. Right now, I am using T3-Higgs mainly for the last stages of the analysis.
• As for the analysis format, the CITAnalyzer is no longer being maintained but is still somewhat alive and is used for private production by Josh. Otherwise, we (me and Yousi) have pretty much completely migrated to the MPAAnalyzer that is used in the H->2photons and QCDPhotons group. If we are to try and share ntuples within the group some effort would have to be invested in producing a sample analyzer and clear explanation for each data format. I am not sure how manageable it would be.

To Do

• Import information from gaewiki and sakai to this new Twiki. Especially important: how to run on the t3-susy and t3-higgs.
• Vacate the raid2 array at t3-susy and merge this array in hadoop (MT/DK + Chris Rogan)
• Delete old data on t3-higgs, merge arrays into hadoop.
• Check how to run directly on data from castor, best strategy on copying to t3-susy
• Add description to Twiki about primary data sets and triggers (Jan Veverka)
• Find out the best strategy for small private production at our T3s
• Compile list of desktop recources at CERN
• SGE: user friendly status of jobs running, waiting (including reason why). Readable response from monitoring.

-- Main.dkcira - 2010-10-05