Software Coordinating Committee Conference Call August 12, 2004 3:00 PM EDT Recorder: C. DeTar Present: Brower, Petrov, Holmgren, Simone, Osborn, Edwards, Watson, DeTar Absent: Mawhinney, Pochinsky, Dru Renner, Gregory, Jung, Mendes, Neff, Watson ** action items ===================================================================== Special Session on I/O RTE and I/O Software DeTar: [see e-mail about what is planned for QIO version 1.0] QIO needs two functions - one that maps a node to its I/O node and one that identifies the master I/O now Osborn: QMP should perform this function. Holmgren: [see his e-mail about dcache, planned at FNAL] It looks like one file per node is the best way to go with dcache. Petrov: My colleagues at Columbia are pushing to have fixed partitions and one queue per partition. In this model the I/O nodes will be known exactly. Edwards: I would have staging jobs share the processors with a running job and write staging software using QIO. Edwards: On our clusters, having one disk per node is the easiest way to designate nodes. On the GigE machine, we submit to fixed partitions, so we always know which disk is associated with which node number. Watson: The scheduler can predict which job runs next and the staging software can respond, possibly moving already-staged files to another disk. DeTar: To assure coordination between QIO and the staging software both QIO and staging utilities should be built simultaneously. DeTar: So is everyone happy with what is planned for QIO 1.0? Petrov: We are not happy that parallel file support has been dropped. That is the way we have done I/O. There is also strong interest at Columbia in being able to append to a file. DeTar I won't be able to add those features for a few months at least. My goal now is to get a minimal working version. Interversion compatibility Edwards: QIO 1.0 supports QMP 2.0. That will break QDP++. Watson: QMP 2.0 in a couple of months. Version control Osborn: Should we be using Subversion instead of CVS? Edwards: I will poll others using the CVS server to see whether any are using Subversion. ===================================================================== Committee conference concluded at 3:30 PM EDT. Next plenary call August 19 at 3 PM EDT. From detar@dirac.physics.utah.edu Thu Aug 12 16:53:34 2004 Date: Thu, 12 Aug 2004 14:53:06 -0600 (MDT) From: Carleton DeTar To: cmendes@cs.uiuc.edu, detar@physics.utah.edu, djholm@fnal.gov, edwards@jlab.org, watson@jlab.org, rdm@phys.columbia.edu, brower@lns.mit.edu, avp@lns.mit.edu, simone@fnal.gov, osborn@physics.utah.edu, petrov@physics.columbia.edu, mdp@fnal.gov, dgr@jlab.org, hneff@bu.edu, jhetrick@uop.edu, gregory@physics.arizona.edu, orginos@lns.mit.edu, chulwoo@phys.columbia.edu, sugar@sarek.physics.ucsb.edu, sg@denali.physics.indiana.edu, doug@klingon.physics.arizona.edu Subject: Software Comm Minutes, July 15, 2004 X-Scanned-By: MIMEDefang 2.43 Software Coordinating Committee Conference Call August 12, 2004 3:00 PM EDT Recorder: C. DeTar Present: Brower, Petrov, Holmgren, Simone, Osborn, Edwards, Watson, DeTar Absent: Mawhinney, Pochinsky, Dru Renner, Gregory, Jung, Mendes, Neff, Watson ** action items ===================================================================== Special Session on I/O RTE and I/O Software DeTar: [see e-mail about what is planned for QIO version 1.0] QIO needs two functions - one that maps a node to its I/O node and one that identifies the master I/O now Osborn: QMP should perform this function. Holmgren: [see his e-mail about dcache, planned at FNAL] It looks like one file per node is the best way to go with dcache. Petrov: My colleagues at Columbia are pushing to have fixed partitions and one queue per partition. In this model the I/O nodes will be known exactly. Edwards: I would have staging jobs share the processors with a running job and write staging software using QIO. Edwards: On our clusters, having one disk per node is the easiest way to designate nodes. On the GigE machine, we submit to fixed partitions, so we always know which disk is associated with which node number. Watson: The scheduler can predict which job runs next and the staging software can respond, possibly moving already-staged files to another disk. DeTar: To assure coordination between QIO and the staging software both QIO and staging utilities should be built simultaneously. DeTar: So is everyone happy with what is planned for QIO 1.0? Petrov: We are not happy that parallel file support has been dropped. That is the way we have done I/O. There is also strong interest at Columbia in being able to append to a file. DeTar I won't be able to add those features for a few months at least. My goal now is to get a minimal working version. Interversion compatibility Edwards: QIO 1.0 supports QMP 2.0. That will break QDP++. Watson: QMP 2.0 in a couple of months. Version control Osborn: Should we be using Subversion instead of CVS? Edwards: I will poll others using the CVS server to see whether any are using Subversion. ===================================================================== Committee conference concluded at 3:30 PM EDT. Next plenary call August 19 at 3 PM EDT.