\documentclass[12pt]{article}
\usepackage{epsfig}
\begin{document}
\begin{center}
{\bfseries\Large A QDP++ Primer}
\end{center}
\section{Introduction}
QDP++ is a implementation of the Level-2 data-parallel QCD-API in C++.
Broadly, the aim is to provide a suite of data-parallel routines and
data types appropriate to lattice gauge theory that enable efficient
writing of C++ lattice QCD codes.  QDP++ codes will be portable, with
details of the low-level communication routine transparent to the
user, and benefit from the SciDAC optimisation developments on the
national facilities available to the lattice community.

Why C++?  C++ encourages an object-orientated programming model, and
the construction of \textit{classes}; a corollary is the concept of
\textit{operator overloading}.  Classes are user-defined data
types, but in addition their definition generally includes a set of
allowed operations on them.  Operator overloading means that the
precise definition of a function or operator depends on the
\textit{class} of the arguments.  Thus, if \verb+A+ and \verb+B+ are
matrices, the multiplication operation might mean something different
from the case where \verb+A+ and \verb+B+ are real numbers.

\subsection{The nitty-gritty}
\subsubsection*{Getting hold of QDP++}
The QDP++ web page is available at
\verb+http://www.jlab.org/~edwards/qdp+.  There are instructions there
for acquiring the source, but the simplest way is via CVS which you
should install on your favourite local linux/cygwin machine.  For the
moment, this method requires that you have an account on one of the JLAB
central machines. To get the latest release of the code, login to your
local workstation, and proceed as follows:
\begin{verbatim}
export CVS_RSH=ssh
cvs -d :ext:<username>@jlab.org:/home/edwards/cvsroot checkout qdp++
\end{verbatim}
Note that JLAB only allows access with ssh2.  This will create a
directory on your machine called \verb-qdp++-, cotaining the basic
code and header files for QDP++ with several further
subdirectories.  Perhaps most important is \verb+tests+, containing
the various test programs that, say, compute the mean plaquettes etc..

To compile QDP++, you need a sufficiently up-to-date version of
\verb+gcc+, say version 3.1 or later.  To check that everything is
installed correctly, go into the top-level qdp++ directory and type
\verb+make all+.  This will create the library \verb+qdp.a+.

\subsubsection*{Documentation}
A \textit{reference manual} can be found at\\
\verb-http://www.jlab.org/~edwards/qcdapi/qdp++/doc/html/-.

This is generated from the code using \verb+doxygen+; it contains the
basic definitions of the classes, and the prototypes for the QDP++
functions.  However, it is for the moment strictly a reference manual
rather than a user guide.

\subsection*{Writing new code}
In order to write code using \verb-qdp++-, you need firstly to include
the header \verb+qdp.h+ that essentially is a wrapper for the headers
that define the data types, and the function prototypes.  Secondly,
you need to link in the library \verb+qdp.a+.  To see how to
accomplish these tasks, it is easiest to use the \verb+tests+
subdirectory as an example.

We will suppose you wish to develop the code in a new directory
\verb+play+.  Copy \verb+Makefile+ from \verb+tests+ into \verb+play+.
Then edit \verb+CXXFLAGS+ so that \verb-qdp++- is included, by
\verb#-I../qdp++# etc, that \verb+libdir+ also points to this
directory.  and make sure that \verb+VPATH+ also includes the
\verb#qdp++# directory.  Finally, you have to ensure that the library
\verb+qdp.a+ in linked in to the final executable.

\section{QDP++ Basics}
\subsection{Data Types}
C++ has a brief set of predefined data types, or \textit{classes} in
C++ parlance, such as \verb+float+, \verb+int+, \verb+char+, \verb+bool+
etc.. QDP++ includes additional ``user-defined'', or rather
predefined, classes that aid the construction of lattice QCD codes.
All QDP-defined classes begin with an upper-case letter.

The basic classes are of two forms:
\subsubsection*{Scalar classes:}
These are lattice objects that essentially live on a single site.
They include
\begin{itemize}
\item \verb+Real+ A floating-point number, which can be defined to be
  either a \textit{float} or a \textit{double}
\item \verb+Complex+ A complex number, constructed from two \verb+Real+
\item \verb+Gauge+ A general SU(\verb+Nc+) matrix
\item \verb+Fermion+ A complex object with \verb+Nc+ colour indices
and \verb+Ns+ spinor indices
\end{itemize}

We can create \textit{arrays}, such an array
\verb+nsize+ that defines the lattice dimensions
\begin{verbatim}
multi1d<int> nsize(Nd)
\end{verbatim}
where here \verb+Nd+ is the number of lattice dimensions.  Similarly,
we can create an array of lattice quantities
\begin{verbatim}
multi1d<LatticeGauge> u(Nd).
\end{verbatim}
which would correspond, say, to all four directions of a gauge field.

\subsubsection*{Lattice classes:}
These are objects that exist on every site of the lattice, and
typically are site-wide version of the scalar classes.  They all begin
with \verb+Lattice+, and some examples corresponding to the scalar
classes are
\begin{itemize}
\item \verb+LatticeReal+
\item \verb+LatticeComplex+
\item \verb+LatticeGauge+
\item \verb+LatticeFermion+
\end{itemize}

\subsection{QDP++ Functions}
The key to QDP++, inherited from C++, is the idea of \textit{operator
  overloading}, the idea that the definition of an operator can depend
  on the \textit{types} of the argument, i.e.\ be object oriented.  As
  a simple example, consider multiplication.  For simple real numbers
  $a, b$ and $c$, then multiplication has the obvious meaning
\[
c = a \times b.
\]
However, in the case of matrices $A, B$ and $C$, we would like
multiplication to be implemented as a matrix operation
\[
C_{ij} = \sum_{k} A_{ik} B_{kj}.
\]
We can extend this concept to multiplication of a vector by a matrix,
or even to the multiplication of objects with indices in different
spaces.  As a simple example, consider an SU(3) gauge field $U$, a
gamma matrix $\Gamma$ and some Dirac spinors $\psi_{\alpha,i}$ and
$(\phi,\psi)_{\alpha,i}$ where the Greek and Latin indices refer to spin and
colour respectively.  Then we would want to define what we mean by
multiplication of a spinor according to the matrix
\[
\psi_{\alpha i} = U_{ij} \Gamma_{\alpha \beta} \phi_{\beta j}
\]

\noindent In QDP++, this is expression can be simply written as
\begin{verbatim}
LatticeFermion psi
LatticeFermion phi
LatticeGauge u

psi = u*Gamma(1)*phi
\end{verbatim}
Note that the operation of the gamma matrices are defined in qdp++ by
the action on spinors, and in most cases you need not worry about the
precise gamma matrix convention.  Furthermore, because the quantities
are \textit{lattice variables}, this is a data parallel operation
performed on all the sites.   All the usual binary operations are
provided, as detailed in the QDP++ manual - but, of course, not
divides in the case of matrices.  Similarly, a set of \textit{unary}
operations such as \verb+sin+, \verb+cos+ etc which can act both on
\textit{scalar reals} and on \textit{lattice reals}.  Certain unitary
operators, notably \verb+conj+ have the interpretation of a
\textit{Hermitian Conjugate} when applied to matrices.

\subsection{Shifts and communication}

We are all familiar with the evaluation of \textit{staples} in lattice
calculations, that enter into the calculation of plaquettes, link
smearing etc..  Let us consider the case of the evaluation of the
upper plaquettes\\
\begin{center}
\psfig{width=3in,file=plaq.eps}
\end{center}

In mathematical language, we want
\[
S(x) = U_{\nu}(x+\mu) * U_{\mu}(x+nu)^{\dagger}*U_{nu}(x)^{\dagger}
\]
and of course we want to evaluate this at every site - or a subset,
say on the even sites.
In QDP++, this has the compact notation
\begin{verbatim}
LatticeGauge S
multi1d<LatticeGauge> u(Nd)
int mu,nu
...
...
...

S = u[nu]*conj(shift(u[mu], FORWARD, nu))*conj(u[nu]);
\end{verbatim}
\verb+shift+ is identical to that of Fortran 90, and the ``FORWARD''
refers to the direciton of the source relative to the destination.
Note that the above operation involves no temporaries, the bugbear of
C++ programming.

\subsection{Checkerboarding}
Efficient algorithms in both the pure-gauge and fermion sectors make
use of \textit{Chequerboards} (to use the British spelling!), in which
operators are performed in parallel on a subset of the lattice sites.
The simplest is a simple red/black checkerboard, in which the sites
are either even ($rb = 0$) or odd ($rb = 1$) according to the value of
\[
rb = (-1)^{x + y + z + t}.
\]
Some algorithms, e.g.\ those for improved gauge actions, use more
involved checkerboarding with more than two checkerboards.  Thus we
need a flexible means of restricting operations to a subset of sites,
i.e.\ those on a particular checkerboard. \textit{\cal N.B.} QDP++
specifies that ``Lattice'' types are defined on \textit{all} the
sites; ``subsets'', the QCD-API terminology we will be using, restrict
the arithmetical operators to a subset of sites, but do not alter the
allocation of memory.

Because we wish subsets to be as general as possible, we will make no
assumptions about whether a given checkerboarding has subsets of equal
size, or indeed whether the sum of subsets contains all the lattice
sites.  This has the important consequence that, in an arithmetic
operator restricted to a subset, it is only necessary, and indeed only
makes sense, to specify the subset of the \textit{target}.  Thus for
the case of the calculation of a generalised Wilson line
\[
A(x) = U(x) U(x+\mu) U(x+\mu+\nu) U(x+\mu+\nu+\rho)
\]
restricted to the case where $x$ is \textit{even} this is written
\begin{verbatim}

cb = 0;
a[rb[cb]] = u*shift(u*shift(u*shift(u,rho),nu), mu);

\end{verbatim}
The index \verb+rb[cb]+ specfies the subset \verb+cb+ of the
checkerboarding \verb+rb+, and the subsets of the sources are
completely specified by the shift operations.
\pagebreak
\section{A simple example}
\begin{verbatim}
#include <iostream>
#include <cstdio>

#include "pure_gauge.h"

// Enable the QDP environment to be accessible from the code.

using namespace QDP;

int main(int argc, char **argv){

  //  Begin by specifying the lattice sizes, of dimension Nd, defined in QDP
  multi1d<int> nsize(Nd);	// Note that the number of dimensions
  // defined in params.h

  //  For those new to C++, note that the vector class contains more 
  //  information than a simple array of elements, and in particular its size

  nsize = {4,4,4,8};

  // We now set up the geometry
  // The hello world of any parallel code
  geom.Init(nsize);

  // Now specify some gauge fields - some of these names may change
  // What we are doing here is creating a Nd array of vectors of type
  // LatticeGauge

  multi1d<LatticeGauge> u(Nd);

  //  Fill the gauge fields with gaussian random numbers

  for(int mu = 0; mu < Nd; mu++) gaussian(u[mu]);

  Double w_plaq, s_plaq, t_plaq, link;

  // Now measure the plaquette

  MesPlq(u, w_plaq, s_plaq, t_plaq, link);

  cerr << "w_plaq = " << w_plaq << endl;
  cerr << "link = " << link << endl;
}
\end{verbatim}
The very first thing is to define the lattice sizes, but then we have
\begin{verbatim}
geom.Init(nsize)
\end{verbatim}
which sets up the geometry.  This essentially is the data-parallel
version of ``hello, world''.  The real work, of course, is performed
in \verb+MesPlq+.
\end{document}

