\documentclass[12pt]{article}
\usepackage{fullpage}
\usepackage{graphicx}
\usepackage{color}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{listings}
\usepackage{framed}
\usepackage[pdftex,pdfborder={0 0 0}]{hyperref}

%opening
\title{Building a Preconditioner}
\author{Eric C. Cyr}

\newcommand{\code}[1]{\lstinline[basicstyle=\footnotesize]!#1!}
\newcommand{\scode}[1]{\lstinline[basicstyle=\small\bfseries]!#1!}
\newcommand{\diag}[1]{\ensuremath{\mbox{diag}(#1)}}

\definecolor{darkgreen}{rgb}{0,0.70,0}
\lstset{%frame=single,
        language=C++,
        numbers=left,
        basicstyle=\scriptsize,
        commentstyle=\color{blue},
        keywordstyle=\bf\color{darkgreen},
        stringstyle=\color{red},
        escapeinside={/*@}{@*/}}

\begin{document}
\maketitle

% This document presents a set of steps that walks through how
% to build a preconditioner in Teko. 
% The document is broken
% down into three steps, each step adding more
% complexity and flexibility.  The appendix serves
% as a partial reference for commands that can be used
% within Teko. It is complemented by the \emph{doxygen}
% generated documentation built directly from the Teko
% source code.

This document is a tutorial on how to construct a 
preconditioner using Teko. In this regard the primary
section of interest is Section~\ref{sec:build-prec}
which presents three steps motivated by an example
preconditioner. Each of the three steps increase
the flexibility and complexity of your preconditioner.
The remaining sections serve as support for Section~\ref{sec:build-prec}.
Section~\ref{sec:design-overview} gives an overview of
the design of Teko, motivating some of the design choices made
and explaining a few of the idioms used. An example for using
your preconditioner is given in Section~\ref{sec:use-prec}.
Section~\ref{sec:thyra-pb}
discusses some of the trickier issues surrounding the use
of Thyra in Teko.  Testing of a newly developed preconditioner
is briefly discussed in Section~\ref{sec:testing}.
Finally, the appendix serves
as a partial reference for commands that can be used
within Teko. It is complemented by the \emph{doxygen}
generated documentation built directly from the Teko
source code.

The example system used is the linear operator
\begin{equation}
A :=  
\begin{bmatrix}
A_{00} & A_{01} \\
A_{10} & A_{11}(v)
\end{bmatrix}
\end{equation}
which operates on the vector $[u,v]^{T}$. The matrix $A_{11}$ 
is parameterized by the solution $v$, indicating that this sub block
has a nonlinear contribution and will be frequently updated in the
context of a nonlinear solve. To precondition this system the
approximation used is
\begin{equation}
\tilde{A} :=  
\begin{bmatrix}
P & 0 \\
A_{10} & H
\end{bmatrix}
\end{equation}
where $P = A_{00} + \alpha A_{01}$ and $H = \diag{A_{11}(v)} \approx A_{11}(v)$. This
entirely artificial linear system and preconditioner is
intended to illustrate the development of a preconditioner
in Teko. Much of the functionality developed in Teko will be used
in this example.

In an effort to not overwhelm someone new to Teko,
and (relatively new) to C++, this document develops the preconditioner code in
three steps. The first step, maintains state internally
in the preconditioner factory (this will be explained).
The second step shows how to implement the state of
a preconditioner that is stored separately from the factory.
The final step, which is optional, adds an additional
level of flexibility to your preconditioner by introducing
the ``strategy'' concept.

\section{Overview of the Design}\label{sec:design-overview}
At the core of the Teko design is the concept of a \emph{preconditioner
factory}. A factory is a commonly used software engineering
pattern for creating C++ objects abstractly~\cite{Go4-DesignPatterns}.
For preconditioners, the goal of the factory is to hide the details
of the particular preconditioner from the user creating it. Thus, 
a Gauss-Seidel preconditioner, for instance, can be repeatedly created
for different operators using the factory. If instead of Gauss-Seidel,
a preconditioner based on AMG is required then a different factory
is passed to the user. The interface for creating the preconditioner
is shared between the Gauss-Seidel factory and the AMG factory, thus
the user of a factory is never aware of the type of preconditioner being
created.

The factory pattern is used again when computing the inverse or approximate
inverse of an operator. This is useful when building a preconditioner based
on a blocked operator. For instance for block Jacobi and Gauss-Seidel methods,
(approximate) inverses are needed for each block on the diagonal. If multiple
preconditioners are needed for different operators, then an inverse operation
needs to be specified for inverting the diagonal blocks. The factory pattern
again abstracts away the specifics of the type and construction of the inverse
operator. All the user needs to know is the operator and factory used to build 
its inverse.

The final major component of the Teko software is the reliance on the Thyra
library. All the fundamental components of Teko are in fact Thyra components. 
We have attempted to hide the more challenging issues in Thyra from
a Teko user, however, in the course of development it may become necessary
to have at least a cursory knowledge of the interaction between the two
packages. For that we refer the reader to Section~\ref{sec:thyra-pb}.

\section{Building your Preconditioner}\label{sec:build-prec}

\subsection{\emph{Step 1}: Simple Preconditioner Factory}
The code in Listing~\ref{lst:step1} shows a bare bones implementation of the
preconditioner factory that builds $\tilde{A}$. The lines beginning on~\ref{lne1:begin-decl}
and ending on~\ref{lne1:end-decl} are the declaration of the 
\code{ExamplePreconditionerFactory} class. This class inherits
from the \code{Teko::BlockPreconditionerFactory} class in Teko which requires
the implementation of the \code{buildPreconditionerOperator} method. 

The next major segment of code is the definition of the constructor on lines
~\ref{lne1:begin-constructor}-\ref{lne1:end-constructor}. This routine initializes
the members of the \code{ExamplePreconditionerFactory} class. In this case,
the desired \code{InverseFactory} is passed in, as is the parameter $\alpha$.

\begin{framed}
\lstinputlisting[caption=\footnotesize ``examples/BuildPreconditioner/step1/ExamplePreconditionerFactory.cpp'',label=lst:step1]
   {../../examples/BuildPreconditioner/step1/ExamplePreconditionerFactory.cpp}
\end{framed}

Construction of the preconditioner is implemented in the member
function \code{buildPreconditionerOperator} on lines~\ref{lne1:begin-bpo}-\ref{lne1:end-bpo}.
This member function takes two arguments. The first argument \code{blockOp} is the blocked operator
that needs to be preconditioned. This is the matrix $A$ in our example. The second argument
\code{state} is a vessel for storing information about the preconditioner. It is discussed 
in detail in Step 2 and can be safely ignored for now.

Lines~\ref{lne1:begin-extraction}-\ref{lne1:end-extraction} extract information about the number
of blocks in the \code{blockOp}, as well as extracting the blocks themselves. The number of rows
in a blocked operator (implemented in Teko as a \code{BlockedLinearOp} object) can be extracted by
calling the \code{blockRowCount} function (similarly for the number of columns). An individual block operator
can be obtained from \code{blockOp} by calling the \code{getBlock} function. This function takes
two integers indicating the location of the desired block in the blocked operator.

The next step is to construct the inverses of the diagonal operators $H$ and $P$. This example
approximates the inverse of the $A_{11}(v)$ block using $H = \diag{A_{11}(v)}$. Consequently,
we see that on line~\ref{lne1:invH} the inverse is constructed by the function \code{getInvDiagonalOp}.
This function takes the linear operator $A_{11}(v)$ and extracts the diagonal and constructs the matrix
$\diag{A_{11}(v)}^{-1}$. The matrix $P = A_{00}+\alpha A_{01}$ is constructed in two steps on
line~\ref{lne1:P}. First we scale $A_{01}$ by $\alpha$ using the \code{scale} function. This
function ``implicitly'' scales the operator, meaning the operator $A_{01}$ is not changed. 
Next, the scaled operator is ``explicitly'' added to the matrix $A_{00}$. The function \code{explicitAdd}
computes the explicit sum of the two arguments and puts them in a new operator (the function
\code{explicitMultiply} computes the explicit product).  Notice how the \code{explicitAdd} function can
handle scaling of the operators.  The reason these operators are
explicitly added is that many inverse factories require the operator to be explicitly formed.
For instance an algebraic multigrid algorithm requires the entries and sparsity pattern of a linear system (compare
this to a Krylov method which only requires matrix-vector products). Once the explicit matrix
$P$ is constructed the inverse needs to be computed (or approximated). The mechanism for doing this,
whether AMG, ILU, GMRES or a direct factorization, is encapsulated in the \code{InverseFactory} class.
Thus the explicit operator object needs to be paired with the \code{InverseFactory} object, this is
performed in line~\ref{lne1:invP} using the \code{buildInverse} function.

Now all the sub blocks required to build the preconditioner have been constructed ($P^{-1}$, $H^{-1}$) or
extracted ($A_{10}$). The final steps, in lines~\ref{lne1:begin-trisolve}-\ref{lne1:end-trisolve},
build the lower triangular preconditioner. First an all zero operator, \code{L}, that has the same dimensions
as \code{blockOp} is constructed using the function \code{zeroBlockedOp}. This is done to guarantee that
the triangular operator will have the same range and domain spaces as the $A$ matrix. The next line
sets the lower triangular component of \code{L} to the matrix $A_{10}$ as seen in the definition of the
preconditioner $\tilde{A}$. In lines~\ref{lne1:begin-invdiags}-\ref{lne1:end-invdiags} the inverse of
the diagonals of $\tilde{A}$ are set.
Finally, on line~\ref{lne1:invLower} the strictly lower triangular matrix \code{L} and \code{invDiag} are passed
to \code{createBlockLowerTriInverseOp} which builds the inverse of the matrix $\tilde{A}$. This is
the preconditioner and the object \code{invTildeA} is returned from the function.

\subsection{\emph{Step 2}: Stateless Preconditioner Factory}
The previous section introduced the basics of implementing a preconditioner. However, as implemented,
the preconditioner factory just developed is severely limited. For instance, it is easy to imagine that
the cost of using the \code{InverseFactory} to build an inverse operator could be substantial (it could
be an LU decomposition!). Therefore it is sensible, if the physics permits, to store those inverse operators
and reuse (or rebuild) them as neccessary. One approach could be to store this inverse in the preconditioner
factory. Thinking carefully about this approach reveals that your preconditioner factory
is limited to building a preconditioner for a single operator, defeating the purpose of the factory pattern. 

To address this problem, Teko pairs each linear operator with a state object capable of storing operator
specific information. This state object is encapsulated in the \code{BlockPreconditionerState} class. The
developer of a preconditioner factory recieves the instantiated object associated with a linear operator 
as the \emph{state} argument to the \code{buildPreconditionerOperator} member function.

The default implementation of the \code{BlockPreconditionerState} class has several useful member functions. 
The first feature of note is how the state object tells the preconditioner factory if it has been
initialized to be paired with a particular linear operator.  Next, the state object \emph{is-a}
\code{Teuchos::ParameterListAcceptor} object.  Therefore, parameter lists can be used to specify the state
of a preconditioner. It is tempting to treat the parameter list as a \code{void*} array, capable of
handling any data type you throw at it. This temptation should be avoided since this does not provide for
the type safety inherit in C++. Furthermore, there are better more flexible ways of adding extra
information to the state object. The last explicit feature allows easy storage of \code{InverseLinearOp}
objects associated with a user defined string. This functionality simplifies what is seen as one of
the most common forms of state required by a preconditioner, mainly the approximate inverse of an operator.

Occasionally, the default implementation of the state object is not sufficient for a preconditioner.
The preconditioner may require more complex or structured storage then is permitted by the original design.
To provide the additional state, a developer creates a new class that inherits from the
\code{BlockPreconditionerState} and implements the functionality required.  The preconditioner factory must
now tell Teko the type of state object that it expects. This is done by implementing the
\code{buildPreconditionerState} member function in the \code{BlockPreconditionerFactory} object. This function
returns a new instance of the inherited state object and is used by Teko to build the state objects (the default
implementation returns the default state object). Now within the \code{buildPreconditionerOperator} function
the state object needs to be dynamically casted to the most specific type used by the preconditioner factory.

\begin{framed}
\lstinputlisting[caption=\footnotesize ``examples/BuildPreconditioner/step2/ExamplePreconditionerFactory.cpp'',label=lst:step2]
   {../../examples/BuildPreconditioner/step2/ExamplePreconditionerFactory.cpp}
\end{framed}


\subsection{Advanced Techniques}

\subsubsection{Setting up parameter list construction for a preconditioner factory}
One nice aspect of Teko is the construction of complex nested preconditioners through
a human readable XML file defining a parameter list. It is possible for a custom
preconditioner to take advantage of this mechanism by registering itself as a preconditioner
with Teko. Here is a brief snippet of code that explains how to do just that.
\begin{framed}
\begin{lstlisting}
RCP<Teko::Clonable> clone = rcp(new Teko::AutoClone<UserDefinedPreconditionerFactory>());
Teko::PreconditionerFactory::addPreconditionerFactory("My Preconditioner",clone);
\end{lstlisting}
\end{framed}
\noindent Notice the use of the \code{Teko::AutoClone} templated class. This class will make a \code{Clonable}
object of any class with a default constructor. The preconditioner can now be constructed
from the parameter list (see the following example) with the preconditioner specfic parameters automatically passed
in to \code{Teko::PreconditionerFactory::initializeFromParameterList}.
\begin{framed}
\begin{lstlisting}
<ParameterList name="Inverse Factory Library">

  <ParameterList name="My Block Prec">
    <Parameter name="Type" type="string" value="My Preconditioner"/>

    <!-- * Parameters for "My Preconditioner" included here to be parsed
         * by the "initializeFromParameterList" function
         * -->

  </ParameterList>

  <!-- Other inverse operations specified here -->
</ParameterList>
\end{lstlisting}
\end{framed}
\noindent Once a \code{Teko::InverseLibrary} is constructed from this parameter list
the user specified preconditioner will be constructed like all other Teko
preconditioners.

%\subsection{The ``strategy'' pattern}
%The use of the strategy pattern is optional. TODO: Write this section

\section{How to use your preconditioner}\label{sec:use-prec}
So you have a preconditioner, now what? How is it to be used? Code listing~\ref{lst:example}
shows how to solve a linear system (from a Q1-Q1 approximation of the Navier-Stokes equation)
using the LSC preconditioner and AztecOO.

\begin{framed}
\lstinputlisting[caption=\footnotesize ``examples/BuildPreconditioner/example-driver.cpp'',label=lst:example]
   {../../examples/BuildPreconditioner/example-driver.cpp}
\end{framed}

Function \code{main} begins with several standard calls, initializing MPI, 
reading the matrix and right hand side vector to be tested from disk, and initializing solution
vector. The matrix used was assembled by Charon from a $2D$ Q1-Q1 discretization on a 
backward facing step. Each node in the mesh has $3$ unknowns ($u$,$v$, and $p$)
with a strided unknown numbering (i.e. the solution vector looks like
$[u_0,v_0,p_0,u_1,v_1,p_1,\ldots]$). Thus the first step in this process is
to break apart the linear system into its sub components.  The two lines starting
on~\ref{lned:define-strided} do this. In particular the first line says
that the first block will be composed of the $u$ and $v$ variables
and the $p$ variable as a second block. This will lead to a block $2\times 2$
system. The second line builds an \code{StridedEpetraOperator} \code{sA} that is a wrapper for
the blocked matrix and has the same interface as an \code{Epetra\_Operator}. The range and
domain spaces have the same maps as the original \code{Epetra\_CrsMatrix} object. Thus to the user both
\code{A} and \code{sA} look the same.

Beginning on line~\ref{lned:construct-prec} the next few lines build the preconditioner.
In LSC and the example discussed above, (approximate) inverses are required for some of the
sub blocks.  Possible parameters for the inverse operation can be read from an XML
file or the defaults provided in Stratimikos can be used (see line~\ref{lned:define-inv-params}). 
This builds a library of inverse \code{InverseFactory}s, so that they can be created
by specifying a string. This operation, constructing an \code{InverseFactory} object, 
is performed in the next step.
%In the next step these parameters are passed into a function which also takes the string describing
%the solver and returns an \code{InverseFactory} object. 
This object when paired with a linear operator creates
the inverse operator. Lines~\ref{lned:const-prec-strategy} and~\ref{lned:const-prec-fact}
construct the preconditioner factory (this is the object you implemented in Section~\ref{sec:build-prec}).
The first step is to build the strategy object needed (this may be omitted depending on
the preconditioner), then this strategy object is passed into the constructor for the
desired preconditioner. Once we have the preconditioner factory in hand we need to
build an Epetra\_Operator. This is done on line~\ref{lned:const-epetra-prec}, the preconditioner
factory is passed into to the constructor for the \code{EpetraBlockPreconditioner}. Then
the function \code{buildPreconditioner} is called passing in the strided operator (so that
it can be used to construct the preconditioner). Similar to the strided operator this operator
behaves exactly like an Epetra\_Operator with the correct range and column spaces.
The remaining lines of the \code{main} function simply invoke the AztecOO solver using
the Epetra wrapped Teko preconditioner.

\section{Improving Performance}

\subsection{Reusing Inverse Operators}
One of the primary wastes from a naive or early implementation of a Teko preconditioner
is in unecessary reconstruction of inverse operators. Often these operators can reuse
allocated memory, avoid recomputing sparsity patterns, or construct prolongation operators
only once. Clearly this depends on the system and the algorithms used.  However, because
the inverse factories are treated abstractly and the preconditioner has no idea about the
specifics of any of them it makes sense to attempt to rebuild all of them. Teko provides
capabilities to make this relatively easy. In this section, we go through how to do this
and provide helpful suggestions on how these operators can be structured.

For instance, on line~\ref{lne1:invP} of Listing~\ref{lst:step1} any previous information 
about \code{invP} that could be reused is simply thrown out. Instead, the code fragment
in listing~\ref{lst:rebuild-inverse} attempts to rebuild the inverse, reusing any available internals.
\begin{framed}
\begin{lstlisting}[caption=\footnotesize ``Rebuilding an inverse'', label=lst:rebuild-inverse]
Teko::LinearOp ExamplePreconditionerFactory
   ::buildPreconditionerOperator(Teko::BlockedLinearOp & blockOp,
                                 Teko::BlockPreconditionerState & state) const
{
   ...

   // try to snag inv(P) from state object
   Teko::ModifiableLinearOp & invP = state.getModifiableOp("invP"); /*@ \label{lne4:from-state} @*/
   
   if(invP==Teuchos::null) // check if it needs to built /*@ \label{lne4:null-check} @*/
      invP  = Teko::buildInverse(*inverse_,P); /*@ \label{lne4:build-inverse} @*/
   else  // rebuild if required
      Teko::rebuildInverse(*inverse_,P,invP); /*@ \label{lne4:rebuild-inverse} @*/
   
   ...
}
\end{lstlisting}
\end{framed}
Listing~\ref{lst:rebuild-inverse} shows how to rebuild an inverse operator. Line~\ref{lne4:from-state}
grabs \emph{a reference} from the state object. Under the hood this is a reference to a
\code{Teuchos::RCP} object stored by the state. As a result, assigning another pointer to this
object will change the reference stored in the \code{Teko::BlockPreconditionerState}. This reference
is stored internally in a way that associates it with the string ``invP''. Line~\ref{lne4:null-check}
checks to see if \code{invP} has been previously initialized, if not the operator is build on
line~\ref{lne4:build-inverse}. Otherwise, the inverse operator is rebuilt using the
\code{Teko::rebuildInverse} function. This is very easy functionality to encapsulate into
a simple function that takes a string, an inverse factory, an operator and a state object. This
function would then return the inverse and store it in the preconditioner state object.

\subsection{Reusing ``Explicit'' Operators}

\section{Overview of Thyra/Teko interactions}\label{sec:thyra-pb}
At the core of Teko is the use of the Thyra abstractions for linear operators
and vectors. For convenience these have been renamed in Teko and a handful of casting
functions were implemented to facilitate their use. 

\subsection{Basic types}
The linear operators and vectors in Teko are all \code{typedef}s of 
\code{RCP} wrapped Thyra types focusing on blocked operators and product
vectors. In particular
\begin{framed}
\begin{lstlisting}
typedef RCP<const Thyra::VectorSpaceBase<double> > VectorSpace;

typedef RCP<Thyra::MultiVectorBase<double> > MultiVector;
typedef RCP<Thyra::ProductMultiVectorBase<double> > BlockedMultiVector;

typedef RCP<const Thyra::LinearOpBase<double> > LinearOp;
typedef RCP<Thyra::PhysicallyBlockedLinearOpBase<double> > BlockedLinearOp;
\end{lstlisting}
\end{framed}

The first thing to notice is that all these types are wrapped in an \code{RCP},
which means that they behave like pointers. This means that method invocation
uses \emph{(object)}\code{->}\emph{(method)} however, most of the commonly used
methods in Teko have been rewritten as C-like function calls.  A reference to an
object can be extracted using the \code{*}-operator and a \emph{raw} pointer can
be obtained by using \code{&*} (get the pointer to the reference).

Another important feature of these types is the \emph{is-a} relationship. This
is a fundamental object-oriented concept that describes the inheritance hierarchy.
For instance a \code{BlockedMultiVector} \emph{is-a} \code{MultiVector}. That means
that the \code{BlockedMultiVector} has all the same functionality and methods as
\code{MultiVector}. However \code{MultiVector} only has a subset of the functionality
of \code{BlockedMultiVector} (in particular \code{MultiVector} is missing the blocking
capability). For the relevant types in Teko the \emph{is-a} relationships are
\begin{center}
\code{BlockedMultiVector} \emph{is-a} \code{MultiVector}, \\
\code{BlockedLinearOp} \emph{is-a} \code{LinearOp}.
\end{center}

\subsection{Casting}
While the \code{RCP} idiom does provide useful features like garbage collection
and type safety~\cite{Bar2007-SAND}, it often obscures object-oriented concepts from the compiler.
The most frequent issue is that the compiler doesn't recognize the \emph{is-a} relationship.
While annoying, this issue is easily overcome by making the relationship explicit using
one of the following casting functions
\begin{framed}
\scode{MultiVector toMultiVector(BlockedMultiVector & blo)}\\
Convert to a MultiVector from a BlockedMultiVector. This explicitly
enforces the \emph{is-a} relationship and always succeeds.

\vspace{10pt}
\scode{BlockedMultiVector toBlockedMultiVector(MultiVector & lo)}\\
Convert to a BlockedMultiVector from a MultiVector. This cast
may fail and will throw an exception if \code{mv} is not a
BlockedMultiVector.

\vspace{10pt}
\scode{LinearOp toLinearOp(BlockedLinearOp & blo)}\\
Convert to a LinearOp from a BlockedLinearOp.
This explicitly enforces the \emph{is-a} relationship
and always succeeds.

\vspace{10pt}
\scode{BlockedLinearOp toBlockedLinearOp(LinearOp & lo)}\\
Convert to a BlockedLinearOp from a LinearOp. This cast
may fail and will throw an exception if \code{lo} is not a
BlockedLinearOp.
\end{framed}
The casting functions that make explicit the \emph{is-a} relationships are
\code{toMultiVector} and \code{toLinearOp}. The functions \code{toBlockedLinearOp}
and \code{toBlockedMultiVector} attempt to perform the cast to a one of the more
specialized types. If the basic type does not contain the correct functionality
(does not implement the right interface) the cast fails and an exception is
thrown.

\subsection{Deep versus shallow copying}
One aspect of C++ that can be challenging is the issue of copying. In Teko there are
two types of copying. \emph{Shallow} copy, expressed with the assignment
operator (\code{=}), copies only the pointer information. Thus modifying either
the source or destination of an assignment operator effects both objects.
\emph{Deep} copy is always explicitly expressed using a function call. 
These copies create a new object that is a separate copy of the original.
Execution of deep copies, unlike shallow copies, are type specific and
typically costly operations that require the allocation of large blocks
of memory and explicit copying of the underlying data. They should be avoided
unless explicitly needed. Currently only a couple of deep copy functions exist
in Teko.
\begin{framed}
\scode{MultiVector deepcopy(const MultiVector & mv)} \\
Perform a deep copy of the vector.

\vspace{10pt}
\scode{BlockedMultiVector deepcopy(const BlockedMultiVector & bmv)} \\
Perform a deep copy of the blocked vector. 
\end{framed}

\section{Testing your preconditioner}\label{sec:testing}
Once your preconditioner has been implemented the next step is to test it.
Unfortunately, a complete test of your preconditioner can be as challenging
if not more difficult then the original implementation. While by no means
comprehensive, the most important test is the verification of the action of your preconditioner
on a vector. Code listing~\ref{lst:step1-test} shows a
program that prints out the action of the preconditioner on a prespecified vector.

\begin{framed}
\lstinputlisting[caption=\footnotesize ``examples/BuildPreconditioner/step1/example-test.cpp'',label=lst:step1-test]
   {../../examples/BuildPreconditioner/step1/example-test.cpp}
\end{framed}

Also included in the ``examples/BuildPreconditioenr/step1'' directory is the file
``example\_test.m''. This file contains a brief Matlab script implementing the example preconditioner
and the linear system used in listing~\ref{lst:step1-test}. This script computes the action of 
the preconditioner on a vector, and servers as a point of reference for the C++ test.

\appendix
\section{Appendix}
Section~\ref{apx:tips-tricks} goes over a few tips and tricks that are useful
in constructing your preconditioner. While Sections~\ref{apx:classes} and~\ref{apx:functions}
provide an incomplete listing of some of the functions and classes in Teko. More details can
be found by building the Teko documentation. All of these functions and classes are
within the Teko namespace.

\subsection{Tips and Tricks}\label{apx:tips-tricks}
\subsubsection{Building an identity operator}
The \code{identity} function takes a \code{VectorSpace} object and builds the corresponding
identity matrix. To extract a \code{VectorSpace} from a linear operator simply call \code{rangeSpace}
or \code{domainSpace}. For example, the code create an identity operator from the range space of an operator 
\code{F} is:
\begin{framed}
\begin{lstlisting}[numbers=none]
   Teko::LinearOp F = buildFOperator()

   // Now build an Identity matrix using the range of F
   Teko::LinearOp id_r = Teko::identity(Teko::rangeSpace(F));
\end{lstlisting}
\end{framed}

\subsubsection{Printing a \code{LinearOp}}
The contents of your operator can be written to any C++ stream by using
the \code{Teuchos::describe} command. For example, this prints the contents
of the operator \code{F} to the standard out.
\begin{framed}
\begin{lstlisting}[numbers=none]
   Teko::LinearOp F = buildFOperator()

   // Print F to standard out
   std::cout << Teuchos::describe(*F, Teuchos::VERB_EXTREME) << std::endl;
\end{lstlisting}
\end{framed}
This is a useful technique. However, the output is also dependent on what the underlying
type of the operator is. For instance, if the concrete implementation is an \code{Epetra_CrsMatrix}
then this calls the \code{Print} routine. For other matrix types the behavior is not
as well defined.

\subsection{Classes}\label{apx:classes}
Several classes (many are just redefines from Thyra) are used within Teko. Here
are the important ones.

\begin{framed}
\begin{flushleft}
\scode{LinearOp} \\
The basic linear operator class. This is object to use when manipulating blocks.

\vspace{10pt}
\scode{BlockedLinearOp} \\
A linear operator that is blocked. That means you can extract sub blocks and
set sub blocks. Also, a \verb!BlockedLinearOp! is a \verb!LinearOp! so a 
\verb!BlockedLinearOp! can use all the same functions as a \verb!LinearOp!.

\vspace{10pt}
\scode{VectorSpace} \\
This object defines the range and domain of a \verb!LinearOp!. It is important
when creating things like identity matrices.

\vspace{10pt}
\scode{InverseFactory} \\
When paired with a \code{LinearOp} this object builds an inverse operator. It specifies
how and what type of inverse is to be created (do we use AMG, ILU, direct factorization, etc...).

\end{flushleft}
\end{framed}

\subsection{Functions}~\label{apx:functions}
Teko includes several functions for querying, manipulating, creating and
using blocked and linear operators. A few are listed here.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Functions for extracting blocked information}
\begin{framed}
\begin{flushleft}
\scode{int blockRowCount(const BlockedLinearOp & blo)} \\
Get the row count in a block linear operator.

\vspace{10pt}
\scode{int blockColCount(const BlockedLinearOp & blo)} \\
Get the col count in a block linear operator.

\vspace{10pt}
\scode{LinearOp getBlock(int i,int j,const BlockedLinearOp & blo)} \\
Get the $i,j$ block in the \code{BlockedLinearOp} object \code{blo}.
\end{flushleft}
\end{framed}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Functions for building blocked operators}
\begin{framed}
\begin{flushleft}
\scode{LinearOp setBlock(int i,int j,const BlockedLinearOp & blo,const LinearOp & lo)} \\
Set the $i,j$ block in a \code{BlockedLinearOp} object \code{blo} to \code{lo}.

\vspace{10pt}
\scode{BlockedLinearOp zeroBlockedOp(const BlockedLinearOp & blo)} \\
Build a zero operator mimicking the block structure of the passed in matrix. 

\vspace{10pt}
\scode{BlockedLinearOp createBlockedOp()} \\
Build a new blocked linear operator.

\vspace{10pt}
\scode{BlockedLinearOp getUpperTriBlocks(const BlockedLinearOp & blo)} \\
Get the strictly upper triangular portion of the matrix.

\vspace{10pt}
\scode{BlockedLinearOp getLowerTriBlocks(const BlockedLinearOp & blo)} \\
Get the strictly lower triangular portion of the matrix. 

\vspace{10pt}
\scode{LinearOp buildInverse(const InverseFactory & factory, const LinearOp & A)} \\
Build an inverse operator using a factory and a linear operator. 

\vspace{10pt}
\scode{const LinearOp getDiagonalOp(const LinearOp &op)} \\
Get a diagonal operator using the diagonal of \code{op}.

\vspace{10pt}
\scode{const MultiVector getDiagonal(const LinearOp &op)} \\
Get the diagonal of \code{op}.

\vspace{10pt}
\scode{const LinearOp getInvDiagonalOp(const LinearOp &op)} \\
Get a diagonal operator using the component-wise inverse of the
diagonal of \code{op}.
\end{flushleft}
\end{framed}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Mathematical operations (Implicit)}
These operations form implicit operators. This technique composes
operators as a chain of matrix-vector multiplication operators. Consequently,
many operators can be composed in this way without having to explicitly
form any of the operators.

\begin{framed}
\begin{flushleft}
\scode{const LinearOp scale(double d,const LinearOp &op)} \\
Scale the matrix.

\vspace{10pt}
\scode{const LinearOp multiply(const LinearOp & opL,const LinearOp & opR)} \\
Multiply the matrices \code{opL} and \code{opR} together. 

\vspace{10pt}
\scode{const LinearOp multiply(const LinearOp & opL,const LinearOp & opM,} \\
\hspace{137pt} \scode{const LinearOp & opR)} \\
Multiply the matrices \code{opL}, \code{opM} and \code{opR} together. 

\vspace{10pt}
\scode{const LinearOp add(const LinearOp & opL,const LinearOp & opR)} \\
Adds the matrices \code{opL} and \code{opR} together. 
\end{flushleft}
\end{framed}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Mathematical operations (Explicit)}
These operations explicitly form the operator. That is (at least) one new operator
is allocated and filled by these operations. Currently all these functions assume
the underlying concrete types are \code{Epetra\_CrsMatrix} objects.

\begin{framed}
\begin{flushleft}
\scode{const LinearOp explicitMultiply(const LinearOp & opL,const LinearOp & opR)} \\
Multiply the matrices \code{opL} and \code{opR} together. (These must be
of type \code{Epetra\_CrsMatrix}!)

\vspace{10pt}
\scode{const LinearOp explicitMultiply(const LinearOp & opL,} \\
\hspace{148pt}\code{const LinearOp & opM,const LinearOp & opR)} \\
Multiply the matrices \code{opL} and \code{opR} together. (Operators \code{opL} and \code{opR}
must be of type \code{Epetra\_CrsMatrix} and \code{opM} must be a \code{Thyra::DiagonalLinearOpBase}
object.)

\vspace{10pt}
\scode{const LinearOp explicitAdd(const LinearOp & opL,const LinearOp & opR)} \\
Add the matrices \code{opL} and \code{opR} together. (These must be
of type \code{Epetra\_CrsMatrix}!)
\end{flushleft}
\end{framed}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Other functions}
These functions don't really have a home yet!

\begin{framed}
\begin{flushleft}
\scode{LinearOp createBlockLowerTriInverseOp(BlockedLinearOp & L,}\\
\hspace{167pt}\scode{const std::vector< LinearOp > & invDiag)} \\
This function takes the strictly lower portion of \code{L} and uses the operators in
\code{invDiag} to perform forward substitution.

\vspace{10pt}
\scode{LinearOp createBlockUpperTriInverseOp(BlockedLinearOp & U,}\\
\hspace{167pt}\scode{const std::vector< LinearOp > & invDiag)} \\
This function takes the strictly upper portion of \code{U} and uses the operators in
\code{invDiag} to perform backward substitution.

\vspace{10pt}
\scode{VectorSpace rangeSpace(const LinearOp & lo)}\\
Get the range space of a linear operator.

\vspace{10pt}
\scode{VectorSpace domainSpace(const LinearOp & lo)}\\
Get the domain space of a linear operator. 

\vspace{10pt}
\scode{LinearOp identity(const VectorSpace & vs)}\\
Based on the vector space \code{vs} build a identity matrix (the domain and range
spaces will both be of type \code{vs}).

\vspace{10pt}
\scode{LinearOp toLinearOp(BlockedLinearOp & blo)}\\
Convert to a LinearOp from a BlockedLinearOp.

\vspace{10pt}
\scode{BlockedLinearOp toBlockedLinearOp(LinearOp & lo)}\\
Convert to a BlockedLinearOp from a LinearOp. This cast
may fail and will throw an exception if \code{lo} is not a
BlockedLinearOp.

\end{flushleft}
\end{framed}

\bibliography{build-precond}
\bibliographystyle{plain}

\end{document}
