http://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&feed=atom&action=historyHybrid MPI+UPC - Revision history2024-03-28T12:10:31ZRevision history for this page on the wikiMediaWiki 1.17.0http://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1389&oldid=prevJim at 21:24, 5 May 20102010-05-05T21:24:44Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:24, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 45:</td>
<td colspan="2" class="diff-lineno">Line 45:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Build and install the Hydra process manager, no special options are needed.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Build and install the Hydra process manager, no special options are needed.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>=<del class="diffchange diffchange-inline">Writing Hybrid Codes</del>=</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>=<ins class="diffchange diffchange-inline">Caveats</ins>=</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>=Running Hybrid Codes=</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">==Writing Hybrid Codes==</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">The Berkeley UPC and MPI runtimes are not fully compatible and mixing MPI and UPC communication can result in deadlock.  Therefore, it is recommended to separate MPI and UPC communication with barriers.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">=</ins>=Running Hybrid Codes=<ins class="diffchange diffchange-inline">=</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">The UPC launcher does not always propagate MPI's process management information in the environment correctly.  To fix this, place the following lines in your code before you initialize MPI:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline"> setenv("PMI_PORT", bupc_getenv("PMI_PORT"), 1);</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline"> setenv("PMI_ID", bupc_getenv("PMI_ID"), 1);</ins></div></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1388&oldid=prevJim at 21:13, 5 May 20102010-05-05T21:13:08Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:13, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 42:</td>
<td colspan="2" class="diff-lineno">Line 42:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Build and install the Hydra process manager.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Hydra is needed for launching nested hybrid programs.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Build and install the Hydra process manager<ins class="diffchange diffchange-inline">, no special options are needed</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Writing Hybrid Codes=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Writing Hybrid Codes=</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Running Hybrid Codes=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Running Hybrid Codes=</div></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1387&oldid=prevJim: /* Berlekey UPC Runtime */2010-05-05T21:08:27Z<p><span class="autocomment">Berlekey UPC Runtime</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:08, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 30:</td>
<td colspan="2" class="diff-lineno">Line 30:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Download the BUPC runtime system [http://upc.lbl.gov/download/source.shtml]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Download the BUPC runtime system [http://upc.lbl.gov/download/source.shtml]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Configure the runtime with the following options:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Configure the runtime with the following options:</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>## .<del class="diffchange diffchange-inline">..</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>## <ins class="diffchange diffchange-inline">''--with-multiconf=opt_gccupc'' Build with the GCCUPC translator (note: ''upc'' binary must be in your path at this point)</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">## ''--with-ibv-spawner=ssh --with-vapi-spawner=ssh --disable-mpi'' Prevent BUPC from using MPI, even in the bootstrap</ins>.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">## ''--prefix=$HOME/opt'' Installation path</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">## ''--disable-aligned-segments'' May also be needed</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Build and install the BUPC runtime</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Build and install the BUPC runtime</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1386&oldid=prevJim: /* GCCUPC */2010-05-05T21:03:53Z<p><span class="autocomment">GCCUPC</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:03, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 21:</td>
<td colspan="2" class="diff-lineno">Line 21:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Download GCCUPC [http://www.gccupc.org/]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Download GCCUPC [http://www.gccupc.org/]</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Untar the source code</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Untar the source code</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div># Make a separate build directory</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div># Make a separate build directory<ins class="diffchange diffchange-inline">.  The build should not be done in the source directory.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Run configure from the build directory, no special options are needed</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Run configure from the build directory, no special options are needed</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">## Example: ../upc-4.2.3.6/configure --prefix=$HOME/opt/gccupc --enable-languages=c,upc</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Build and install GCCUPC</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div># Build and install GCCUPC</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1385&oldid=prevJim: /* Building the Environment */2010-05-05T21:01:32Z<p><span class="autocomment">Building the Environment</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:01, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 11:</td>
<td colspan="2" class="diff-lineno">Line 11:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Building the Environment=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Building the Environment=</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Many hybrid MPI and UPC setups are possible.  In our work, we have used the GCCUPC compiler with the Berkeley UPC runtime.  The instructions here are focused on building the hybrid setup on an Infiniband cluster.  MPI and the BUPC runtime must be <del class="diffchange diffchange-inline">build </del>with a compiler that is ABI compatible (i.e. produces binaries that can be linked by) GCC.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Many hybrid MPI and UPC setups are possible.  In our work, we have used the GCCUPC compiler with the Berkeley UPC runtime.  The instructions here are focused on building the hybrid setup on an Infiniband cluster.  MPI and the BUPC runtime must be <ins class="diffchange diffchange-inline">built </ins>with a compiler that is ABI compatible (i.e. produces binaries that can be linked by) <ins class="diffchange diffchange-inline">with </ins>GCC<ins class="diffchange diffchange-inline">.  MPI does not need to be compiled with the UPC compiler, it only needs to be able to be linked in using the UPC compiler</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>==<del class="diffchange diffchange-inline">WARNING</del>==</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>==<ins class="diffchange diffchange-inline">DISCLAIMER</ins>==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>The current state of the hybrid MPI+UPC environment is in the proof-of-concept phase.  We are working to improve the level of support and interoperability, but at present there are many caveats and challenges involved in setting up the <del class="diffchange diffchange-inline">environment</del>.  Expect things to be broken right now.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>The current state of the hybrid MPI+UPC environment is <ins class="diffchange diffchange-inline">still </ins>in the proof-of-concept phase.  We are working to improve the level of support and interoperability, but at present there are many caveats and challenges involved in setting up <ins class="diffchange diffchange-inline">and using </ins>the <ins class="diffchange diffchange-inline">model.  These instructions are intended for hybrid MPI+UPC developers, not end users</ins>.  Expect things to be broken right now.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==GCCUPC==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==GCCUPC==</div></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1384&oldid=prevJim at 20:59, 5 May 20102010-05-05T20:59:03Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:59, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The Hybrid MPI+UPC programming model is a new approach to writing parallel programs that combines MPI and UPC in the same program.  This model has several important benefits.  It provides MPI programs with access to a large distributed shared global address space with significant advantages when compared with MPI-2 one-sided communication.  It provides UPC programmers with an additional level of locality control that can be used to create multiple UPC spaces connected via MPI.  Finally, it can provide access to libraries written in either model, e.g. access to PETSc and ScaLAPACK for UPC programmers.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The Hybrid MPI+UPC programming model is a new approach to writing parallel programs that combines MPI and UPC in the same program.  This model has several important benefits.  It provides MPI programs with access to a large distributed shared global address space with significant advantages when compared with MPI-2 one-sided communication.  It provides UPC programmers with an additional level of locality control that can be used to create multiple UPC spaces connected via MPI.  Finally, it can provide access to libraries written in either model, e.g. access to PETSc and ScaLAPACK for UPC programmers.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>This page is intended to serve as early documentation on hybrid parallel programming using MPI and Unified Parallel C.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>This page is intended to serve as <ins class="diffchange diffchange-inline">a collection of </ins>early documentation on hybrid parallel programming using MPI and Unified Parallel C.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Papers=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Papers=</div></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1383&oldid=prevJim at 20:58, 5 May 20102010-05-05T20:58:42Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:58, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">The Hybrid MPI+UPC programming model is a new approach to writing parallel programs that combines MPI and UPC in the same program.  This model has several important benefits.  It provides MPI programs with access to a large distributed shared global address space with significant advantages when compared with MPI-2 one-sided communication.  It provides UPC programmers with an additional level of locality control that can be used to create multiple UPC spaces connected via MPI.  Finally, it can provide access to libraries written in either model, e.g. access to PETSc and ScaLAPACK for UPC programmers.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>This page is intended to serve as early documentation on hybrid parallel programming using MPI and Unified Parallel C.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>This page is intended to serve as early documentation on hybrid parallel programming using MPI and Unified Parallel C.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1382&oldid=prevJim at 20:50, 5 May 20102010-05-05T20:50:23Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:50, 5 May 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 10:</td>
<td colspan="2" class="diff-lineno">Line 10:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Many hybrid MPI and UPC setups are possible.  In our work, we have used the GCCUPC compiler with the Berkeley UPC runtime.  The instructions here are focused on building the hybrid setup on an Infiniband cluster.  MPI and the BUPC runtime must be build with a compiler that is ABI compatible (i.e. produces binaries that can be linked by) GCC.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Many hybrid MPI and UPC setups are possible.  In our work, we have used the GCCUPC compiler with the Berkeley UPC runtime.  The instructions here are focused on building the hybrid setup on an Infiniband cluster.  MPI and the BUPC runtime must be build with a compiler that is ABI compatible (i.e. produces binaries that can be linked by) GCC.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">==WARNING==</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">The current state of the hybrid MPI+UPC environment is in the proof-of-concept phase.  We are working to improve the level of support and interoperability, but at present there are many caveats and challenges involved in setting up the environment.  Expect things to be broken right now.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==GCCUPC==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==GCCUPC==</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 31:</td>
<td colspan="2" class="diff-lineno">Line 35:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Build and install the Hydra process manager.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Writing Hybrid Codes=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Writing Hybrid Codes=</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Running Hybrid Codes=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Running Hybrid Codes=</div></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1381&oldid=prevJim: /* MPI */2010-03-27T04:22:42Z<p><span class="autocomment">MPI</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 04:22, 27 March 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 28:</td>
<td colspan="2" class="diff-lineno">Line 28:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==MPI==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==MPI==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Build and install MVAPICH.  No special options are needed.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Build and install MVAPICH <ins class="diffchange diffchange-inline">[http://mvapich.cse.ohio-state.edu/]</ins>.  No special options are needed.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td></tr>
</table>Jimhttp://hpcrl.cse.ohio-state.edu/wiki/index.php?title=Hybrid_MPI%2BUPC&diff=1380&oldid=prevJim: /* Building the Environment */2010-03-27T04:21:33Z<p><span class="autocomment">Building the Environment</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 04:21, 27 March 2010</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 8:</td>
<td colspan="2" class="diff-lineno">Line 8:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Building the Environment=</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=Building the Environment=</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Many hybrid MPI and UPC setups are possible.  In our work, we have used the GCCUPC compiler with the Berkeley UPC runtime.  The instructions here are focused on building the hybrid setup on an Infiniband cluster.  MPI and the BUPC runtime must be build with a compiler that is ABI compatible (i.e. produces binaries that can be linked by) GCC.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==GCCUPC==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==GCCUPC==</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Download GCCUPC [http://www.gccupc.org/]</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Untar the source code</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Make a separate build directory</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Run configure from the build directory, no special options are needed</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Build and install GCCUPC</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Berlekey UPC Runtime==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Berlekey UPC Runtime==</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Download the BUPC runtime system [http://upc.lbl.gov/download/source.shtml]</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Configure the runtime with the following options:</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">## ...</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"># Build and install the BUPC runtime</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==MPI==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==MPI==</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Build and install MVAPICH.  No special options are needed.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>==Hydra Process Manager==</div></td></tr>
</table>Jim