Swarm and MinGW

From Swarm
Revision as of 17:49, 2 December 2015 by SwarmWikiSysop (Talk | contribs) (Using Swarm Models Compiled with MinGW)

Jump to: navigation, search

Overview

This documentation provides information on how to use Swarm on Windows with MinGW compilers. The following topics will be covered:

  • How to run a Swam model compiled with MinGW on Windows: Once you produce a Swarm model using MinGW (or someone gives one to you), how do you run it on other machines without installing Swarm there?
  • How to use MinGW to compile your own Swarm models, either during model development or so you can distribute working models to others. This requires installing MinGW and our binary Swarm libraries.
  • How to package a model compiled with MinGW so it can be installed and run by itself on Windows machines.
  • How to use MinGW to compile the Swarm libraries, in case you want to build your own Swarm binaries.

Each topic is more difficult than the one before it. However the topics are independent in that it is not necessary to know how to compile the Swarm library in order to run a pre-compiled Swarm model or compile your own model against a pre-compiled version of Swarm.

Using Swarm Models Compiled with MinGW

A model that has been compiled using MinGW can be installed and run from a Windows computer without installing MinGW or Swarm. An example code is this build of Heatbugs:

http://download.savannah.gnu.org/releases/swarm/src/testing/heatbugs-MinGW-win32.exe

However, running a Swarm model requires some support programs and libraries. These are available as <tt>swarm-2.4.1-win32.zip</tt>, available here:

http://download.savannah.gnu.org/releases/swarm/binaries/w32/MinGW/Swarm-2.4.1-win32.zip

Unzipping that file will create a folder called <tt>swarm</tt> which should be moved somewhere that is easy to remember. The rest of this tutorial will assume this Swarm folder has been placed at <tt>C:\swarm</tt>.

Before a Swarm model can be run, the swarm\bin folder must be added to the Windows PATH so that Heatbugs can find necessary libraries when it runs. The swarm\bin folder can be temporarily added to the path by executing the following in the Windows command line (the command line is accessed using the program cmd.exe):

path=%PATH%;C:\swarm\bin

However, it is possible to have conflicts if other versions of libraries used by Swarm, such as HDF5, are already installed and in your path. And some Swarm programs require that the environment variable SWARMHOME be defined, which will typically not be true. These problems can be avoided by using the following statements instead of the above path command:

<pre>SWARMHOME=C:\swarm path=%SWARMHOME%\bin;%PATH%</pre>

You can alternatively add the Swarm folder to your <tt>PATH</tt> permanently by editing the Windows PATH environment variable. A tool like RapidEE makes doing so easy, or you can use use Windows control panel > System > Advanced > Environment Variables to add <tt>swarm\bin</tt> to the PATH system variable.

Once the path has been set to include the <tt>swarm\bin</tt> folder, Swarm models can be started through the Windows command line, or by double-clicking on them from the Windows Explorer.

The Heatbugs model can now be run by changing the directory of the command line to the folder holding <tt>heatbugs.exe</tt> and then executing the program. For example, if <tt>heatbugs.exe</tt> is located in <tt>C:\Users\example_user\Downloads</tt>, then the model may be run as follows:

<pre>cd C:\Users\example_user\Downloads heatbugs.exe</pre>

(or you can just double-click on <tt>heatbugs.exe</tt> from Windows Explorer, which temporarily opens a Windows command terminal while the model runs.)

Running the Heatbugs model

You can also create a simple Windows batch file that sets SWARMHOME and the path and then executes the model. Create a file called, for example <tt>heatbugs.bat</tt> in the same directory where <tt>heatbugs.exe</tt> is, containing these lines:

<pre>SWARMHOME=C:\swarm path=%SWARMHOME%\bin;%PATH% heatbugs.exe</pre>

Then, just double-clicking on <tt>heatbugs.bat</tt> will start the model.

If you try this and get an error that some .dll file is not found, then the problem is likely that the path is not set correctly to include the swarm\bin folder. If instead the program crashes without an error statement, then the problem is more likely that SWARMHOME needs to be set.

Installing Swarm and MinGW to compile models

To compile Swarm models from source using MinGW, the following simple steps need to be taken to set up the build environment:

NOTE:

The archived Swarm binaries MUST be placed at <tt>C:\swarm</tt> to successfully compile Swarm models. This is because some paths had to be hard-coded by <tt>libtool</tt> during the creation of the binaries.

The final step before you can compile a Swarm model is to set the <tt>SWARMHOME</tt> environment variable so that the "make" program knows where to find the Swarm libraries when compiling your code. There are several ways to do this.

  • You can temporarily set SWARMHOME by entering this command in the MinGW shell: <pre>export SWARMHOME=/c/swarm</pre>
  • You can edit your Makefile so it starts with: <pre>SWARMHOME=/c/swarm</pre>
  • Set SWARMHOME permanently as a Windows environment variable, either by using RapidEE or Windows control panel > System > Advanced > Environment Variables to add a new (User or System) variable called SWARMHOME with the value <tt>/c/swarm</tt>.

After MinGW has been installed and the Swarm binaries have been downloaded, Swarm models may be compiled using the MinGW shell. Start up the program MinGW Shell, cd to the directory containing your code and Makefile, and enter the command "Make".

An archive of official Swarm example programs can be downloaded from:

http://ftp.swarm.org/pub/swarm/src/swarm/swarmapps-2.3.0.tar.gz

We will use the &quot;Heatbugs&quot; model as an example of how to compile a Swarm application:

<pre>tar xzf /path/to/downloaded/swarmapps-2.3.0.tar.gz cd swarmapps-2.3.0/objc/heatbugs

make</pre> VERY IMPORTANT NOTE: MinGW produces two executables called <tt>heatbugs.exe</tt>---one in the <tt>heatbugs</tt> folder and another in the <tt>heatbugs/.libs</tt> folder. You should ignore the executable in the top-level folder, <tt>heatbugs/heatbugs.exe</tt>, as it is merely a compiled wrapper script that executes <tt>heatbugs/.libs/heatbugs.exe</tt>. When distributing compiled models, only use binaries from the <tt>.libs</tt> folder.

The resulting executable, <tt>.libs/heatbugs.exe</tt> may be executing by following the steps described above at Using Swarm Models Compiled with MinGW. These steps must include setting the Windows Path to include <tt>C:\swarm\bin</tt>.

Distributing a Swarm Model for Windows Users

A major benefit of the MinGW version of Swarm is that models can be distributed for people to use without installing Swarm. A simple way to do this, after the model has been compiled and tested using methods in the previous section, is to create a ZIP archive that contains all the files to run the model.

  • Create a directory tree that includes the Swarm binary files. For example, create C:\MySwarmModel and add the Swarm binaries (as described above) to it as a subdirectory tree.
  • Create another subdirectory that includes just the compiled model.
  • Create a third subdirectory that includes any input files that the model needs and a batch file that sets the path and executes the model.
  • Zip the whole tree up for distribution.

The directory tree in the zip file looks like this:

<pre> C:\MySwarmModel\swarm

                     \bin
                     \etc
                     \include
                     \lib
                     \man
                     \share
              \executable\MySwarmModel.exe
              \Runfiles\MySwarmModel.bat (and any input files)

</pre>

where the file <tt>MySwarmModel.bat</tt> contains:

<pre> set SWARMHOME=C:\MySwarmModel\swarm path=%SWARMHOME%\bin;%PATH% C:\MySwarmModel\executable\MySwarmModel.exe </pre>

Then users can run the model just by double-clicking on <tt>MySwarmModel.bat</tt>.

Building Swarm Using MinGW

This tutorial provides directions and code for compiling your own Swarm libraries for Windows. It is a bit advanced---some familiarity with compiling software from source in a Linux environment and knowledge of the UNIX command line is assumed.

Install Required Tools

The first step is to download and install tools required for compiling Swarm. These links provide directions:

Build Support Libraries

The next steps are to build and install the libraries that Swarm depends on. A ZIP archive containing all the source code required can be found at:

http://ftp.swarm.org/pub/swarm/src/swarm/Swarm-2.4.1-MinGW-sources.zip

This archive contains two subdirectories:

  • sources: This folder contains the source code for each component.
  • patches: This folder contains patchfiles that modfy source code so that MinGW can compile it.

After downloading and extracting the source code archive, open up a MinGW shell and run the following commands:

<pre>cd /path/to/where/swarm/source/was/extracted mkdir build</pre> The following steps will all assume your MinGW shell is operating out of the <tt>build</tt> directory. All compiled software will be installed to <tt>/c/swarm</tt> (The Windows Explorer will see the folder appear at <tt>C:\swarm</tt>).

ZLib

ZLib will be the first library to be built as it is a common dependency for many of the other software components. From the <tt>build</tt> directory, execute the following to build Zlib:

<pre>tar xzf ../sources/zlib-1.2.5.tar.gz cd zlib-1.2.5 make -f win32/Makefile.gcc</pre> And then the following commands to install ZLib to <tt>/c/swarm</tt>:

<pre>mkdir -p /c/swarm/bin mkdir -p /c/swarm/include mkdir -p /c/swarm/lib

cp zlib1.dll /c/swarm/bin/ cp zconf.h zlib.h /c/swarm/include/ mv libzdll.a libz.dll.a cp *.a /c/swarm/lib</pre>

LibPNG

LibPNG will be built next as its only dependency is ZLib. From the <tt>build</tt> directory, execute the following steps to build and install LibPNG:

<pre>tar xjf ../sources/libpng-1.5.4.tar.bz2 cd libpng-1.5.4

export CPPFLAGS='-I/c/swarm/include' export LDFLAGS='-L/c/swarm/lib' ./configure --prefix=/c/swarm --with-zlib-prefix=/c/swarm

make make install</pre>

XPM

Starting from the <tt>build</tt> directory, the XPM library can be built and installed as follows:

<pre>tar xjf ../sources/xpm-nox-4.2.0.tar.bz2 cd xpm-nox-4.2.0 patch -p1 &lt; ../../patches/xpm-nox-4.2.0-mingw-tcl.patch

make CC=gcc make install prefix=/c/swarm</pre> Summary of patchfile changes:

The patchfile is inherited from the SUSE builds of XPM for MinGW and adds Makefiles and other critical components.

One major modification has been made to the SUSE patchfile---<tt>tcl.h</tt> is included in <tt>xpm.h</tt>. This is required in order to use XPM while Tk is installed. Conversely, this means that this build of XPM cannot be used without having Tk installed.

LibFFI

Starting from the <tt>build</tt> directory, LibFFI can be built and installed as follows:

<pre>tar xzf ../sources/libffi-3.0.9.tar.gz cd libffi-3.0.9

./configure --prefix=/c/swarm make make install</pre> The LibFFI headers get installed in a weird place.

<pre>mv /c/swarm/lib/libffi-3.0.9/include/*.h /c/swarm/include rm -rf /c/swarm/lib/libffi-3.0.9</pre>

Tcl

Starting from the <tt>build</tt> directory, Tcl can be built and installed as follows:

<pre>tar xzf ../sources/tcl8.4.19-src.tar.gz cd tcl8.4.19 patch -p1 &lt; ../../patches/tcl-8.4.19-fix-lvalues.patch cd win

./configure --prefix=/c/swarm make make install</pre> Summary of patchfile changes:

At two places in the TCL source code, an assignment is made while simultaneously performing a typecast. The typecast and the assignment had to be split into separate operations in order to be acceptable to the MinGW compilers.

Tk

Starting from the <tt>build</tt> directory, Tk can be built and installed as follows:

<pre>tar xzf ../sources/tk8.4.19-src.tar.gz cd tk8.4.19/win

./configure --prefix=/c/swarm make make install</pre>

BLT

Starting from the <tt>build</tt> directory, BLT can be built and installed as follows:

<pre>tar xzf ../sources/BLT2.4z.tar.gz cd blt2.4z

./configure --prefix=/c/swarm --without-x make make install</pre>

HDF5 (Optional)

Getting HDF5 to work with MinGW is a little tricky. It is an optional dependency, so if support for reading and writing <tt>.hdf</tt> files is not needed this section can be skipped.

As of version 1.8.7, HDF5 currently does not compile under MinGW. So, we must wrap pre-compiled binaries for use with MinGW. In order to wrap the binaries, we will need an additional tool called <tt>pexports</tt>. This can be installed using <tt>mingw-get</tt>:

<pre>mingw-get install mingw-utils</pre> Download 32-bit Windows binaries from the HDF website compiled for use with the Visual Studio compilers (these files have &quot;VS&quot; listed under the Compilers section). After extracting the archive, the HDF5 libraries can be wrapped for use with MinGW by executing the following commands:

<pre>cd path/to/where/HDF5/was/extracted

for dll in `ls dll/*dll`; do

 def_file=`basename $dll .dll`.def
 lib_file=lib`basename $dll dll.dll`.a
 pexports $dll &gt; $def_file
 dlltool -d $def_file -l lib/$lib_file

done</pre> Next, move the libraries and header files needed to use HDF5 from C into <tt>C:\swarm</tt>:

<pre>cp dll/hdf5dll.dll dll/szip.dll /c/swarm/bin cp include/* /c/swarm/include cp lib/libhdf5.a /c/swarm/lib</pre> Finally, a patch must be applied to the HDF header file <tt>H5public.h</tt> so that it doesn't define <tt>ssize_t</tt>---a constant already defined by MinGW. The required patch is contained in the Swarm MinGW source archive:

<pre>cd /path/to/where/swarm/source/was/extracted patch -p1 -d /c/swarm &lt; patches/hdf5-1.8.7-mingw.patch</pre>

Build Swarm

Finally, the Swarm libraries themselves can be built. From the <tt>build</tt> directory, execute the following

<pre>tar xzf ../sources/swarm-2.4.1.tar.gz cd swarm-2.4.1 patch -p1 ../../patches/swarm-2.4.1-mingw.patch

export CPPFLAGS='-I/c/swarm/include' export LDFLAGS='-L/c/swarm/lib' ./configure --prefix=/c/swarm \

 --with-zlibdir=/c/swarm \
 --with-pngdir=/c/swarm \
 --with-xpmdir=/c/swarm \
 --with-ffidir=/c/swarm \
 --with-tcl=/c/swarm/lib \
 --with-tk=/c/swarm/lib \
 --with-bltdir=/c/swarm

cd avcall gcc -S avcall-i386.c cd ..

make make install</pre> If you wrapped HDF5 up for use with MinGW, add <tt>--with-hdf5dir=/c/swarm</tt> to the list of arguments passed to <tt>configure</tt>.

Summary of patchfile changes:

<tt>swarm-2.4.1-mingw.patch</tt> modifies the Swarm source code in the following ways:

  • <tt>gettimeofday</tt> is no longer defined in the Swarm sources as MinGW includes this function in its standard library.
  • Swarm GUI functions are updated to be compatible with LibPNG 1.5.
  • The Swarm <tt>configure</tt> script defines <tt>DATADIR</tt> as a preprocessor macro---this conflicts with variables in the GCC Objective-C library. <tt>DATDIR</tt> is <tt>undef</tt>ed in critical parts of the source code.
  • The <tt>sleep</tt> function is aliased to <tt>usleep</tt>---which is the name of the MinGW implementation.
  • The way in which Makefiles invoke <tt>gperf</tt> is amended so that it actually produces output.
  • NULL checks are added to `zstrdup` in src/defobj/internal.m. Some functions in the Swarm library may attempt to call `zstrdup` on a NULL pointer. (This happens if the SWARMHOME environment variable is not set.) Failing to check for this condition can lead to segfaults. `zstrdup` now returns NULL if passed a NULL string or if `alloc` fails to allocate memory for a duplicate.

Full details are contained in the patchfile.

Tools

MinGW

Homepage: http://www.mingw.org

Download Area: http://sourceforge.net/projects/mingw/files

Version Used: 20110530

License: GPL

MinGW is a set of software that aims to provide an environment where software developed on UNIX-like systems can be easily compiled for Windows. Since Swarm is primarily developed on Linux (a UNIX clone), MinGW can ease the task of creating Swarm models that run on Windows.

MinGW is similar to the Cygwin project. The major difference is that MinGW produces executables that depend only on native Windows libraries while Cygwin executables require special compatibility libraries that are part of the Cygwin distribution.

Installing MinGW

MinGW can be installed by downloading the &quot;Automated MinGW Installer&quot;, which usually has a name like <tt>mingw-get-inst-&lt;date&gt;.exe</tt> where <tt>&lt;date&gt;</tt> is the datestamp for when the installer was packaged. Running this executable will start the installation process. The installer lets you select which optional components to install; make sure all boxes are checked so you install all components.

Install all available MinGW components

The installer downloads all components from the internet. This will take a while, so have a cup of coffee or walk the dog.

See the following section about also installing an optional, more-functional shell window for MinGW.

Using MinGW

The MinGW compilers are accessible through the &quot;MinGW Shell&quot; which should be located under the MinGW folder in the Start Menu:

Location of MinGW Shell

or by running <tt>C:\MinGW\msys\1.0\msys.bat</tt> (if MinGW was installed to its default location).

By default the MinGW shell executes inside of a Windows <tt>cmd.exe</tt> terminal. There are a couple major drawbacks to this:

  • You cannot resize the window to full-screen.
  • The shell provides limited history scrollback.

MinGW provides a couple of alternative terminals for the MinGW shell which do not have these drawbacks; one of which is called MinTTY. To install MinTTY execute the following in a MinGW shell:

<pre>mingw-get install mintty</pre> Then the <tt>mintty</tt> command can be used to launch a MinTTY terminal. The MinGW Shell can also be set to launch MinTTY by default by editing the file <tt>C:\MinGW\msys\1.0\msys.bat</tt> and changing the following line (approximately line 58):

<pre>if &quot;x%MSYSCON%&quot; == &quot;xunknown&quot; set MSYSCON=sh.exe</pre>

to:

<pre>if &quot;x%MSYSCON%&quot; == &quot;xunknown&quot; set MSYSCON=mintty.exe</pre>

A Note About Paths

The MinGW Shell emulates a UNIX <tt>bash</tt> shell and thus uses UNIX-style paths, <tt>/path/to/file</tt>, instead of Windows-style paths, <tt>C:\path\to\file</tt>. One peculiarity of this process is that the shell must deal with the fact that Windows filesystems have multiple roots (<tt>C:</tt>, <tt>D:</tt>, etc.) while a UNIX filesystem only has one root called <tt>/</tt>. The way this is reconciled by the <tt>bash</tt> shell provided with MinGW is that <tt>/c</tt> refers to the root of the <tt>C:</tt> drive, <tt>/d</tt> refers to the root of the <tt>D:</tt> drive and so on and so forth.

The root of the filesystem itself, <tt>/</tt>, also has some caveats. <tt>/</tt> refers to the <tt>mysy</tt> folder, <tt>C:/MinGW/mysys/1.0</tt> in a default installation. <tt>/usr</tt> also refers to the same location. Thus, <tt>/usr/local</tt> and <tt>/local</tt> both refer to <tt>C:/MinGW/mysys/1.0/local</tt>.

A Note About MinGW-64

Currently Swarm has been developed and tested against the 32-bit version of MinGW. Hence Swarm models produced this way should run in any version of Windows.

The [MinGW-w64][mingw_65] project produces 64-bit MinGW compilers that can produce 64-bit executables that will only run on 64-bit versions of Windows. These executables benefit from:

  • Access to more memory. A 32-bit executable will only be able to allocate ~3 GB of RAM---even if the system has more RAM available.
  • Possibly quicker execution speeds due to matching the native architecture of the operating system---however this depends on the types of tasks that the program executes. It has been reported that 64 bit Swarm executables run quicker than 32 bit Swarm executables on Linux.

Currently, no attempt has been made to compile Swarm 2.4.1 with the 64 bit MinGW compilers. If a successful attempt is made, please report it on the Swarm mailing list.


Emacs

Homepage: http://www.gnu.org/software/emacs

Version Used: 23.3

Download Area: http://ftp.gnu.org/gnu/emacs/windows

License: GPL

GNU Emacs is a text editor that is used by many programmers to edit code. However, Emacs can also be scripted to perform a variety of complicated transformations on text files. The scripts that build the Swarm libraries use Emacs to automate the process of generating some source files. Thus Emacs has to be installed in order to build Swarm from source.

Installing Emacs

Install Emacs by downloading one of the <tt>emacs-&lt;version number&gt;-bin-i386.zip</tt> files from the download area. Don't use any of the files containing <tt>barebin</tt>. After extracting the ZIP archive there will be an Emacs folder containing a subfolder called <tt>bin</tt>. Add this folder to your Windows <tt>PATH</tt>. RapidEE can help edit the <tt>PATH</tt> variable.


GPerf

Homepage: http://www.gnu.org/software/gperf/

Download Area: http://sourceforge.net/projects/gnuwin32/files/gperf

Version Used: 3.0.1

License: GPL

GPerf is a program that produces C code containing hash functions. The scripts that build the Swarm libraries use GPerf to generate some source code files. Download <tt>gperf.exe</tt> from the GnuWin32 site on Sourceforge and place it somewhere on your Windows <tt>PATH</tt>. Again, RapidEE can help set the <tt>PATH</tt> variable.


RapidEE

Homepage: http://www.rapidee.com

Version Used: 5.0

License: Freeware

The Rapid Environment Editor (RapidEE) is a spiffy tool that makes it easy to inspect and alter the value of Windows environment variables, such as the <tt>PATH</tt>. RapidEE may be used to edit &quot;System Variables&quot; which will affect all user accounts or &quot;User Variables&quot; which will affect just the current account. Administrative privileges may be required to edit the System Variables.

The different variables RapidEE can edit

Right-clicking on the <tt>Path</tt> variable in the System Variables or User Variables will bring up a menu that presents the option to &quot;Add value&quot;:

Adding a new value to the PATH

Once the new entry has been added, RapidEE must be told to save the changes. Command line shells, such as <tt>cmd.exe</tt>, must be restarted in order for them to notice the changes.

Adding and saving a value

Depends

Homepage: http://www.dependencywalker.com/

Version Used: 2.2

License: Freeware

Depends is a usefull utility that scans Windows executables and reports information concerning the Dynamic-link Libraries (DLLs) that those executables require in order to run.

Software Used to Compile Swarm 2.4.1

Name Verson Used License Homepage Download Area
ZLib 1.2.5 ZLib/PNG License http://www.zlib.net --
LibFFI 3.0.9 MIT http://sourceware.org/libffi --
LibPNG 1.5.4 ZLib/PNG http://www.libpng.org/pub/png/libpng.html --
XPM-NoX 4.2.0 MIT unknown http://build.opensuse.org/package/files?package=mingw32-xpm-nox&project=windows%3Amingw%3Awin32
Tcl 8.4.19 Tcl http://www.tcl.tk --
Tk 8.4.19 Tcl http://www.tcl.tk --
BLT 2.4z BSD http://blt.sourceforge.net --
HDF5 1.8.7 BSD http://www.hdfgroup.org/HDF5 --
Swarm 2.4.1 GPL http://www.swarm.org http://ftp.swarm.org/pub/swarm/src/swarm/swarmapps-2.3.0.tar.gz