Swarm and MinGW

From Swarm
Revision as of 14:06, 16 November 2017 by Railsback (Talk | contribs) (MinGW)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Overview

This documentation provides information on how to use Swarm on Windows with MinGW compilers. The following topics will be covered:

  • How to run a Swam model compiled with MinGW on Windows: Once you produce a Swarm model using MinGW (or someone gives one to you), how do you run it on other machines without installing Swarm there?
  • How to use MinGW to compile your own Swarm models, either during model development or so you can distribute working models to others. This requires installing MinGW and our binary Swarm libraries.
  • How to package a model compiled with MinGW so it can be installed and run by itself on Windows machines.
  • How to use MinGW to compile the Swarm libraries, in case you want to build your own Swarm binaries.

Each topic is more difficult than the one before it. However the topics are independent in that it is not necessary to know how to compile the Swarm library in order to run a pre-compiled Swarm model or compile your own model against a pre-compiled version of Swarm.

Using Swarm Models Compiled with MinGW

A model that has been compiled using MinGW can be installed and run from a Windows computer without installing MinGW or Swarm. An example code is this build of Heatbugs:

http://download.savannah.gnu.org/releases/swarm/src/testing/heatbugs-MinGW-win32.exe

However, running a Swarm model requires some support programs and libraries. These are available as swarm-2.4.1-win32.zip, available here:

http://download.savannah.gnu.org/releases/swarm/binaries/w32/MinGW/Swarm-2.4.1-win32.zip

Unzipping that file will create a folder called swarm which should be moved somewhere that is easy to remember. The rest of this tutorial will assume this Swarm folder has been placed at C:\swarm.

Before a Swarm model can be run, the swarm\bin folder must be added to the Windows PATH so that Heatbugs can find necessary libraries when it runs. The swarm\bin folder can be temporarily added to the path by executing the following in the Windows command line (the command line is accessed using the program cmd.exe):

path=%PATH%;C:\swarm\bin

However, it is possible to have conflicts if other versions of libraries used by Swarm, such as HDF5, are already installed and in your path. And some Swarm programs require that the environment variable SWARMHOME be defined, which will typically not be true. These problems can be avoided by using the following statements instead of the above path command:

SWARMHOME=C:\swarm
path=%SWARMHOME%\bin;%PATH%

You can alternatively add the Swarm folder to your PATH permanently by editing the Windows PATH environment variable. A tool like RapidEE makes doing so easy, or you can use use Windows control panel > System > Advanced > Environment Variables to add swarm\bin to the PATH system variable.

Once the path has been set to include the swarm\bin folder, Swarm models can be started through the Windows command line, or by double-clicking on them from the Windows Explorer.

The Heatbugs model can now be run by changing the directory of the command line to the folder holding heatbugs.exe and then executing the program. For example, if heatbugs.exe is located in C:\Users\example_user\Downloads, then the model may be run as follows:

cd C:\Users\example_user\Downloads
heatbugs.exe

(or you can just double-click on heatbugs.exe from Windows Explorer, which temporarily opens a Windows command terminal while the model runs.)

Running the Heatbugs model

You can also create a simple Windows batch file that sets SWARMHOME and the path and then executes the model. Create a file called, for example heatbugs.bat in the same directory where heatbugs.exe is, containing these lines:

SWARMHOME=C:\swarm
path=%SWARMHOME%\bin;%PATH%
heatbugs.exe

Then, just double-clicking on heatbugs.bat will start the model.

If you try this and get an error that some .dll file is not found, then the problem is likely that the path is not set correctly to include the swarm\bin folder. If instead the program crashes without an error statement, then the problem is more likely that SWARMHOME needs to be set.

Installing Swarm and MinGW to compile models

To compile Swarm models from source using MinGW, the following simple steps need to be taken to set up the build environment:

NOTE:

The archived Swarm binaries MUST be placed at C:\swarm to successfully compile Swarm models. This is because some paths had to be hard-coded by libtool during the creation of the binaries.

The final step before you can compile a Swarm model is to set the SWARMHOME environment variable so that the "make" program knows where to find the Swarm libraries when compiling your code. There are several ways to do this.

  • You can temporarily set SWARMHOME by entering this command in the MinGW shell:
    export SWARMHOME=/c/swarm
  • You can edit your Makefile so it starts with:
    SWARMHOME=/c/swarm
  • Set SWARMHOME permanently as a Windows environment variable, either by using RapidEE or Windows control panel > System > Advanced > Environment Variables to add a new (User or System) variable called SWARMHOME with the value /c/swarm.

After MinGW has been installed and the Swarm binaries have been downloaded, Swarm models may be compiled using the MinGW shell. Start up the program MinGW Shell, cd to the directory containing your code and Makefile, and enter the command "Make".

An archive of official Swarm example programs can be downloaded from:

http://download.savannah.gnu.org/releases/swarm/src/swarm/swarmapps-2.3.0.tar.gz

We will use the "Heatbugs" model as an example of how to compile a Swarm application:

tar xzf /path/to/downloaded/swarmapps-2.3.0.tar.gz
cd swarmapps-2.3.0/objc/heatbugs

make

VERY IMPORTANT NOTE: MinGW produces two executables called heatbugs.exe---one in the heatbugs folder and another in the heatbugs/.libs folder. You should ignore the executable in the top-level folder, heatbugs/heatbugs.exe, as it is merely a compiled wrapper script that executes heatbugs/.libs/heatbugs.exe. When distributing compiled models, only use binaries from the .libs folder.

The resulting executable, .libs/heatbugs.exe may be executing by following the steps described above at Using Swarm Models Compiled with MinGW. These steps must include setting the Windows Path to include C:\swarm\bin.

Distributing a Swarm Model for Windows Users

A major benefit of the MinGW version of Swarm is that models can be distributed for people to use without installing Swarm. A simple way to do this, after the model has been compiled and tested using methods in the previous section, is to create a ZIP archive that contains all the files to run the model.

  • Create a directory tree that includes the Swarm binary files. For example, create C:\MySwarmModel and add the Swarm binaries (as described above) to it as a subdirectory tree.
  • Create another subdirectory that includes just the compiled model.
  • Create a third subdirectory that includes any input files that the model needs and a batch file that sets the path and executes the model.
  • Zip the whole tree up for distribution.

The directory tree in the zip file looks like this:

C:\MySwarmModel\swarm
                      \bin
                      \etc
                      \include
                      \lib
                      \man
                      \share
               \executable\MySwarmModel.exe
               \Runfiles\MySwarmModel.bat (and any input files)

where the file MySwarmModel.bat contains:

set SWARMHOME=C:\MySwarmModel\swarm
path=%SWARMHOME%\bin;%PATH%
C:\MySwarmModel\executable\MySwarmModel.exe

Then users can run the model just by double-clicking on MySwarmModel.bat.

Building Swarm Using MinGW

This tutorial provides directions and code for compiling your own Swarm libraries for Windows. It is a bit advanced---some familiarity with compiling software from source in a Linux environment and knowledge of the UNIX command line is assumed.

Install Required Tools

The first step is to download and install tools required for compiling Swarm. These links provide directions:

Build Support Libraries

The next steps are to build and install the libraries that Swarm depends on. A ZIP archive containing all the source code required can be found at:

http://download.savannah.gnu.org/releases/swarm/src/swarm/Swarm-2.4.1-MinGW-sources.zip

This archive contains two subdirectories:

  • sources: This folder contains the source code for each component.
  • patches: This folder contains patchfiles that modfy source code so that MinGW can compile it.

After downloading and extracting the source code archive, open up a MinGW shell and run the following commands:

cd /path/to/where/swarm/source/was/extracted
mkdir build

The following steps will all assume your MinGW shell is operating out of the build directory. All compiled software will be installed to /c/swarm (The Windows Explorer will see the folder appear at C:\swarm).

ZLib

ZLib will be the first library to be built as it is a common dependency for many of the other software components. From the build directory, execute the following to build Zlib:

tar xzf ../sources/zlib-1.2.5.tar.gz
cd zlib-1.2.5
make -f win32/Makefile.gcc

And then the following commands to install ZLib to /c/swarm:

mkdir -p /c/swarm/bin
mkdir -p /c/swarm/include
mkdir -p /c/swarm/lib

cp zlib1.dll /c/swarm/bin/
cp zconf.h zlib.h /c/swarm/include/
mv libzdll.a libz.dll.a
cp *.a /c/swarm/lib

LibPNG

LibPNG will be built next as its only dependency is ZLib. From the build directory, execute the following steps to build and install LibPNG:

tar xjf ../sources/libpng-1.5.4.tar.bz2
cd libpng-1.5.4

export CPPFLAGS='-I/c/swarm/include'
export LDFLAGS='-L/c/swarm/lib'
./configure --prefix=/c/swarm --with-zlib-prefix=/c/swarm

make
make install

XPM

Starting from the build directory, the XPM library can be built and installed as follows:

tar xjf ../sources/xpm-nox-4.2.0.tar.bz2
cd xpm-nox-4.2.0
patch -p1 < ../../patches/xpm-nox-4.2.0-mingw-tcl.patch

make CC=gcc
make install prefix=/c/swarm

Summary of patchfile changes:

The patchfile is inherited from the SUSE builds of XPM for MinGW and adds Makefiles and other critical components.

One major modification has been made to the SUSE patchfile---tcl.h is included in xpm.h. This is required in order to use XPM while Tk is installed. Conversely, this means that this build of XPM cannot be used without having Tk installed.

LibFFI

Starting from the build directory, LibFFI can be built and installed as follows:

tar xzf ../sources/libffi-3.0.9.tar.gz
cd libffi-3.0.9

./configure --prefix=/c/swarm
make
make install

The LibFFI headers get installed in a weird place.

mv /c/swarm/lib/libffi-3.0.9/include/*.h /c/swarm/include
rm -rf /c/swarm/lib/libffi-3.0.9

Tcl

Starting from the build directory, Tcl can be built and installed as follows:

tar xzf ../sources/tcl8.4.19-src.tar.gz
cd tcl8.4.19
patch -p1 < ../../patches/tcl-8.4.19-fix-lvalues.patch
cd win

./configure --prefix=/c/swarm
make
make install

Summary of patchfile changes:

At two places in the TCL source code, an assignment is made while simultaneously performing a typecast. The typecast and the assignment had to be split into separate operations in order to be acceptable to the MinGW compilers.

Tk

Starting from the build directory, Tk can be built and installed as follows:

tar xzf ../sources/tk8.4.19-src.tar.gz
cd tk8.4.19/win

./configure --prefix=/c/swarm
make
make install

BLT

Starting from the build directory, BLT can be built and installed as follows:

tar xzf ../sources/BLT2.4z.tar.gz
cd blt2.4z

./configure --prefix=/c/swarm --without-x
make
make install

HDF5 (Optional)

Getting HDF5 to work with MinGW is a little tricky. It is an optional dependency, so if support for reading and writing .hdf files is not needed this section can be skipped.

As of version 1.8.7, HDF5 currently does not compile under MinGW. So, we must wrap pre-compiled binaries for use with MinGW. In order to wrap the binaries, we will need an additional tool called pexports. This can be installed using mingw-get:

mingw-get install mingw-utils

Download 32-bit Windows binaries from the HDF website compiled for use with the Visual Studio compilers (these files have "VS" listed under the Compilers section). After extracting the archive, the HDF5 libraries can be wrapped for use with MinGW by executing the following commands:

cd path/to/where/HDF5/was/extracted

for dll in `ls dll/*dll`; do
  def_file=`basename $dll .dll`.def
  lib_file=lib`basename $dll dll.dll`.a
  pexports $dll > $def_file
  dlltool -d $def_file -l lib/$lib_file
done

Next, move the libraries and header files needed to use HDF5 from C into C:\swarm:

cp dll/hdf5dll.dll dll/szip.dll /c/swarm/bin
cp include/* /c/swarm/include
cp lib/libhdf5.a /c/swarm/lib

Finally, a patch must be applied to the HDF header file H5public.h so that it doesn't define ssize_t---a constant already defined by MinGW. The required patch is contained in the Swarm MinGW source archive:

cd /path/to/where/swarm/source/was/extracted
patch -p1 -d /c/swarm < patches/hdf5-1.8.7-mingw.patch

Build Swarm

Finally, the Swarm libraries themselves can be built. From the build directory, execute the following

tar xzf ../sources/swarm-2.4.1.tar.gz
cd swarm-2.4.1
patch -p1 ../../patches/swarm-2.4.1-mingw.patch

export CPPFLAGS='-I/c/swarm/include'
export LDFLAGS='-L/c/swarm/lib'
./configure --prefix=/c/swarm \
  --with-zlibdir=/c/swarm \
  --with-pngdir=/c/swarm \
  --with-xpmdir=/c/swarm \
  --with-ffidir=/c/swarm \
  --with-tcl=/c/swarm/lib \
  --with-tk=/c/swarm/lib \
  --with-bltdir=/c/swarm

cd avcall
gcc -S avcall-i386.c
cd ..

make
make install

If you wrapped HDF5 up for use with MinGW, add --with-hdf5dir=/c/swarm to the list of arguments passed to configure.

Summary of patchfile changes:

swarm-2.4.1-mingw.patch modifies the Swarm source code in the following ways:

  • gettimeofday is no longer defined in the Swarm sources as MinGW includes this function in its standard library.
  • Swarm GUI functions are updated to be compatible with LibPNG 1.5.
  • The Swarm configure script defines DATADIR as a preprocessor macro---this conflicts with variables in the GCC Objective-C library. DATDIR is undefed in critical parts of the source code.
  • The sleep function is aliased to usleep---which is the name of the MinGW implementation.
  • The way in which Makefiles invoke gperf is amended so that it actually produces output.
  • NULL checks are added to `zstrdup` in src/defobj/internal.m. Some functions in the Swarm library may attempt to call `zstrdup` on a NULL pointer. (This happens if the SWARMHOME environment variable is not set.) Failing to check for this condition can lead to segfaults. `zstrdup` now returns NULL if passed a NULL string or if `alloc` fails to allocate memory for a duplicate.

Full details are contained in the patchfile.

Tools

MinGW

Homepage: http://www.mingw.org

Current versions of MinGW include a compiler version (gcc 6.x) that is incompatible with Swarm. Swarm needs a version no newer than 4.6. IF you use a current version of MinGW your Swarm code may compile but then produce incorrect results. You can find on the web (a) instructions for replacing the gcc in MinGW with an older version, and (b) old versions of MinGW that contain older gcc versions. However, some of us have not had success with either of those options. At least temporarily, there is a version of MinGW that works in Windows 10 and with Swarm, available here.

Version Used: 20110530

License: GPL

MinGW is a set of software that aims to provide an environment where software developed on UNIX-like systems can be easily compiled for Windows. Since Swarm is primarily developed on Linux (a UNIX clone), MinGW can ease the task of creating Swarm models that run on Windows.

MinGW is similar to the Cygwin project. The major difference is that MinGW produces executables that depend only on native Windows libraries while Cygwin executables require special compatibility libraries that are part of the Cygwin distribution.

Installing MinGW

MinGW can be installed by downloading the "Automated MinGW Installer", which usually has a name like mingw-get-inst-<date>.exe where <date> is the datestamp for when the installer was packaged. Running this executable will start the installation process. The installer lets you select which optional components to install; make sure all boxes are checked so you install all components.

Install all available MinGW components

The installer downloads all components from the internet. This will take a while, so have a cup of coffee or walk the dog.

See the following section about also installing an optional, more-functional shell window for MinGW.

Using MinGW

The MinGW compilers are accessible through the "MinGW Shell" which should be located under the MinGW folder in the Start Menu:

Location of MinGW Shell

or by running C:\MinGW\msys\1.0\msys.bat (if MinGW was installed to its default location).

By default the MinGW shell executes inside of a Windows cmd.exe terminal. There are a couple major drawbacks to this:

  • You cannot resize the window to full-screen.
  • The shell provides limited history scrollback.

MinGW provides a couple of alternative terminals for the MinGW shell which do not have these drawbacks; one of which is called MinTTY. To install MinTTY execute the following in a MinGW shell:

mingw-get install mintty

Then the mintty command can be used to launch a MinTTY terminal. The MinGW Shell can also be set to launch MinTTY by default by editing the file C:\MinGW\msys\1.0\msys.bat and changing the following line (approximately line 58):

if "x%MSYSCON%" == "xunknown" set MSYSCON=sh.exe

to:

if "x%MSYSCON%" == "xunknown" set MSYSCON=mintty.exe

A Note About Paths

The MinGW Shell emulates a UNIX bash shell and thus uses UNIX-style paths, /path/to/file, instead of Windows-style paths, C:\path\to\file. One peculiarity of this process is that the shell must deal with the fact that Windows filesystems have multiple roots (C:, D:, etc.) while a UNIX filesystem only has one root called /. The way this is reconciled by the bash shell provided with MinGW is that /c refers to the root of the C: drive, /d refers to the root of the D: drive and so on and so forth.

The root of the filesystem itself, /, also has some caveats. / refers to the mysy folder, C:/MinGW/mysys/1.0 in a default installation. /usr also refers to the same location. Thus, /usr/local and /local both refer to C:/MinGW/mysys/1.0/local.

A Note About MinGW-64

Currently Swarm has been developed and tested against the 32-bit version of MinGW. Hence Swarm models produced this way should run in any version of Windows.

The [MinGW-w64][mingw_65] project produces 64-bit MinGW compilers that can produce 64-bit executables that will only run on 64-bit versions of Windows. These executables benefit from:

  • Access to more memory. A 32-bit executable will only be able to allocate ~3 GB of RAM---even if the system has more RAM available.
  • Possibly quicker execution speeds due to matching the native architecture of the operating system---however this depends on the types of tasks that the program executes. It has been reported that 64 bit Swarm executables run quicker than 32 bit Swarm executables on Linux.

Currently, no attempt has been made to compile Swarm 2.4.1 with the 64 bit MinGW compilers. If a successful attempt is made, please report it on the Swarm mailing list.

Emacs

Homepage: http://www.gnu.org/software/emacs

Version Used: 23.3

Download Area: http://ftp.gnu.org/gnu/emacs/windows

License: GPL

GNU Emacs is a text editor that is used by many programmers to edit code. However, Emacs can also be scripted to perform a variety of complicated transformations on text files. The scripts that build the Swarm libraries use Emacs to automate the process of generating some source files. Thus Emacs has to be installed in order to build Swarm from source.

Installing Emacs

Install Emacs by downloading one of the emacs-<version number>-bin-i386.zip files from the download area. Don't use any of the files containing barebin. After extracting the ZIP archive there will be an Emacs folder containing a subfolder called bin. Add this folder to your Windows PATH. RapidEE can help edit the PATH variable.

GPerf

Homepage: http://www.gnu.org/software/gperf/

Download Area: http://sourceforge.net/projects/gnuwin32/files/gperf

Version Used: 3.0.1

License: GPL

GPerf is a program that produces C code containing hash functions. The scripts that build the Swarm libraries use GPerf to generate some source code files. Download gperf.exe from the GnuWin32 site on Sourceforge and place it somewhere on your Windows PATH. Again, RapidEE can help set the PATH variable.


RapidEE

Homepage: http://www.rapidee.com

Version Used: 5.0

License: Freeware

The Rapid Environment Editor (RapidEE) is a spiffy tool that makes it easy to inspect and alter the value of Windows environment variables, such as the PATH. RapidEE may be used to edit "System Variables" which will affect all user accounts or "User Variables" which will affect just the current account. Administrative privileges may be required to edit the System Variables.

The different variables RapidEE can edit

Right-clicking on the Path variable in the System Variables or User Variables will bring up a menu that presents the option to "Add value":

Adding a new value to the PATH

Once the new entry has been added, RapidEE must be told to save the changes. Command line shells, such as cmd.exe, must be restarted in order for them to notice the changes.

Adding and saving a value

Depends

Homepage: http://www.dependencywalker.com/

Version Used: 2.2

License: Freeware

Depends is a usefull utility that scans Windows executables and reports information concerning the Dynamic-link Libraries (DLLs) that those executables require in order to run.

Software Used to Compile Swarm 2.4.1

Name Verson Used License Homepage Download Area
ZLib 1.2.5 ZLib/PNG License http://www.zlib.net --
LibFFI 3.0.9 MIT http://sourceware.org/libffi --
LibPNG 1.5.4 ZLib/PNG http://www.libpng.org/pub/png/libpng.html --
XPM-NoX 4.2.0 MIT unknown Try: http://gnuwin32.sourceforge.net/packages/xpm-nox.htm
Tcl 8.4.19 Tcl http://www.tcl.tk --
Tk 8.4.19 Tcl http://www.tcl.tk --
BLT 2.4z BSD http://blt.sourceforge.net --
HDF5 1.8.7 BSD http://www.hdfgroup.org/HDF5 --
Swarm 2.4.1 GPL https://savannah.nongnu.org/projects/swarm/ http://download.savannah.gnu.org/releases/swarm/src/swarm/swarm-2.4.1.tar.gz