corosync/INSTALL

----------------------------------------------
The Corosync Cluster Engine Installation Guide
----------------------------------------------
Please read LICENSE for a description of the licensing of this software.

All cryptographic software in this package is subject to the following legal
notice:
This package includes publicly available encryption source code which,
together with object code resulting from the compiling of publicly
available source code, may be exported from the United States under License
Exception TSU prsuant to 15 C.F.R Section 740.13(e).

---------------------------------
* Platforms Built and Tested On *
---------------------------------
Corosync has been tested on:
Linux: Fedora 11, Fedora 12 (rawhide), Debian Lenny, Debian Unstable,
       Ubuntu Hardy, Red Hat Enterprise Linux 5
Solaris: version 10 CSW
BSD: FreeBSD and OpenBSD
MacOSX: Darwin latest update

It should build and run properly on the tested platforms as well as possibly
others with minimal effort.  Corosync requires a coherent mmap() system call
and will not operate on platforms which don't support coherent mmap().

Corosync has been tested with recent gcc compilers and Intel's C compiler.
SunStudio compilers fail to build with various internal compiler errors.

----------------------------
* Building from subversion *
----------------------------
When building and installing from subversion, automake 2.61 or later is
required.  Prior versions will result in build failures.

Step 1: check out a read only copy of the repository
svn checkout http://svn.fedoraprojects.org/svn/corosync

Find the version you want to build.  Usually this will be the "trunk" version
located in the trunk directory.  If you want to build a specific released
version check in the tags directory.

Step 2: Generate the makefiles
balance:~/corosync/trunk% ./autogen.sh

Step 3: Run the configure script
balance:~/corosync/trunk% ./configure

Step 4: Install the binaries
balance:~/corosync/trunk% su
balance:~/corosync/trunk# make install

-------------------------
* Building from tarball *
-------------------------
The tarball is distributed with pregenerated makefiles.  There is no need
to run the autogen.sh script in this case.

Step 1: Run the configure script
balance:~/corosync/trunk% ./configure

Step 2: Install the binaries
balance:~/corosync/trunk% su
balance:~/corosync/trunk# make install

-------------------------------
* A notice about dependencies *
-------------------------------
We have strived very hard to avoid dependencies as much as possible, and in
fact corosync can be compiled without any dependencies.

Optional dependencies are support for NSS and support for Infiniband RDMA.

NSS encryption performs more poorly then the default encryption, but may be
 a good choice if your organization has policies regarding encryption technology
requiring the use of a common cryptographic library such as Netscape Security Services.

To enable nss during building:
balance:~/corosync/trunk% ./configure --enable-nss

To disable nss during buildling:
balance:~/corosync/trunk% ./configure --disable-nss

The nss packages required to build corosync with nss support can usually be
found by installing the "nss-devel" and "nspr-devel" packages for your
distribution.  Note nss is enabled by default, so if you don't want nss support
or don't want to install nss devel packages, make sure to set the appropriate
configure option as described above.

-----------------------------------------
* Building with Infiniband RDMA support *
-----------------------------------------
Infiniband RDMA build support is disabled by default.  To enable rdma during
building:
balance:~/corosync/trunk% ./configure --enable-rdma

Infiniband rdma support requires the libibverbs and librdmacm packages (and
their -devel counterparts).  Corosync requires the use of pkgconfig in these
packages.  The Corosync maintainers have sent patches for this feature upstream
to the Infiniband maintainers.

If your version of these packages don't have pkgconfig support, you will see an
error during configure such as:

checking for ibverbs... configure: error: Package requirements (ibverbs) were not met:

No package 'ibverbs' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables ibverbs_CFLAGS
and ibverbs_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

** Follow the directions:
balance:~/corosync/trunk% export ibverbs_CFLAGS=-I/usr/include/infiniband
balance:~/corosync/trunk% export ibverbs_LIBS=-libverbs
balance:~/corosync/trunk% export rdmacm_CFLAGS=-I/usr/include/rdma
balance:~/corosync/trunk% export rdmacm_LIBS=-lrdmacm
balance:~/corosync/trunk% ./configure --enable-rdma

Hopefully pkgconfig support is added in your distribution's version of
libibverbs and librdmacm soon if you run into this problem.

-----------------------------------------
* Building with SNMP/DBUS support *
-----------------------------------------

You can get SNMP traps on the following corosync events:
1) node joine/leave
2) application connect/dissconnect from corosync
3) quorum gain/lost

There are 2 modes of achieving this DBUS + foghorn and snmp-agentx.

Setting up to get dbus events.
------------------------------
foghorn (http://git.fedorahosted.org/git/foghorn.git) converts
dbus signals into snmp traps. So install foghorn.

$ ./configure --enable-dbus
$ make && sudo make install
$ /etc/init.d/corosync start
$ echo "OPTIONS=\"-d\"" > /etc/sysconfig/corosync-notifyd
$ /etc/init.d/corosync-notifyd start
Start foghorn

to see the dbus signals getting sent try:
$ dbus-monitor --system

Setting up snmp-agentx.
-----------------------
If you don't want to use dbus then you can use snmp-agentx.

$ ./configure --enable-snmp
$ make && sudo make install
$ /etc/init.d/corosync start
$ vim /etc/snmp/snmptrapd.conf

Add the following:
authCommunity   log,execute,net public
$ /etc/init.d/snmptrapd start
$ echo "OPTIONS=\"-s\"" > /etc/sysconfig/corosync-notifyd
$ /etc/init.d/corosync-notifyd start

I start up wireshark to see if there are any snmp traps been sent
as I am too lazy to setup a manager to receive traps.

run a program that talks to corosync e.g.
$ corosync-objctl

And you should get traps


------------------------
* Configuring Corosync *
------------------------
The corosync executive will automatically determine cluster membership by
communicating on a specified multicast address and port.

The directory conf contains the file corosync.conf.example

# Please read the corosync.conf.5 manual page
totem {
	version: 2
	secauth: off
	threads: 0
	interface {
		ringnumber: 0
		bindnetaddr: 192.168.1.1
		mcastaddr: 226.94.1.1
		mcastport: 5405
	}
}

logging {
	fileline: off
	to_stderr: yes
	to_file: yes
	to_syslog: yes
	logfile: /tmp/corosync.log
	debug: off
	timestamp: on
}

The totem section contains three values.  All three values must be set
or the corosync executive wll exit with an error.

bindnetaddr specifies the address which the corosync Executive should bind to.
This address should always end in zero.  If the local interface taffic
should routed over is 192.168.5.92, set bindnetaddr to 192.168.5.0.

mcastaddr is a multicast address.  The default should work but you may have
a different network configuration.  Avoid 224.x.x.x because this is a "config"
multicast address.

mcastport specifies the UDP port number.  It is possible to use the same
multicast address on a network with the corosync services configured for
different UDP ports.

The timeout section contains seven values.  This section is not normally used,
but rather used to override the program defaults for the purposes of fine
tuning for a given networking/processor combination or for debugging purposes.
Be careful to use the same timeout values on each of the nodes in the cluster
or unpredictable results may occur.

Do not use DOS style termination.  This breaks the parser.

Generate a private key
----------------------
corosync uses cryptographic techniques to ensure authenticity and privacy of
messages.  A private key must be generated and shared by all processors for
correct operation.

First generate the key on one of the nodes:

balance# corosync-keygen
Corosync Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Writing corosync key to /etc/ais/authkey.

After this is complete, a private key will be in the file /etc/ais/authkey.
This private key must be copied to every processor that will be a member of
the cluster.  If the private key isn't the same for every node, those nodes
with nonmatching private keys will not be able to join the same configuration.

Copy the key to some transportable storage or use ssh to transmit the key
from node to node.  Then install the key with the command:

balance# install -D --group=0 --owner=0 --mode=0400 /path_to_authkey/authkey /etc/ais/authkey

If the message invalid digest appears, the keys are not the same on each node.

Run the corosync executive
-------------------------
Get one or more nodes and run the corosync executive on each node.  Run the
corosync daemon after following the previous directions.  The daemon must be
run as UID 0(root).

please read SECURITY to understand the threat model assumed by corosync
and the techniques corosync use to overcome these threats.

Before running any of the test programs
---------------------------------------
The corosync executive will ensure security by only allowing the UID 0(root) or
GID 0(root) to connect to it.  To allow other users to access the corosync
executive, create a directory called /etc/corosync/uidgid.d and place a file in
it named in some way that is identifiable to you.  All files in this directory
will be scanned and their contents added to the allowed uid gid database.  The
contents of this file should be
uidgid {
	uid: username
	gid: groupname
}

Please note that these users then have full ability to transmit and receive
messages in the cluster and are not bound by the threat model described in
SECURITY.

Try out the corosync cpg functionality
--------------------------------------
After corosync is running

Run test/testcpg on multiple nodes or on the same node.  Messages can be typed
which will then be sent to other testcpg applications in the cluster.

To see a hashed verified output of data on all nodes, test/cpgverify can be
run.