corosync/man/corosync_overview.8

.\"/*
.\" * Copyright (c) 2005 MontaVista Software, Inc.
.\" * Copyright (c) 2006 Red Hat, Inc.
.\" *
.\" * All rights reserved.
.\" *
.\" * Author: Steven Dake (sdake@redhat.com)
.\" *
.\" * This software licensed under BSD license, the text of which follows:
.\" *
.\" * Redistribution and use in source and binary forms, with or without
.\" * modification, are permitted provided that the following conditions are met:
.\" *
.\" * - Redistributions of source code must retain the above copyright notice,
.\" *   this list of conditions and the following disclaimer.
.\" * - Redistributions in binary form must reproduce the above copyright notice,
.\" *   this list of conditions and the following disclaimer in the documentation
.\" *   and/or other materials provided with the distribution.
.\" * - Neither the name of the MontaVista Software, Inc. nor the names of its
.\" *   contributors may be used to endorse or promote products derived from this
.\" *   software without specific prior written permission.
.\" *
.\" * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
.\" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
.\" * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
.\" * THE POSSIBILITY OF SUCH DAMAGE.
.\" */
.TH COROSYNC_OVERVIEW 8 2006-05-10 "corosync Man Page" "Corosync Cluster Engine Programmer's Manual"
.SH OVERVIEW
The corosync project is a project to implement a production quality "Revised BSD"
licensed implementation of the most recent SA Forum's Application Interface
Specification.  The Application Interface Specification is a software API and
policies which are used to develop applications that maintain service during
faults.  The API consists of Availability Management Framework (AMF) which
provides application failover, Cluster Membership (CLM), Checkpointing (CKPT),
Eventing (EVT), Messaging (MSG), and Distributed Locking (DLOCK).

Currently Messaging is unimplemented.

Faults occur for various reasons:
.PP
* Application Faults
.PP
* Middleware Faults
.PP
* Operating System Faults
.PP
* Hardware Faults

The major focus of high availability in the past has been to mask hardware
faults. Faults in other components of the system have gone unsolved until
AIS. AIS can mask many types of faults in applications, middleware,
operating systems, or even hardware by providing a simple framework
for allowing developers to create redundant applications. These redundant
applications can be distributed over multiple nodes such that if any one
node faults, another node can recover.

Application programmers develop applications to periodically record their
state using the checkpointing service. When an active application fails,
a standby application recovers the state of the application. This
technique, called stateful application failover, provides the fundamental
difference between corosync and other systems that have come before it.
With stateful application failover, the end-application user doesn't
have to reload the application or redial a telephone. The full state
is recorded, so the end-application user sees no interruption in service.

Because programmers can now distribute applications across multiple
processes or nodes, a mechanism must exist for them to communicate.
This mechanism is provided by two services. The event service provides
a publish/subscribe model for events. The messaging service provides
end to end messaging. Finally a mechanism to synchronize access is
provided by the distributed lock service.

The corosync project also provides a group messaging toolkit called EVS.
The EVS service implements a messaging model known as Extended Virtual
Synchrony.  This model allows one sender to transmit to many receivers.
Certain guarantees are provided for message and membership delivery
which make virtual synchrony ideal for developing distributed applications.

.SH QUICKSTART
The corosync executive must be configured.  In the directory conf in the
source distribution are several files that must be copied to the /etc/ais
directory.  If corosync is packaged by a distro, this may be complete.

The directory contains the file corosync.conf.  Please read the corosync.conf(5)
man page for details on the configuration options.  The corosync project will
work out of the box with the default configuration options, although the
administrator may desire different options.

An user and group of the name "ais" must be added to the system.  If corosync
is packaged from a distro, this step should already be completed.
This can be achieved by executing:

[root@slickdeal root]# adduser ais -g ais

The corosync executive uses cryptographic techniques to ensure authenticity
and privacy of the messages.  In order for corosync to be secure and operate,
a private key must be generated and shared to all processors.

First generate the key on one of the nodes:

unix# ais-keygen
corosync authentication key generator.
.PP
Gathering 1024 bits for key from /dev/random.
.PP
Writing corosync key to /etc/ais/authkey.
.PP

After this operation, a private key will be in the file /etc/ais/authkey.
This private key must be copied to every processor in the cluster.  If the
private key isn't the same for every node, those nodes with nonmatching private
keys will not be able to join the same configuration.

Copy the key to some security transportable storage or use ssh to transmit the
key from node to node.  Then install the key with the command:

unix#: install -D --group=0 --owner=0 --mode=0400 /path_to_authkey/authkey /etc/ais/authkey

If a message "Invalid digest" appears from the corosync executive, the keys
are not consistent between processors.

Finally run the corosync executive.  If corosync is packaged from a distro, it
may be set to start on system start.  It may also be turned off by default in
which case the init script for corosync must be enabled.

After running aisexec, a list of all processors IP addresses running the ais
executive and configured on the same multicast address will appear.  If they
don't appear, there may be a problem with multicast in the distro or hardware.
If this happens, participation in the corosync mailing list may help solve the
problem.  The email address is corosync@lists.osdl.org.

.SH USING LIBRARIES
The corosync AIS libraries have header files which must be included in the
developer's application.  Once the header file is included, the developer can
reference the AIS interfaces.

The corosync project recommends to distros to place include files in
/usr/include/corosync.  The following include lines must be added to
the application to use each of the following services:

#include <corosync/saClm.h> For the Cluster Membership B.01.01 service.
.PP
#include <corosync/saCkpt.h> For the Checkpointing B.01.01 service.
.PP
#include <corosync/saEvt.h> For the Eventing B.01.01 service.
.PP
#include <corosync/ais_amf.h> For the AMF A.01.01 service.
.PP

The corosync project recommends to distros to place library files in
/usr/lib.  The following link lines must be added to the LDFLAGS section
of the makefile.

-lsaClm For the Cluster Membership B.01.01 service
.PP
-lsaCkpt For the Checkpointing B.01.01 service
.PP
-lsaEvt For the Eventing B.01.01 service
.PP
-lsaAmf For the AMF A.01.01 service
.PP
-lais Specify this to get access to all AIS libraries without specifying
each library individually.

.SH IPv6
The corosync project supports both IPv4 and IPv6 network addresses.  The entire
cluster must use either IPv4 or IPv6 for the cluster communication mechanism.
In order to use IPv6, IPv6 addresses must be specified in the bindnetaddr and
mcastaddr fields in the configuration file.  The nodeid field must also be
set.

An example of this is:
nodeid: 2
bindnetaddr: fec0::1:a800:4ff:fe00:20
mcastaddr: ff05::1

To configure a host for IPv6, use the ifconfig program to add interfaces:
box20: ifconfig eth0 add fec0::1:a800:4ff:fe00:20/64
box30: ifconfig eth0 add fec0::1:a800:4ff:fe00:30/64

If the /64 is not specified, a route for the IPv6 network will not be configured
which will cause significant problems.  Make sure a route is available for
IPv6 traffic.

.SH ARCHITECTURE
The AIS libraries are a thin IPC interface to the corosync executive.  The
corosync executive provides services for the SA Forum AIS libraries as well
as the EVS and CPG libraries.

The corosync executive uses the Totem extended virtual synchrony protocol.  The
advantage to the end user is excellent performance characteristics and a proven
protocol with excellent reliability.  This protocol connects the processors
in a configuration together so they may communicate.

.SH ENVIRONMENT VARIABLES
The corosync executive process uses four environment variables during startup.
If these environment variables are not set, defaults will be used.

.TP
COROSYNC_MAIN_CONFIG_FILE
This specifies the fully qualified path to the corosync configuration file.

The default is /etc/ais/corosync.conf.

.TP
COROSYNC_AMF_CONFIG_FILE
This specifies the fully qualified path to the corosync Availability Management
Framework configuration file.

The default is /etc/ais/amf.conf.

.TP
COROSYNC_DEFAULT_CONFIG_IFACE
This specifies the LCRSO that is used to parse the configuration file.  This
allows other configuration file parsers to be implemented within the system.

The default is to use the default corosync configuration file parser which
parses the format specified in corosync.conf (5).

.TP
COROSYNC_TOTEM_AUTHKEY_FILE
This specifies the fully qualified path to the shared key used to
authenticate and encrypt data used within the Totem protocol.

The default is /etc/ais/authkey.

.SH SECURITY
The corosync executive optionally encrypts all messages sent over the network
using the SOBER-128 stream cipher.  The corosync executive uses HMAC and SHA1 to
authenticate all messages.  The corosync executive library uses SOBER-128
as a pseudo random number generator.  The EVS library feeds the PRNG using
the /dev/random Linux device.

If membership messages can be captured by intruders, it is possible to execute
a denial of service attack on the cluster.  In this scenario, the cluster is
likely already compromised and a DOS attack is the least of the administration's
worries.

The security in corosync does not offer perfect forward secrecy because the keys
are reused.  It may be possible for an intruder by capturing packets in an
automated fashion to determine the shared key.  No such automated attack has
been published as of yet.  In this scenario, the cluster is likely already
compromised to allow the long-term capture of transmitted data.

For security reasons, the corosync executive binary aisexec should NEVER
be setuid or setgid in the filesystem.

.PP
.SH SAFTEST COMPLIANCE
The corosync libraries are now nearly compliant with every aspect of the SA
Forum's AIS specification.  The AMF service, however, is not compliant with the
B.01.01 specification.  The remaining services pass most of the tests of the
saftest suite against the B.01.01 specification.

.SH BUGS
The messaging service is partially implemented and not suitable for deployment.
The distributed locking service is buggy and not suitable for deployment.
The Availability Management Framework is under development and not suitable for
deployment..

.SH "SEE ALSO"
.BR corosync.conf (5),
.BR evs_overview (8)
.PP