.\"/* .\" * Copyright (c) 2005 MontaVista Software, Inc. .\" * .\" * All rights reserved. .\" * .\" * Author: Steven Dake (sdake@mvista.com) .\" * .\" * This software licensed under BSD license, the text of which follows: .\" * .\" * Redistribution and use in source and binary forms, with or without .\" * modification, are permitted provided that the following conditions are met: .\" * .\" * - Redistributions of source code must retain the above copyright notice, .\" * this list of conditions and the following disclaimer. .\" * - Redistributions in binary form must reproduce the above copyright notice, .\" * this list of conditions and the following disclaimer in the documentation .\" * and/or other materials provided with the distribution. .\" * - Neither the name of the MontaVista Software, Inc. nor the names of its .\" * contributors may be used to endorse or promote products derived from this .\" * software without specific prior written permission. .\" * .\" * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" .\" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE .\" * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR .\" * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF .\" * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS .\" * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN .\" * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) .\" * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF .\" * THE POSSIBILITY OF SUCH DAMAGE. .\" */ .TH OPENAIS_CONF 5 2005-06-08 "openais Man Page" "Openais Programmer's Manual" .SH NAME openais.conf - openais executive configuration file .SH SYNOPSIS /etc/ais/openais.conf .SH DESCRIPTION The openais.conf instructs the openais executive about various parameters needed to control the openais executive. The configuration file consists of bracketed top level directives. The possible directive choices are .IR totem { } " and " logging { } ". These directives are described below. .TP totem { } This top level directive contains configuration options for the totem protocol. .TP logging { } This top level directive contains configuration options for logging. .PP .PP Within the .B totem directive, there are four configuration options which are all required: .TP version This specifies the version of the configuration file. Currently the only valid version for this directive is 1. .TP bindnetaddr This specifies the address which the openais executive should bind. This address should always end in zero. If the totem traffic should be routed over 192.168.5.92, set bindnetaddr to 192.168.5.0. Multiple bindnetaddr directives may be specified. When multiple bindnetaddr directives are specified, the totem redundant ring protocol will use multiple interfaces to replicate the network traffic. This may also be an IPV6 address, in which case IPV6 networking will be used. In this case, the full address must be specified and there is no automatic selection of the network interface within a specific subnet as with IPv4. If IPv6 networking is used, the nodeid field must be specified. .TP mcastaddr This is the multicast address used by openais executive. The default should work for most networks, but the network administrator should be queried about a multicast address to use. Avoid 224.x.x.x because this is a "config" multicast address. This may also be an IPV6 multicast address, in which case IPV6 networking will be used. If IPv6 networking is used, the nodeid field must be specified. .TP mcastport This specifies the UDP port number. It is possible to use the same multicast address on a network with the openais services configured for different UDP ports. .TP nodeid This configuration option is optional when using IPv4 and required when using IPv6. This is a 32 bit value specifying the node identifier delivered to the cluster membership service. If this is not specified with IPv4, the node id will be determined from the 32 bit IP address the system is bound to. The node identifier value of zero is reserved and should not be used. .PP Within the .B totem directive, there are four configuration options which are all optional. These control secrecy & authentication, the redundant ring mode of operation, and network MTU, and number of sending threads. .TP secauth This specifies that HMAC/SHA1 authentication should be used to authenticate all messages. It further specifies that all data should be encrypted with the sober128 encryption algorithm to protect data from eavesdropping. Enabling this option adds a 36 byte header to every message sent by totem which reduces total throughput. Encryption and authentication consume 75% of CPU cycles in aisexec as measured with gprof when enabled. For 100mbit networks with 1500 MTU frame transmissions: A throughput of 9mb/sec is possible with 100% cpu utilization when this option is enabled on 3ghz cpus. A throughput of 10mb/sec is possible wth 20% cpu utilization when this optin is disabled on 3ghz cpus. For gig-e networks with large frame transmissions: A throughput of 20mb/sec is possible when this option is enabled on 3ghz cpus. A throughput of 60mb/sec is possible when this option is disabled on 3ghz cpus. The default is on. .TP redundantring This specifies the mode of redundant ring, which may be none, active, or passive. Active replication offers slightly lower latency from transmit to delivery in faulty network environments but with poorer performance. Passive replication may nearly double the speed of the totem protocol if the protocol doesn't become cpu bound. The final option is none, in which case only one network interface will be used to operate the totem protocol. At this time redundant ring is only partially implemented and not yet available. The default is none. .TP netmtu This specifies the network maximum transmit unit. To set this value beyond 1500, the regular frame MTU, requires ethernet devices that support large, or also called jumbo, frames. If any device in the network doesn't support large frames, the protocol will not operate properly. The hosts must also have their mtu size set from 1500 to whatever frame size is specified here. Please note while some NICs or switches claim large frame support, they support 9000 MTU as the maximum frame size including the IP header. Setting the netmtu and host MTUs to 9000 will cause totem to use the full 9000 bytes of frame room. Then Linux will add a 18 byte header moving the full frame size to 9018. As a result some hardware will not operate properly with this size of data. A netmtu of 8982 seems to work for the few large frame devices that have been tested. Some manufacturers claim large frame support when in fact they support frame sizes of 4500 bytes. Increasing the MTU from 1500 to 8982 doubles throughput performance from 30MB/sec to 60MB/sec as measured with evsbench with 175000 byte messages with the secauth directive set to off. When sending multicast traffic, if the network frequently reconfigures, chances are that some device in the network doesn't support large frames. Choose hardware carefully if intending to use large frame support. The default is 1500. .TP threads This directive controls how many threads are used to encrypt and send multicast messages. If secauth is off, the protocol will never use threaded sending. If secauth is on, this directive allows systems to be configured to use multiple threads to encrypt and send multicast messages. A thread directive of 0 indicates that no threaded send should be used. This mode offers best performance for non-SMP systems. The default is 0. Within the .B totem directive, there are several configuration options which are used to control the operation of the protocol. It is generally not recommended to change any of these values without proper guidance and sufficient testing. Some networks may require larger values if suffering from frequent reconfigurations. Some applications may require faster failure detection times which can be achieved by reducing the token timeout. .TP token This timeout specifies in milliseconds until a token loss is declared after not receiving a token. This is the time spent detecting a failure of a processor in the current configuration. Reforming a new configuration takes about 50 milliseconds and is independent of this timeout. .TP token_retransmit This timeout specifies in milliseconds after how long before receiving a token the token is retransmitted. This will be automatically calculated if token is modified. It is not recommended to alter this value without guidance from the openais community. .TP hold This timeout specifies in milliseconds how long the token should be held by the representative when the protocol is under low utilization. It is not recommended to alter this value without guidance from the openais community. .TP retransmits_before_loss This value identifies how many token retransmits should be attempted before forming a new configuration. If this value is set, retransmit and hold will be automatically calculated from retransmits_before_loss and token. .TP join This timeout specifies in milliseconds how long to wait for join messages in the membership protocol. .TP consensus This timeout specifies in milliseconds how long to wait for consensus to be achieved before starting a new round of membership configuration. .TP merge This timeout specifies in milliseconds how long to wait before checking for a partition when no multicast traffic is being sent. If multicast traffic is being sent, the merge detection happens automatically as a function of the protocol. .TP downcheck This timeout specifies in milliseconds how long to wait before checking that a network interface is back up after it has been downed. .TP fail_to_recv_const This constant specifies how many rotations of the token without receiving any of the messages when messages should be received may occur before a new configuration is formed. .TP seqno_unchanged_const This constant specifies how many rotations of the token without any multicast traffic should occur before the merge detection timeout is started. .TP heartbeat_failures_allowed [HeartBeating mechanism] Configures the optional HeartBeating mechanism for faster failure detection. Keep in mind that engaging this mechanism in lossy networks could cause faulty loss declaration as the mechanism relies on the network for heartbeating. So as a rule of thumb use this mechanism if you require improved failure in low to medium utilized networks. This constant specifies the number of heartbeat failures the system should tolerate before declaring heartbeat failure e.g 3. Also if this value is not set or is 0 then the heartbeat mechanism is not engaged in the system and token rotation is the method of failure detection default value = 0. .TP max_network_delay [HeartBeating mechanism] This constant specifies in Milliseconds the approximate delay that your network takes to transport one packet from one machine to another. This value is to be set by system engineers and please dont change if not sure as this effects the failure detection mechanism using heartbeat. default value = 50 milliseconds. .PP Within the .B logging directive, there are four configuration options which are all optional: .TP logoutput This specifies the logging output. The choices are file, which logs to a file, stderr, which logs to stderr, and syslog which logs to the system log. It is possible to have multiple targets by including this directive with different options multiple times in the top level directive. .TP logfile If the logoutput: file directive is set, this option specifies where the log file is written to. .TP debug This specifies whether debug output is logged. This is generally a bad idea, unless there is some specific bug or problem that must be found in the executive. Set the value to on to debug, off to turn of debugging. .TP timestamp This specifies that a timestamp is placed on all log messages. .SH "FILES" .TP /etc/ais/openais.conf The openais executive configuration file. .SH "SEE ALSO" .BR openais_overview (8) .PP