corosync/man/cmap_keys.8

.\"/*
.\" * Copyright (c) 2012-2017 Red Hat, Inc.
.\" *
.\" * All rights reserved.
.\" *
.\" * Author: Jan Friesse (jfriesse@redhat.com)
.\" *
.\" * This software licensed under BSD license, the text of which follows:
.\" *
.\" * Redistribution and use in source and binary forms, with or without
.\" * modification, are permitted provided that the following conditions are met:
.\" *
.\" * - Redistributions of source code must retain the above copyright notice,
.\" *   this list of conditions and the following disclaimer.
.\" * - Redistributions in binary form must reproduce the above copyright notice,
.\" *   this list of conditions and the following disclaimer in the documentation
.\" *   and/or other materials provided with the distribution.
.\" * - Neither the name of the Red Hat, Inc. nor the names of its
.\" *   contributors may be used to endorse or promote products derived from this
.\" *   software without specific prior written permission.
.\" *
.\" * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
.\" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
.\" * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
.\" * THE POSSIBILITY OF SUCH DAMAGE.
.\" */
.TH "CMAP_KEYS" 8 "10/10/2012" "corosync Man Page" "Corosync Cluster Engine Programmer's Manual"

.SH NAME
.P
cmap_keys \- Overview of keys stored in the Configuration Map

.SH OVERVIEW
.P
There are 3 main types of keys stored in CMAP:
.PP
* Mapping of values stored in the config file.
.PP
* Runtime statistics.
.PP
* Other user created values.

In this man page, wild-cards have the usual meaning.

.SH ICMAP KEYS
These keys are in the icmap (default) map
.TP
internal_configuration.*
Internal configuration data. All keys in this prefix are read only.
It's only useful for getting a list of loaded services.

.TP
logging.*
Values read from the configuration file. It's possible to change them at runtime.
If subsystem specific configuration is needed, the key must be in the form
logging.logger_subsys.SERVICE.key, where SERVICE is upper case name of the service and
key is same as in the configuration file. All values are of string type.

.TP
nodelist.*
Values read from the configuration file. Each node element in the configuration file gets
assigned it's position starting from zero. So the first node from the config file has
nodelist.node.0. prefix. To be a valid entry, each node must have
.B ring0_addr
key.
To change the
.B nodeid
key, use a u32 data type.

Local node position is stored in
.B local_node_pos
key (RO), so it's easy to find
out nodeid/ring addresses of the local node directly from cmap.

.TP
runtime.blackbox.*
Trigger keys for storing fplay data. It's recommended that you the corosync-blackbox command
to change keys in this prefix.

.TP
runtime.config.*
Contains the values actually in use by the totem membership protocol.
Values here are either taken from the Corosync configuration file,
defaults or computed from entries in the config file. For information
on individual keys please refer to the man page
.BR corosync.conf (5).

.TP
runtime.services.*
Prefix with statistics for service engines. Each service has it's own
.B service_id
key in the prefix with the name runtime.services.SERVICE., where SERVICE is the lower case
name of the service. Inside the service prefix is the number of messages received and sent
by the corosync engine in the format runtime.services.SERVICE.EXEC_CALL.rx and
runtime.services.SERVICE.EXEC_CALL.tx, where EXEC_CALL is the internal id of the service
call (so for example 3 in cpg service is receive of multicast message from other
nodes).

.TP
runtime.totem.members.*
Prefix containing members of the totem single ring protocol. Each member
keys has format runtime.totem.members.NODEID.KEY, where key is
one of:

.B config_version
Config version of the member node.

.TP
resources.process.PID.*
Prefix created by applications using SAM with CMAP integration.
It contains the following keys:

.B recovery
Recovery policy of the process. Can be one of quit or restart.

.B poll_period
Value passed in sam_initialize as a time_interval.

.B last_updated
Last time SAM received a heartbeat from the client.

.B state
State of the client. Can be one of failed, stopped, running and waiting for quorum.

.TP
uidgid.*
Information about users/groups which are allowed to make IPC connections to
corosync. Entries loaded from configuration file are stored with
uidgid.config.* prefix and are pruned on configuration file reload. Dynamic
entries has uidgid.* prefix and a configuration file reload doesn't affect them.

.TP
quorum.cancel_wait_for_all
Tells votequorum to cancel waiting for all nodes at cluster startup. Can be used
to unblock quorum if notes are known to be down. For pcs use only.

.TP
config.reload_in_progress
This value will be set to 1 (or created) when a corosync.conf reload is started,
and set to 0 when the reload is completed. This allows interested subsystems
to do atomic reconfiguration rather than changing each key. Note that
individual add/change/delete notifications will still be sent during a reload.

.TP
config.totemconfig_reload_in_progress
This key is similar to
.B config.totemconfig_reload_in_progress
but changed after the totem config trigger is processed. It is useful (mainly)
for situations when
.B nodelist.local_node_pos
must be correctly reinstated before anything else.

.SH STATS KEYS
These keys are in the stats map. All keys in this map are read-only.
Modification tracking of individual keys is supported in the stats map, but not
prefixes. Add/Delete operations are supported on prefixes though so you can track
for new ipc connections or knet interfaces.
.TP
stats.srp.*
Prefix containing statistics about totem.
Typical key prefixes:

.B commit_entered
Number of times the processor entered COMMIT state.

.B commit_token_lost
Number of times the processor lost token in COMMIT state.

.B consensus_timeouts
How many times the processor timed out forming a consensus about membership.

.B continuous_gather
How many times the processor was not able to reach consensus.

.B firewall_enabled_or_nic_failure
Set to 1 when processor was not able to reach consensus for long time. The usual
reason is a badly configured firewall or connection failure.

.B gather_entered
Number of times the processor entered GATHER state.

.B gather_token_lost
Number of times the processor lost token in GATHER state.

.B mcast_retx
Number of retransmitted messages.

.B mcast_rx
Number of received multicast messages.

.B mcast_tx
Number of transmitted multicast messages.

.B memb_commit_token_rx
Number of received commit tokens.

.B memb_commit_token_tx
Number of transmitted commit tokens.

.B memb_join_rx
Number of received join messages.

.B memb_join_tx
Number of transmitted join messages.

.B memb_merge_detect_rx
Number of received member merge messages.

.B memb_merge_detect_tx
Number of transmitted member merge messages.

.B orf_token_rx
Number of received orf tokens.

.B orf_token_tx
Number of transmitted orf tokens.

.B recovery_entered
Number of times the processor entered recovery.

.B recovery_token_lost
Number of times the token was lost in recovery state.

.B rx_msg_dropped
Number of received messages which were dropped because they were not expected
(as example multicast message in commit state).

.B token_hold_cancel_rx
Number of received token hold cancel messages.

.B token_hold_cancel_tx
Number of transmitted token hold cancel messages.

.B mtt_rx_token
Mean transit time of token in milliseconds. In other words, time between
two consecutive token receives.

.B avg_token_workload
Average time in milliseconds of holding time of token on the current processor.

.B avg_backlog_calc
Average number of not yet sent messages on the current processor.

.TP
stats.knet.nodeX.linkY.*
Statistics about the network traffic to and from each node and link when using
tke kronosnet transport

.B connected
Whether the link is connected or not

.B up_count
Number of times this link has changed state to UP

.B down_count
Number of times this link has changed state to DOWN

.B latency_ave / latency_max / latency_max
Calculated latencies of this link. Note that if there has been no traffic
on the link then latency_min will show a very large number.

.B latency_samples
The number of samples used to calculate the latency figures, so you have
some idea of their precision.

.B rx_data_packets / tx_data_packets
The number of packets sent/received on this link

.B rx_data_bytes / tx_data_bytes
The number of bytes sent/received on this link

.B rx_pmtu_packets / tx_pmtu_packets
The number of packets sent/received by the PMTUd subsystem

.B rx_pmtu_bytes / tx_pmtu_bytes
The number of bytes sent/received by the PMTUd subsystem

.B rx_ping_packets / tx_ping_packets
The number of packets sent/received as pings

.B rx_ping_bytes / tx_ping_bytes
The number of bytes sent/received as pings

.B rx_pong_packets / tx_pong_packets
The number of packets sent/received as pongs

.B rx_pong_bytes / tx_pong_bytes
The number of bytes sent/received as pongs

.B rx_total_packets / tx_total_packets
The total number of packets sent/received. The aggregate of all of the above packet stats

.B rx_total_bytes / tx_total_bytes
The total number of bytes sent/received. The aggregate of all of the above bytes stats

.B tx_data_retries / tx_pmtu_retries / tx_ping_retries / tx_pong_retries / tx_total_retries
Number of times a transmit operation had to be retried due to the socket returning EAGAIN

.TP
stats.ipcs.*
There is information about total number of active connections from client programs
at the time the request was made.
.B active
number of closed connections during whole runtime of corosync
.B closed
Total number of connnections that have been made since corosync was started

.TP
stats.ipcs.ID.*
Each IPC connection has a unique ID. This is in the form [[serviceX:][PID:]internal_id.

Typical keys in this prefix are:

.B proc_name
process name of connected process (unavailable on some platforms)

.B dispatched
number of dispatched messages.

.B invalid_request
number of requests made by IPC which are invalid (calling non-existing call, ...).

.B name
contains short name of the IPC connection (unavailable on some platforms).

.B overload
is number of requests which were not processed because of overload.

.B queue_size
contains the number of messages in the queue waiting for send.

.B recv_retries
is the total number of interrupted receives.

.B requests
contains the number of requests made by IPC.

.B responses
is the number of responses sent to the IPC client.

.B send_retries
contains the total number of interrupted sends.

.B service_id
contains the ID of service which the IPC is connected to.

.TP
stats.clear.*
These are write-only keys used to clear the stats for various subsystems

.B totem
Clears the pg & srp totem stats.

.B knet
Clears the knet stats

.B ipc
Clears the ipc stats

.B all
Clears all of the above stats


.SH DYNAMIC CHANGE USER/GROUP PERMISSION TO USE COROSYNC IPC
Is the same as in the configuration file. eg: to add UID 500 use

.br
# corosync-cmapctl -s uidgid.uid.500 u8 1

GID is similar, so to add a GID use

.br
# corosync-cmapctl -s uidgid.gid.500 u8 1

For removal of permissions, simply delete the key

.br
# corosync-cmapctl -d uidgid.gid.500

.SH DYNAMIC ADD/REMOVE OF UDPU NODE
Eg. To add the node with address 10.34.38.108
and nodeid 3. This node is called NEW and it's not running corosync yet.

.PP
* Find a node position in the node list which is not used yet. It's recommended that you
use highest_number + 1. Let's say output of corosync-cmapctl looks like:

.br
nodelist.local_node_pos (u32) = 1
.br
nodelist.node.0.nodeid (u32) = 1
.br
nodelist.node.0.ring0_addr (str) = 10.34.38.106
.br
nodelist.node.1.nodeid (u32) = 2
.br
nodelist.node.1.ring0_addr (str) = 10.34.38.107

So next node position will be 2.
.PP
* Add all entries needed for the node on all running nodes, as:

.br
# corosync-cmapctl -s nodelist.node.2.nodeid u32 3
.br
# corosync-cmapctl -s nodelist.node.2.ring0_addr str 10.34.38.108

Always add the ring0_addr key last. The Corosync engine on all nodes should reply
with
.I notice  [TOTEM ] adding new UDPU member {10.34.38.108}
message.
.PP
* Add node information to the configuration file on all nodes so that it
will survive a restart of corosync.
.PP
* Copy and edit configuration file to the NEW node.
.PP
* Start corosync on the NEW node.

Removal of a UDPU node is a very similar, slightly reversed action, so
.PP
* Stop corosync on the OLD node.
.PP
* Remove the relevant entries from cmap on all nodes.
.PP
* Change the configuration file on all nodes.

.SH "SEE ALSO"
.BR corosync_overview (7),
.BR corosync.conf (5),
.BR corosync-cmapctl (8)