REST API Specifications ####################### Key Concepts ============ Clusters -------- Redis Labs clusters are a set of nodes, typically two or more, providing database services. Clusters are inherently multi-tenant, and a single cluster can manage multiple databases accessed through individual endpoints. Protocol and Headers ==================== JSON Requests and Responses --------------------------- The Redis Labs REST API uses the JavaScript Object Notation (JSON) for requests and responses. Some responses may have an empty body, but indicate the response with standard HTTP codes. For more information, see RFC 4627 (http://www.ietf.org/rfc/rfc4627.txt) and www.json.org. Both requests and responses may include zero or more objects. In case the request is for a single entity, the response shall return a single JSON object, or none. In case the request if for a list of entities, the response shall return a single JSON array with 0 or more elements. Requests may be delivered with some JSON object fields missing. In this case, these fields will be assigned default values (often indicating they are not in use). Request Headers --------------- The Redis Labs REST API supports the following HTTP headers: +-------------------+-----------------------------------------------------+ | Header | Supported/Required Values | +===================+=====================================================+ | Accept | application/json | +-------------------+-----------------------------------------------------+ | Content-Length | Length (in bytes) of request message. | +-------------------+-----------------------------------------------------+ | Content-Type | application/json | +-------------------+-----------------------------------------------------+ Response Headers ---------------- The Redis Labs REST API supports the following HTTP headers: +------------------+------------------------------------------------------+ | Header | Supported/Required Values | +==================+======================================================+ | Content-Type | application/json | +------------------+------------------------------------------------------+ | Content-Length | Length (in bytes) of response message. | +------------------+------------------------------------------------------+ API Versions ============ All RLEC API operations are versioned, in order to minimize the impact of backwards-incompatible API changes and to coordinate between different versions operating in parallel. Authentication ============== Authentication to RLEC API occurs via `Basic Auth `_. Provide your RLEC username and password as the basic auth credentials. All calls must be made over SSL, to port 9443. Example Request:: curl -u "demo@redislabs.com:password" https://localhost:9443/v1/bdbs Common Responses ================ The following are common responses which may be returned in some cases regardless of any specific request. +------------------+------------------------------------------------------+ | Response | Condition / Required handling | +==================+======================================================+ | 503 (Service | Contacted node is currently not a member of any | | Unavailable) | active cluster. | +------------------+------------------------------------------------------+ | 505 (HTTP | An unsupported X-API-Version was used, see API | | Version Not | Versions above. | | Supported) | | +------------------+------------------------------------------------------+ Object Attributes ================= *Note: Some objects include fields that are for internal use and are not documented. These fields might be returned as part of the object's json format and should be ignored.* .. _object-attributes: .. automodule:: cnm.apidoc bdb -> status ------------- The bdb status field is a read-only field representing database status and can have one of the following possible values: - ‘pending’: temporary status while the database is initially created Can change to: - ‘active’ - ‘creation-failed’ - ‘creation-failed’ - status if the initial database creation failed Can change to: - N/A - ‘active’ - status when the database is active and no special action it taking place Can change to: - ‘active-change-pending’ - ‘import-pending’ - ‘delete-pending’ - ‘active-change-pending’ Can change to: - ‘active’ - ‘import-pending’ - status while a dataset import is taking place Can change to: - ‘active’ - ‘delete-pending’ - status while the database is being deleted Can change to: - Not applicable - ‘recovery’ - not relevant, intended for future use. .. graphviz:: digraph bdb_status { pending -> active; pending -> "creation-failed"; active -> "active-change-pending"; active -> "import-pending"; active -> "delete-pending"; "active-change-pending" -> active; "import-pending" -> active; recovery; } bdb -> replica_sync ------------------- The bdb replica_sync field is related to the “Replica of” feature which enables enables creating a Redis database (single or multi shard) that keeps synchronizing data from another Redis database (single or multi shard). You can read more about the replica of concepts, statuses, and errors, in the `Replica of documentation `_. The bdb crdt_sync field has a similar purpose for the Redis CRDB. The field represents the replica of sync process and using it you can enable, disable or pause the process. The bdb sync field can have one of the following possible values: - ‘disabled’ - default value. Used to disable the sync process and represents that no sync is currently configured or running. Can change to: - ‘enabled’ - ‘enabled’ - used to enable the sync process and represents that the process is currently active. Can change to: - ‘stopped’ - ‘paused’ - ‘paused’ - used to pause the sync process and represents that the process is configured but is currently not executing the sync commands. Can change to: - ‘enabled’ - ‘stopped’ - ‘stopped’ - represents that an unrecoverable error occurred in the sync process that caused the sync to be stopped by the system. Can change to: - ‘enabled’ .. graphviz:: digraph bdb_replica_sync { disabled -> enabled; enabled -> stopped; enabled -> paused; paused -> enabled; paused -> stopped; stopped -> enabled; } When the sync is in ‘stopped’ or ‘paused’ state, then the the last_error field in the relevant source entry in the sync_sources “status” field holds the specific error message. bdb -> replica_sources -> “status” ---------------------------------- The status field is related to the “Replica of” feature which enables enables creating a Redis database (single or multi shard) that keeps synchronizing data from another Redis database (single or multi shard). You can read more about the replica of concepts, statuses, and errors, in the `Replica of documentation `_. The field represents the replica of sync process for a specific sync source and can have one of the following possible values: - ’out-of-sync’ - temporary status when the sync process is disconnected from source and trying reconnect Can change to: - ‘syncing’ - ‘syncing’ - temporary status when the sync process is running the initial sync and valid until the initial sync is done Can change to: - ‘in-sync’ - ’out-of-sync’ - ‘in-sync’ - initial sync process finished successfully and currently new commands are syncing on a regular basis. Can change to: - ‘syncing’ - ’out-of-sync’ .. graphviz:: digraph bdb_replica_sources_status { "out-of-sync" -> syncing; syncing -> "in-sync"; syncing -> "out-of-sync"; "in-sync" -> syncing; "in-sync" -> "out-of-sync"; } .. _backup-location: bdb -> backup_location, bdbs/(int: uid)/actions/export -> export_location ========================================================================= You can backup or export a database's dataset to one of the following location types: - FTP/S - Amazon S3 - Microsoft Azure Storage - Google Cloud Storage - OpenStack Object Storage ("Swift") - SFTP - NAS/Local Storage Each of these location types requires a different set of parameters to be sent. First you need to define the location type by setting the "type" (string) field to one of the following possible values:. - "url" - for FTP/S - "s3" - for Amazon S3 - "abs" - for Microsoft Azure Storage - "gs" - for Google Cloud Storage - "swift" - OpenStack Object Storage ("Swift") - "sftp" - for SFTP - "mount_point" - for NAS/Local Storage Then, depending on the option you are using there is a different set of values you need to provide: - FTP - "url" (string) - a uri representing FTP/S location with the following format: ftp://user:password@host:port/path/. The user and password can be omitted if not needed. - AWS S3 - "bucket_name" (string) - S3 bucket name - "subdir" (string) - (Optional) Path to the backup directory in the S3 bucket - "access_key_id" (string) - The AWS Access Key ID with access to the bucket - "secret_access_key" (string) - The AWS Secret Access Key that matches the Access Key ID - Azure Blob Storage - "container" (string) - Blob Storage container name - "subdir" (string) - (Optional) Path to the backup directory in the Blob Storage container - "account_name" (string) - Storage account name with access to the container - "account_key" (string) - Access key for the storage account - Google Cloud Storage - "bucket_name" (string) - Cloud Storage bucket name - "subdir" (string) - (Optional) Path to the backup directory in the Cloud Storage bucket - "client_id" (string) - Cloud Storage client ID with access to the Cloud Storage bucket - "client_email" (string) - Email address for the Cloud Storage client ID - "private_key_id" (string) - Cloud Storage private key ID with access to the Cloud Storage bucket - "private_key" (string) - Cloud Storage private key that matches the private key ID - Swift - "auth_url" (string) - Swift service authentication URL. - "user" (string) - Swift service username to use to access the storage. - "key" (string) - Swift service key corresponding to the username to use to access the storage. - "container" (string) - Swift object store container name to backup to. - "prefix" (string) - (Optional) Swift path used a prefix for the file name for the backup files to be created. - SFTP - "sftp_url" (string) - SFTP URL in the format 'sftp://user:password@host[:port][/path/]'). The default port number is 22 and the default path is '/'. - "key" (string) - (Optional) SSH private key to secure the SFTP server connection. If you do not specify an SSH private key, the auto-generated private key of the cluster is used, and you must add the SSH public key of the cluster to the SFTP server configuration. - NAS/Local Storage - "path" (string) - Path to the local mount point. You must create the mount point on all nodes, and the redislabs:redislabs user must have read and write permissions on the local mount point. .. _import-sources: bdbs/(int: uid)/actions/import -> dataset_import_sources ======================================================== You can import data to a database from one of the following location types: - HTTP/S - FTP - Amazon S3 - Microsoft Azure Storage - Google Cloud Storage - OpenStack Object Storage ("Swift") - SFTP - NAS/Local Storage Each of these location types requires a different set of parameters to be sent. The file to import from should be in the RDB format. It can also be in a compressed (gz) RDB file. You can import data to a database from multiple files by supplying an array of the object described below. First you need to define the location type by setting the "type" (string) field to one of the following possible values:. - "url" - for FTP/S - "s3" - for Amazon S3 - "abs" - for Microsoft Azure Storage - "gs" - for Google Cloud Storage - "swift" - OpenStack Object Storage ("Swift") - "sftp" - for SFTP - "mount_point" - for NAS/Local Storage Then, depending on the option you are using there is a different set of values you need to provide: - FTP - "url" (string) - a uri representing FTP/S location with the following format: ftp://user:password@host:port/path/. The user and password can be omitted if not needed. - AWS S3 - "bucket_name" (string) - S3 bucket name - "subdir" (string) - (Optional) Path to the backup directory in the S3 bucket - "access_key_id" (string) - The AWS Access Key ID with access to the bucket - "secret_access_key" (string) - The AWS Secret Access that matches the Access Key ID - Azure Blob Storage - "container" (string) - Blob Storage container name - "subdir" (string) - (Optional) Path to the backup directory in the Blob Storage container - "account_name" (string) - Storage account name with access to the container - "account_key" (string) - Access key for the storage account - Google Cloud Storage - "bucket_name" (string) - Cloud Storage bucket name - "subdir" (string) - (Optional) Path to the backup directory in the Cloud Storage bucket - "client_id" (string) - Cloud Storage client ID with access to the Cloud Storage bucket - "client_email" (string) - Email address for the Cloud Storage client ID - "private_key_id" (string) - Cloud Storage private key ID with access to the Cloud Storage bucket - "private_key" (string) - Private key for the Cloud Storage matching the private key ID - Swift - "auth_url" (string) - Swift service authentication URL. - "user" (string) - Swift service username to use to access the storage. - "key" (string) - Swift service key corresponding to the username to use to access the storage. - "container" (string) - Swift object store container name holding the file to be imported. - "objname" (string) - Swift object name (file name) of the file to be imported. - "prefix" (string) - (Optional) Swift path used a prefix to the object name for the file be imported. - SFTP - "sftp_url" (string) - SFTP URL in the format of 'sftp://user:password@host[:port]/path/filename.rdb'). The default port number is 22 and the default path is '/'. - "key" (string) - (Optional) SSH private key to secure the SFTP server connection. If you do not specify an SSH private key, the auto-generated private key of the cluster is used, and you must add the SSH public key of the cluster to the SFTP server configuration. - NAS/Local Storage - "path" (string) - Path to the locally mounted file name to import. You must create the mount point on all nodes, and the redislabs:redislabs user must have read permissions on the local mount point. Statistics ========== Statistics Overview ------------------- Statistics returned from API calls will always have the following fields: * **interval** – a string the time interval represented in this result. This can be one of the following values: 1sec / 10sec / 5min / 15min / 1hour / 12hour / 1week. * **stime** – a timestamp representing the beginning of the interval. Format is: "2015-05-27T12:00:00Z". * **etime** – a timestamp representing the end of the interval. Format is: "2015-05-27T12:00:00Z". The difference between the stime and the etime is equal to the interval. In addition, there is a set of fields representing the values of the different metrics for this interval and object. The set of metrics relevant to each object is different, and listed below for each object type. Optionally, you can pass an stime and/or an etime to the different GET stats calls. When provided, the returned results will fall within the range provided, meaning no earlier than the provided stime, and no later than the provided etime. For each interval type (1sec / 10sec / 5min / 15min / 1hour / 12hour / 1week) there is a maximum number of samples (X) kept in the system, see the exact maximum number of values available for each interval type in the table below. These X samples are kept from the most current sample back. The actual number of samples returned by a stats GET call can vary depending on the number of samples available in the system, and the number of samples that fall between stime and the etime, if provided. For example, if an object (Cluster / Node / DB / Shard) was just created, not all samples will be available yet, depending on how much time elapsed since creation. Or, if the stime and etime provided are pretty close, only samples that fall within this range, out of all the available samples, will be returned. An exception to the above is that in order to reduce load generated by stats collection, for relatively inactive DBs or Shards (less than 5 ops/sec), 1sec stats are not collected at a 1sec interval. For those inactive DBs or shards 1sec stats are collected at a reduced interval of once every 2 to 5 seconds, but the same number of maximum number of samples (X) is kept. **Note**: There are certain statistics that are returned from the API that are intended for internal use. These statistics are not documented below and should be ignored. **Note**: There are certain statistics that are not always returned by the API, but are returned only when they are relevant, see more details in the specific statistics below. You should not expect all statistics to always be returned. Maximum number of samples per interval -------------------------------------- +----------+---------------------------------------+ | Interval | Maximum available number of samples | +==========+=======================================+ | 1sec | 10 | +----------+---------------------------------------+ | 10sec | 30 | +----------+---------------------------------------+ | 5min | 12 | +----------+---------------------------------------+ | 15min | 96 | +----------+---------------------------------------+ | 1hour | 168 | +----------+---------------------------------------+ | 12hour | 62 | +----------+---------------------------------------+ | 1week | 53 | +----------+---------------------------------------+ Object Metrics -------------- **Cluster Metrics** +--------------------------+--------------------------------------------------+-----------------+ | Metric Name | Description | Type | +==========================+==================================================+=================+ | free_memory | sum of free memory in all cluster nodes (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | available_memory | sum of available_memory in all nodes (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | provisional_memory | sum of provisional_memory in all nodes (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | available_flash | sum of available_flash in all nodes (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | provisional_flash | sum of provisional_flash in all nodes (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | cpu_user | cpu time portion spent by users-pace processes | | | | on cluster, the value is weighted between all |float | | | nodes based on number of cores in each node | | | | (0-1, multiply by 100 to get percent) | | +--------------------------+--------------------------------------------------+-----------------+ | cpu_system | cpu time portion spent in kernel on cluster, the | | | | value is weighted between all nodes based on |float | | | number of cores in each node (0-1, multiply by | | | | 100 to get percent) | | +--------------------------+--------------------------------------------------+-----------------+ | cpu_idle | cpu idle time portion, , the value is weighted | | | | between all nodes based on number of cores in |float | | | each node (0-1, multiply by 100 to get percent) | | +--------------------------+--------------------------------------------------+-----------------+ | ephemeral_storage_avail | sum of disk space available to RLEC processes on | | | | configured ephemeral disk on all cluster nodes |float | | | (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | persistent_storage_avail | sum of disk space available to RLEC processes on | | | | configured persistent disk on all cluster nodes |float | | | (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | ephemeral_storage_free | sum of free disk space on configured ephemeral |float | | | disk on all cluster nodes (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | persistent_storage_free | sum of free disk space on configured persistent |float | | | disk on all cluster nodes (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | total_req | request rate handled by all endpoints on cluster |float | | | (ops/sec) | | +--------------------------+--------------------------------------------------+-----------------+ | avg_latency | average latency of requests handled by all | | | | cluster endpoints (micro-sec); returned only |float | | | when there is traffic | | +--------------------------+--------------------------------------------------+-----------------+ | conns | total number of clients connected to all |float | | | cluster endpoints | | +--------------------------+--------------------------------------------------+-----------------+ | ingress_bytes | sum of rate of incoming network traffic on all |float | | | cluster nodes (bytes/sec) | | +--------------------------+--------------------------------------------------+-----------------+ | egress_bytes | sum of rate of outgoing network traffic on all |float | | | cluster nodes (bytes/sec) | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_kv_ops | rate of value read/write operations against | | | | back-end flash for all shards which are part of |float | | | a flash based DB (BigRedis) in cluster (ops/sec);| | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_iops | rate of i/o operations against back-end flash | | | | for all shards which are part of a flash based |float | | | DB (BigRedis) in cluster (ops/sec); | | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_throughput | throughput i/o operations against back-end flash | | | | for all shards which are part of a flash based |float | | | DB (BigRedis) in cluster (bytes/sec); | | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_free | sum of free space of back-end flash (used by | | | | flash DB's [BigRedis]) on all cluster nodes |float | | | (bytes); returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ **Node Metrics** +--------------------------+--------------------------------------------------+-----------------+ | Metric Name | Description | Type | +==========================+==================================================+=================+ | free_memory | free memory in node (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | available_memory | available ram in node (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | provisional_memory | amount of ram available for new shards on this |float | | | node, taking into account overbooking, max redis | | | | servers, reserved memory and provision and | | | | migration thresholds (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | available_flash | available flash in node (bytes) |float | +--------------------------+--------------------------------------------------+-----------------+ | provisional_flash | amount of flash available for new shards on this |float | | | node, taking into account overbooking, max redis | | | | servers, reserved flash and provision and | | | | migration thresholds (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | cpu_user | cpu time portion spent by users-pace processes |float | | | (0-1, multiply by 100 to get percent) | | +--------------------------+--------------------------------------------------+-----------------+ | cpu_system | cpu time portion spent in kernel (0-1, multiply |float | | | by 100 to get percent) | | +--------------------------+--------------------------------------------------+-----------------+ | cpu_idle | cpu idle time portion (0-1, multiply by 100 to |float | | | get percent) | | +--------------------------+--------------------------------------------------+-----------------+ | ephemeral_storage_avail | disk space available to RLEC processes on |float | | | configured ephemeral disk (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | persistent_storage_avail | disk space available to RLEC processes on |float | | | configured persistent disk (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | ephemeral_storage_free | free disk space on configured ephemeral disk |float | | | (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | persistent_storage_free | free disk space on configured persistent disk |float | | | (bytes) | | +--------------------------+--------------------------------------------------+-----------------+ | total_req | request rate handled by endpoints on node |float | | | (ops/sec) | | +--------------------------+--------------------------------------------------+-----------------+ | avg_latency | average latency of requests handled by endpoints | | | | on node (micro-sec); |float | | | returned only when there is traffic | | +--------------------------+--------------------------------------------------+-----------------+ | conns | number of clients connected to endpoints on node |float | +--------------------------+--------------------------------------------------+-----------------+ | ingress_bytes | rate of incoming network traffic to node |float | | | (bytes/sec) | | +--------------------------+--------------------------------------------------+-----------------+ | egress_bytes | rate of outgoing network traffic to node |float | | | (bytes/sec) | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_kv_ops | rate of value read/write operations against | | | | back-end flash for all shards which are part of |float | | | a flash based DB (BigRedis) on node (ops/sec); | | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_iops | rate of i/o operations against back-end flash | | | | for all shards which are part of a flash based |float | | | DB (BigRedis) on node (ops/sec); | | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_throughput | throughput of i/o operations against back-end | | | | flash for all shards which are part of a flash |float | | | based DB (BigRedis) on node (bytes/sec); | | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | bigstore_free | free space of back-end flash (used by flash DB's | | | | [BigRedis]) (bytes) |float | | | returned only when BigRedis is enabled | | +--------------------------+--------------------------------------------------+-----------------+ | cur_aof_rewrites | number of aof rewrites that are currently |float | | | performed by shards on this node | | +--------------------------+--------------------------------------------------+-----------------+ **DB Metrics** +----------------------------+--------------------------------------------------+-----------------+ | Metric Name | Description | Type | +============================+==================================================+=================+ | no_of_keys | number of keys in DB |float | +----------------------------+--------------------------------------------------+-----------------+ | no_of_expires | number of volatile keys in the DB |float | +----------------------------+--------------------------------------------------+-----------------+ | evicted_objects | rate of key evictions from DB (evictions/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | expired_objects | rate keys expired in DB (expirations/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | instantaneous_ops_per_sec | request rate handled by all shards of DB |float | | | (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | read_hits | rate of read operations accessing an existing |float | | | key (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | read_misses | rate of read operations accessing a non-existing |float | | | key (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | write_hits | rate of write operations accessing an existing |float | | | key (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | write_misses | rate of write operations accessing a non- |float | | | existing key (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | pubsub_channels | Count the pub/sub channels with subscribed |float | | | clients. | | +----------------------------+--------------------------------------------------+-----------------+ | pubsub_patterns | Count the pub/sub patterns with subscribed |float | | | clients. | | +----------------------------+--------------------------------------------------+-----------------+ | mem_not_counted_for_evict | Portion of used_memory (in bytes) that's not |float | | | counted for eviction and OOM error. | | +----------------------------+--------------------------------------------------+-----------------+ | mem_frag_ratio | RAM fragmentation ratio (RSS / allocated RAM) |float | +----------------------------+--------------------------------------------------+-----------------+ | disk_frag_ratio | Flash fragmentation ratio (used / required) |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_kv_ops | rate of value read/write/del operations against | | | | back-end flash for all shards of DB (BigRedis) |float | | | (key access / sec); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_iops | rate of i/o operations against back-end flash | | | | for all shards of DB (BigRedis) (ops/sec); |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_throughput | throughput of i/o operations against back-end | | | | flash for all shards of DB (BigRedis) (bytes/sec)|float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_fetch_ram | rate of key reads / updates for keys that happen | | | | to be in ram (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+-------------------------------------------------+-----------------+ | big_fetch_flash | rate of key reads / updates for keys that happen | | | | to be on flash (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_write_ram | rate of key writes for keys that happen | | | | to be in ram (bigredis) (key access / sec); |float | | | this includes write misses (new keys created); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_write_flash | rate of key writes for keys that happen | | | | to be on flash (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_del_ram | rate of key deletes for keys that happen | | | | to be in ram (bigredis) (key access / sec); |float | | | this includes write misses (new keys created); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_del_flash | rate of key deletes for keys that happen | | | | to be on flash (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_io_ratio_redis | rate of redis operations on keys. can be used | | | | to compute ratio of io operations) (key access / |float | | | sec); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | big_io_ratio_flash | rate of key operations on flash. can be used to | | | | compute ratio of io operations (key access / |float | | | sec); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_reads | rate of key reads from flash (key access / sec); |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_writes | rate of key writes from flash (key access / sec);|float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_dels | rate of key deletions from flash (key access / |float | | | sec); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_read_bytes | throughput of i/o read operations against | | | | back-end flash for all shards of DB (BigRedis) |float | | | (bytes/sec); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_write_bytes | throughput of i/o write operations against | | | | back-end flash for all shards of DB (BigRedis) |float | | | (bytes/sec); | | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | ram_overhead | Non values RAM overhead (bigredis) (bytes) |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_objs_ram | value count in ram (bigredis) |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | bigstore_objs_flash | value count on flash (bigredis) |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | used_memory | memory used by db (in bigredis this includes |float | | | flash) (bytes) | | +----------------------------+--------------------------------------------------+-----------------+ | mem_size_lua | redis lua scripting heap size (bytes) |float | +----------------------------+--------------------------------------------------+-----------------+ | used_ram | RAM used by db (bigredis) (bytes) |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | used_bigstore | Flash used by db (bigredis) (bytes) |float | | | returned only when BigRedis is enabled | | +----------------------------+--------------------------------------------------+-----------------+ | last_req_time | last request time received to DB (ISO format |date, | | | 2015-07-05T22:16:18Z")"; |ISO_8601 format | | | when not available returns 1/1/1970 | | +----------------------------+--------------------------------------------------+-----------------+ | last_res_time | last response time received from DB (ISO format |date, | | | 2015-07-05T22:16:18Z")"; |ISO_8601 format | | | when not available returns 1/1/1970 | | +----------------------------+--------------------------------------------------+-----------------+ | conns | number of client connections to DB's endpoints. |float | +----------------------------+--------------------------------------------------+-----------------+ | total_connections_received | rate of new client connections to DB |float | | | (connections/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | ingress_bytes | rate of incoming network traffic to DB's |float | | | endpoint (bytes/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | egress_bytes | rate of outgoing network traffic to DB's |float | | | endpoint (bytes/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | read_req | rate of read requests on DB (ops/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | read_res | rate of read responses on DB (ops/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | write_req | rate of write requests on DB (ops/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | write_res | rate of write responses on DB (ops/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | other_req | rate of other (non read/write) requests on DB |float | | | (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | other_res | rate of other (non read/write) responses on DB |float | | | (ops/sec) | | +----------------------------+--------------------------------------------------+-----------------+ | total_req | rate of all requests on DB (ops/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | total_res | rate of all responses on DB (ops/sec) |float | +----------------------------+--------------------------------------------------+-----------------+ | monitor_sessions_count | number of client connected in monitor mode to |float | | | the DB | | +----------------------------+--------------------------------------------------+-----------------+ | avg_read_latency | average latency of read operations (microsecond);|float | | | returned only when there is traffic | | +----------------------------+--------------------------------------------------+-----------------+ | avg_write_latency | average latency of write operations | | | | (microsecond); returned only when |float | | | there is traffic | | +----------------------------+--------------------------------------------------+-----------------+ | avg_other_latency | average latency of other (non read/write) | | | | operations (microsecond); returned only when |float | | | there is traffic | | +----------------------------+--------------------------------------------------+-----------------+ | avg_latency | average latency of operations on the DB | | | | (microsecond); returned only when |float | | | there is traffic | | +----------------------------+--------------------------------------------------+-----------------+ | shard_cpu_user | % cores utilization in user mode for the |float | | | redis shard process | | +----------------------------+--------------------------------------------------+-----------------+ | shard_cpu_system | % cores utilization in system mode for all |float | | | redis shard processes of this database | | +----------------------------+--------------------------------------------------+-----------------+ | main_thread_cpu_user | % cores utilization in user mode for all |float | | | redis shard main threads of this database | | +----------------------------+--------------------------------------------------+-----------------+ | main_thread_cpu_system | % cores utilization in system mode for all |float | | | redis shard main threads of this database | | +----------------------------+--------------------------------------------------+-----------------+ | fork_cpu_user | % cores utilization in user mode for all |float | | | redis shard fork child processes of this database| | +----------------------------+--------------------------------------------------+-----------------+ | fork_cpu_system | % cores utilization in system mode for all |float | | | redis shard fork child processes of this database| | +----------------------------+--------------------------------------------------+-----------------+ **Shard Metrics** +-----------------------------+--------------------------------------------------+-----------------+ | Metric Name | Description | Type | +=============================+==================================================+=================+ | aof_rewrite_inprog | The number of simultaneous AOF rewrites that are |float | | | in progress. | | +-----------------------------+--------------------------------------------------+-----------------+ | avg_ttl | Estimated average time to live of a random key |float | | | (msec). | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_iops | rate of i/o operations against back-end flash | | | | for all shards of DB (BigRedis) (ops/sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_throughput | throughput of i/o operations against back-end | | | | flash for all shards of DB (BigRedis) (bytes/sec)|float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_kv_ops | rate of value read/write/del operations against | | | | back-end flash for all shards of DB (BigRedis) |float | | | (key access / sec); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_objs_flash | key count on flash (bigredis); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_objs_ram | key count in ram (bigredis); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_fetch_ram | rate of key reads / updates for keys that happen | | | | to be in ram (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_fetch_flash | rate of key reads / updates for keys that happen | | | | to be on flash (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_write_ram | rate of key writes for keys that happen | | | | to be in ram (bigredis) (key access / sec); |float | | | this includes write misses (new keys created); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_write_flash | rate of key writes for keys that happen | | | | to be on flash (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_del_ram | rate of key deletes for keys that happen | | | | to be in ram (bigredis) (key access / sec); |float | | | this includes write misses (new keys created); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_del_flash | rate of key deletes for keys that happen | | | | to be on flash (bigredis) (key access / sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_io_ratio_redis | rate of redis operations on keys. can be used | | | | to compute ratio of io operations) (key access / |float | | | sec); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | big_io_ratio_flash | rate of key operations on flash. can be used to | | | | compute ratio of io operations (key access / |float | | | sec); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_reads | rate of key reads from flash (key access / sec); |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_writes | rate of key writes from flash (key access / sec);|float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_dels | rate of key deletions from flash (key access / |float | | | sec); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_read_bytes | throughput of i/o read operations against | | | | back-end flash for all shards of DB (BigRedis) |float | | | (bytes/sec); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | bigstore_io_write_bytes | throughput of i/o write operations against | | | | back-end flash for all shards of DB (BigRedis) |float | | | (bytes/sec); | | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | blocked_clients | Count the clients waiting on a blocking call. |float | +-----------------------------+--------------------------------------------------+-----------------+ | connected_clients | number of client connections to the specific |float | | | shard. | | +-----------------------------+--------------------------------------------------+-----------------+ | evicted_objects | rate of key evictions from DB (evictions/sec) |float | +-----------------------------+--------------------------------------------------+-----------------+ | expired_objects | rate keys expired in DB (expirations/sec) |float | +-----------------------------+--------------------------------------------------+-----------------+ | last_save_time | Time of the last RDB save. |float | +-----------------------------+--------------------------------------------------+-----------------+ | used_memory | memory used by shard (in bigredis this includes |float | | | flash) (bytes) | | +-----------------------------+--------------------------------------------------+-----------------+ | mem_size_lua | redis lua scripting heap size (bytes) |float | +-----------------------------+--------------------------------------------------+-----------------+ | used_memory_peak | The largest amount of memory used by this shard |float | | | (bytes). | | +-----------------------------+--------------------------------------------------+-----------------+ | used_memory_rss | resident set size of this shard (bytes) |float | +-----------------------------+--------------------------------------------------+-----------------+ | no_of_keys | number of keys in DB |float | +-----------------------------+--------------------------------------------------+-----------------+ | no_of_expires | number of volatile keys on the shard |float | +-----------------------------+--------------------------------------------------+-----------------+ | pubsub_channels | Count the pub/sub channels with subscribed |float | | | clients. | | +-----------------------------+--------------------------------------------------+-----------------+ | pubsub_patterns | Count the pub/sub patterns with subscribed |float | | | clients. | | +-----------------------------+--------------------------------------------------+-----------------+ | rdb_changes_since_last_save | Count changes since last RDB save. |float | +-----------------------------+--------------------------------------------------+-----------------+ | read_hits | rate of read operations accessing an existing |float | | | key (ops/sec) | | +-----------------------------+--------------------------------------------------+-----------------+ | read_misses | rate of read operations accessing a non-existing |float | | | key (ops/sec) | | +-----------------------------+--------------------------------------------------+-----------------+ | total_req | rate of operations on DB (ops/sec) |float | +-----------------------------+--------------------------------------------------+-----------------+ | write_hits | rate of write operations accessing an existing |float | | | key (ops/sec) | | +-----------------------------+--------------------------------------------------+-----------------+ | write_misses | rate of write operations accessing a non- |float | | | existing key (ops/sec) | | +-----------------------------+--------------------------------------------------+-----------------+ | mem_not_counted_for_evict | Portion of used_memory (in bytes) that's not |float | | | counted for eviction and OOM error. | | +-----------------------------+--------------------------------------------------+-----------------+ | mem_frag_ratio | RAM fragmentation ratio (RSS / allocated RAM) |float | +-----------------------------+--------------------------------------------------+-----------------+ | disk_frag_ratio | Flash fragmentation ratio (used / required) |float | | | returned only when BigRedis is enabled | | +-----------------------------+--------------------------------------------------+-----------------+ | shard_cpu_user | % cores utilization in user mode for the |float | | | redis shard process | | +-----------------------------+--------------------------------------------------+-----------------+ | shard_cpu_system | % cores utilization in system mode for the |float | | | redis shard process | | +-----------------------------+--------------------------------------------------+-----------------+ | main_thread_cpu_user | % cores utilization in user mode for the |float | | | redis shard main thread | | +-----------------------------+--------------------------------------------------+-----------------+ | main_thread_cpu_system | % cores utilization in system mode for the |float | | | redis shard main thread | | +-----------------------------+--------------------------------------------------+-----------------+ | fork_cpu_user | % cores utilization in user mode for the |float | | | redis shard fork child process | | +-----------------------------+--------------------------------------------------+-----------------+ | fork_cpu_system | % cores utilization in system mode for the |float | | | redis shard fork child process | | +-----------------------------+--------------------------------------------------+-----------------+ Actions ======= .. _rest-api-actions: The cluster provides support for invoking general maintenance actions such as rebalance cluster, take a node offline by moving all of its entities to other nodes, etc. Actions are implemented in the cluster as tasks. Every task has a a unique `task_id` which is assigned by the cluster, a task name which describes the task, status, and additional task-specific parameters. The REST API provides a simplified interface that allows callers to invoke actions and query their status without explicitly `task_id` handles. This is described in detail on a per-request basis. Action life-cycle is based on the following status and status transitions: .. graphviz:: digraph status { queued -> cancelled; queued -> starting; starting -> running; running -> completed; running -> cancelling -> cancelled; running -> failed; } +------------------+------------------------------------------------------+ | Field | Description | +==================+======================================================+ | status | - `queued`: operation was requested and is waiting | | | to start. | | | - `starting`: operating was picked up from the queue | | | and processing begins. | | | - `running`: operation is currently executing. | | | - `cancelling`: operation cancel was requested and | | | is in progress. | | | - `cancelled`: operation was cancelled. | | | - `completed`: operation has completed. | | | - `failed`: operation has failed. | +------------------+------------------------------------------------------+ | progress | Integer (0-100) representing percentage completed. | +------------------+------------------------------------------------------+ When a task is failed, the `error_code` and `error_message` fields may be present in order to describe the encountered error. The following `error_code` field value are used: +---------------------------+------------------------------------------------------+ | Value | Description | +===========================+======================================================+ | `internal_error` | An internal error that cannot be mapped to a more | | | precise error code has been encountered. | +---------------------------+------------------------------------------------------+ | `insufficient_resources` | The cluster does not have sufficient resources to | | | complete the required operation. | +---------------------------+------------------------------------------------------+ Alerts ====== .. _rest-api-alerts: The cluster supports getting, configuring and enabling various alerts. Alerts can go on or off depending on the state of the cluster. And alert is bound to a cluster object (such as a bdb or node). Alerts objects are retrieved and in JSON format. The following table describes the Alert json object: +-----------------+-----------------------------------------------------------------+----------+ | Field | Description | Writable | +=================+=================================================================+==========+ | `enabled` | true if alert is enabled. | x | +-----------------+-----------------------------------------------------------------+----------+ | `state` | true if alert is currently triggered. | | +-----------------+-----------------------------------------------------------------+----------+ | `threshold` | String representing an alert threshold when applicable. | x | +-----------------+-----------------------------------------------------------------+----------+ | `change_time` | Timestamp when alert state last changed. | | +-----------------+-----------------------------------------------------------------+----------+ | `severity` | The alert's severity: DEBUG, INFO, WARNING, ERROR, CRITICAL | | +-----------------+-----------------------------------------------------------------+----------+ | `change_value` | object containing data relevant to the evaluation time when the | | | | alert went on/off (thresholds/sampled values/etc.). | | +-----------------+-----------------------------------------------------------------+----------+ Supported Requests ================== .. autoflask:: cnm.http_services.cluster_api.cnm_http:http_app :undoc-static: :endpoints: certificate_api.get_cluster_certificates, certificate_api.update_cluster_cert, certificate_api.delete_cluster_certificate, certificate_api.rotate_internal_cluster_certs, password_api.reset_user_password, password_api.add_user_password, password_api.delete_user_password, bdb_api.get_all_bdbs, bdb_api.get_bdb, bdb_api.create_bdb, bdb_api.update_bdb, bdb_api.update_bdb_with_action, bdb_api.add_bdb_password, bdb_api.delete_bdb_password, bdb_api.reset_bdb_password, bdb_api.delete_bdb, bdb_api.bdb_export, bdb_api.bdb_reset_export_status, bdb_api.bdb_reset_backup_status, bdb_api.bdb_import, bdb_api.recover_bdb, bdb_api.bdb_get_recovery_plan, bdb_api.post_db_command, bdb_api.get_optimized_shards_placement, bdb_api.bdb_upgrade, bdb_api_v2.create_bdb_with_params, bdb_api_v2.update_bdb_with_params, alert_api.get_bdbs_alerts, alert_api.get_bdb_alerts, alert_api.get_bdb_alert, alert_api.update_bdb_alerts, alert_api.get_nodes_alerts, alert_api.get_node_alerts, alert_api.get_node_alert, alert_api.get_cluster_alerts, alert_api.get_cluster_alert, alert_api.get_bdbs_crdt_sources_alerts, alert_api.get_bdb_crdt_sources_alerts, alert_api.get_bdb_crdt_source_alerts, alert_api.get_bdb_crdt_source_alert, alert_api.get_bdbs_replica_sources_alerts, alert_api.get_bdb_replica_sources_alerts, alert_api.get_bdb_replica_source_alerts, alert_api.get_bdb_replica_source_alert, node_api.get_all_nodes, node_api.get_node, stats_api.get_bdb_stats, stats_api.get_all_bdbs_stats, stats_api.get_bdb_last_stats, stats_api.get_all_bdbs_last_stats, stats_api.get_shard_stats, stats_api.get_all_shards_stats, stats_api.get_node_stats, stats_api.get_all_nodes_stats, stats_api.get_cluster_stats, bootstrap_api.get_bootstrap, bootstrap_api.post_bootstrap, cluster_api.get_cluster, cluster_api.update_cluster, cluster_api.post_cluster_action, cluster_api.get_cluster_action, cluster_api.delete_cluster_actions, cluster_api.get_services_configuration, cluster_api.update_services_configuration, api_v1.get_log, api_v1.get_jsonschema, bdb_api.bdb_reset_import_status, node_api.post_node_action, node_api.get_node_action, node_api.get_all_node_actions, node_api.delete_node_actions, license_api.get_license, license_api.put_license, node_api.update_node, bootstrap_api.post_bootstrap_validate, debuginfo_api.get_aggregate_debuginfo, debuginfo_api.get_aggregate_debuginfo_for_bdb, debuginfo_api.get_node_debuginfo, debuginfo_api.get_node_debuginfo_for_bdb, stats_api.get_cluster_last_stats, stats_api.get_shard_last_stats, stats_api.get_all_shards_last_stats, stats_api.get_node_last_stats, stats_api.get_all_nodes_last_stats, stats_api.get_all_endpoint_stats, stats_api.get_bdb_peer_stats, stats_api.get_bdb_all_peer_stats, stats_api.get_bdb_sync_source_stats, stats_api.get_bdb_all_sync_source_stats, role_api.create_role, role_api.update_role, role_api.delete_role, role_api.get_role, role_api.get_roles, redis_acl_api.create_redis_acl, redis_acl_api.update_redis_acl, redis_acl_api.delete_redis_acl, redis_acl_api.get_redis_acl, redis_acl_api.get_all_redis_acl, api_v1.create_user, api_v1.update_user, api_v1.delete_user, api_v1.get_user, api_v1.get_users, api_v1.authorize_user, api_v1.refresh_jwt_token, suffix_api.get_all_suffixes, suffix_api.get_suffix, module_api.list_modules, module_api.get_module, module_api.remove_module, module_api.list_module_capabilities, module_api.upload_module, module_api.bdb_upgrade_module, module_api.bdb_configure_module_in_runtime, ldap_api.update_ldap_config, ldap_api.get_ldap_config, ldap_api.delete_ldap_config, ldap_mapping_api.create_ldap_mapping, ldap_mapping_api.get_ldap_mapping, ldap_mapping_api.get_ldap_mappings, ldap_mapping_api.update_ldap_mapping, ldap_mapping_api.delete_ldap_mapping Roles ===== .. _users-roles: Each user in the cluster has a role assigned to it, which defines the permissions the user holds. There are 5 types of roles: * **Db_viewer** - Can view bdbs related info. * **Db_member** - Can create and modify bdbs, and view bdbs related info. * **Cluster_viewer** - Can view cluster and bdbs related info. * **Cluster_member** - Can create and modify bdbs, and view bdbs and cluster related info. * **Admin** - Can modify all view and modify all elements of the cluster. .. automodule:: cnm.roles_doc