CockroachDB v19.2 is a required Regular Release.
Refer to Major release types before installing or upgrading for release timing and support details.
On this page, you can read about changes and find downloads for all production and testing releases of CockroachDB v19.2
For key feature enhancements in v19.2 and other upgrade considerations, refer to the notes for v19.2.0.
For details about release types, naming, and licensing, refer to the Releases page.
Be sure to also review the Release Support Policy.
After downloading a supported CockroachDB binary, learn how to install CockroachDB or upgrade your cluster.
Get future release notes emailed to you:
v19.2.12
Release Date: January 19, 2021
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Bug fixes
- Fixed a panic in protobuf decoding. #58876
- Fixed a bug where the
agefunction did not normalize the duration for large day orH:M:Svalues in the same way PostgreSQL does. #55528
Contributors
This release includes 5 merged PRs by 5 authors. We would like to thank the following contributors from the CockroachDB community:
- kev (first-time contributor)
v19.2.11
Release Date: October 12, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
General changes
- Links to unimplemented issues that are returned in error messages now use the CockroachLabs redirect/short-link server. #53955
SQL language changes
Bug fixes
- The Cluster Node Map and the Debug Page that lists cluster localities are now available again to non-admin users. This fixes a bug introduced in v19.2.3. #53332
- Fixed a bug that could cause backfills to fail with a vague
TransactionStatuserror as they are completing. #53765 - Fixed a bug that could cause CockroachDB to crash when evaluating queries with window functions with the
GROUPSmode of framing whenOFFSET FOLLOWINGboundary was used and when the offset was a very large value such that it could result in an integer overflow. #53758 - Fixed a bug that could cause CockroachDB to crash when using a
RANGE-mode window function with an offset in some cases (e.g,OVER (PARTITION BY b ORDER BY a RANGE 1 PRECEDING). #54074 - Adding a
NOT NULLconstraint no longer fails with the errorcheck ... does not exist. #54290 - A
NOT VALIDforeign key constraint no longer fails with the internal errortable descriptor is not valid: duplicate constraint name. #54290 - Fixed a bug that could cause columns used in an index containing foreign key columns as a prefix to lead to all of the index columns being set to
NULLor the default value onCASCADE. #54592 - Fixed a rare bug that could lead to index backfills failing on transaction restarts. #54860
Contributors
This release includes 18 merged PRs by 11 authors.
v19.2.10
Release Date: August 24, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Bug fixes
- Fixed "column not in input" internal error in some corner cases. #50860
- Previously, CockroachDB could crash when internal memory accounting hit a discrepancy. Now it will report an error instead. #51015
- Fixed incorrect results in some cases involving joins on interleaved tables with limits. #51435
- Previously, CockroachDB could hit an internal error when executing
regexp_replacebuiltin. This has been fixed. #51348 - Previously, CockroachDB could hit a "command is too large" error when performing an
UPSERToperation with many values. Internally, we attempt to perform such operation by splitting it into "batches", but the batching mechanism was broken. This has been fixed. #51612 - CockroachDB previously could crash on some queries with merge joins. This has been fixed. #52074
- Fixed an internal error that could occur when an aggregate function argument contained a correlated subquery with another aggregate function referencing the outer scope. This now returns an appropriate user-friendly error, "aggregate function calls cannot be nested". #52141
- Previously, subtracting months from a
TIMESTAMP/TIMESTAMPTZorDATEwhose date value is greater than 28 could subtract a further year off. This has been fixed. #52155 - Large write requests no longer have a chance of erroneously throwing a "transaction with sequence has a different value" error. #52268
- Previously, CockroachDB could return incorrect results on query that encountered a
ReadWithinUncertaintyIntervalerror. This has been fixed. #52252 - Previously, CockroachDB would erroneously restart the execution of empty, unclosed portals after they had been fully exhausted. This has been fixed. #52444
- Previously, executing a large number of statements in a transaction without committing could crash a CockroachDB server. This has been fixed. #52403
Performance improvements
- Queries no longer block during planning if cached table statistics have become stale and the new statistics have not yet been loaded. Instead, the stale statistics are used for planning until the new statistics have been loaded. This improves performance because it prevents latency spikes that may occur if there is a delay in loading the new statistics. #52189
Contributors
This release includes 24 merged PRs by 9 authors.
v19.2.9
Release Date: July 6, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
General changes
- Schema changes are now logged in greater detail. #50375
Bug fixes
- Fixed a bug that occurred during incremental backups with revision history on a database (or full cluster) when a table in the database being backed up was dropped. When other tables were later created, dropped, or altered, future incremental backups would return an error. #50208
- Fixed a bug in
TRUNCATEthat could leave tables in a state where they could not be renamed. #50714 #50756#50756]
Contributors
This release includes 4 merged PRs by 3 authors.
v19.2.8
Release Date: June 29, 2020
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
SQL language changes
- The
pg_databasetable inpg_catalogno longer requires privileges on any database in order for the data to be visible. #48766
Bug fixes
- Fixed a bug where
cockroach dumpon a table with collated strings would omit the collation clause for the data insertion statements. #48833 - Manually writing a
NULLvalue into thesystem.userstable for the "hashedPassword" column will no longer cause a server crash during user authentication. #48837 - Fixed a memory leak which could affect changefeeds performing scans of large tables. #49162
- Previously, when the value passed to
--drain-waitwas very small, but non-zero,cockroach quitin certain cases would not proceed to perform a hard shutdown. This has been corrected. This bug was present in v19.1.9, v19.2.7, and v20.1.1. #49364 - Previously, some benign errors were reported as unexpected internal errors by the vectorized execution engine. This is now fixed. #49535
- Fixed a bug causing file descriptors to be leaked during GSS authentication. #49655
- Casting to width-limited strings now works correctly for strings containing Unicode characters. #50160
- Fixed cases in which casting a string to a width-limited string array was not truncating the string. #50169
- Fixed a RocksDB bug that could result in inconsistencies in rare circumstances. #50498
Performance improvements
- Improved the optimizer's estimation of the selectivity of some filters involving a disjunction (
OR) of predicates over multiple columns. This results in more accurate cardinality estimation and enables the optimizer to choose better query plans in some cases. #50473
Build changes
- Release Docker images are now built on Debian 9.12. #50481
Doc updates
- Updated guidance on node decommissioning. #7304
- Renamed "whitelist/blacklist" terminology to "allowlist/blocklist". #7535
- Updated the Releases navigation in the sidebar to expose the latest Production and Testing releases. #7550
- Fixed scrollbar visibility on Chrome. #7487
Contributors
This release includes 19 merged PRs by 16 authors. We would like to thank the following contributors from the CockroachDB community:
- Drew Kimball (first-time contributor, CockroachDB team member)
- Jackson Owens (first-time contributor, CockroachDB team member)
- James H. Linder (first-time contributor, CockroachDB team member)
v19.2.7
Release Date: May 20, 2020
This page lists additions and changes in v19.2.7 since v19.2.6.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Backward-incompatible changes
- Prior to this patch, the phase of server shutdown responsible for range lease transfers to other nodes would give up after 10000 attempts of transferring replica leases away, regardless of the value of
server.shutdown.lease_transfer_wait. The limit of 10,000 attempts has been removed, so that now only the maximum durationserver.shutdown.lease_transfer_waitapplies. #47691 - The textual error and warning messages displayed by
cockroach quitunder various circumstances have been updated. The messageokstill indicates that the operation has likely succeeded. #47691 cockroach quitnow prints out progress details on its standard error stream, even when--logtostderris not specified. Previously, nothing was printed on standard error. Scripts that wish to ignore this output can redirect the standard error stream. #47691- The copy of
systemandcrdb_internaltables extracted bycockroach debug zipis now written using the TSV format (inside the ZIP file), instead of an ASCII-art table as previously. #48098
General changes
- Prior to this patch, the phase of server shutdown responsible for range lease transfers to other nodes had a hard timeout of 5 seconds. This timeout is now configurable with the new cluster setting
server.shutdown.lease_transfer_wait. #47691
SQL language changes
- Columns in the process of being added or removed to a table are now always set to their default or computed value if another transaction concurrently inserts, updates, or upserts a row. This fixes an issue where a column being backfilled would not be properly set by concurrent transactions. #47411
- The
pg_collation,pg_proc, andpg_typetables inpg_catalogno longer require privileges on any database in order for the data to be visible. #48081 - Histogram collection with
CREATE STATISTICSis no longer supported on columns with typeARRAY. Only row count, distinct count, and null count are collected forARRAY-type columns. #48344 - CockroachDB now correctly reports the type length for the
CHARdata type. #48643
Command-line changes
- The time that
cockroach quitwaits client-side for the node to drain (i.e., remove existing clients and push range leases away) is now configurable with the flag--drain-wait. The default value for--drain-waitis10 minutes. Note that separate server-side timeouts still apply. Check theserver.shutdown.*cluster settings for details. #47691 - It is now possible to drain a node without shutting down the process, using
cockroach node drain. This makes it easier to integrate CockroachDB with service managers and orchestration. It is safe to issuecockroach node drainand then separately stop the service via a process manager or orchestrator. Without this new command, there is a risk of misconfiguring the service manager to auto-restart the node after it shuts down viacockroach quit.cockroach node drainalso recognizes the new--drain-waitflag. #47691 - The commands
cockroach quitandcockroach node drainnow report a "work remaining" metric on their standard error stream. The value reduces until it reaches 0 to indicate that the graceful shutdown has completed server-side. #47691 - To support clusters that have been configured to use a cluster name, client commands such as
cockroach initandcockroach quitnow support the--cluster-nameand--disable-cluster-name-verificationflags. Previously it was impossible to run such commands against nodes configured with the--cluster-nameflag. #48017 cockroach debug zipnow properly skips over fully decommissioned nodes. #48095cockroach debug zipnow tries multiple times to retrieve data using SQL if it encounters retry errors. #48098cockroach debug zipnow supports two command-line parameters:--nodesand--exclude-nodes. When specified, they control which nodes are inspected when gathering the data. This makes it possible to focus on a group of nodes of interest in a large cluster, or to exclude nodes thatcockroach debug zipwould have trouble reaching otherwise. Both flags accept a list of individual node IDs or ranges of node IDs, e.g.,--nodes=1,10,13-15. #48098
Bug fixes
- Previously, CockroachDB could incorrectly consider a non-
NULLvalue from an interleaved parent table to beNULLwhen the interleaved child has aNULLvalue in the row with the corresponding index key. This bug has been fixed. #47104 - A data race on AST nodes for
SELECTstatements that include aWINDOWdefinition has been fixed. Previously, these queries could have returned incorrect results. #47176 - Previously, a panic could occur when
SHOW RANGESorSHOW RANGE FOR ROWwas called with a virtual table. This bug has been fixed. #47516 - Fixed incorrect results that could occur when casting negative intervals or timestamps to type decimal. #47521
- Previously, when the query was executed with the row-by-row engine, CockroachDB could incorrectly return no output instead of returning a parsing error in queries with
count(*). This bug has been fixed. #47486 - Previously, CockroachDB incorrectly released the memory used in hash aggregation operations, which could lead to node crashes. This bug has been fixed. #47519
- Previously,
PARTITION BYandORDER BYcolumns in window specifications lost qualifications when used inside views. This bug has been fixed. #47716 - Fixed a bug where target column specifications in
IMPORT INTOwere ignored. #47796 - Columns named with reserved keywords and/or other special characters are now correctly handled. #47796
- CockroachDB will now avoid producing a severe "internal error" upon certain privilege check failures with
pg_catalogbuilt-in functions. #48243 - Fixed a bug causing cascading deletes to fail when deleting thousands of rows across interleaved tables. #48287
- Fixed a rare bug that could lead to data corruption or data loss if a replica was both the source of a snapshot, was being concurrently removed from the range, and if certain specific conditions exist inside RocksDB. #48319
- Fixed a rare bug where stats were not automatically generated for a new table. #48028
- Fixed a rare bug where stats refreshes could fail. CockroachDB now triggers stats refreshing after completing an import job, after the transaction finishes the job commits. #48028
- Fixed an error that occurred when statistics collection was explicitly requested on a column with type
ARRAY. #48344 - Fixed a rare bug causing a range to deadlock and all the writes to the respective range to timeout. #48304
- Fixed a bug causing HTTP requests to fail with error 503 "
transport: authentication handshake failed: io: read/write on closed pipe" and continue failing until the node was restarted. #48482 cockroach debug zipcan now successfully avoid out-of-memory errors when extracting very largesystemorcrdb_internaltables. #48098cockroach debug zipwill now properly report an error if an error is encountered while writing the end of the output ZIP file. #48098- Fixed a bug that caused vectorized queries on composite data types to occasionally return invalid data. #48464
- Previously,
SHOW STATISTICS USING JSONoutput contained incorrect single quotes for strings with spaces inside histograms. This bug has been fixed. #48545 - Fixed a bug preventing timestamps from being closed, which could result in failed follower reads or failure to observe resolved timestamps in changefeeds. #48683
- Fixed a bug that falsely indicated that
kv.closed_timestamp.max_behind_nanoswas almost always growing. #48690 - CockroachDB now prevents dropping databases that contain tables that are currently offline due to
IMPORTorRESTORE. Previously, dropping a database in this state could lead to a corrupted schema, which prevented running backups. #48612 - Fixed a bug that caused
cockroach dumpto erroneously include an additionalCREATE UNIQUE INDEX "primary" ... INTERLEAVE IN PARENTstatement in the dump output on tables with interleaved primary keys. This bug made it impossible to re-import dumped data without manual editing. #48777 - Fixed a bug that caused
SHOW CREATEstatements to sometimes show a partitioning step for an index that was previously dropped. #48767 - Fixed a bug that caused changefeeds that were created with a cursor, and that encounter a retryable error before writing their first checkpoint, to perform full-table backfills upon restart. #48752
- Fixed a bug that could cause canceled changefeeds to not stop. #48752
Performance improvements
- Fixed a bug in the histogram filtering logic of the optimizer which was causing inaccurate cardinality estimates for queries with equality predicates on UUIDs and strings, as well as some other types. This bug has existed since histograms were first introduced into the optimizer in the v19.2.0 release. Fixing it improves the optimizer's cardinality estimates and results in better query plans in some cases. #48625
- Histograms used by the optimizer for query planning now have more accurate row counts per histogram bucket, particularly for columns that have many
NULLvalues. This results in better plans in some cases. #48645 - Fixed a performance inefficiency in the vectorized execution engine. This fix speeds up all queries that are executed with the vectorized engine, with most noticeable gains on the queries that output many rows. #48733
Contributors
This release includes 47 merged PRs by 21 authors. We would like to thank the following contributors from the CockroachDB community:
- Artem Barger (first-time contributor)
- Jason Brown (first-time contributor)
v19.2.6
Release Date: April 13, 2020
This page lists additions and changes in v19.2.6 since v19.2.5.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Backward-incompatible changes
cockroach initnow waits for server readiness and thus no longer fails when a mistaken server address is provided. #46477
Security updates
- The non-authenticated
/healthHTTP endpoint was previously exposing the private IP address of the node, which can be privileged information in some deployments. This has been corrected. Deployments using automation to retrieve a node build details and address details should use/_status/details/localinstead and use a valid admin authentication cookie. #46477 - All users can now view any comments on any object (bypassing other privileges), but modifying comments require write privilege on the target object. Previously, any user could modify any database/table/view/index comment via direct SQL updates to
system.comments. This was unintended and a form of privilege escalation, and is now prevented. The privileges required for theCOMMENTstatement,pg_description,col_description(),obj_description(), andshobj_description()are operating as in PostgreSQL and are unaffected by this change. #46833
SQL language changes
- The type checking code now prefers aggregate overloads with string inputs if there are multiple possible candidates due to arguments of unknown type. #46807
- Added an unimplemented error when attempting to
ADD CONSTRAINTwith theEXCLUDE USINGsyntax. #46910 - Added support for
CREATE INDEX .... INCLUDE (col1, col2, ...), which is an alias that PostgreSQL uses that is analogous to CockroachDB'sSTORING (col1, col2, ...)syntax. #46910 - Added support for parsing the
REINDEXsyntax, which results in an unimplemented error that explains thatREINDEXing is not required in CockroachDB. #46910 CREATE INDEX CONCURRENTLYandDROP INDEX CONCURRENTLYare now parsed but return errors. #46805- Improved the error message for unsupported interactions between correlated subqueries and
WITHclauses. #46971
Command-line changes
- The
cockroach initcommand now waits until the node at the provided server address is ready to accept initialization. This also waits for network readiness. This makes it easier to implement initialization scripts by removing the need for a loop. In addition, implementing such a loop is operationally unsafe and is not recommended. #46477 - CockroachDB will now dump the stacks of all goroutines upon receiving
SIGQUITprior to terminating. This feature is intended for use while troubleshooting misbehaving nodes. #46478 - The
cockroach startandcockroach start-single-nodecommands no longer initiate a 1-minute hard shutdown countdown after a request to gracefully terminate. This means that graceful shutdowns are now free to take longer than one minute. It also means that deployments where a maximum shutdown time must be enforced must now use a service manager that is suitably configured to do so. #46483 - The
cockroach debug zipcommand now creates valid zip files even if some of its requests encounter an error. #46635
Admin UI changes
- Metrics relating to SQL transaction restarts and rollbacks are now properly captured and exported. #46272
- The display options are now saved separately for each authenticated user. Note: When upgrading to a version with this change, all current display customizations for admin users are lost. #46479
Bug fixes
- Fixed a bug where
pg_catalog.pg_indexesshowed the wrong index definition for inverted indexes. #46564 - Fixed a performance bug where
TRUNCATEwould take 2*num columns round trips. This could lead to slow truncate performance in distributed clusters. #46648 - Fixed an internal error that could occur in the optimizer when a
WHEREfilter contained at least one correlated subquery and one non-correlated subquery. #46167 - CockroachDB no longer returns an internal error in some cases when running queries with
CASE,AND, orORoperators via the vectorized execution engine. #46088 - Fixed a bug where various session variables could not be set to the values "on" or "off", only true or false. #46219
- Accesses to
/healthusing a non-root authentication token no longer hang when a node is currently under load, or if a system range is unavailable. #46477 - Fixed a race in the vectorized execution engine. #46486
- The
cockroach debug zipcommand now properly collects heap profiles. #46482 - The goroutine dump facility now functions properly when logging to files is disabled, e.g., via
--log-dir=or--logtostderr. #46482 - Fixed a bug where the vectorized engine could sometimes give an incorrect result when reading from interleaved parents or children. #46495
- Fixed an internal error that could occur when an aggregate inside the right-hand side of a
LATERALjoin was scoped at the level of the left-hand side. #46544 - Fixed an error that incorrectly occurred when an aggregate was used inside the
WHEREorONclause of a subquery but was scoped at an outer level of the query. #46544 - Added support for queries with qualified stars that refer to tables in outer scopes. #46538
- Fixed an incorrect "no data source matches prefix" error in some cases involving subqueries that use views. #46537
- Expressions of the form
NOT(a && b)no longer fail. #46539 - Fixed a bug where operations on an index that contained a collated string in descending order would fail.#46578
- Fixed a bug where the vectorized engine would throw an internal error when executing a query that utilized an inverted index. #46506
- Fixed an internal error that could happen during planning when a column with a histogram was filtered with a predicate of a different data type. #46667
- CockroachDB no longer incorrectly accounts for some RAM usage when computing aggregate functions. #46622
- Fixed an incorrect query result that could occur when a scalar aggregate was called with a null input. #46807
- Benign "outbox next" errors are now only logged when log verbosity is set to 1 or greater. #46837
- Fixed incorrect result with count(*) when grouping on constant columns. #46894
- Customizations of the Admin UI are once again properly saved across sessions. #46479
- Fixed a rare bug causing transactions that have performed schema changes to deadlock after they restart. #46904
Performance improvements
- Less memory is loaded into memory for
crdb_internal.jobs. #46452 - Virtual tables that access all table descriptors now make fewer round trips. #46950
Contributors
This release includes 42 merged PRs by 16 authors.
v19.2.5
Release Date: March 23, 2020
This page lists additions and changes in v19.2.5 since v19.2.4.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
- Non-licensed users are now able to add more principals to the special superuser role/group
admin. Creation of additional roles is still a licensed feature. #45395
SQL language changes
- The SQL:2008 syntax
OFFSET <x> ROWSandFETCH FIRST <y> ROWS ONLYnow accepts parameter values. #45166 - It is now possible to use
GRANTandREVOKEto add users to theadminrole without a valid license. This change aims to enable use of the Admin UI and other privileged features without a license. #45395
Command-line changes
cockroach debug zipnow includes secondary log files (e.g., the RocksDB logs) in the main log directory. Log files in alternate log directories, if different from the main log directory, are not included. #45202cockroach debug zipnow displays its progress differently on the terminal. #45207cockroach debug zipnow properly supports special characters in database and table names. #45207cockroach debug zipwill now apply the--timeoutparameters to the SQL queries it performs. Previously, there was no timeout, which could causecockroach debug zipto hang. #45207cockroach debug zipis now able to tolerate more forms of cluster unavailability. In case system ranges are unavailable, we recommend runningcockroach debug zipon each node's address, to maximize useful data collection. #45207- The
debug.zipfile generated bycockroach debug zipnow contains hex a representation of the marshaled jobs payload, progress, and table descriptors. #46040
Admin UI changes
- The "Log file list" endpoint now includes secondary log files (e.g., the RocksDB logs) in the main log directory. Log files in alternate log directories, if different from the main log directory, are not included. #45202
Bug fixes
- Fixed a "cannot map variable" error that could occur in rare cases involving joins. #44859
- Fixed a bug where
DROP INDEXjobs waiting for GC TTL might be erroneously deleted early. Usually aDROP INDEXjob would appear as "waiting for GC TTL" to indicate that the data was still on disk. However, this bug might have caused the job to be deleted before the data was actually removed from disk. #44873 - Fixed occasional "concurrent map write" crash. #44874
- Previously, CockroachDB would return an internal error when a
substringfunction with non-INT8start and length arguments was executed with the vectorized execution engine. This is now fixed. #44902 - Fixed incorrect deduplication of impure expressions (e.g.,
gen_random_uuid) in projections and default values. #44915 - Previously, CockroachDB could return an internal error on the queries that return
INTcolumns when the default integer size has been changed. This is now fixed. #44980 - Previously, CockroachDB would return an internal error when a merge join operation was performed with the vectorized execution engine in the case when two sides of the join had comparable but different types in the equality columns (e.g.,
INT2on the left andINT4on the right). This is now fixed. #45032 - Previously, CockroachDB could crash when running
EXPLAIN (VEC)in some edge cases. Now, an internal error is returned instead. #45004 - Fixed an internal error that could occur when
NULLIFwas called with one null argument. #45390 - Some vectorized execution plans that used lookup joins with
DECIMALs would previously return incorrect results. This is now fixed. #45548 - AbortSpan records are now cleaned up more aggressively when it is known to be safe. #45553
- It is now possible to create inverted indexes on columns whose names are mixed-case. #45660
- Fixed a bug causing distributed deadlock detection between transactions to stall and fail to resolve a deadlock. #45664
- Fixed a bug where large ranges with slow send rates would hit the timeout in several storage system queues by making the timeout dynamic based on the current rate limit and the size of the data being sent. This affects several storage system queues: the Raft snapshot queue, the replication queue, and the merge queue. #44952
- Previously, drivers that did not truncate trailing zeroes for decimals in the binary format end up having inaccuracies of up to 10^4 during the decode step. Trailing zeroes are now truncated as appropriate. This fixes known incorrect decoding cases with Postgrex in Elixir. #45670
- Range garbage collection will now trigger based on a large abort span, adding defense-in-depth against ranges growing large and eventually unstable. #45744
- Previously, an internal error could occur in CockroachDB when executing queries that contained unordered synchronizers with the vectorized execution engine. This is now fixed. #45739
- Previously, CockroachDB would crash when using a window function with the
RANGEmode of framing anoffset PRECEDINGoroffset FOLLOWINGboundary where the single column in anORDER BYclause was also in thePARTITION BYclause. This is now fixed. #45805 - Previously, CockroachDB could crash when computing window functions with the
RANGEmode of framing when one of the bounds was eitheroffset PRECEDINGoroffset FOLLOWINGand there wereNULLvalues in the single column of anORDER BYclause. Additionally,RANGEmode bounds0 PRECEDINGand0 FOLLOWINGcould have been handled incorrectly. This is now fixed. #45805 - When considering if a job should be orphaned, CockroachDB used to take the conservative approach when a descriptor ID pointing to non-existent descriptor was found. This caused jobs to hang and be garbage collected. We now disregard these IDs when considering if a job still has work to do. #45895
- The
--setparameter forcockroach sqlandcockroach demois now able to properly override all client-side options, as advertised. #46130 - Expected errors from the vectorized execution engine are no longer mistakenly annotated as unexpected errors. #46092
Performance improvements
- Execution plans are now faster in some cases that involve
IN/NOT INwith an empty tuple, or= ANYwith an empty array. #45213 - Improved the selectivity estimation of some predicates containing
OR, leading to better plan selection by the optimizer, in some cases. #45972 - Improved cardinality estimation in the optimizer for relations with a small number of rows. This may lead to the optimizer choosing a better query plan. #45972
Contributors
This release includes 36 merged PRs by 15 authors.
v19.2.4
Release Date: February 11, 2020
This page lists additions and changes in v19.2.4 since v19.2.3.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
- Previous versions of CockroachDB were incorrectly enabling non-admin SQL users to use the statements details in the Admin UI and the HTTP endpoint
/_status/statements. This information is sensitive because the endpoint does not hide data that the requester does not have privilege over. This has been corrected by requiring a SQLadminuser to access the statements details page and the HTTP endpoint. #44354
Bug fixes
- Fixed a bug where CockroachDB would return an internal error when the
substringfunction with a negative length was executed via the vectorized engine. CockroachDB now returns a regular query error on executing the function. #44629 - Fixed "no output column equivalent to.." and "column not in input" errors in some cases involving
DISTINCT ONandORDER BY. #44598 - Fixed "expected constant FD to be strict" internal error. #44604
- Fixed possibly incorrect query results in various cornercases, especially when
SELECT DISTINCTis used. #44604 - Fixed a bug where running a query with the
LIKEoperator using the customESCAPEsymbol when the pattern contained Unicode characters could result in an internal error in CockroachDB. #44648 - CockroachDB no longer repeatedly looks for non-existing jobs, which may cause high memory usage, when cleaning up schema changes. #44698
- Fixed "no indexes" internal error in some cases when we
GROUP BYon a virtual table. #44723 - Fixed invalid query results in some corner cases where part of a
WHEREclause is incorrectly discarded. #44749 - Fixed a typechecking error where
BETWEENwould sometimes allow boundary expressions of a different type. #44810 CASEoperators with an unknownWHENtype no longer return an error. #44818
Contributors
This release includes 15 merged PRs by 10 authors.
v19.2.3
Release Date: February 3, 2020
This page lists additions and changes in v19.2.3 since v19.2.2.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
- SQL admins can now use the new command
cockroach auth-session loginto create authentication tokens with an arbitrary expiration date. Operators should be careful to monitorsystem.web_sessionsand enforce policy-mandated expirations using either SQL queries or the new commandcockroach auth-session logout. #44110
Enterprise edition changes
- CockroachDB is now more resilient to connection/network errors when reading data from HTTP servers. CockroachDB now attempts to resume the download after such errors. #43687
SQL language changes
extract()now supports millennium, century, decade, isoyear, isodow, and julian fordate,timestamp, andtimestamptz.DATE_TRUNCnow supports millennium, century, and decade fordate,timestamp, andtimestamptz. #42957- The output of
EXPLAINnow shows joins where there are no equality columns as "cross" instead of "hash". Cross joins can be very expensive and should be avoided. #43239 - Previously, we error-coded duplicate foreign key constraint names with
42830or23503. We changed this to be42710to be in line with PostgreSQL. #43425 - Vectorized queries that execute only on supported types, even if those types form part of a table with unsupported types are now run through the vectorized engine. This would previously fall back to the row-by-row execution engine. #44014
SHOW RANGESnow shows locality information consistent with the range descriptor when node ID and store ID do not match. #44312- Previously, CockroachDB would return an internal error when using
SET TRACINGwith any type other thanSTRING. Now it will return a regular query error. Additionally, boolean arguments are now supported inSET TRACING, andtrueis mapped toonmode of tracing whereasfalseis mapped tooff. #44347
Command-line changes
- Telemetry is now recorded for whenever the command
cockroach demois used. #43819 - Three new CLI commands,
cockroach auth-session login,cockroach auth-session list, andcockroach auth-session logout, are now provided to facilitate the management of web sessions. The commandauth-session loginalso produces a HTTP cookie which can be used by non-interactive HTTP-based database management tools. It also can generate such a cookie for therootuser, who would not otherwise be able to do so using a web browser. #44110
Admin UI changes
- We previously introduced a fix on the Admin UI to prevent non-admin users from executing queries. However, this inadvertently caused certain pages requiring table details not to display. This issue has now been resolved. #44191
Bug fixes
- Fixed a bug during planning for some queries that could cause an infinite loop and prevent the query from being cancelled. #43082
- Some incorrect issue links referenced by error hints have been corrected. #43233
EXPLAINcan now be used with statements that useAS OF SYSTEM TIME. #43303- This bug may cause jobs to be left indefinitely in
PENDINGstate and never run. #43421 - Fixed a bug that prevented error reporting while generating shell autocomplete scripts. #43507
- Fixed a bug that prevented error reporting while reading from
crdb_internal.gossip_nodeswhich could lead to lost writes or other undefined behavior. #43502 - Fixed a bug that prevented error reporting while generating plan diagrams rather than silently reporting no result. #43506
- Fixed a bug where an internal error could occur when
CASEoperator operating on distinct although compatible types was executed via the vectorized engine. For example, a query similar toSELECT CASE WHEN false THEN 0:::INT2 ELSE 1:::INT8 ENDpreviously would error out. #43566 - Fixed a bug that prevented error reporting while looking up table descriptors during schema changes and other transactional interactions with schema which could cause lost writes or other undefined behavior. #43503
- Fixed a bug in the parsing logic for
server.host_based_authentication.configuration, where both single-character strings, and quoted strings containing spaces and separated by commas, were not properly parsed. #43714 - Fixed a bug that prevented error reporting while failing to parse URIs for multi-locality backup. #43501
- It is now possible to perform
ALTER COLUMN SET/DROP NULLon multiple (different) columns of the same table inside a single transaction. #43645 - CockroachDB now properly rejects
ALTER COLUMN DROP NOT NULLon a column that is part of the primary key. #43645 - Fixed a bug where errors from JSON containment operations were silently ignored rather than being returned. #43505
- Previously, CockroachDB could incorrectly report that some values were
NULLwhen, in fact, they weren't. This could occur only when vectorized execution engine was used withvectorize=experimental_onand now has been fixed. #43823 - Fixed a bug where CockroachDB returned incorrect results for some aggregate functions when used as window functions with non-default window frame. Note that
MIN,MAX,SUM,AVG, as well as "pure" window functions (i.e., non-aggregates) were not affected. #43854 - Fixed a bug where CockroachDB could, in some cases, return an internal error when running a query with a
CASToperation (:::) if vectorized execution engine is used. #43890 - A SQL row write that is re-issued after already succeeding will no longer throw a duplicate key error when the previous write in its transaction deleted the row. #43937
- On Linux machines, CockroachDB now respects the available memory limit set by the
cgrouplimits which apply to the cockroach process. #43860 - Fixed a bug in which a query shutdown mechanism could fail to fully cleanup the infrastructure when the query was executed via the vectorized engine and the query plan contained wrapped row-by-row processors. #43944
- Fixed a bug introduced in 19.2 that would allow foreign keys to use a unique index on the referenced columns that indexed more columns than were included in the columns used in the foreign key constraint, which allows potentially violating uniqueness in the referenced columns themselves. #43796
- A benign error previously logged at the
ERRORlevel is now logged at theINFOlevel behind a verbosity(2) flag. This error might have been observed ascontext canceled: readerCtx in Inbox stream handler. #44037 - Fixed a bug where an internal error could occur when a query with an aggregate function
MINorMAXwas executed via the vectorized engine when the input column was eitherINT2orINT4type. #44055 cockroach zipnow emits thegoroutinefile in the proper sub-directory when the corresponding call fails with an error. #44065cockroach debug zipis again able to operate correctly and continue to iterate over all nodes if one of the nodes does not deliver its goroutine dumps. It would previously prematurely and incorrectly stop with an incomplete dump; this was a regression introduced in 19.2. #44065- CDC is no longer susceptible to a bug where a resolved timestamp might be published before all events that precede it have been published in the presence of a range merge. #44077
- Converted a panic in
golang.org/x/text/language/tags.gowhen using collated strings to an error. #44117 - CockroachDB now properly supports using
--urlwith query options (e.g.,application_name) but without specifyingsslmode. The default ofsslmode=disableis assumed in that case. #44115 - Removed statistics information from backup jobs' payload information to avoid excessive memory utilization when issuing commands such as
SHOW JOBS. #44190 - Fixed a bug where CockroachDB could crash in special circumstances when vectorized execution engine is used (it was more likely to happen if
vectorize=experimental_onsetting was used). #44196 - Fixed a planning bug related to
FULLjoins between single-row relations. #44228 - Fixed
CopyFrom requires empty destinationinternal error. #44245 - Fixed a bug where multiple nodes attempted to populate the results for
CREATE TABLE ... ASleading to duplicate rows. #44300 - Fixed internal error when mixed types are used with
BETWEEN. #44319 - Fixed a bug where CockroachDB would return an internal error when a query with
CASEoperator that returns onlyNULLvalues was executed via the vectorized engine. #44353 - Fixed an internal error in the planner when table statistics were collected manually using
CREATE STATISTICSfor different columns at different times. #44442 - Fixed a bug that prevented
adminusers from usingBACKUP,RESTORE, andIMPORT. #44456 - Fixed incorrect plans in very rare cases involving filters that aren't constant folded in the optimizer but that can be evaluated statically when running a given query. #44419
Performance improvements
- Adjusted the optimizer's cost of lookup join when the lookup columns aren't a key in the table. This will now cause some queries to switch to using a hash or merge join instead of a lookup join, improving performance in most cases. #43059
- Improved the estimated row count for some lookup joins during planning, leading to better plans in some cases. #43364
- A transaction running into multiple intents from an abandoned conflicting transaction now cleans them up more efficiently. #43588
- Histograms are now collected automatically for all boolean columns, resulting in better query plans in some cases. For tables that aren't being modified frequently, it might be necessary to run
CREATE STATISTICSmanually to see the benefit. #44171
Allocator
- The kv.allocator.range_rebalance_threshold setting, which controls how far away from the mean a store's range count must be before it is considered for rebalance, is now subject to a 2-replica minimum. If, for example, the mean number of replicas per store is 5.6 and the setting is 5%, the store will not be considered for rebalance unless the number of replicas is lesser than 3 or greater than 8. Previously, the bounds would have been 5 and 6. #44359
Doc updates
- Added a Django app development tutorial. #6359 #6365
- Updated the Hibernate app development tutorial with client-side retry logic. #5760
- Documented how to use keyset pagination to iterate through query results. #6114
- Improved the GSSAPI authentication instructions for configuring Active Directory and MIT and for configuring the client. #6115
- Expanded the Kubernetes tutorial to show how to use a custom CA instead of Kubernetes built-in CA when using manual configs. #6232
- Updated the Kubernetes tutorial for compatibility with Helm 3.0. #6121
- Added language-specific connection strings to the instructions on connecting to a CockroachCloud cluster. #6077
- Added Docker as a download option on the full release notes list. #5792
- Updated the
IMPORTdocumentation with an example usage ofDELIMITEDwith escaping, a note aboutDEFAULTvalues, and an explanation of thestrict_quotesoption. #6244 - Added an FAQ on why Cockroach Labs changed the license for CockroachDB. #6154
- Corrected the description of the possible result of clock skew outside the configured clock offset bounds. #6329
- Expanded the data types overview to indicate whether or not a type supports vectorized execution. #6327
Contributors
This release includes 58 merged PRs by 18 authors. We would like to thank the following contributors from the CockroachDB community:
- Roga Pria Sembada (first-time contributor)
v19.2.2
Release Date: December 16, 2019
This page lists additions and changes in v19.2.2 since v19.2.1.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
Security updates
CockroachDB previously allowed non-authenticated access to privileged HTTP endpoints like
/_admin/v1/events, which operate usingrootuser permissions and can thus access (and sometimes modify) any and all data in the cluster. This security vulnerability has been patched by disallowing non-authenticated access to these endpoints and restricting access to admin users only.Note:Users who have built monitoring automation using these HTTP endpoints must modify their automation to work using an HTTP session token for an admin user.
Some Admin UI screens (e.g., Jobs) were previously incorrectly displayed using
rootuser permissions, regardless of the logged-in user's credentials. This enabled insufficiently privileged users to access privileged information. This security vulnerability has been patched by using the credentials of the logged-in user to display all Admin UI screens.Privileged HTTP endpoints and certain Admin UI screens require an admin user. However,
rootis disallowed from logging in via HTTP and it is not possible to create additional admin accounts without an Enterprise license. This is further discussed here and will be addressed in an upcoming patch revision.Note:Users without an Enterprise license can create an additional admin user using a temporary evaluation license, until an alternative is available. A user created this way will persist beyond the license expiry.
Some Admin UI screens currently display an error or a blank page when viewed by a non-admin user (e.g., Table Details). This is a known limitation mistakenly introduced by the changes described above. This situation is discussed further here and will be addressed in an upcoming patch revision. The list of UI pages affected includes but is not limited to:
- Job details
- Database details
- Table details
- Zone configurations
Note:Users can access these Admin UI screens using an admin user until a fix is available.
The list of HTTP endpoints affected by the first change above includes:
| HTTP Endpoint | Description | Sensitive information revealed | Special (see below) |
|---|---|---|---|
/_admin/v1/data_distribution |
Database-table-node mapping | Database and table names | |
/_admin/v1/databases/{database}/tables/{table}/stats |
Table stats histograms | Stored table data via PK values | |
/_admin/v1/drain |
API to shut down a node | Can cause DoS on cluster | |
/_admin/v1/enqueue_range |
Force range rebalancing | Can cause DoS on cluster | |
/_admin/v1/events |
Event log | Usernames, stored object names, privilege mappings | |
/_admin/v1/nontablestats |
Non-table statistics | Stored table data via PK values | |
/_admin/v1/rangelog |
Range log | Stored table data via PK values | |
/_admin/v1/settings |
Cluster settings | Organization name | |
/_status/allocator/node/{node_id} |
Rebalance simulator | Can cause DoS on cluster | yes |
/_status/allocator/range/{range_id} |
Rebalance simulatoor | Can cause DoS on cluster | yes |
/_status/certificates/{node_id} |
Node and user certificates | Credentials | |
/_status/details/{node_id} |
Node details | Internal IP addresses | |
/_status/enginestats/{node_id} |
Storage statistics | Operational details | |
/_status/files/{node_id} |
Retrieve heap and goroutine dumps | Operational details | yes |
/_status/gossip/{node_id} |
Gossip details | Internal IP addresses | yes |
/_status/hotranges |
Ranges with active requests | Stored table data via PK values | |
/_status/local_sessions |
SQL sessions | Cleartext SQL queries | yes |
/_status/logfiles/{node_id} |
List of log files | Operational details | yes |
/_status/logfiles/{node_id}/{file} |
Server logs + entries | Many: names, application data, credentials, etc. | yes |
/_status/logs/{node_id} |
Log entries | Many: names, application data, credentials, etc. | yes |
/_status/profile/{node_id} |
Profiling data | Operational details | |
/_status/raft |
Raft details | Stored table data via PK values | |
/_status/range/{range_id} |
Range details | Stored table data via PK values | |
/_status/ranges/{node_id} |
Range details | Stored table data via PK values | |
/_status/sessions |
SQL sessions | Cleartext SQL queries | yes |
/_status/span |
Statistics per key span | Whether certain table rows exist | |
/_status/stacks/{node_id} |
Stack traces | Application data, stored table data | |
/_status/stores/{node_id} |
Store details | Operational details |
"Special" endpoints are subject to the cluster setting server.remote_debugging.mode. Unless the setting was customized, clients are only able to connect from the same machine as the node.
General changes
- Improved the consistency checker's log output. #42149
- Nodes that have been terminated as the result of a failed consistency check now refuse to restart, making it more likely that the operator notices that there is a persistent issue in a timely manner. #42149
- Client usernames can now be defined to start with a digit; in particular, all-digit usernames are now permitted. #42989
- To curb the spread of corrupted data, outliers are now terminated when the replicas within a range are corrupted. Previously, the leaseholder replica would terminate, regardless of which replicas disagreed with each other. #42149
Enterprise edition changes
SQL language changes
- Added the ability to run
avg()over intervals. #42679 - Some columns (
typinput,typoutput,typreceive,typsend,typmodin,typmodout,typanalyze) inpg_typewere incorrectly typed asOIDinstead ofREGPROC. This issue has been resolved. #42784 - It is now valid for
SELECTandHAVINGto refer to ungrouped columns in the special case when the grouped columns contain the primary key of the table containing the ungrouped column. #42694 - It is now supported to specify selection target aliases as
GROUP BYcolumns. Note that theFROMcolumns take precedence over the aliases, which are only used if there is no column with that name in the current scope. #42694 - CockroachDB will now report a more accurate error message, hint, and error code if an error is encountered while adding a new column. #42530
Admin UI changes
- Certain web UI pages (like the list of databases or tables) now restrict their content to match the privileges of the logged-in user. #42726
- The event log now presents all cluster settings changes, unredacted, when an admin user uses the page. #42726
- Customization of the UI by users is now only properly saved if the user has write privilege to
system.ui(i.e., is an admin user). Also, all authenticated users share the same customizations. This is a known limitation and should be lifted in a future version. #42726 - Access to table statistics are temporarily blocked from access by non-admin users until further notice, for security reasons. #42726
- Certain debug pages have been blocked from non-admin users for security reasons. #42726
Bug fixes
- The
TIMESTAMPdata type is now fully supported by the vectorized engine. Previously, an "unsupported type" error was thrown. #42576 - Previously, casting the same type into two or more different precisions/widths from a table in the same
SELECTquery would result in only the first precision specified. For example,SELECT a::decimal(10, 3), a::decimal(10, 1) FROM twould return both results asa::decimal(10, 3). This PR fixes that behavior. #42578 - Fixed a bug that would produce a spurious failure with the error message "incompatible
COALESCEexpressions" when adding or validatingMATCH FULLforeign key constraints involving composite keys with columns of differing types. #42651 - It is now possible to transfer range leases to lagging replicas. #42761
- Long running transactions which attempt to
TRUNCATEcan now be pushed and will commit in cases where they previously failed or retried forever. #42772 - Fixed a case in which CockroachDB incorrectly determined that a query (or part of a query) that contains an
IS NULLconstraint on a unique index column returns at most one row, possibly ignoring aLIMIT 1clause. #42791 ALTER INDEX IF EXISTSno longer fails when using an unqualified index name that does not match any existing index. Now it is a no-op. #42838- Fixed a bug with incorrect handling of Top K sort by the vectorized engine when K is greater than 1024. #42843
- Prevent internal error in some cases when a NULL literal is passed to the
OVERLAPSoperator #42878 - Fixed a bug that caused rare crashes when using built-in functions. #42674
- Other callers to
acquireNodeLeasenow do not get erroneously cancelled just because the context of the first caller was cancelled. #42893 - Changefeeds now emit backfill row updates for a dropped column when the table descriptor drops that column. #42669
- Tables that are being restored or imported are no longer considered valid targets in backups and changefeeds. #42735
- Fixed a bug where selecting columns by forcing an
INTERLEAVINGindex would return an error instead of returning the correct results. #42901 - CockroachDB now prevents a number of panics from the SQL layer caused by an invalid range split. These would usually manifest with messages containing encoding errors like "found null on not null column". #42859
- Fixed the row count estimate during query planning for some queries with multiple predicates in which the selectivity of one predicate was calculated using a histogram. #42962
- Previously, when executing a built-in function via the vectorized engine that returns an output type unsupported by the vectorized engine, CockroachDB could return an error instead of falling back to the row-by-row execution engine. This is now fixed. #42990
- Empty arrays are now correctly encoded and decoded over the binary protocol. #42974
- Fixed some existing caching issues surrounding role memberships, where users could sometimes see out-of-date role membership information. #43004
- Previously, attempting to parse
0000-01-01 00:00when involvingtimedid not work aspgdatedoes not understand0000as a year. This is now fixed. #42918 - The allocator now considers stores with very close diversity scores equal (all other things being the same) and doesn't attempt to rebalance. #43077
- The allocator now considers the new store being added when looking for a target in the case of a rebalance. #43077
- CockroachDB did not previously handle date casts from
TIMESTAMP/TIMESTAMPZwith time attached for times before the UNIX epoch correctly. For example,'1969-12-30 01:00:00'::timestampwould round to'1969-12-31'instead of'1969-12-30'. This is now fixed. #43022 - CockroachDB is now less likely to hang in an inconvenient or inoperative state if it attempts to access an external HTTP server that is blocked or overloaded. Nodes failing to shut down with
cockroach quitwere a symptom of this bug. #42539
Contributors
This release includes 38 merged PRs by 24 authors. We would like to thank the following contributors from the CockroachDB community:
- Adam Pantel (first-time contributor, CockroachDB team member)
- Oliver Tan (first-time contributor, CockroachDB team member)
- georgebuckerfield (first-time contributor)
v19.2.1
Release Date: November 25, 2019
This page lists additions and changes in v19.2.1 since v19.2.0.
- For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
- To upgrade to v19.2, see Upgrade to CockroachDB v19.2
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
SQL language changes
- Dropping a unique index that was created with
CREATE UNIQUE INDEXno longer requires theCASCADEoption. #42413 - The vectorized execution engine now supports the
UUIDandTIMESTAMPdata types. #42414 #42416 - The vectorized execution engine now supports
IS NULLandIS NOT NULLoperators. #42433
Command-line changes
- When restarting a node, CockroachDB now provides a hint to ensure that all
cockroach startcommands include the--joinflag. #42325
Bug fixes
- Reduced the likelihood of out-of-memory errors during histogram collection. #42370
- Fixed an out-of-memory error that could occur when collecting statistics on tables with a string index column. #42380
- Fixed some casts from
OIDtoTEXT. #42257 OUTER HASHjoins are now executed correctly via the vectorized engine whenNULLvalues are present. #42438- Fixed a bug in the
crdb_internal.encode_key()built-in function that could causeSHOW RANGE FOR ROWstatements on tables with dropped indexes to return incorrect results or an error. #42461 - Prevented memory leaks when counting rows during
BACKUP. #42535
Performance improvements
- The
./cockroach debug check-storecommand is now faster. #42150 - Improved statistics estimation during query planning for columns with many
NULLvalues. #42436
Contributors
This release includes 12 merged PRs by 8 authors.
v19.2.0
Release Date: November 12, 2019
With the release of CockroachDB v19.2, we’ve made a variety of performance, resiliency, and usability improvements. Check out a comprehensive summary of the most significant user-facing changes and then upgrade to CockroachDB v19.2. You can also read more about these changes in the v19.2 blog post or watch our 19.2 release demo and overview.
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Cockroach Labs has discovered a bug relating to incremental backups, for CockroachDB v19.2.0 - v19.2.12. If a backup coincides with an in-progress index creation (backfill), RESTORE, or IMPORT, it is possible that a subsequent incremental backup will not include all of the indexed, restored or imported data.
Users are advised to upgrade to v20.1.15 or v20.2.8 or later, which includes resolutions.
For more information, including other affected versions, see Technical Advisory 63162.
In addition to v19.2, we're launching Cockroach University, a free, online learning tool for developers and architects who want to gain a fundamental understanding of distributed databases and deep knowledge of CockroachDB’s functionality and architecture. The first course, “Getting Started with CockroachDB,” uses videos, exercises, and quizzes to teach the key characteristics of a distributed SQL database, walking you through how to spin up a CockroachDB instance, run basic queries, and test out CockroachDB’s unique capabilities. For additional information or to register, visit university.cockroachlabs.com.
Summary
This section summarizes the most significant user-facing changes in v19.2.0. For a complete list of features and changes, including bug fixes and performance improvements, see the release notes for previous testing releases.
- CockroachCloud
- Core features
- Enterprise features
- Backward-incompatible changes
- Known limitations
- Documentation
In addition to the features below, SQL support for timestamp objects with a precision value, present in earlier testing releases of v19.2, has been reverted.
CockroachCloud
CockroachCloud will soon offer a completely self-service account creation and management workflow as well as single-region cluster creation in AWS and GCP using a credit card. Sign up for the beta program.
Core features
These features are freely available in the core version and do not require an enterprise license.
The core version of CockroachDB v19.2 uses the Business Source License instead of the Apache 2.0 License. For more information on why we changed our licensing approach and some practical questions and implications, see our blog post and Licensing FAQs. The full details of the license can be found on Github.
| Feature | Description |
|---|---|
| Parallel Commits | CockroachDB's new optimized atomic commit protocol cuts the commit latency of a transaction in half, from two rounds of consensus down to one. Combined with transaction pipelining, parallel commits brings the latency incurred by common OLTP transactions to near the theoretical minimum: the sum of all read latencies plus one round of consensus latency. This especially lowers latency for transactions involving secondary indexes both in multi-region and single-region deployments. |
| Vectorized Query Execution | CockroachDB now supports column-oriented ("vectorized") query execution for operations that are guaranteed to execute in memory on tables with supported data types. |
| Bulk Import Improvements | CockroachDB now supports efficiently loading large amounts of CSV data into existing tables using the new IMPORT INTO statement.Also, for the previously existing IMPORT statement for bulk import of new tables, using as many files as you have nodes and ordering data in imported files by primary key now increases parallelism and performance. |
| Data Replication Reports | Several new and updated system tables can help you query the status of your cluster's data replication, data placement, and replication zone constraint conformance. For example, you can use these reports to see what data is under-replicated or unavailable, to show which of your localities (if any) are vulnerable to data unavailability in more common failure scenarios, and to see if any of your cluster's data placement constraints are being violated. |
| Multi-Use CTEs | Common Table Expressions can now be re-used multiple times in the same query via a WITH operator. |
| Comprehensive Cost-Based Optimizer | All SQL queries now leverage the cost-based optimizer to choose the lowest cost plans, including DDL statements and window functions that previously leveraged the legacy heuristic planner. |
| Ordering Aggregations | Non-commutative aggregate functions are sensitive to the order in which rows are processed. This order can now be controlled with an ORDER BY clause within the function argument list. |
Index Hints for UPDATE & DELETE |
It's now possible to force the use of a specific index for deleting rows and updating rows. |
| Streaming with JDBC | CockroachDB now provides limited support for Postgres wire-protocol cursors for implicit and explicit transactions executed to completion. The Java JDBC driver can use this protocol to stream queries with large result sets, providing much faster performance than result paginating with LIMIT ... OFFSET. |
| Transaction Latency Graphs | The SQL Dashboard in the Admin UI now provides timeseries graphs of p90 and p99 transaction latencies to complement the per-statement metrics on the Statements page. |
CREATE TABLE AS Improvements |
When using the CREATE TABLE AS statement to create a new table from a selection query, you can now specify the primary key and define column families for the new table. The latter is especially helpful for modifying the primary key of tables for geo-partitioning. Also, CREATE TABLE AS is now usable for larger tables than before. |
Adding NOT NULL Constraint |
It's now possible to add the NOT NULL constraint to an existing column, in addition to the previous ability to set this constraint during table creation. |
| Local Testing Improvements | CockroachDB v19.2 includes several usability improvements to running CockroachDB locally for SQL testing and app development. First, the cockroach demo command, which starts a temporary, in-memory CockroachDB cluster with immediate access to an interactive SQL shell, now comes pre-loaded with the movr dataset and a temporary enterprise license that expires after an hour. This means users can try out enterprise features quickly before requesting a license for use in a persistent cluster.Also, the --nodes flag now lets you specify the number of in-memory nodes to create for the demo, with the new --demo-locality flag setting locality information for each node. Alternatively, the --geo-partitioned-replicas flag lets you start a 9-node demo cluster with the Geo-Partitioned Replicas data topology applied automatically to the movr database.Finally, the --with-load command lets you run movr as an active workload against the in-memory cluster. |
| Cluster Startup Improvements | There are now distinct methods for starting single-node and multi-node clusters. For multi-node clusters, start each node with the cockroach start command including a --join flag and then run the one-time cockroach init command. Using cockroach start without the --join flag is now deprecated and will be removed in a future release.For single-node clusters, use the new cockroach start-single-node command to start just a single node with replication disabled and without the need for any extra commands. |
| Built-in Workload Improvements | Cockroach Labs' fictional vehicle-sharing app, MovR, is now available as a sample workload using the cockroach workload CLI command. Also, the --display-every and --display-format flags now make it possible to control the frequency and format for printing per-operation statistics for several workloads. |
| Data Type Improvements | DATE values in CockroachDB are now fully PostgreSQL-compatible, including support for special values (e.g., +/- infinity). Existing dates outside of the PostgreSQL date range (4714-11-24 BC to 5874897-12-31) are converted to +/- infinity dates.Also, you can now use the timezone() and AT TIME ZONE functions to convert a TIMESTAMPTZ into a TIMESTAMP at a specified timezone, or vice versa. |
| Interactive SQL Shell Commands | Within the interactive SQL shell, the l, dt, d, and d <table> commands now list all databases in the CockroachDB cluster, show the tables of the current schema in the current database, list the users for all databases, and show details about columns in the specified table, respectively. |
| Removing Manual Splits | The new UNSPLIT AT statement removes a split enforcement on a key-value range, at a specified row in a table or index, allowing CockroachDB to merge the range as needed. |
Locking and FOR UPDATE |
Because CockroachDB only supports SERIALIZABLE isolation, locking is not required to order concurrent transactions relative to each other. However, the FOR UPDATE locking clause is now supported in selection queries as a no-op for database migration compatibility only. |
| Showing Row Location | The new SHOW RANGE ... FOR ROW statement shows information about a range for a particular row of data. This information is useful for verifying how SQL data maps to underlying ranges, and where the replicas for a range are located. However, note that this is an experimental feature, and the interface and output are subject to change. |
| Viewing Node Locality | It's now easy to retrieve the localities of nodes for setting zone configuration constraints via the SHOW LOCALITY statement and the cockroach node status command. |
| Viewing Complete Jobs | By default, the SHOW JOBS statement lists all of the types of long-running tasks your cluster has performed in the last 12 hours. You can now use the WHEN COMPLETE option to return only after all specified job ID(s) have completed. |
| Viewing Comments for Virtual Tables | Using SHOW TABLES ... WITH COMMENTS on virtual tables in the pg_catalog, information_schema, and crdb_internal schemas now returns useful comments, often with links to further documentation. |
Enterprise features
These features require an enterprise license. Register for a 30-day trial license here, or consider testing enterprise features locally using the cockroach demo CLI command, which starts an in-memory CockroachDB cluster with a temporary enterprise license pre-loaded.
| Feature | Description |
|---|---|
| Backup & Restore Improvements | CockroachDB now supports locality-aware backup and restore such that each node writes to and restores from files in its locality. This can reduce cloud storage data transfer costs by keeping data within cloud regions and help you comply with data domiciling requirements. Also, using SHOW BACKUP with the new SCHEMAS option adds the number of rows and the schema of the backed up table to the output, and using RESTORE with the new skip_missing_views option restores a table from a view without restoring the view's dependencies. |
| Geo-Partitioning Improvements | CockroachDB v19.2 includes several usability improvements to geo-partitioning. First, it's now possible to name partitions identically across indexes of a table (e.g., us-west, us-east) and use a single command to create zone configurations for all partitions of a given name, saving you multiple steps.Next, it's easy to retrieve the localities of nodes for setting zone configuration constraints via the SHOW LOCALITY statement and the cockroach node status command, both of which do not require an enterprise license.There are now also several ways to view the details of partitions and confirm they are in effect, from the outputs of SHOW CREATE TABLE and the table schemas listed on the Databases page in the Admin UI to the output of SHOW PARTITIONS.Finally, it's now much easier to efficiently query partitioned data; when filtering by the column directly following the partitioned prefix in the primary key, the cost-based optimizer creates a query plan that scans each partition in parallel, rather than performing a costly sequential scan of the entire table. Filtering by the partition value itself can further improve performance by limiting the scan to the specific partition(s) that contain the data that you are querying. |
Backward-incompatible changes
Before upgrading to CockroachDB v19.2.0, be sure to review the following backward-incompatible changes and adjust your application as necessary.
The
cockroach zoneCLI command has been removed. Use the equivalentCONFIGURE ZONEandSHOW ZONE CONFIGURATIONSSQL statements for managing replication zones instead.The
IMPORTSQL statement no longer accepts quotes inside unquoted CSV fields.CONFIGURE ZONESQL statements now fail if the user does not have sufficient privileges. If the target is asystemrange, thesystemdatabase, or a table in thesystemdatabase, the user must have an admin role. For all other databases and tables, the user must have theCREATEprivilege on the target database or table.This change might be backward-incompatible for users running scripted
CONFIGURE ZONEstatements with restricted permissions. To add the necessary permissions, useGRANT<privileges> orGRANT<roles> as a user with an admin role. For example, to grant a user the admin role, runGRANT admin TO <user>. To grant theCREATEprivilege on a database or table, runGRANT CREATE ON [DATABASE | TABLE] <name> TO <user>.INTandFLOATcolumns of less than the max width will now be returned as their own type via the binary protocol. For example, anint4column will be returned in 32 bits over the pgwire binary protocol instead of 64 bits.The
changefeed.min_high_watermetric has been removed from the Prometheus Endpoint. For detecting high changefeed latency, use thechangefeed.max_behind_nanosmetric instead.
Known limitations
For information about new and unresolved limitations in CockroachDB v19.2, with suggested workarounds where applicable, see Known Limitations.
Documentation
| Topic | Description |
|---|---|
| Performance Benchmarking | Added an overview of CockroachDB's performance profiles (scaling, throughput, latency), based on Cockroach Labs's extensive testing using the TPC-C industry-standard benchmark, as well as detailed instructions for reproducing our TPC-C benchmarking results at different scales. |
| Multi-Region Deployment | Updated the tutorial on getting low latency reads and writes in a multi-region cluster to feature two of the most important multi-region data topologies for dramatically reducing the impact of network latency, Geo-Partitioned Replicas and Duplicate Indexes. |
| Orchestration with Kubernetes | Expanded the tutorial on Kubernetes single-region deployment to cover running on Amazon's hosted EKS and naming CSR naming requirements for secure deployments. Also updated and expanded the instructions on using Helm. |
| Client-Side Transaction Retries | Updated and simplified the client-side transaction logic in the Java, Python, and Go getting started tutorials and code samples. Also added pseudocode to help with the implementation of this logic in other languages as well as instructions for authors of database drivers and ORMs who would like to implement client-side retries in their database driver or ORM for maximum efficiency and ease of use by application developers. |
SQL Tuning with EXPLAIN |
Added a tutorial on how to use EXPLAIN to identify and resolve common SQL performance problems. |
| Testing with MovR Dataset | Added an overview of MovR, CockroachDB's fictional vehicle-sharing dataset and application, and updated several SQL pages and examples to use the built-in MovR dataset, for example, Learn CockroachDB SQL. |
| Migration from Oracle | Added guidance on migrating from Oracle, including the process of converting schema and exporting data for loading into CockroachDB. |
| App Deployment on CockroachCloud | Added a tutorial on running a sample To-Do app in Kubernetes with CockroachCloud as the datastore. The app is written in Python with Flask as the web framework and SQLAlchemy for working with SQL data, and the code is open-source and forkable. |
v19.2.0-rc.4
Release Date: November 7, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
SQL language changes
- CockroachDB now uses the zig-zag join algorithm only in cases where it can benefit performance. #42186
- Added the
sql.defaults.zigzag_join.enabledcluster setting, which can be used to control the default value of theenable_zigzag_joinsession variable and, thus, whether zig-zag joins are allowed. #42182 - The error message generated when a transaction containing DDL is both partially committed and partially rolled back (
XXAOO) now contains a link to GitHub issue #42061 where this situation is discussed further. #42090
Bug fixes
- CockroachDB now supports GSSAPI authentication from Postgres 11.5 and 12.0 clients. #42189
- Fixed a "cannot map variable to an indexed var" error for certain queries against partitioned tables. #42183
- Fixed "output columnTypes unset after planning" and "inconsistent post-processing" errors for certain queries involving
UNION ALL. #42184 - Fixed an "invalid indexIdx" error for certain queries involving zig-zag joins. #42182
Contributors
This release includes 6 merged PRs by 3 authors.
v19.2.0-rc.3
Release Date: November 4, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
General changes
- Go 1.12.12 is now required to build CockroachDB from source. #41994
SQL language changes
- Changed the data type of the
system.reports_meta.generatedandsystem.replication_constraint_stats.violation_startcolumns fromTIMESTAMPtoTIMESTAMPTZ. #41915
Bug fixes
- Fixed a rare corruption bug when files are ingested that do not overlap with the in-memory buffer and compaction has fallen behind. #42009
- CockroachDB no longer erroneously transfers leases to replicas that are in the process of being removed, which could lead to ranges being unavailable due to an invalid lease. #42079
Doc updates
- Added a CockroachDB Performance page describing the performance profiles you can expect (scaling, throughput, and latency), based on Cockroach Labs' extensive testing using the industry-standard benchmark TPC-C. #5595
- Documented how to run the built-in
movrworkload. #5713 - Documented how to show comments for virtual tables, including links to further documentation. #5695
- Refreshed the Deploy a Python To-Do App with Flask, Kubernetes, and CockroachCloud. #5694
Contributors
This release includes 4 merged PRs by 3 authors.
v19.2.0-rc.2
Release Date: October 28, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
General changes
- Go 1.12.10 is now required to build CockroachDB from source. #41647
- Named ports in URLs are no longer supported due to a URL parsing security fix in Go 1.12.8. Use port numbers instead. #41647
SQL language changes
- Mutations under
UNIONorUNION ALLare once again supported. #41500 - Added new
skipandnullifoptions forIMPORT ... DELIMITED DATA, which ignore the specified number of lines at the beginning of the file and force all empty string fields to be inserted asNULLin the imported table, respectively. #41656 - It is no longer possible to use
IMPORT INTOwith an interleaved table. This restriction is temporary and will be lifted in a future release. #41651 IMPORTis no longer allowed during a rolling upgrade. #41733
Bug fixes
- Fixed multiple bugs relating to
COPY FROM PARENTin replication zones. #41677 - The
system.replication_statstable no longer incorrectly shows some ranges belonging to table partitions as over-replicated. #41700 - Fixed a bug that resulted in some queries returning an "index join must be against primary index" error. #41723
- Fixed a bug where backfills could lead to certain changefeed row updates never being emitted. #41721
- Fixed a bug that could cause changefeed row updates to be emitted at a timestamp less than or equal to an already forwarded resolved timestamp. #41714
- Fixed a bug in cloud storage sink file naming for changefeeds that could violate ordering in the presence of schema changes. #41693
Performance improvements
- Increased
BACKUPparallelism in some cases. #41729 - Improved the performance of generating the
system.replication_*tables for large clusters. #41757 - Improved performance of bulk ingestion. #41768
Doc updates
- Clarified that to upgrade to the next major version upgrade path for rolling upgrades, a cluster must be on a production release of the previous version, as opposed to a testing release (alpha/beta). #5644
- Documented the experimental
SHOW RANGE FOR ROWstatement, which shows information about a range for a particular row of data. This information is useful for verifying how SQL data maps to underlying ranges, and where the replicas for a range are located. #5612 - Improved the usability of the tutorial on deploying a Python To-Do app with Flask, Kubernetes, and CockroachCloud. #5694
- Updated Helm-specific instructions for Kubernetes version 1.16.0 and fixed various errors. #5657, #5659
- Documented the
FOR UPDATElocking clause, which is supported in selection queries for database migration compatibility. #5658 - Documented using
timezone()andAT TIME ZONEfunctions to convert aTIMESTAMPTZinto aTIMESTAMP, or vice versa, in a specified timezone. #5614 - Clarified the required privileges for the
CONFIGURE ZONEstatement. #5609
Contributors
This release includes 32 merged PRs by 19 authors.
v19.2.0-rc.1
Release Date: October 21, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
SQL language changes
INTERVAL(6)is now supported as an alias forINTERVAL, for compatibility with third-party tools. #41447- Added the
pg_available_extensionstable, for compatibility with third-party tools. #41447 - The
pg_attrdef()andpg_get_indexdef()built-in functions no longer include type annotations on default expressions, andpg_get_indexdef()now includes the index method in the returnedCREATE INDEXstatement, for compatibility with third-party tools. #41447
Bug fixes
- The
generate_subscripts()built-in function now behaves properly onint2vectorandoidvectortypes. #41447 - Fixed a bug causing the
cluster_logical_timestamp()built-in function to sometimes return incorrect results. #41439 - Prevented an unlikely race condition where a WAL pending recycling could be deleted, leading to a crash. This bug was present in past 19.2 alphas/betas only, not in any earlier release. #41523
Performance improvements
- Metadata queries that use the
'string'::regclassform are now much more efficient in the presence of a large number of tables. #41447 - The
pg_get_coldesc()built-in function now retrieves comments more efficiently. #41447
Doc updates
- Clarified epoch-based vs. expiration-based leases in the Replication Layer architecture documentation. #5578
- Updated the list of Enterprise features. #5599
- Documented the new
--geo-partitioned-replicasflag of thecockroach democommand, which starts a 9-node in-memory cluster with the Geo-Partitioned Replicas data topology applied to themovrdatabase. This lets you quickly simulate one of the most effective data topology patterns for getting low latency in a multi-region cluster. #5587
Contributors
This release includes 12 merged PRs by 10 authors.
v19.2.0-beta.20191014
Release Date: October 14, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
SQL language changes
- The new
SHOW RANGE ... FOR ROWstatement identifies the range containing a specific row of a table or secondary index. This is particularly useful for validating the location of a specific piece of data when using partitions to control data location for improved performance or GDPR compliance. Note that this new statement is experimental and its interface and output are subject to change. #40954 - The
SHOW PARTITIONSstatement now displays inherited as well as explicitly set zone configurations for partitions. Also, it is now possible to retrieve full zone configurations via thefull_config_sqlcolumn in thecrdb_internal.zonestable, and the newzone_idandsubzone_idcolumns in thecrdb_internal.partitionstable point to partitions' corresponding zone configurations incrdb_internal.zones. #40493
Command-line changes
- When
vmoduleis set toexec_logs=2to log queries executed by a node, CockroachDB-generated statements are no longer logged alongside client-generated statements. To log CockroachDB-generated statements as well, you must now useexec_logs=3. #41149
Bug fixes
- Fixed a bug that would cause queries combining SQL mutation statements in sub-queries inside
UNIONto return invalid results or to modify the wrong rows if the transaction is forced to retry. #41307 - Fixed a bug that prevented changes to the
kv.bulk_io_write.concurrent_addsstable_requestscluster setting from taking effect. #41195 - Fixed a memory leak that affected secondary logging (SQL audit logs, statement execution, and RocksDB logging). #41231
- Fixed a bug where zone configuration changes on tables with existing index zone configurations would not take effect unless the
num_replicasfield was also set. #41261 - Fixed a bug where zone configuration changes on secondary indexes wouldn't perform setting verification, such as constraints with lease preferences or setting only one of
min_range_sizeandmax_range size. #41216 - Fixed a rare data corruption bug in RocksDB caused by newer Linux kernels' handling of
i_generationon certain filesystems. #41346
Performance improvements
- Stabilized performance of
IMPORTandRESTOREby changing the default value of therocksdb.ingest_backpressure.pending_compaction_thresholdcluster setting from 64 GiB to 2 GiB. #41263
Doc updates
- Expanded the architecture documentation to describe the new Parallel Commits protocol, which cuts the commit latency of a transaction in half, from two rounds of consensus down to one. #5394
- The tutorial on getting Low Latency Reads and Writes in a Multi-Region Cluster now features two of the most important multi-region data topologies, Geo-Partitioned Replicas and Duplicate Indexes, to dramatically reduce the impact of network latency. #5558
- Documented the new
cockroach start-single-nodecli command, which starts a single-node cluster with replication disabled for quick SQL testing or app development. #5445 - Documented how to use
SPLIT AT ... WITH EXPIRATIONSQL statement to set an expiration on a range split as well as how to remove a split enforcement viaUNSPLIT AT. #5360 - Updated hardware recommendations in the Production Checklist. #5472
- Clarified the benefit of merge joins over hash joins. #5464
- Updated the recommendations for improving
IMPORTperformance. #5452 - Documented the
ALTER COLUMN ... SET NOT NULLSQL statement, which adds a non-null constraint to an existing column. #5456 - Documented how to use the
pg_collation_for()built-in function, or its alternative syntax formCOLLATION FOR, to return the locale name of a collated string. #5576 - Documented how to use
SHOW BACKUP SCHEMASSQL statement to list the schemas of enterprise backups alongside their contents. #5455 - Updated the SQL statements that are allowed as row sources for table expressions.
- Documented how to use the
SHOW JOBS WHEN COMPLETEDSQL statement to list only just that have completed. #5395 - Documented the
skip_missing_viewsoption for theRESTORESQL statement, which restores a table with a view but does not restore the view's dependencies. - Documented the
--display-everyand--display-formatflags for thecockroach workloadcli command, which control the frequency and format of per-operation statistics. #5424 - Documented the naming requirements for CSR names when orchestrating CockroachDB with Kubernetes. #5422
- Documented the
--watchflag for thecockroach sqlcli command, which repeats the SQL statements specified with--executeor-euntil a SQL error occurs or the process is terminated. #5543 - Documented how to use the JDBC driver to retrieve large data sets in chunks using cursors. #5544
- Clarified that
CHANGEFEEDcursors must be in quotes. #5529 - Documented how to show the locality of a node using the
crdb_internal.locality_value()built-in function. #5540 - Documented how to reference a common table expression (CTE) multiple times in a single query, using a
WITHoperator. #5485
Contributors
This release includes 57 merged PRs by 23 authors.
v19.2.0-beta.20190930
Release Date: September 30, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
In addition to bug fixes and various general, enterprise, SQL, and command-line enhancements, this release includes several major highlights:
- Locality-aware backup and restore: You can now back up or restore a subset of a table's data according to zone configurations (e.g., send your EU backup partition to an EU cloud storage location).
- Import performance improvements: Internal improvements have been made to
IMPORTto enable more efficient loading of large data sets. - Show partitions using SQL: CockroachDB now supports the
SHOW PARTITIONSstatement, which can be used to identify partitions. It also enhancesSHOW CREATE TABLEto show constraints as described. - Vectorized Execution: Queries are now automatically faster for users. This work expands upon the SQL vectorized execution prototype, and allows it to be used in production. Note that this does not cover the long-tail of features necessary to use vectorization on all queries. Vectorized execution is turned on by default for a subset of queries and defaults back to DistSQL execution for unsupported queries.
- Improve Streaming Behavior on JDBC Driver: PostgreSQL wire protocol cursors have been implemented. As a result, PGJDBC is able to use streaming behavior to return pagination-related results. Limitations still apply. For more information, see Build a Java App with CockroachDB.
- Improve partitioning user experience: CockroachDB now prevents users from specifying a non-partitioned secondary index on a partitioned table by default to improve performance. Users will have the ability to override this default as it may be necessary in some rare use cases. Additionally, users can now specify the same name for the partition for tables and for all corresponding indexes.
- Improve setting of zone configs via SQL:
ALTER TABLEnow returns an error when users select conflicting or impossible zone constraints due to user error. - Make it easy to focus queries on partitioned data: CockroachDB now automatically limits a query to only the applicable partition without having to specify the particular partition for a geo-distributed table. This prevents the query from conducting a full table scan.
- Better insight into constraint conformance: You can now tell if your cluster is in a healthy state based on the status of conformance to constraints like zone configurations, lease preferences, and replica diversification.
General changes
- Replicas are now added using a Raft learner and go through the normal Raft snapshot process to catch them up, eliminating technical debt. No user-facing changes are expected. #39034
- Atomic replication changes are now enabled by default. #40464
- RocksDB
INFOlogs are now stored with CockroahDB debug logs. #40681
Backward-incompatible changes
- Quotes are no longer accepted inside unquoted CSV fields. #40424
CONFIGURE ZONEstatements now fail if the user does not have sufficient privileges. If the target is asystemrange, thesystemdatabase, or a table in thesystemdatabase, the user must have an admin role. For all other databases and tables, the user must have theCREATEprivilege on the target database or table. This change might be backward-incompatible for users running scriptedCONFIGURE ZONEstatements with restricted permissions. To add the necessary permissions, useGRANT<privileges> orGRANT<roles> as a user with an admin role. For example, to grant a user the admin role, runGRANT admin TO <user>. To grant theCREATEprivilege on a database or table, runGRANT CREATE ON [DATABASE | TABLE] <name> TO <user>. #40879
Enterprise edition changes
- Added support for client certificates for Kafka changefeed authentication. #39832
- Added support for locality-aware backups across multiple backup destinations. #39250
- Added support for displaying creation statements of relations stored in a backups. #39323
- Table statistics for the tables that are being backed up are now also backed up. #39644
- Corrected behavior for unmatched quotes in enclosed fields. #40960
- Added
auth=implicitsetting, which removes access key requirement when listing S3 files for bulk I/O operations. #40836
SQL language changes
- Zone configurations now error out when they conflict with each other. #39081
- Renamed
SHOW EXPERIMENTAL_RANGEStoSHOW RANGEScommand, as it is no longer an experimental feature. #39153 - Added the
SHOW RANGES FROM DATABASEcommand. #39194 - Added partial support for row limits during portal execution in
pgwire. #39085 - Added the
SHOW LOCALITYcommand. #39058 - Added the
SHOW PARTITIONScommand. #39053 CREATE VIEWnow supports all queries supported by the optimizer, including those containing correlated subqueries. #39169- Mutations are no longer allowed inside views. #39169
- Added the
COLLATION FORsyntax andpg_collation_for()built-in function. #39198 - Implemented the
timezone()built-in andAT TIME ZONEsyntax, which allows converting atimestamptzinto atimezoneat a specified timezone, and converting atimestampinto atimestamptzat a specified timezone. #39186 - The new
vectorize_row_count_thresholdcluster setting controls when the vectorized execution engine is used. #39329 - Added a
VECoption to theEXPLAINcommand, which shows detailed plan information for vectorized plans. #39280 - Partition names can now be reused between different indexes on the same table. #39332
- Columns without an explicit default value now support foreign keys with the
SET DEFAULTaction, in the same way as theSET NULLandSET DEFAULT NULLcases. #39136 - Added
SHOW JOB <jobid>,SHOW JOB <a_expr>, andSHOW JOBS <selectclause>. #39324 - Added
SHOW JOB WHEN COMPLETE <a_expr>andSHOW JOBS WHEN COMPLETE <selectclause>. #39457 IMPORT INTOnow cleans up any imported rows if it fails. #39459- Updated
SHOW CREATEto display information about partition and table zone configurations. #39102 - Explicitly closing portals over the wire after partial use is now supported. #39639
- Added progress tracking for new implementation of
IMPORT. #39646 - CockroachDB now supports an optional frame exclusion clause in the specification of window frames. #28262
- Enabled collection of histograms by default during automatic statistics collection. #39743
- The
force_retry()function, which previously required admin privileges, is now available to all users to test their transaction retry loops. #39246 - When
DROP CONSTRAINTis executed in a transaction on a validated constraint, it no longer takes effect immediately, and subsequent writes will now fail if they violate that constraint. This is to enforce consistency when dropping constraints asynchronously in the schema changer. #39648 - The
ALTER PARTITIONstatement now supports applying a zone configuration to all the partitions of a table and its indexes that share the same partition name. The syntax for this isALTER PARTITION <partition name> OF INDEX <table name>@*. #39750 - CockroachDB now supports the
FOR UPDATEmodifier onSELECTstatements for compatibility with third-party tools. It has no effect becauseSERIALIZABLEtransactions do not require explicit write locks to maintain consistency. However, this means thatSELECT FOR UPDATEcannot be used as an advisory exclusive lock in CockroachDB, as used by some tools.#40206 SHOW PARTITIONSnow shows the full zone configuration statement used to configure the partition. #39814- Added the
pg_my_temp_schema()built-in for Postgres compatibility. #40194 - Added the overlaps (
&&), contains (@>), and contained_by (<@) operators for arrays. #40194 - Added
pg_get_function_identity_arguments()built-in. #40194 - Added the
width_bucket()builtin function. #39263 - Added range size information to
SHOW RANGES. #40281 - Added replica locality information to
SHOW RANGES. #39813 - Reordered columns in
SHOW RANGESoutput #40501 SHOW CREATE TABLEnow warns users if they have a partitioned table that does not contain zone configurations #40475- Added the
replication_constraint_stats,replication_stats, andreplication_critical_localitiessystem tables. These tables contain information about replication status, constraint conformance, and critical localities. #40625 - Common table expressions that are used a single time no longer create an optimization fence. #40673
EXPLAINnow has additional output that shows if a query will run with the vectorized execution engine, and if the query will be distributed across nodes. #40652- Column references are no longer allowed in
ROWSandRANGEclauses in window functions. #40832 - Added the
pg_function_is_visibleandpg_get_function_resultbuilt-in functions. #40902 - Added parsing for
FOR UPDATE [ OF tables ]form. #40962 - Mutations under
UNIONorUNION ALLare now disallowed;WITHshould be used on top of the union operation instead. This restriction is temporary and will be lifted in a future release. #40975 - Added support for the syntax for
oidwrapper arrays, likeREGCLASS[]. #40949 EXPLAIN ANALYSEcan now be used as an alias toEXPLAIN ANALYZE. #41090SHOW PARTITIONSnow displays inherited zone configurations. #40493- Added the
zone_idandsubzone_idcolumns tocrdb_internal.partitions. These columns form a link to the zone configurations incrdb_internal.zonesthat apply to the partitions. #40493 - Renamed the
config_yaml,config_sql, andconfig_protocolumns incrdb_internal.zonestoraw_config_yaml,raw_config_sql, andraw_config_proto. #40493 - Added the columns
full_config_sqlandfull_config_yamlto thecrdb_internal.zonestable. #40493 - Added
FOR ROWclause toSHOW RANGESto show range information for a particular row. #40954
Command-line changes
cockroach demonow accepts a--nodesflag, which permits customization of the size of the demo cluster. The default remains at1. #39088cockroach demonow accepts a--demo-localityflag, which makes it possible to set node localities. #39454cockroach demowithout an argument is now equivalent tocockroach demo movr. The previous behavior (no pre-defined dataset) is still available viacockroach demo --empty. #39298cockroach startnow writes the client URL to the file specified via--listen-url-fileas soon as the server is ready to accept connections. This also happens before the server detaches from the terminal when--backgroundis specified. #39300- Using
cockroach startwithout--joinis now deprecated and this mode of execution will be removed in a later version of CockroachDB. Consider usingcockroach start-single-nodeinstead or combinecockroach startwithcockroach init. #28495 - The advisory/informative check performed by
cockroach starton the validity of addresses contained in the node certificate is now relaxed to focus on the advertised node address, and to tolerate cases when the cert contains an IP address but a hostname is specified as advertised address. #39166 - Added locality information to the output of
cockroach node status. #39059 - Removed the deprecated
cockroach zoneCLI commands. Please use the equivalentSHOW ZONE CONFIGURATIONSandCONFIGURE ZONEcommands in a SQL client. #39177 - Added a new
cockroach start-single-nodecommand to start single-node clusters with replication disabled. #28495 - The
cockroach usersub-commands are now deprecated. Users and roles can be managed using SQL statements instead. This functionality will be removed in the next release. #39297 cockroach startnow accepts a--cluster-nameflag. If specified, it must match when a node connects to a cluster. This feature is meant for use in combination with the cluster ID, which is still checked in any case, to protect newly created nodes (that do not have a cluster ID yet) from joining the wrong cluster. To introduce a cluster name into an existing cluster without one, the new flag can be temporarily paired with--disable-cluster-name-verification. #39270cockroach startnow accepts a--sql-addrflag, which makes it possible to accept connections by clients on a separate TCP address and/or port number from the one used for intra-cluster (node-node) connections. This is aimed to enable firewalling client traffic from server traffic. #39305cockroach workload runandcockroach workload initnow support loading initial table data usingIMPORT. #35312cockroach workloadnow supports additional command-line parameters to customize the output, to facilitate the integration with third-party testing tools:- For tools that wish to observe the metrics more frequently than every second, a new flag
--display-everyis now supported, which can be used to specify the period between metric reports. This applies to both the JSON and textual output. - For tools that require a different output format than the default, a new
--display-formatargument is supported. For now only the formats "simple" (original output format) and "incremental-json" (RFC3339 timestamps, no summary row) are supported. #37929
- For tools that wish to observe the metrics more frequently than every second, a new flag
- Added
d,dt,du,lmetacommands to the cockroach sql interactive shell. #39141 - Added default cluster locality topologies for multi-node
cockroach democlusters. #40221 - Errors in
cockroach demoare no longer logged to the command-line interface. #40196 - Added
dclient-side command with no arguments as an alias fordt, for more ease of adoption bypsqlusers. #40309 - Added a MovR workload to run with
cockroach demo. #40271 - Added location information for default localities to populate Admin UI for
cockroach demo. #40229 cockroach demonow attempts to contact a license server to obtain a temporary license for demo usage. #40273cockroach demonow enables telemetry for demo clusters. You can opt out of telemetry by setting theCOCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTINGenvironment variable. For more information, see Diagnostics Reporting. #40273- Deprecated the
--initflag on theworkload runcommand. Useworkload initinstead. #40426 - Added support for a new
DELIMITEDdata format to theIMPORTstatement. #40482 - Added an option for
cockroach demoto automatically apply the geo-partitioned replicas topology to themovrdataset using the--geo-partitioned-replicasflag. #40355 cockroach sqlnow supports the--watchflag. When used to specified a duration, the client will repeat the statement(s) specified by-euntil an error occurs. This is intended for simple monitoring scenarios during development and testing. #40594- Added support for redirecting license acquisition to a replacement server (e.g., a suitably configured HTTP proxy), with the environment variable
COCKROACH_DEMO_LICENSE_URL. This enablescockroach demowith enterprise features in firewalled network environments. #41029
Admin UI changes
- Metrics about p90 and p99 percentiles of per-application average transaction time are now exposed in the Admin UI. #39429
Bug fixes
- Fixed a potential infinite loop in queries involving reverse scans. #39084
- Upgraded the
grpclibrary to fix connection state management bug. #39041 - Properly initialize proposal quota tracking to prevent quota leak which can hang imports or other AddSSTable operations. #39135
crdb_internal.rangescan now be used inside views. Note that such views can become invalid in future releases ifcrdb_internal.rangeschanges. #39195- Prevented panic due to recording into finished tracing spans caused by acknowledging an earlier failed re-proposal when a later re-proposal exists. #39203
- CockroachDB now properly computes the default port number if an IPv6 address is provided without port number with
--locality-advertise-addr. #39334 - Unary negatives in constant arithmetic expressions are no longer ignored. #39245
pg_catalognow shows correct type lengths forINTandFLOAT. #39337- Fixed a rare panic (message:
"raft group deleted") that could occur during splits. #39571 - Fixed a panic due to incorrect type inference of some
ARRAY(...)expressions. #39469 - Zone configurations now propagate to non-gossiped system tables. #39638
cockroach dumpnow works properly when handlingINTERLEAVEDtables, printing them outside ofCREATE TABLEstatements. #39486- Fixed internal errors generated during execution of some complicated cases of correlated subqueries. #39818
- Using
ORDER BYinside aggregation functions is now fully supported. #40242 - Lookup joins now spill to disk if the index lookups return more rows than can be stored in memory. #40208
- Fixed an internal planning error that could occur when a
DISTINCTorGROUP BYexpression was contained in a subquery. #40468 - Fixed an optimizer panic when building array access expressions. #40469
- Fixed faulty error when trying to delete a table and an unrelated index in the same transaction. #40516
ordinal_positionininformation_schema.columnsnow matchespg_attribute.attnumafter a column is dropped. #40485- Fixed fatal error with message
replica descriptor of local store not found. #39796 - Intents in a read's uncertainty interval are now considered as uncertain as committed values. This removes the potential for stale reads when a causally dependent transaction runs into unresolved intents from a causal ancestor. #40600
- Fixed a cluster migration bug that could occur in clusters running a mixture of v19.1 and v19.2. This bug was signaled by messages of the form:
X at key /Table/54 failed: unexpected value: ...Affected clusters should be updated to 19.2 or, if 19.1 is preferred, recreated from backup. #40464 - Fixed behavior of
SERIALcolumns on mixed-case tables whenexperimental_serial_normalization=sql_sequence. #40965 width_bucketfunction no longer errors on 0-length arrays #40626- Fixed an error that could occur when the optimizer creates a plan with a lookup semi- or anti-join nested inside another join. #40669
- Fixed a memory corruption error in the optimizer that could cause stack overflow. #40644
- The
cockroachcommands are now able to connect to a server with the environment variableCOCKROACH_URL. #40824 - Index hints used with
DELETEandUPDATEare now taken into consideration. #40715 - Fixed an issue with creating table indexes when the server is running on Windows and the store is on a file system that does not support hard links (e.g., FAT32 or network share) #40875
- Restore correct result generation for queries with index disjunctions on tables with multiple column families. #40898
- Fixed a stack overflow that could occur via use of
WITH. #40937 - Detailed crash reports now always appear in the main log file. If SQL audit reporting or statement logging was activated, crash reports (i.e., "panic messages") could previously be reported in the wrong file. #40941
- Users must now have admin privileges to cancel, pause, or resume jobs. #40978
- CockroachDB now properly removes excess secondary log files (SQL audit logging, statement execution logging, and RocksDB events). #41020
- CockroachDB now avoids the internal re-use of Replica objects to fix the following crashes:
- #38772 "
found rXXX:{-} [, next=0, gen=0?] in place of the RHS" - #39796 "
replica descriptor of local store not found in right hand side of split" - #40470 "
split trigger found right-hand side with tombstone" - #40257 "
snapshot widens existing replica, but no replica exists for subsumed key" #40892
- #38772 "
- Fixed panic in
cockroach workload movr run. #41072 - Resolved an issue with server hanging on
cockroach quiton Windows. #40994 - The
cockroach zip,cockroach nodeandcockroach usercockroachcommands now work properly when thedefaultdbdatabase is manually dropped and the connection URL does not specify a database.cockroach useris deprecated in 19.2. #41121 cockroach workload init ycsbnow works with--data-loader=import. #41071- Fixed bug where zone configurations on indexes could leak into configurations on partitions. [#41089][#41089
- Statements inside
[ ... ]can no longer refer to outer columns or common table expressions. #41158 - The SQL for non-
SELECTcommon table expressions now shows up inEXPLAINoutput. #41159 - Fixed bug that prevented changes to
kv.bulk_io_write.concurrent_addsstable_requestsfrom taking effect. #41195 - Fixed a memory leak that affected secondary logging (SQL audit logs, statement execution, and RocksDB logging). #41231
Performance improvements
- Sped up file-writing during bulk-ingestion. #37528
- Snapshots sent between replicas are now applied more performantly and use less memory. #38932
- Raft entries no longer wait to be applied to the RocksDB storage engine before signaling their success to clients, they now only wait until they are committed in their Raft log. #38954
- Adjusted load-based splitting QPS threshold to avoid over-splitting. #39687
IMPORTnow writes unsorted data directly, reducingIMPORT's overhead (but potentially increasing RocksDB compaction overhead). #39487- Improved performance for mutation statements with
RETURNINGthat are not inside an explicit transaction. #40871
Doc updates
- Added an Oracle migration guide, which guides users through the process of converting their schema and then exporting their data so they can load it into CockroachDB via
IMPORT. #5902 - Added an overview of MovR, CockroachDB's fictional vehicle-sharing dataset and application, and updated several SQL pages and examples to use the built-in MovR dataset, for example, Learn CockroachDB SQL. #5075, #5216
- Added a tutorial on how to use
EXPLAINto identify and resolve common SQL performance problems. #5178
Contributors
This release includes 724 merged PRs by 53 authors. We would like to thank the following contributors from the CockroachDB community:
- Arseni Lapunov
- Elliot Courant
- HyeonGyu Lee (Vazrupe) (first-time contributor)
- Jaewan Park
- Kevin Barbour (first-time contributor)
- Niels Hofmans (first-time contributor)
- Shishir Jaiswal (first-time contributor)
- Taufiq Rahman (first-time contributor)
- Zeming YU (first-time contributor)
- 贾德星 (first-time contributor)
v19.2.0-alpha.20190805
Release Date: August 5, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
Enterprise edition changes
- Added a
skip_missing_viewsoption forRESTOREto skip restoring views which cannot be restored because their dependencies are not being restored at the same time. #38597
SQL language changes
- Added support for a table reader that performs a loose index scan over the underlying table. The index scan table reader uses information about the index being scanned to skip unnecessary rows while scanning the table, allowing for some optimizations to be used for some types of queries. #38216
- Enabled automatic detection of compression schemes on filenames in
IMPORTthat have parameters after the path. #38521 - Added
check_constraintstable to theinformation_schemadatabase. #38545 - Added vectorized support for the
INselection operator. #38546 - Made some
pgwireerror codes for binary parameter decoding better match Postgres. #38365 - Added support for projection of the
INoperator. #38613 EXPLAINreturns more information about lookup joins. #38748- The first statement of a transaction will no longer return a transaction retry error if it is an
UPDATEorDELETE(this was already true forINSERT). #38668 - Non-fatal errors updating jobs while dropping a table are now ignored. #38617
- Added support for
NOT VALIDoption for Foreign Key constraints. #38663 - Added cluster setting
sql.stats.histogram_collection.enabledto enable collection of histograms during statistics collection. #38812 - Common Table Expressions (CTEs) may now be referenced from multiple locations in a query. #38670
CREATE TABLE ASnow allows user-defined primary keys and column families. #38904, #38589EXPLAIN (OPT,ENV)now returns a URL with the data encoded in the fragment portion. Opening the URL shows a page with the decoded data. Note that the data is processed in the local browser session and is never sent out. #38973
Command-line changes
cockroach dumpwill now more clearly refer to issue 37768 when it encounters a table with no visible columns, which (currently) cannot be dumped successfully. #38725- CockroachDB now requires square brackets to specify IPv6 addresses in
--join,--host,--listen-addrand similar flags, for example--listen-addr=[::1]. The syntax without square brackets had been deprecated since CockroachDB 2.1. #38924
Admin UI changes
- Dashboard graphs now offer a "Last 2 days" option in the time selector. #38852
- Corrected the units displayed for the
MVCC Intent Ageand theMVCC GC Bytes Agefields in the Range Status page. #38283
Bug fixes
- The
ALTER PARTITION x OF INDEX y CONFIGURE ZONE USINGsyntax no longer erroneously modifies the entire index's zone configuration. #38423 - Fixed an issue that prevented restoring some backups if they included tables that were partitioned by columns of a certain types while also interleaved by child tables. #38341
- Restoring multi-line CLI histories in the SQL shell no longer produces terminal anomalies when the cursor is on the last line of the screen. #38505
- Fixed a panic where generators in an aggregate function were not appropriately rejected. #38554
- Handled
database_nameand composite primary key forUNSPLIT ALL. #38338 - Ranges consisting of only one row (and historical versions of that row) are now correctly up-replicating. #38529
- Nodes that have been down now recover more quickly when they rejoin, assuming they weren't down for much more than the value of the
server.time_until_store_deadcluster setting (which defaults to 5 minutes). #38484 cockroach sqlfmtno longer stripsORDER BYfrom aggregates. #38623- Corrected issue that caused certain plans that contained both offsets and limits to return more rows than desired. #38667
- Fixed a bug that prevented inverted indexes from being created on JSON columns containing
NULLvalues. #38738 - Transaction retries in schema changes are again handled correctly. #38668
- On macOS, the server process can once again start in environments where the
kern.maxfilesperprocsysctl has been lowered. #38472 - Fixed a bug where dropping an index that a foreign key depends on could cause an inconsistent table state if the schema change were rolled back. #38783
- Fixed an issue where Cost-based optimizer-planned upserts that used lookup join that were run during column mutations on the table being upserted into could cause crashes or other issues. #38637
- Fixed incorrect results, or "unordered span" errors, in some cases involving exclusive inequalities with non-numeric types. #38880
- Prevented OOM conditions during lookup joins between tables with a very large n:1 relationship. #38837
sum_int()now errors on overflow instead of silently wrapping. #38921- Fixed bug in the optimizer causing a bad index for lookup join in some cases. #39016
- Backfills are no longer performed when a user explicitly sets the default value of a new column to be
NULL. #38910
Performance improvements
- Introduced new concurrent Raft proposal buffer, which increases the degree of write concurrency supported on a single range. #38343
- Grow stack prior to decoding grpc BatchRequest messages. #38107
- Batched raft entry application and coalesce writes to applied state for the batch. #38568
- Improved performance of the
BACKUPprocess. #37887
Build changes
- Building CockroachDB from source now requires node version 10. This was previously implicitly required. Now the requirement is explicit. #38955
- Building CockroachDB from source now requires Go 1.12 or higher. #38379
Doc updates
- Expanded single-region Kubernetes tutorial to cover running on Amazon’s hosted EKS. #5063
- Build an App with CockroachDB examples now include C++ and Clojure application examples for secure clusters. In addition, the client-side transaction logic has been updated in the Java, Python, and Go code examples. #5028, #5084, #5085, #5088
- Updated Common Errors document with specific error strings to aid user searches. #5033
- Documented a known limitation about hanged requests to a restarted node that needs Raft snapshots. #5018
- Updated the
BACKUPandRESTOREdocs to specify that thesystem.userstable is not included with a backup. To restore the table, you must explicitly include it. #5072
Contributors
This release includes 285 merged PRs by 33 authors. We would like to thank the following contributors from the CockroachDB community:
- Arseni Lapunov (first-time contributor)
- Elliot Courant (first-time contributor)
v19.2.0-alpha.20190701
Release Date: July 1, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
This release contains enhancements and bug fixes to the 19.2.0 series, including:
- Experimental support for using
IMPORT INTOto ingest CSV data into an existing table. Usage:IMPORT INTO t(a, b) CSV DATA ('gs://acme-co/employees.csv'). - Adding non-null constraints to existing columns with
ALTER TABLE ... SET NOT NULL. - Various updates to improve Postgres compatibility.
For more information, see below.
Backward-incompatible changes
- Integer and float columns of less than the max width will now be returned as their own type via the binary protocol. For example, an
int4column will be returned in 32 bits over the pgwire binary protocol instead of 64 bits. #37295
General changes
- Removed
kv.range_merge.manual_split_ttlcluster setting. #38004 - Go 1.12.5 is now required to build CockroachDB from source.
SQL language changes
- Added experimental support for using
IMPORT INTOto ingest CSV data into an existing table. Usage:IMPORT INTO t(a, b) CSV DATA ('gs://acme-co/employees.csv'). #37451 - Implemented SQL support for timestamp objects to have an optional precision value. #37920
- The logging format for statement execution (as enabled by
sql.trace.log_statement_execute) and audit logs (as enabled byALTER TABLE .. EXPERIMENTAL AUDIT SET ...) is extended with one column that reports internal execution retries performed automatically by the server. The valuezerois logged for the first retry attempt and thus corresponds to statements directly received from a client. #38028 - Added
WITH EXPIRATIONoption forSPLIT AT. #38004 - The
DISTINCTkeyword is now rejected when used in a window function. It was previously accepted, but ignored. #37912 - Added
UNSPLIT ALLstatement. #38023 - The
relamcolumn of thepg_catalogtablespg_classandpg_indexis now filled in correctly. #38164 - CockroachDB now supports the special case
set timezone = 'utc'as a special alias forset timezone = 'UTC'. The other time zone names are still case-sensitive as previously, pending resolution of issue #36864. #38195 - Added support for
ALTER TABLE ... ALTER COLUMN ... SET NOT NULL, which adds a non-null constraint to an existing column. #37554 pg_get_constraintdefnow outputsCHECKconstraints exactly the same way that Postgres does. #38318- The
format_typePostgres builtin now properly respects its second argument. #38318 SHOW CREATE TABLEnow prints foreign keys more like Postgres does. #38318- The
pg_type_is_visiblebuiltin from Postgres is now implemented. #38318 - Casting an
int2vectorto a string now produces a Postgres-compatible result. #38318
Command-line changes
- CockroachDB will now print out an error message and an informative hint if the time zone database is unusable. #38197
Admin UI changes
- The snapshots graph on the Replication page in the Web UI now includes learner snapshots. #38147
- Removed version information from login screen. #38140
- Added
Implicit Transactionflag to Statements and Statement Details page. #36341 - Checking the "skip should queue" checkbox in the Manual Enqueue Range advanced debugging page now works for the GC Queue. #38249
Bug fixes
- Fixed help text that erroneously labeled Encryption at Rest as experimental. #38228
- The
max_retriescolumn in the statement statistics now properly displays the maximum number of times a statement was retried. #38028 - Previously, due to a bug when estimating result set sizes in the optimizer, queries involving
intranges that were very large could result in poor plans being generated. #38036 - Fixed a bug where MVCC value at future timestamp is returned after a transaction restart. #38085
- The YCSB workload will no longer ignore the
--dboption. #38236 - Localities in the
crdb_internaltables were displayed JSON objects instead of strings. This format stripped them of their order, which is important in localities. This string format now exactly matches what is used in the command line for setting localities. #37944 - Fixed an incorrect type mismatch error when empty array values are used as default values (and potentially in other contexts). #38294
- Fixed a panic that could occur when decoding decimals as query parameters. #38304
- Nulls are now correctly handled by
MIN,SUM, andAVGaggregate functions when used as window functions. #38110 - Prevented a possible panic during
JSONBbinary query parameter parsing. #38346 - Prevented a possible panic during
INETbinary query parameter parsing. #38346 - Prevented a possible panic during
VARBITbinary query parameter parsing. #38346 - Prevented a possible panic during array binary query parameter parsing. #38346
- Fixed possible deadlock when storage engine write fails. #38170
- Fixed potential reappearance of deleted timeseries data, which could trip consistency checker. #38170
- Removed dependency on
sync_file_rangeon Linux platforms on which it returnsENOSYS, such as WSL (Windows Subsystem for Linux). #38170
Build changes
- A recent
libeditis now bundled with CockroachDB, which fixes some line editing bugs in the CockroachDB console. On platforms that include libedit as part of the base system, like macOS and FreeBSD, CockroachDB no longer links against the system libedit. #32623
Security
- Only check
CNon first certificate in file. #38163
Contributors
This release includes 125 merged PRs by 30 authors. We would like to thank the following contributors from the CockroachDB community:
- Utkarsh Gupta (first-time contributor)
v19.2.0-alpha.20190606
Release Date: June 6, 2019
A denial-of-service (DoS) vulnerability is present in CockroachDB v19.2.0 - v19.2.11 due to a bug in protobuf. This is resolved in CockroachDB v19.2.12 and later releases. When upgrading is not an option, users should audit their network configuration to verify that the CockroachDB HTTP port is not available to untrusted clients. We recommend blocking the HTTP port behind a firewall.
For more information, including other affected versions, see Technical Advisory 58932.
In addition to various enhancements and bug fixes, with this first alpha release of the v19.2.0 series, we would like to highlight the following changes:
Parallel Commit: The parallel commit feature is a major optimization that halves the latency of distributed transactions, including those caused by secondary indexes. Previously, transactions would involve two synchronous rounds of replication consensus - one for write operations performed in parallel via transaction pipelining and then another for the commit. Now, replication for the commit runs in parallel with other write operations, avoiding an extra synchronous round trip and dramatically increasing overall transaction performance. This is achieved via a new "implicit" commit status that is distributed yet atomic and durable and that resolves to an "explicit" centralized status quickly for the efficiency of conflicting transactions. Detailed documentation is coming soon; in the meantime, see the Parallel Commit RFC.
Business Source License (BSL): This release uses the Business Source License instead of the Apache 2.0 License. For more information on why we changed our licensing approach and some practical questions and implications, see our blog post and Licensing FAQs. The full details of the license can be found on Github.
Backward-incompatible changes
- Removed the deprecated
changefeed.min_high_watermetric; usechangefeed.max_behind_nanosinstead. #36915
General changes
- Added the
kv.range_merge.manual_split_ttlcluster setting. #37728 - CockroachDB now uses the Business Source License instead of the Apache License. See our blog post for details. #38007
Enterprise edition changes
- You can now alter the zone configuration for a secondary index partition using the syntax
ALTER PARTITION OF INDEX <tablename>@<indexname> CONFIGURE ZONE .... #36868
SQL language changes
- Zigzag joins will now be planned by the optimizer in certain cases. They were previously available via the
experimental_enable_zigzag_joinsession variable and can still be disabled by setting that value tofalse. #35911 - The
LATERALkeyword in aFROMclause is now supported. #36613 - The
information_schema.columnstable now gives more conformant output forvarbit,char,timestamp, andtimetypes. #36598 - Extended the list of advertised OIDs in
pg_typeto include additional array-related types, such as_int2vector,_regproc, etc. #36598 - CockroachDB now displays correct array type
oidsforoidvectorandint2vectortypes in thepg_typetable, for better PG compatibility. #36598 - CockroachDB now uses the Postgres-compatible name when auto-generating a column alias for a
CASTexpression. #36598 - Locale names in
COLLATEclauses now have dash characters mapped to underscore when printed as part of SQL syntax. #36598 - Parallel statement execution has been removed because the performance improvement it provided was absorbed by transactional pipelining, which doesn't require client buy in. The
RETURNING NOTHINGsyntax will still be accepted but becomes a no-op. #35959 - Only
SELECT,INSERT,UPDATE,UPSERT,DELETE,SHOW,EXPLAINare now supported as data sources using theSELECT ... FROM [ ... ]syntax. #36977 - The
atttypmodcolumn of thepg_attributecatalog table is now supported. #37230 - Dates are now fully Postgres-compatible, including support for sentinel values (
+/- infinity) and the Postgres date range (4714-11-24 BCto5874897-12-31). Existing dates outside of this range will be converted to the+/- infinitydates. #36938 - Added an overload for
now()that returns a date. #37284 - Added supported for the empty
default_tablespacesession variable for Postgres compatibility. #37333 - Permitted casting empty arrays to any array type. #37356
- Removed the legacy batch-by-batch index backfiller, as well as the
schemachanger.bulk_index_backfill.enabledcluster setting that could be disabled to activate it. #37213 - Removed the
transformoption fromIMPORT. #37338 - Added a new
locality_valuebuilt-in function that returns the value of the locality key given as its argument. #37369 - CockroachDB now returns
NaNinstead of an error during invalid decimal operations. #36964 - Foreign keys that are added to an existing table using
ALTER TABLEwill now be validated for existing rows, with improved performance compared to runningADD CONSTRAINTfollowed byVALIDATE CONSTRAINTpreviously. #37433 SHOW COLUMNS ... WITH COMMENTSnow supports printing out column comments using the optional phraseWITH COMMENT, e.gSHOW COLUMNS FROM mytable WITH COMMENT. #37174SHOW DATABASES WITH COMMENTSnow supports printing out database comments using the optional phraseWITH COMMENT, e.gSHOW DATABASES WITH COMMENT. #37175CREATE TABLEstatements now always trigger automatic statistics collection on a newly created table. #37611- Added the
split_timecolumn tocrdb_internal.rangesandcrdb_internal.ranges_no_leases. #37610 - Corrected interval math when multiplying or dividing by floats or ints. #37744
- Added the
ALTER TABLE/INDEX ... UNSPLIT AT ...statement. #37603 - Added support for the time value
24:00and variants. #37898 - CockroachDB now reports a dedicated SQL error code in the particular situation where a multi-statement transaction is committed but one of the schema change (DDL) operations failed. As in previous versions, CockroachDB leaves the transaction partly committed and partly aborted in that case; the new error code aims to facilitate the recognition of this case in client code. #37791
- Added support for a
SHOW INDEXES FROM DATABASEcommand. #37942
Command-line changes
- Included collected goroutine dumps in
cockroach debug zip. #36813 - Added the
movrworkload to thecockroach workloadcommand. This workload simulates a fictional ride sharing company and is based on our open-source MovR GitHub repository. #37709
Admin UI changes
- CockroachDB now includes only successfully executed statements in the statement counters. #37264
Bug fixes
- Fixed a roundtrip typecasting issue where an
INETcasted to aSTRINGand back to anINETwould fail. #37938 - Fixed a panic when
json_build_objectis called withBIT/VARBITvalues. #37057 - Fixed a panic when planning zigzag joins against unique indexes. #36139
RESTOREnow balances the work of ingesting data more evenly between nodes. #36665IMPORTnow balances the work of ingesting data more evenly between nodes. #36665- CockroachDB now computes the result of shifting bit arrays to the right properly and avoids generating invalid bit arrays. #36715
SHOW ZONE CONFIGURATIONno longer emits invalidALTERsyntax in its output when displaying the zone configuration for a table or index partition that is inheriting from the database or the default configuration. #36868SHOW ZONE CONFIGURATION FOR TABLE t PARTITION pno longer ignores the clausePARTITION pand now properly displays the zone configuration for that partition instead. #36868- When a node is restarted with another node's address, CockroachDB now performs fewer attempts to communicate with the wrong node. #34197
- It is now possible to write to columns of type
name[]. #37170 - Permitted planning of window functions within mutation statements (and others that cannot be distributed). #37171
- Avoided a source of internal connectivity problems that would resolve after restarting the affected node. #37204
- Permitted arrays and other sequences to be type-checkable in certain contexts when all of their elements are null. #36673
- CockroachDB now correctly escapes non-finite JSON number values when marshaling to text. #37331
- The
cockroach initcommand will now always properly report when a cluster is already initialized, even after the node that it's connecting to is restarted. #37399 - Automated table statistics no longer encounter "batch timestamp must be after replica GC threshold" errors on configurations with low TTL. #36500
- Fixed type inference of columns in subqueries for some expressions of the form
scalar IN (subquery). #37578 - Fixed a crash in apply join. #37597
- The
age()function is now correctly marked as impure, causing it to be unavailable in certain contexts. #37609 - Fixed a panic when constructing the error message for an invalid partitioning. #37689
- Fixed incorrect multiplication of
floatandintervaltypes. #37582 - Fixed a case in which
cockroach quitwould return success even though the server process was still running in a severely degraded state. #37668 - Fixed a potential source of (faux) replica inconsistencies that can be reported while running a mixed v19.1 / v2.1 cluster. This error (in that situation only) is benign and can be resolved by upgrading to the latest v19.1 patch release. Every time this error occurs a "checkpoint" is created which will occupy a large amount of disk space and which needs to be removed manually (see
<store directory>/auxiliary/checkpoints). #37668 - Prevented unlimited memory usage during SQL range deletions #36728
- Dividing floats by zero now returns an error instead of
-Inf. #37774 - Fixed incorrect results or "incorrectly ordered stream" error in some cases of queries with aggregations. #37713
- A null right operand now causes the sub-operator expression to return null. #37775
- Certain binary encodings of numeric/decimal values no longer result in values that are an order of magnitude off. #35115
- Fixed
GROUP BYfor empty arrays. #37901 - Fixed a race condition that could cause a panic during query planning. #37972
- The
cockroachcommand line utilities that internally use a RPC connection (e.g.,cockroach quit,cockroach init, etc.) once again properly support passing an IPv6 address literal via the--hostargument. #37977
Performance improvements
- Improved memory efficiency of bulk ingestion and index backfills buffering. #36743
- Fixed a significant performance regression when building CockroachDB with go1.12. #37967
Security improvement
- Stack memory used by CockroachDB is now marked as non-executable, improving security and compatibility with SELinux. #37939
Contributors
This release includes 728 merged PRs by 50 authors. We would like to thank the following contributors from the CockroachDB community:
- George Papadrosou
- Jaewan Park
- Simo Kinnunen (first-time contributor)
- Txiaozhe
- Vijay Karthik
- joowon (first-time contributor)
- lanzao