• Ubuntu 18.10 (cosmic)
• RHEL/CentOS 8
• Debian 10 (buster)

How can a system update cause index corruption?

By default, PostgreSQL uses the operating systems collations (as provided by the GNU C library). Since collations determine how strings are compared, changing them can affect the result of the ORDER BY clause in SELECT statements and the order of the keys in B-tree indexes. Changes to the behavior of ORDER BY can be annoying, but changing the order of an index can lead to incorrect query results and duplicate entries.

You may also see rows being stored in the wrong partition if using columns of type text, varchar, char, or citext in the partition key on range-partitioned tables.

What indexes are affected?

Only indexes involving columns of type text, varchar, char, or citex are affected. Databases or table columns using the “C” or “POSIX” locales and table columns using collations with the ICU provider are not affected.

You can identify affected indexes in all databases with the following query:

SELECT indrelid::regclass::text, indexrelid::regclass::text, collname, pg_get_indexdef(indexrelid)
FROM (SELECT indexrelid, indrelid, indcollation[i] coll FROM pg_index, generate_subscripts(indcollation, 1) g(i)) s
 JOIN pg_collation c ON coll=c.oid
WHERE collprovider IN ('d', 'c') AND collname NOT IN ('C', 'POSIX');

How can I correct this?

The simplest method is to rebuild the indexes by running REINDEX index_name for all affected indexes. However, this will block all access to the index and prevent DDL on the table during the process. Starting with PostgreSQL version 12 (to be released this fall), you can use REINDEX CONCURRENTLY to avoid these limitations. For older versions of PostgreSQL you can work around these limitations by creating a copy of the index using CREATE INDEX CONCURRENTLY, then dropping the original index as shown below:

CREATE INDEX CONCURRENTLY myindex2 ON mytable (textcol);
DROP INDEX myindex;
ALTER INDEX myindex2 RENAME TO myindex;

It should be noted that the above will not work for primary keys. Additionally sufficient storage for both indexes will be needed for the duration of the process described above.

You can find more information about the locale data changes here: https://wiki.postgresql.org/wiki/Locale_data_changes

and the Postgres documentation on managing collations here: https://www.postgresql.org/docs/10/collation.html#COLLATION-MANAGING ]]> https://blog.crunchydata.com/blog/glibc-collations-and-data-corruption Fri, 02 Aug 2019 05:00:00 EDT 2019-08-02T09:00:00.000Z 2019-08-02T09:00:00.000Z https://www.crunchydata.com/blog/performing-a-major-postgresql-upgrade-with-pg_dumpall For most major upgrades using a utility such as pg_upgrade or a replication tool such as pglogical will be the best solution. However if these options are not available, pg_dumpall can be used to perform a major upgrade. What follows is a guide on how you can safely upgrade your database to a newer version of PostgreSQL with pg_dumpall.

Install and Initialize New System

You will first need to install the latest PostgreSQL binaries on the new system (hostname new in this example). Once the binaries are installed you will need to initialize the new instance with the initdb command:

initdb -D /path/to/pgdata/

The new instance can now be started

pg_ctl -D /path/to/pgdata/ -l logfile start

Confirm Connectivity

You can confirm the new instance is started and is accessible by running the following on the new instance:

psql -U postgres

To test connectivity to the old system (hostname old in this example) by running the following on the new instance:

psql -h old -U postgres

Migrate Data

Once connectivity is confirmed the data can be migrated from old to new with the following command run on the new instance:

pg_dumpall -h old -U postgres | psql ---single-transaction --no-psqlrc -h new -U postgres

By running the restore in a single transaction, if any one command fails the entire migration will be rolled back. This helps maintain data consistency. Check out the documentation for more info on psql.

After the command completes all data has been migrated, however the configuration from the old system will need to be migrated manually. You will also need to update any application settings to point to the new instance. The old system should be kept until sufficient testing is performed on the new instance. ]]> https://blog.crunchydata.com/blog/performing-a-major-postgresql-upgrade-with-pg_dumpall Mon, 26 Nov 2018 04:00:00 EST 2018-11-26T09:00:00.000Z 2018-11-26T09:00:00.000Z