<?xml version="1.0" encoding="UTF-8" ?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" version="2.0"><channel><title>Greg Sabino Mullane | CrunchyData Blog</title>
<atom:link href="https://www.crunchydata.com/blog/author/greg-sabino-mullane/rss.xml" rel="self" type="application/rss+xml" />
<link>https://www.crunchydata.com/blog/author/greg-sabino-mullane</link>
<image><url>https://www.crunchydata.com/build/_assets/greg-sabino-mullane.png-JXL7XWOF.webp</url>
<title>Greg Sabino Mullane | CrunchyData Blog</title>
<link>https://www.crunchydata.com/blog/author/greg-sabino-mullane</link>
<width>835</width>
<height>834</height></image>
<description>PostgreSQL experts from Crunchy Data share advice, performance tips, and guides on successfully running PostgreSQL and Kubernetes solutions</description>
<language>en-us</language>
<pubDate>Thu, 11 Dec 2025 08:00:00 EST</pubDate>
<dc:date>2025-12-11T13:00:00.000Z</dc:date>
<dc:language>en-us</dc:language>
<sy:updatePeriod>hourly</sy:updatePeriod>
<sy:updateFrequency>1</sy:updateFrequency>
<item><title><![CDATA[ Postgres 18 New Default for Data Checksums and How to Deal with Upgrades ]]></title>
<link>https://www.crunchydata.com/blog/postgres-18-new-default-for-data-checksums-and-how-to-deal-with-upgrades</link>
<description><![CDATA[ Postgres 18 defaults to checksums on. This is a good feature for data integrity but might catch you off guard with an upgrade.  ]]></description>
<content:encoded><![CDATA[ <p>In a recent Postgres <a href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=04bec894a04">patch</a> authored by Greg Sabino Mullane, Postgres has a new step forward for data integrity: <strong>data checksums are now enabled by default.</strong><p>This appears in the release notes as a fairly minor change but it significantly boosts the defense against one of the sneakiest problems in data management - silent data corruption.<p>Let’s dive into what this feature is, what the new default means for you, and how it impacts upgrades.<h2 id=what-is-a-data-checksum><a href=#what-is-a-data-checksum>What is a data checksum?</a></h2><p>A data checksum is a simple but powerful technique to verify the integrity of data pages stored on disk. It's like a digital fingerprint for every 8KB block of data (a "page") in your database.<ul><li><strong>Creation:</strong> When Postgres writes a data page (table and indexes) to disk, it runs an algorithm on the page's contents to calculate a derived, small value—the <strong>checksum</strong>.<li><strong>Storage:</strong> This checksum is stored in the page header alongside the data.<li><strong>Verification:</strong> Whenever Postgres reads that page back from disk, it immediately recalculates the checksum from the data and compares it to the stored value.</ul><p>If the two values do not match, it means the data page has been altered or corrupted since it was last written. This is important because data corruption can happen <em>silently.</em> By detecting a mismatch, Postgres can immediately raise an error and alert you to a potential problem. Checksums are also an integral part of <a href=https://github.com/pgbackrest/pgbackrest>pgBackRest</a> which uses these checksums to verify backups.<h2 id=what-is-initdb-and-why-does-it-matter><a href=#what-is-initdb-and-why-does-it-matter>What is initdb and why does it matter?</a></h2><p>The <code>initdb</code> command in Postgres is the utility used to create a new Postgres database cluster and initializes the data directory where Postgres stores all the permanent data. When you run initdb, it does things like:<ol><li>create the directory structure<li>create the template databases like <code>template1</code> and <code>postgres</code><li>populate the initial system catalog tables<li>create the initial version of the server configuration files<li>enable and start keeping track of checkums</ol><p>The syntax often looks something like this:<pre><code class=language-bash>/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
</code></pre><p>As an end user who uses cloud managed Postgres or even a local tool like Postgres.app, you generally never see the <code>initdb</code> command because it is a one-time administrative setup task.<h2 id=the-new-default---data-checksums-for-initdb><a href=#the-new-default---data-checksums-for-initdb>The new default <code>--data-checksums</code> for initdb</a></h2><p>In the past database admins had to manually add the <code>--data-checksums</code> flag when running initdb to enable this feature. If you forgot or didn’t know about this feature, the new cluster was created without these built-in integrity checks.<p>The default behavior of initdb is now to <strong>enable data checksums</strong> every time Postgres is initiated.<ul><li>old command - checksums OFF by default: <code>initdb -D /data/pg14</code><li>new default command - checksums ON by default: <code>initdb -D /data/pg18</code></ul><p>This is generally a win for Postgres best practices. Every new database cluster is now automatically equipped with this corruption defense, requiring no extra effort.<h3 id=--no-data-checksums><a href=#--no-data-checksums><code>--no-data-checksums</code></a></h3><p>You might have a very specific reason to disable checksums and you can explicitly opt out using the new flag:<pre><code class=language-sbash>initdb --no-data-checksums -D /data/pg18
</code></pre><h2 id=checksums-and-pg_upgrade><a href=#checksums-and-pg_upgrade>Checksums and <code>pg_upgrade</code></a></h2><p>While the new default is great, it may introduce a compatibility issue for those doing a major version upgrade using the <code>pg_upgrade</code> utility.<p>pg_upgrade works by connecting an old data directory to a new data directory and a fundamental requirement is that both clusters must have the same checksum setting—either both ON or both OFF.<p>If you are upgrading an older Postgres cluster that was created before this change, chances are it has checksums disabled and pg_upgrade will fail because the settings mismatch.<p>In an upgrade pinch, to upgrade a non-checksum-enabled cluster, you can use the new <code>--no-data-checksums</code> flag when initializing the new cluster to make the settings align.<h3 id=upgrading-an-existing-postgres-database-to-checksums><a href=#upgrading-an-existing-postgres-database-to-checksums>Upgrading an existing Postgres database to checksums</a></h3><p>Instead of continuing forever with no data checksums, the better long term solution is to add checksums to your database before the next upgrade. Sadly, there’s really no way to do this without some downtime and a restart. Adding checksums to an existing database can be a slow process with a large database. There’s a <a href=https://www.crunchydata.com/blog/fun-with-pg_checksums>pg_checksums utility</a> to help with this which is well documented.<p>We have helped a few folks with this issue. For larger no-downtime environments, you can add the checkums on a replica machine and then fail over to that.<h2 id=summary><a href=#summary>Summary</a></h2><p>Postgres checksums are a great feature - and will be the default in the future. If you haven’t used checksums in the past, you may want to start planning now for adding them, especially since a self managed major version upgrade will require a bit of extra thinking. ]]></content:encoded>
<category><![CDATA[ Postgres 18 ]]></category>
<author><![CDATA[ Greg.Sabino.Mullane@crunchydata.com (Greg Sabino Mullane) ]]></author>
<dc:creator><![CDATA[ Greg Sabino Mullane ]]></dc:creator>
<guid isPermalink="false">fa1787ed297110b99885d60008c312cb0ebd13f901f1167f84a7af3a4dcf9755</guid>
<pubDate>Thu, 11 Dec 2025 08:00:00 EST</pubDate>
<dc:date>2025-12-11T13:00:00.000Z</dc:date>
<atom:updated>2025-12-11T13:00:00.000Z</atom:updated></item>
<item><title><![CDATA[ Postgres Troubleshooting: Fixing Duplicate Primary Key Rows ]]></title>
<link>https://www.crunchydata.com/blog/postgres-troubleshooting-fixing-duplicate-primary-key-rows</link>
<description><![CDATA[ Corrupted unique index? A recent glibc update invalidate a primary key index? Greg has seen a few duplicate primary key issues recently and steps through how to fix these with some special tricks and functions. ]]></description>
<content:encoded><![CDATA[ <p>Someone recently asked on the Postgres mailing lists about how to remove unwanted duplicate rows from their table. They are “unwanted” in that sense that the same value appears more than once in a column designated as a primary key. We’ve seen an uptick in this problem since glibc was kind enough to change the way they sorted things. This can lead to invalid indexes when one upgrades their OS and modifies the underlying glibc library.<p>One of the main effects of a corrupted unique index is allowing rows to get added which should be caught by the primary key. In other words, for a table with a primary key on a column named “id”, you might observe things like this:<pre><code class=language-sql>-- Can you spot the problem?
SELECT id FROM mytable ORDER BY id LIMIT 5;

 id
----
  1
  2
  3
  3
  4
(5 rows)
</code></pre><p>Without knowing anything else about the problem, what’s the first step to solving it? If you said take a backup, you are correct! Make a fresh backup anytime you think something is wrong with your database, or before any attempt to fix such a problem.<p><strong>Aside: can we simply reindex?</strong> No - a unique index cannot be created (or recreated) as long as there are duplicate rows. Postgres will simply refuse to do so, as it violates the uniqueness we are trying to enforce. So we must delete the rows from the table.<p>Here is one careful recipe for fixing the problem of duplicated primary key entries in your Postgres table. Obviously you need to adjust this as needed for your situation, but just walk through it slowly and make sure you understand each step. Especially if this happens in production (spoiler: it almost always happens in production).<h3 id=1-debugging-aids><a href=#1-debugging-aids>1. Debugging aids</a></h3><p>The first thing we want to do may look a little strange:<pre><code class=language-sql>-- Encourage not using indexes:
set enable_indexscan = 0;
set enable_bitmapscan = 0;
set enable_indexonlyscan = 0;
</code></pre><p>Because bad indexes are the primary way bad rows like this get into our database, we cannot trust them. These low-level debug aids are a way of telling the Postgres planner to prioritize other ways of getting the data. In our case, that means hitting the table directly and not looking things up in the indexes.<h3 id=2-run-a-quick-sanity-check><a href=#2-run-a-quick-sanity-check>2. Run a quick sanity check</a></h3><pre><code class=language-sql>-- Sanity check. This should return a number greater than 1. If not, stop.
set search_path = public;
select count(*) from mytable where id = 3;
</code></pre><p>Before we begin, we want to make sure we have the correct table. The search_path is a safety measure, as is the lookup on the table. Since the id is the primary key of the table, it should return a count of 1 for each value. In our case, we know this table has multiple entries for id “3”, so this is mostly a sanity check that we are about to operate on the correct patient. We expect to get back a number greater than “1”. For a working primary key, the only values ever returned should be 0 or 1.<h3 id=3-create-a-backup-always-><a href=#3-create-a-backup-always->3. Create a backup (always 😉)</a></h3><pre><code class=language-sql>-- Make a backup:
create table mytable_backup as select * from mytable;
</code></pre><p>Before we begin, we want to copy all of the existing rows from the table into a new backup table. Again, this does not take the place of a full and complete backup of your entire database (always step 0), but it’s another good safety feature.<h3 id=4-make-a-test-table><a href=#4-make-a-test-table>4. Make a test table</a></h3><pre><code class=language-sql>--  Test out the process on a subset of the data:
create table test_mytable as select * from mytable where id &#60 30;
create table test_mytable_duperows_20250317 (like mytable);
</code></pre><p>It’s always a good idea to test things first on a test table. In our case, in a smaller version of the actual table. Because we know that there are problematic rows at id 3, we created a new table that contains those rows. We also create a new empty table called test_mytable_20250317 which is going to hold the duplicate rows that get removed. The date at the end tells future viewers when the table was created.<h3 id=5-start-the-cleanup-in-a-replica-session><a href=#5-start-the-cleanup-in-a-replica-session>5. Start the cleanup in a replica session</a></h3><p>From here on out, we are going to start the actual cleanup. We start a transaction, and then set our session_replication_role to replica, which is an advanced (and dangerous) command that disables all triggers and rules. Normally this is not a good practice, but we want to do this in case there are any foreign keys that may prevent us from removing the bad rows. In addition, we do this as <code>SET LOCAL</code> instead of just <code>SET</code> which ensures that this setting goes back to normal at the next <code>COMMIT</code> or <code>ROLLBACK</code>.<pre><code class=language-sql>begin;
set local session_replication_role = 'replica';
</code></pre><p>Because we just created this test table, we know that it has no triggers and is not linked to any other tables via foreign keys, but we want to make this test as close to possible as the actual table modification, so we leave the session_replication_role modification in place.<h3 id=6-cleanup-duplicate-rows-with-a-function><a href=#6-cleanup-duplicate-rows-with-a-function>6. Cleanup duplicate rows with a function</a></h3><pre><code class=language-sql>begin;

set local session_replication_role = 'replica';

with goodrows as (
  select min(ctid) from TEST_mytable group by id
)
,mydelete as (
  delete from TEST_mytable
  where not exists (select 1 from goodrows where min=ctid)
  returning *
)
insert into TEST_mytable_duperows_20250317 select * from mydelete;

reset session_replication_role;

commit;
</code></pre><p>So we issue a begin, set the session_replication_role, run a single SQL statement, reset the session_replication_role, and finally commit. That SQL statement is doing some heavy lifting, so let’s break it down.<p><code>select min(ctid) from TEST_mytable group by id</code> The first thing we need to do is figure out a way to ferret out which rows are duplicated. As the <code>id</code> column is supposed to be unique (as all primary key columns are), we know that any ids appearing more than once need a tie-breaker. Each row in Postgres has a hidden column named “ctid” that stands for column tuple identifier, and is basically a pointer to exactly where the actual physical row is located. Therefore, it is always unique. If we group by the id column, we can pull a single ctid for each unique id by asking for the “lowest” ctid (it doesn’t really matter if we use min() or max() or something else, just as long as we pick only one).<p>We are going to store that information and use it to help with the delete, so we start a cte via the <code>WITH</code> command, and name this one as <code>goodrows</code> .<pre><code class=language-sql>delete from TEST_mytable
where not exists (select 1 from goodrows where min=ctid)
returning *
</code></pre><p>The next step is to delete all duplicate rows that do NOT come from our goodrows list which was just created. So each of the duplicated rows will have distinct ctids, and we are going to delete all but one of them for each id. The final <code>RETURNING *</code> bit tells delete to send back complete information about every row that was deleted.<pre><code class=language-sql>insert into test_mytable_duperows_20250317 select * from mydelete;
</code></pre><p>Finally, we take the output of the delete and store it into our table. In this way, the rows are deleted, but we still have a complete list of which rows were removed, for debugging and forensics.<p>At this point, the duplicated rows should be removed, and inside the “duperows” tables. It is probably best to examine both this table and the test_mytable one, to make sure this all worked as expected.<h3 id=7-run-the-function-on-the-live-table><a href=#7-run-the-function-on-the-live-table>7. Run the function on the live table</a></h3><p>When ready, you can re-run the same code, but replace the test table with the actual one:<pre><code class=language-sql>create table mytable_duperows_20250317 (like mytable);

begin;

set local session_replication_role = 'replica';

with goodrows as (
  select min(ctid) from mytable group by id
)
,mydelete as (
  delete from mytable
  where not exists (select 1 from goodrows where min=ctid)
  returning *
)
insert into mytable_duperows_20250317 select * from mydelete;

reset session_replication_role;

commit;
</code></pre><h3 id=8-reindex><a href=#8-reindex>8. Reindex</a></h3><p>As a final step, we want to rebuild those suspect indexes. Even though the duplicated rows are gone, the index may still have wrong information. The <code>REINDEX</code> command is basically a drop and recreate, so we do that for all indexes that may exist in our table:<pre><code class=language-sql>reindex table mytable;
</code></pre><p>Now we can also return Postgres to normal plan settings with <code>enable_indexscan</code>, <code>enable_bitmapscan</code>, and <code>enable_indexonlyscan</code> all set to <code>1</code>.<p>That’s all the steps! When in doubt, reach out to your local Postgres expert if you are not 100% sure of your steps, as corruption is not something to take lightly. ]]></content:encoded>
<category><![CDATA[ Production Postgres ]]></category>
<author><![CDATA[ Greg.Sabino.Mullane@crunchydata.com (Greg Sabino Mullane) ]]></author>
<dc:creator><![CDATA[ Greg Sabino Mullane ]]></dc:creator>
<guid isPermalink="false">274e2c30eb9fb36b07f66770214c20bdb3acb3b0f973ca61f125287b18dd0f75</guid>
<pubDate>Wed, 19 Mar 2025 10:00:00 EDT</pubDate>
<dc:date>2025-03-19T14:00:00.000Z</dc:date>
<atom:updated>2025-03-19T14:00:00.000Z</atom:updated></item>
<item><title><![CDATA[ When Does ALTER TABLE Require a Rewrite? ]]></title>
<link>https://www.crunchydata.com/blog/when-does-alter-table-require-a-rewrite</link>
<description><![CDATA[ Greg found a way to test if tables get a full rewrite or not by using some crafty Postgres internals. ]]></description>
<content:encoded><![CDATA[ <p>It is rare that a Postgres table keeps the exact same structure year after year. New columns get added. Old columns get dropped. Column data types need to change. Those are all done with the ALTER TABLE command. One big drawback to these changes is that they may force a complete table rewrite. A rewrite means a completely new copy of the table is created, and then the old one is dropped. This can take a very long time for large tables. Worse, everything else is blocked/locked from using the table, so your application may need downtime.<p>So which commands need a full table rewrite? Which only needs a split-second lock? The <a href=https://www.postgresql.org/docs/current/sql-altertable.html>alter table documentation</a> has some guidelines, but at the end of the day, you want to know for sure if your production application is going to require major downtime or not. Presented here is a quick recipe for safely determining if a rewrite will happen. In short make a copy of the table, modify that copy, see if the underlying file on disk for that table has changed.<p>Let’s look at what happens when a table is modified. For this, we will use the good old reliable <em>pgbench_accounts</em> table, which gets created when you run <code>pgbench -i</code>. We can simulate the effects of a long-running table rewrite by putting our statement into a transaction. For this first one, let’s add a new column to it and see what locks are being held:<pre><code class=language-sql>greg=> begin;
BEGIN

greg=*> alter table pgbench_accounts add chocolates int;
ALTER TABLE

greg=*> select locktype, mode from pg_locks
        where relation::regclass::text = 'pgbench_accounts';
 locktype |        mode
----------+---------------------
 relation | AccessExclusiveLock
</code></pre><p>The <strong>AccessExclusiveLock</strong> is on the entire table, and is a very, very strong lock that blocks almost all other access to the table from other processes. Let’s start another process and see what happens:<pre><code class=language-sql>-- Without the lock_timeout, this update would hang forever,
-- or until the other transaction commits:
greg=> set lock_timeout TO '5s';
SET
greg=> update pgbench_accounts set bid = bid;
ERROR:  canceling statement due to lock timeout
LINE 1: update pgbench_accounts set bid = bid;

-- This lock also stops SELECT statements as well!
-- (another way to not wait forever is with a statement_timeout)
greg=> set statement_timeout = '500ms';
SET
greg=> select * from pgbench_accounts limit 10;
ERROR:  canceling statement due to statement timeout
</code></pre><p>This confirms that ALTER TABLE can prevent your application from accessing the table. Yet, all is not lost, watch what happens when we run it without a commit:<pre><code class=language-sql>greg=> \timing on
Timing is on.
greg=> alter table pgbench_accounts add chocolates bigint;
ALTER TABLE
Time: 2.149 ms
</code></pre><p>Two milliseconds is quite fast. Most applications should be able to easily deal with that. One caveat is that the lock still needs to be acquired. To do this, it needs to have complete solo control of the table for a split second. For a very busy table, this may take some coordination with your application. The table rewrite may require that your application be blocked for hours, or even days!<h2 id=create-a-test-table-with-create-table-like><a href=#create-a-test-table-with-create-table-like>Create a test table with CREATE TABLE LIKE</a></h2><p>To start, we always want to work on a simulacrum of your important production table. In other words, a copy of the table structure, but without all the table data. Postgres provides a handy way to do this with the LIKE clause of the CREATE TABLE command. You may specify which optional parts of the table to copy over. For our purposes, we only need the column defaults, so we can write this:<pre><code class=language-sql>CREATE UNLOGGED TABLE gregtest (LIKE pgbench_accounts INCLUDING defaults)
</code></pre><p>Might as well make this <a href=https://www.crunchydata.com/blog/postgresl-unlogged-tables>an unlogged table</a>, as this will be a very temporary construct. If we compare the two tables, their structure is identical:<pre><code>greg=> \d pgbench_accounts
              Table "public.pgbench_accounts"
  Column  |     Type      | Collation | Nullable | Default
----------+---------------+-----------+----------+---------
 aid      | integer       |           | not null |
 bid      | integer       |           |          |
 abalance | integer       |           |          |
 filler   | character(84) |           |          |

 greg=> \d gregtest
             Unlogged table "public.gregtest"
  Column  |     Type      | Collation | Nullable | Default
----------+---------------+-----------+----------+---------
 aid      | integer       |           | not null |
 bid      | integer       |           |          |
 abalance | integer       |           |          |
 filler   | character(84) |           |          |
</code></pre><h2 id=was-a-table-rewritten><a href=#was-a-table-rewritten>Was a table rewritten?</a></h2><p>Now, we can ALTER our new table. But how do we determine if the table was rewritten? As this table has no rows, we need another way to detect it rather than seeing how long it takes. Every table in Postgres is mapped to one or more physical files in the Postgres data directory. We can see the file for the table with the <strong>pg_relation_filenode</strong> function. If the output changes, we know that it has been rewritten. First, let’s add a simple BIGINT column with no default, which should not cause a rewrite:<pre><code class=language-sql>greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495135
greg=> alter table gregtest add monkeys bigint;
ALTER TABLE

greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495135
</code></pre><p>Notice that our filenode has not changed. So we still need the Access Exclusive lock, but a rewrite did not happen. Let’s run an example that does rewrite the table:<pre><code class=language-sql>greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495135

greg=> alter table gregtest add foobar4 bigint default random(1,10);
ALTER TABLE

greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495140
</code></pre><p>Per the documentation, a non-static default will force a rewrite. In this case, all the existing rows will have a random number from 1 to 10 inserted into them. Although this table has zero rows, the rewrite happened anyway.<p>Another common use case is changing the data type of an existing column. As I know from bitter experience, changing from an int to a bigint always requires a rewrite:<pre><code class=language-sql>greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495140
greg=> alter table gregtest alter column bid type bigint;
ALTER TABLE

greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495144
</code></pre><p>There are a few data type changes that do not require a rewrite, so you should always TIAS ("try it and see"). Does going from an 8-byte bigint to a 4-byte force a rewrite?<pre><code class=language-sql>greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495144

greg=> alter table gregtest alter bid type integer;
ALTER TABLE

greg=> select pg_relation_filenode('gregtest');
 pg_relation_filenode
----------------------
              1495147
</code></pre><p>Yep! Even though anything we put into a bigint can “fit” inside an integer, they are represented differently on disk and thus need a table rewrite.<h2 id=summary><a href=#summary>Summary</a></h2><p>Hopefully this helps you to learn if a rewrite is needed or not, without actually messing with your production table! Don’t forget to drop that test table when you are done. If you need to make a change that requires a table rewrite, but cannot afford the downtime, there are <a href=https://learn.microsoft.com/en-us/shows/citus-con-postgres-2022/advanced-int-to-bigint-conversions>more complex options</a> - such as using logical replication - that can help.<p>There are a few other operations that can cause a full table rewrite, but they are mostly things you probably will not encounter very often:<ul><li>VACUUM FULL<li>CLUSTER<li>REFRESH MATERIALIZED VIEW<li>Changing a table from LOGGED to UNLOGGED, or vice-versa.</ul> ]]></content:encoded>
<category><![CDATA[ Production Postgres ]]></category>
<author><![CDATA[ Greg.Sabino.Mullane@crunchydata.com (Greg Sabino Mullane) ]]></author>
<dc:creator><![CDATA[ Greg Sabino Mullane ]]></dc:creator>
<guid isPermalink="false">6df87b3ea8132002a71840d298aa7adfe9f1c73b830b200104a924c1bff0a81d</guid>
<pubDate>Mon, 27 Jan 2025 09:30:00 EST</pubDate>
<dc:date>2025-01-27T14:30:00.000Z</dc:date>
<atom:updated>2025-01-27T14:30:00.000Z</atom:updated></item>
<item><title><![CDATA[ Enhanced Postgres Release Notes ]]></title>
<link>https://www.crunchydata.com/blog/enhanced-postgres-release-notes</link>
<description><![CDATA[ Just out in Postgres 17, the Postgres release notes now link to git commit messages! ]]></description>
<content:encoded><![CDATA[ <p>There is something new you may not have seen in the <a href=https://www.postgresql.org/docs/17/release-17.html>release notes for Postgres 17</a>. No, not a new feature - I mean inside the actual release notes themselves! The Postgres project uses the git program to track commits to the project, and now each item in the release notes has a link to the actual commit (or multiple commits) that enabled it.</p><!--more--><p>You may have missed it if you were scanning the release notes, but after the end of each specific item in the release note is a small “section” symbol which looks like this: <strong>§</strong>. Each of these symbols is a link leading to the relevant commit for that item. Here’s what it looks like on the Postgres 17 release notes page:<p><img alt=pg_commit_plain.png loading=lazy src=https://imagedelivery.net/lPM0ntuwQfh8VQgJRu0mFg/3e213b8a-045b-4db0-befc-ff20f4076500/public><p>Clicking the section symbol will send you to the GIT link for each individual patch, for example, this one:<p><img alt="hackers email messsage"loading=lazy src=https://imagedelivery.net/lPM0ntuwQfh8VQgJRu0mFg/a7514091-2a9a-4478-f087-0c9f4c92a000/public><p>Note that there’s a “Discussion” link in each commit linking back to the full thread on the hackers channel.<p>Writing the release notes is hard work, and involves a good bit of debate in the community. We have to make sure we list all the changes, in a concise yet comprehensible manner, and decide what level of detail to include. Oftentimes, this level is not sufficient for people interested in learning about this feature. That’s where these new commit links are invaluable. They link to the git commit, which not only lets you see the exact code changes that were made, but show the actual commit message, which has more detail than what can be provided in the release notes.<h3 id=postgres-notes-in-lots-of-places><a href=#postgres-notes-in-lots-of-places>Postgres Notes in Lots of Places</a></h3><p>Postgres release notes appear in lots of different places so this addition will surely make its way into other downstream projects. This new link also now appears on “<a href="https://www.notion.so/cff5e34f441140d687828e40e50f603c?pvs=21">postgres all versions</a>” - a project that I maintain that collates the information from all of the release notes for every version of Postgres (over 500 now!) into a single page. To make the link more visible and easier to use, I converted it to a larger Unicode scroll symbol, then added some tooltip tech to make it show information about the link like so:<p><img alt=pg_commit_tooltip.gif loading=lazy src=https://imagedelivery.net/lPM0ntuwQfh8VQgJRu0mFg/2f220f04-b12d-4fe3-bd85-d58ef452ee00/public><p>There was some community debate about including this, and about how prominent to make it. I think we ended up a little too cryptic and subtle with this section mark, but I welcome the addition and find it extraordinarily useful. I’d love to see the www Postgres project adopt a bigger symbol and tooltips in the future too!<p>The Postgres project takes the quality of its code very seriously, and that extends to the git commit messages as well. You will find them quite detailed; they not only describe the change that has been made, but have a link back to the mailing list discussion, as well as giving credit to the people who authored the change, discovered the bug, or otherwise helped out. A great new addition to the release notes! ]]></content:encoded>
<category><![CDATA[ Postgres 17 ]]></category>
<author><![CDATA[ Greg.Sabino.Mullane@crunchydata.com (Greg Sabino Mullane) ]]></author>
<dc:creator><![CDATA[ Greg Sabino Mullane ]]></dc:creator>
<guid isPermalink="false">fe2e381fe3d770d3ce23b77158ba10800b5b32895408db4affbd897f334bd59c</guid>
<pubDate>Wed, 09 Oct 2024 17:30:00 EDT</pubDate>
<dc:date>2024-10-09T21:30:00.000Z</dc:date>
<atom:updated>2024-10-09T21:30:00.000Z</atom:updated></item>
<item><title><![CDATA[ Understanding the Postgres Hackers Mailing List Language ]]></title>
<link>https://www.crunchydata.com/blog/understanding-the-postgres-hackers-mailing-list</link>
<description><![CDATA[ Greg analyzed a year's worth of email exchanges on the Postgres hackers email list. He's got a primer on all the lingo and abbreviations used to help you follow along. ]]></description>
<content:encoded><![CDATA[ <p>The Postgres hackers mailing list (<a href=mailto:pgsql-hackers@postgresql.org>pgsql-hackers@postgresql.org</a>) is an invaluable resource for anyone wanting to contribute to the PostgreSQL code. The Postgres project does not use PRs (pull requests) or GitHub issues. So if you want to contribute an idea, or help with code reviews, the hackers mailing list is the canonical way to do so. More information on contributing is on the Postgres wiki at: <a href=https://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer>https://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer</a>? My colleague Elizabeth Christensen has short primer <a href="https://www.youtube.com/watch?v=ZtSgdgPzv4k">talk on contributing to Postgres</a> as well.<p>HOWEVER...the -hackers mailing list is busy. <a href=https://www.postgresql.org/list/pgsql-hackers/>Very, very busy.</a> And dense. And filled with jargon and obscure terms. In short, it does not make for light reading. But here's the most important rule: you don't have to read everything. You don't even have to read every subject line. Just do your best.<p>Do not worry if you do not understand everything you read. Very few people (hello to my colleague Tom! ) know the code so well they can easily figure out what every post on -hackers is talking about. I am not one of those people. You will not be either, and that is perfectly okay. Focus on the parts of Postgres that are of interest to you. Over time, you will be able to understand more and more.<p>Here are my rules for staying sane while reading the Postgres hackers mailing list:<ul><li>Do not try to read everything; skim the subject lines<li>Use a good email client that does proper threading<li>Set a routine - if you are reading even a small portion of the messages, you likely need to do it at least once a day<li>Be willing to let threads go<li>If your backlog gets too big, just "mark all read”!</ul><h2 id=jargon><a href=#jargon>Jargon</a></h2><p>The mailing lists are full of acronyms and jargon that might not be familiar to younger people who did not grow up on email (although text messages have inherited many of the abbreviations). If you are a non-native English speaker, or under the age of 30, or not steeped in the world of tech, I offer some solutions below.<p>To do this, I downloaded the last year's worth of hackers email, wrote a program to strip out all the non-human stuff (headers, code blocks, attachments, etc.), and then did some data analysis on the results.<h3 id=version-control><a href=#version-control>VERSION CONTROL</a></h3><p>Postgres uses the <a href=https://wiki.postgresql.org/wiki/Working_with_Git><strong>git</strong> version control system</a>, which means there is quite a bit of assumed git knowledge and git-related phrases, such as:<ul><li><strong>diff</strong> = the output of a "diff" or "git diff" command, showing what has changed <em>"Looking at the diffs of these, I wonder if …"</em><li><strong>1f47afd8</strong> = reference to a git object / commit ID Always at least a 7 digit hexadecimal, often 9, or 12, or sometimes longer <em>"30e7c175b81 removed support for dumping..."</em><li><strong>HEAD</strong> - the current main, active branch within git <em>"patch was not applying on top of HEAD because"</em><li><strong>rebase</strong> - the process of using 'git rebase' to create a new version of a patch, which takes into account code changes since the time the patch was created <em>"Please find attached v2, mandatory rebase due to cd312adc56"</em></ul><h3 id=building-testing-and-patching><a href=#building-testing-and-patching>BUILDING, TESTING, and PATCHING</a></h3><p>Making <a href=https://wiki.postgresql.org/wiki/Creating_Clean_Patches>a patch</a> means compiling Postgres with your changes, submitting one or more patches to the mailing list, entering it in the <a href=https://commitfest.postgresql.org/>commitfest app</a>, and having it go through the CI (continuous integration) system. Here are some related terms:<ul><li><strong>+1</strong> = a general stamp of approval for someone's idea or patch. Sometimes seen as disapproval via <strong>-1</strong><li><strong>meson</strong> = a program used to build Postgres. The successor to the old "configure" system<li><strong>gcc, clang</strong> = two main compilers used to build Postgres. There are others.<li><strong>backpatching</strong> = the process of taking a patch and applying the changes to older (but still supported) major versions of Postgres. Not everything can be backpatched - the default is to NOT backpatch unless it is deemed important enough.<li><strong>0001, 0002</strong> = refers to the patch attached in the email thread. Most people create patches using "git format-patch", which creates one file per commit, and numbers them as 0001, 0002, 0003, etc.<li><strong>commitfest</strong> - An external site that lives here: <a href=https://commitfest.postgresql.org/>https://commitfest.postgresql.org/</a> It's job is to store the current state of all patches, and it contains many links back to the mailing lists.<li><strong>CF</strong> = commitfest<li><strong>CFM</strong> = commitfest manager<li><strong>RMT</strong> = release management team<li><strong>buildfarm</strong> - An external site that lives here: <a href=https://buildfarm.postgresql.org/>https://buildfarm.postgresql.org/</a> This is a distributed CI (continuous integration) system, run by volunteers all over the world, that tests Postgres after each commit.<li><strong>curculio wrasse lorikeet</strong> - every server in the buildfarm gets assigned a unique animal name, so you may see some obscure animals mentioned on the lists.<li><strong>cfbot</strong> - an automated system that creates git threads based on the commitfest entries, and gathers the results of CI testing (<a href=http://cfbot.cputube.org/>http://cfbot.cputube.org/</a>)<li><strong>Coverity</strong> = an external tool used to try and detect problems with the code<li><strong>s/This/That/</strong> = someone is suggesting changing the text to replace "This" with "That"<li><strong>OOM</strong> = out of memory</ul><h3 id=acronyms-used-in-postgres-development><a href=#acronyms-used-in-postgres-development>Acronyms Used in Postgres Development</a></h3><p>These are acronyms that are often used by people on the mailing list. The more common ones are at the top. Each has a sample usage as actually used on the list in the last year. The most common acronym is IMO aka "in my opinion". This proves that us hackers are a very opinionated bunch!<ul><li><strong>IMO</strong> = in my opinion "<em>IMO there should be some simple test cases</em>"<li><strong>FWIW</strong> = for what it's worth "<em>FWIW I think it's a fairly serious issue</em>"<li><strong>IIUC</strong> = if I understand correctly <em>"IIUC what you're saying is that we should"</em><li><strong>BTW</strong> = by the way <em>"BTW some drivers also send Describe even before Bind"</em><li><strong>IMHO</strong> = in my humble opinion. A slightly politer way to say IMO <em>"IMHO, this is not a good direction"</em><li><strong>LGTM</strong> = looks good to me (especially in regards to a patch) <em>"Did a quick check and still LGTM"</em><li><strong>PFA / PSA</strong> = please find attached / please see attached <em>"PFA the small patch that implements this"</em><li><strong>AFAICT</strong> = as far as I can tell <em>"AFAICT, this world target doesn't include the man target"</em><li><strong>OTOH</strong> = on the other hand <em>"OTOH, maybe the current code is more readable"</em><li><strong>IIRC</strong> = if I recall correctly <em>"IIRC, we have some similar issues in other hooks"</em><li><strong>AFAICS</strong> = as far as I can see <em>"AFAICS, only assert-enabled LLVM builds crash"</em><li><strong>WIP</strong> = work in progress <em>"Here's a new WIP version of the patch”</em><li><strong>ISTM</strong> = it seems to me <em>"ISTM that the fix here is to not use a spinlock"</em><li><strong>AFAIK</strong> = as far as I know <em>"AFAIK that guarantees it happens after"</em><li><strong>TBH</strong> = to be honest <em>"TBH, I am wondering what is the purpose of this sentence"</em><li><strong>FYI</strong> = for your information <em>"FYI, I also ran the patch"</em><li><strong>WFM</strong> = works for me <em>"Both suggestions WFM"</em><li><strong>IMV</strong> = in my view <em>"You really need diagrams for something like this IMV"</em><li><strong>IOW</strong> = in other words <em>"IOW, search path is a bandaid for this kind of thing”</em><li><strong>POV</strong> = point of view <em>"From my POV the idea seems reasonable"</em><li><strong>OP</strong> = original poster (the person who started the email thread) <em>"The OP suggests archiving the timeline history file"</em><li><strong>AFAIU</strong> = as far as I understand <em>"AFAIU currently we do not add Memoize nodes"</em><li><strong>YMMV</strong> = your mileage may vary <em>"I find it easy to read if the GUC parameters are quoted, but YMMV"</em><li><strong>FTR</strong> = for the record. Also <strong>JFTR</strong> = "just for the record"* <em>"FTR this has been discussed in the past"</em><li><strong>IME</strong> = in my experience <em>"often that's hard, and IME is rarely done"</em><li><strong>AFAIR</strong> = as far as I recall <em>"AFAIR, we don't prevent similar invalidations"</em><li><strong>ASAP</strong> = as soon as possible <em>"something merged that fixes the bug ASAP"</em><li><strong>AIUI</strong> = as I understand it <em>"which AIUI has never been committed"</em><li><strong>TBD</strong> = to be determined ”<em>something TBD at time of implementation</em>"<li><strong>WRT</strong> = with regards to <em>"discussion about this WRT renaming macros"</em><li><strong>WDYT?</strong> = what do you think? <em>"I expanded that into the following. WDYT?"</em><li><strong>IDK</strong> = I don't know <em>"IDK, to me something like this seems to promise more than we can actually use"</em></ul><h3 id=common-postgres-acronyms><a href=#common-postgres-acronyms>Common Postgres Acronyms</a></h3><p>While this is far from a canonical list, here are some of the more common Postgres-related terms you might run across. Again, these are in order of frequency:<ul><li><strong>WAL</strong> = write-ahead log<li><strong>GUC</strong> = grand unified configuration. Basically, a configuration variable<li><strong>LSN</strong> = log sequence number, a specific address in the WAL stream<li><strong>API</strong> = application programming interface<li><strong>OID</strong> = object identifier<li><strong>TOAST</strong> = <a href=https://www.crunchydata.com/blog/postgres-toast-the-greatest-thing-since-sliced-bread>the oversized attribute storage technique</a><li><strong>FSM</strong> = free space map<li><strong>SAOP</strong> = scalar array operator (note: not a misspelling of SOAP!)<li><strong>ABI</strong> = application binary interface<li><strong>RLS</strong> = row-level security<li><strong>DSM</strong> = dynamic shared memory<li><strong>TPS</strong> = transactions per second<li><strong>TLI</strong> = timeline ID<li><strong>EOL</strong> = end of life<li><strong>2PC</strong> = two-part commit<li><strong>LRU</strong> = least recently used strategy<li><strong>PITR</strong> = point-in-time recovery<li><strong>CTAS</strong> = create table as select<li><strong>CIC</strong> = concurrent index creation<li><strong>TAM</strong> = table access method<li><strong>RNG</strong> = random number generator<li><strong>LTO</strong> = link time optimization<li><strong>POLA</strong> = principle of least astonishment<li><strong>LR</strong> = logical replication</ul><h2 id=funny-phrases><a href=#funny-phrases>FUNNY PHRASES</a></h2><p>Finally, there are some phrases that might not have a direct translation from English, but are common on the list:<ul><li><strong>spitballing</strong> = throwing ideas out without much concern of feasibility or correctness.<li><strong>footgun</strong> = something dangerous that might cause you to shoot your own foot off, in other words, something that may cause serious problems too easily<li><strong>paint into a corner</strong> = getting into a difficult situation with no way out<li><strong>bikeshedding</strong> = focusing on minor details of something (such as the color of a bike shed) rather than focusing on the larger, more important details (such as the material to use, how sturdy the shed is, etc.)<li><strong>firehose</strong> = a strong, excessive amount of information that can be hard to consume. Such as the pgsql-hackers mailing list!</ul><p>That’s my collection! Again, this is not meant to be a canonical list, but I hope it is useful for those that may not be familiar with some of the terms above. If you are posting to the list, consider limiting the use of acronyms and jargon to only the most popular ones, or eschew them completely. TIA! (thanks in advance) ]]></content:encoded>
<category><![CDATA[ Production Postgres ]]></category>
<author><![CDATA[ Greg.Sabino.Mullane@crunchydata.com (Greg Sabino Mullane) ]]></author>
<dc:creator><![CDATA[ Greg Sabino Mullane ]]></dc:creator>
<guid isPermalink="false">3d4af8f33df143e8f207c0f3952fd08ee89d9a196d79bbdc67f1eb570ac71eda</guid>
<pubDate>Thu, 29 Aug 2024 15:30:00 EDT</pubDate>
<dc:date>2024-08-29T19:30:00.000Z</dc:date>
<atom:updated>2024-08-29T19:30:00.000Z</atom:updated></item></channel></rss>