Postgresql on Doug's Dabblings | Quick thoughts on stuff I'm playing with

I Heard You Like Releases in Your Releases

Sat, 01 Mar 2025 16:03:48 +0000

On Thursday, Feb 13th, the PostgreSQL Global Development Group (PGDG) released their regularly scheduled minor releases of PostgreSQL: 17.3, 16.7, 15.11, 14.16, and 13.19. These releases, among other bugfixes, resolved a CVE within PostgreSQL, so a sense of urgency in releasing them to our customers was bred. This, in turn, kicked off a flurry of activity within Crunchy as we readied Crunchy Postgres via Automation (CPA) for release to ship these new PostgreSQL releases. As we dove into the release process and began our internal testing, word came down from PGDG that the CVE fix was flawed and new PostgreSQL releases would be forthcoming.

On CVEs, BoMs, and Releases

Fri, 15 Nov 2024 17:10:07 +0000

As previously mentioned in this Crunchy Postgres via Automation series, we now use a Bill of Materials (BoM) to ship tested-together versions of the components that comprise the CPA product. Accordingly, when the PostgreSQL Global Development Group (PGDG) releases new versions of PostgreSQL as they did yesterday, we have to iterate our BoMs to include these releases.

Crunchy Postgres via Automation V2.2

Fri, 08 Nov 2024 17:23:23 +0000

Continuing our series on Crunchy Postgres via Automation, we’re here this week to discuss the highlights of our latest release line, v2.2. If you haven’t already, you can catch up on the differences between v1 and v2 and then come back, we’ll wait for you.

Automating Highly Available PostgreSQL Clusters

Fri, 04 Oct 2024 14:07:29 +0000

In March of ‘23, I took over as the Lead Architect of my employer’s Ansible-based automation for creating highly-available PostgreSQL clusters.¹ Since then, I’ve been responsible for advancing the product: refactoring the code, adding functionality, rethinking some of its core attributes, etc. I’ve also taken some steps to informally restructure the team that works on things to better divide up responsibilities and make everyone more productive.

tuned, PG, and you

Tue, 23 Jun 2020 18:22:44 +0000

We’ve had a small flurry of customers asking about tuning their OS for the best PostgreSQL performance. While the answer to this question is always ’that depends on your hardware and workload’ and involves a lot of iteration between changing a setting and benchmarking, I thought I’d take a moment to point out that once you do manage to dial-in the settings, you should be writing a profile and deploying to your systems for tuned to make use of. Please, for the love of $diety, stop editing sysctl.conf and friends!

pgBackRest 2.08 and macOS Mojave

Fri, 04 Jan 2019 08:30:44 +0000

UPDATE: My reasoning was incorrect below. It wasn’t the moving of some of the lock code to C that caused the issue. It was moving -D_POSIX_C_SOURCE up to the Makefile that caused the problem. The solution below is still the same though.

pgBackRest 2.07 and macOS Mojave

Fri, 16 Nov 2018 06:12:31 +0000

pgBackRest 2.07 was announced today. As usual, I immediately downloaded it and tried to get it up and running on my MacBook (currently running Mojave). It wasn’t as straightforward as one might hope, and the online instructions assume a Linux system, so I figured I’d write this up for anyone else attempting the same.

Monitoring pgBackRest with tail_n_mail

Mon, 24 Sep 2018 10:58:22 +0000

After a lively discussion at work today about monitoring tools and use cases, I decided to see if I could use tail_n_mail, which I already use to monitor my PostgreSQL logs, to monitor my pgBackRest logs. It turns out that it can, and can do so fairly trivially.

pgBouncer and auth pass-through

Tue, 07 Aug 2018 12:53:28 +0000

I’ve noticed several individuals inquiring lately about pgBouncer and how they can avoid putting all users and their passwords in it’s auth_file. After the most recent such inquiry (hi Richard!) I decided I’d write this"post"to hopefully make it clearer how to use ‘pass-through auth’ and avoid maintaining your users and their passwords in an external file. So let’s see what this takes, shall we?

Upgrading PostgreSQL from 9.4 to 10.3 with pglogical

Thu, 05 Apr 2018 13:15:46 +0000

I recently helped a customer upgrade their PostgreSQL instance from 9.4.x on RHEL to 10.x on Ubuntu. While it initially sounded daunting, the use of pglogical and some planning actually made it rather straightforward. While there’s nothing new or original in this post, I still felt compelled to write it up both for posterity’s sake and for anyone else that might find the info useful as an example in their own endeavors.

updated PostgreSQL homebrew script

Mon, 16 Oct 2017 13:09:23 +0000

With the release of PostgreSQL 10, I’ve updated my pg script. You might recall from previous posts that this script is for Homebrew users that have tapped Peter’s brew recipes. It allows for installing and switching between multiple version of PostgreSQL seemlessly. While I was in there adding v10 support, I tweaked and tuned the code a bit and tidyied up the output significantly. I’m pretty pleased with the new version actually.

When you cannot get there from here

Tue, 25 Jul 2017 11:42:47 +0000

Connecting to a PostgreSQL instance isn’t hard generally, but sometimes you can run into issues. Sometimes a port isn’t open on a firewall, or the server is in a VLAN that you can’t get to, or perhaps the server isn’t running on the network interface you think it is. More commonly, you can reach the PostgreSQL instance but you’re connection isn’t authorized (which is not the same as being unable to authenticate). Fortunately, the error messages returned in these different failure scenarios are fairly verbose and distinct so you can easily tell which scenario you’re facing. Let’s dive into each scenario and see what the error looks like, shall we?

Installing pgBackRest on OSX

Wed, 14 Jun 2017 12:40:30 +0000

If you’ve followed my previous posts (here and here), then you already have one or more versions of PostgreSQL installed on your Mac. Maybe these are solely for test or dev purposes and you don’t really care about any of the data therein, but if you do, let me guide you to pgBackRest.

EXPLAINing intermittent perf problems

Mon, 28 Nov 2016 12:24:12 +0000

We’ve all gotten the dreaded email/call from a user stating that a query is “slow sometimes”. If you’re lucky, the “sometimes” actually ends up being fairly consistent and you can fairly easily determine what’s happening (an errant cron job, for example). All too often though, the issue really is sporadic, fleeting, and indeterministic. So how do you track these down? And more importantly what do you do about them once found?

PostgreSQL logging, strftime, and you

Mon, 21 Nov 2016 12:17:03 +0000

PostgreSQL has a pretty extensive logging facility. I’ve talked briefly about configuring it to get the most out of pgBadger before, but today I’m gonna talk a bit about the naming of the log file itself. The chosen filename doesn’t have to be static. You can, in fact, have the name dynamically created by using strftime() escapes. But what exactly are those?

Upgrading PostgreSQL 5x faster

Fri, 18 Nov 2016 12:18:28 +0000

Upgrading your PostgreSQL database from one major version (e.g. 9.4.x) to another major version (e.g. 9.5.x) used to a painful and exceedingly slow process. You essentially had two options: dump / reload the data or use one of the complex logical replication tools.

Managing multiple PostgreSQL installs via Homebrew

Wed, 16 Nov 2016 13:45:27 +0000

Following on from this post, you probably have multiple versions of PostgreSQL installed on your Mac. In that post, I added an example function to help you manage all these concurrent installs. Today, I’m back with a full-fledged shell script to help manage all this. Without further ado, the script:

Getting fancy with PostgreSQL and Homebrew

Thu, 10 Nov 2016 14:58:15 +0000

In our previous post, we installed PostgreSQL via Homebrew and got our own little cluster up and running under our userid. That is probably good enough for 90% of the users out there who just want to play with or devel on PostgreSQL, but for those of us who need a little more flexibility in our installs, we’re going to take the next step.

PostgreSQL, Homebrew, and You

Fri, 04 Nov 2016 13:30:09 +0000

If you need to install PostgreSQL onto your Macbook, you have several options available to you nowadays. You could use the BigSQL package, or you could use Postgres.app, or several others. However, if you’re a geek running OSX, you’ve probably already installed Homebrew and it has a wonderful PostgreSQL package. So let’s use it, shall we?

Inaugural pgCMH mtg scheduled

Thu, 03 Nov 2016 11:45:32 +0000

As announced on our Twitter, we’ve scheduled our inaugural meeting for Jan 24, 2017 at 1800 hrs. The folks at CoverMyMeds will graciously provide the meeting space (and parking garage) while OpenSCG is buying the pizza!

Announcing pgCMH

Mon, 31 Oct 2016 11:35:22 +0000

I’ve been kicking around the idea of founding a Columbus-based PostgreSQL User Group for a while now. I even went so far as to float the idea to people at OLF in ‘14. After much hemming and hawing (and no one else stepping up in the interim), I’ve finally gone and done it.

Incremental pgBadger

Wed, 26 Oct 2016 14:46:09 +0000

You’re probably already running pgBadger to monitor your PostgreSQL logs. However, you’re probably not running it incrementally throughout the day. Most likely, you’ve setup a cron.daily job that runs pgBadger against yesterday’s log(s). And that’s great. Except when you get the dreaded “what just happened on the db?” email. Are you going to wait until tonight’s normal run of pgBadger to see what happened? Are you going to run a ‘one off’ pgBadger against today’s logfile and wait for it to process the entire log? Or are you going to copy the log off somewhere, edit it to cut it down, and then run pgBadger against this cut-down version (hoping you left enough in the log to see proper trending)?

I'm syndicated

Sat, 08 Oct 2016 15:19:57 +0000

Someone at work thought it would be a good idea to give me access to the corporate blog so that I might post PostgreSQL-related things there and have them syndicated to Planet PostgreSQL. So my PostgreSQL ramblings will show up there now instead of here. This should be fun!

Where Not To Put Your Tablespaces

Wed, 24 Aug 2016 11:55:12 +0000

From the PostgreSQL docs:

Tablespaces in PostgreSQL allow database administrators to define locations in the file system where the files representing database objects can be stored. Once created, a tablespace can be referred to by name when creating database objects.

Happy 20th Birthday, PostgreSQL

Thu, 12 May 2016 10:11:31 +0000

Just a quick shout out to PostgreSQL, the world’s most advanced open source database, as it celebrates it’s 20^th year in existence. Happy birthday, y’all!

PostgreSQL Partitioning Quick Tip

Thu, 21 Apr 2016 09:25:32 +0000

Partitioning in PostgreSQL can be a little daunting at times. In fact, you should probably just use pg_partman and be done with it. However, if you’re trying to learn, can’t use pg_partman, or are a masochist you’ll probably be following the docs and thinking ‘seriously? i have to create indexes on each child? why don’t they copy the indexes of the parent? why isn’t this easier?’. Here’s a little tip to make things slightly easier:

Logical Replication with Skytools3

Tue, 19 Apr 2016 13:55:11 +0000

UPDATE: My coworker Richard liked this write up, and Skytools, so much he threw together a demo script. You can get it here.

PostgreSQL Streaming Replication in 10 Minutes

Mon, 18 Apr 2016 14:47:10 +0000

While there’s absolutely nothing new in this blog"post"that isn’t covered by the wonderful docs I’ve been asked multiple times now by customers if we had some kind of ‘crib notes’ format for how to get replication up and running. And since I just had to set this up and document it for a customer, I figured I might as well"post"it so that I can simply point people to it in the future. So here we are.