Scaling Postgres

In this episode of Scaling Postgres, we discuss the Beta 1 release of Postgres 15, optimizing a query by breaking it down, improvements to sort performance and prepared statements for partitioned tables. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-beta-1-released-2453/ https://www.crunchydata.com/blog/postgres-query-optimization-left-join-vs-union-all https://www.citusdata.com/blog/2022/05/19/speeding-up-sort-performance-in-postgres-15/ https://amitlan.com/2022/05/16/param-query-partition-woes.html https://pganalyze.com/blog/5mins-postgres-partition-pruning-prepared-statements-generic-vs-custom-query-plans https://www.ongres.com/blog/solving-postgresql-statistic-corruption/ https://www.craigkerstiens.com/2022/05/18/unfinished-business-with-postgres/ https://ptc-it.de/collate-docker-postgres-bug/ https://geopet85.github.io/2022/05/15/rds-partitioned-tables-export.html https://www.crunchydata.c

In this episode of Scaling Postgres, we discuss parallel server-side backup compression, IO in Postgres, parquet files and the new pg_stat_monitor extension. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-143-137-1211-1116-and-1021-released-2449/ https://cloud.google.com/blog/products/databases/introducing-alloydb-for-postgresql https://www.youtube.com/watch?v=o5pKfH0Tonw https://www.cybertec-postgresql.com/en/time-zone-management-in-postgresql/ https://www.enterprisedb.com/blog/pg-phriday-what-you-should-know-about-basic-postgres-authentication https://www.crunchydata.com/blog/the-vectors-of-database-encryption https://www.crunchydata.com/blog/demystifying-database-performance-for-developers https://pganalyze.com/blog/5mins-postgres-for-app-developers-tables-indexes https://kiranrao.ca/2022/05/04/zero-downtime-migrations.html https://www.se-radio.net/2022/05/episode-511-ant-wilson-on-supabase-post

In this episode of Scaling Postgres, we discuss parallel server-side backup compression, IO in Postgres, parquet files and the new pg_stat_monitor extension. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: http://rhaas.blogspot.com/2022/05/parallel-server-side-backup-compression.html https://www.youtube.com/watch?v=3Oj7fBAqVTw https://www.crunchydata.com/blog/parquet-and-postgres-in-the-data-lake https://www.percona.com/blog/understand-your-postgresql-workloads-better-with-pg_stat_monitor/ https://www.timescale.com/blog/point-in-time/ https://www.cybertec-postgresql.com/en/postgresql-15-using-merge-in-sql/ https://www.highgo.ca/2022/04/30/upcoming-feature-in-postgresql-15-merge-statement/ https://andreas.scherbaum.la/blog/archives/1110-Its-beer-oclock-for-PostgreSQL.html https://pganalyze.com/blog/5mins-postgres-15-beta1-incremental-materialized-views-pg-ivm https://www.highgo.ca/2022/05/06/atomic-commit-with-2-phase-commit-in-fdw-distributed-setu

In this episode of Scaling Postgres, we discuss ways you can optimize the planner, becoming part of the Postgres Community, considerations for large databases and mapping three different types of spatial data. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.aha.io/engineering/articles/optimizing-with-the-postgresql-deterministic-query-planner https://pganalyze.com/blog/5mins-postgres-tuning-deterministic-query-planner-extended-statistics-join-collapse-limits https://www.timescale.com/blog/the-future-of-community-in-light-of-babelfish/ https://www.percona.com/blog/working-with-large-postgresql-databases/ https://blog.rustprooflabs.com/2022/04/postgis-h3-intro https://selectfrom.dev/mapping-fracking-wells-in-colorado-with-postgresql-db17fc3b8812 https://www.cybertec-postgresql.com/en/analyzing-historical-flight-data-with-mobilitydb/ https://www.highgo.ca/2022/04/29/global-deadlock-in-a-distributed-database-cluster/ https://www.endpointde

In this episode of Scaling Postgres, we discuss dealing with slow queries, upgrading a 4TB database, running Postgres on ZFS and handling a storage stampede. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pgdash.io/blog/slow-queries-postgres.html https://retool.com/blog/how-we-upgraded-postgresql-database/ https://lackofimagination.org/2022/04/our-experience-with-postgresql-on-zfs/ https://www.enterprisedb.com/blog/steady-storage-stampede https://tech.instacart.com/the-vanishing-thread-and-postgresql-tcp-connection-parameters-93afc0e1208c https://www.cybertec-postgresql.com/en/find-and-fix-a-missing-postgresql-index/ https://www.percona.com/blog/postgresql-14-b-tree-index-reduced-bloat-with-bottom-up-deletion/ https://pganalyze.com/blog/5mins-postgres-hot-updates-vs-bottom-up-index-deletion https://www.youtube.com/watch?v=oJj-pltxBUM https://www.postgresql.fastware.com/blog/two-phase-commits-for-logical-replication-publications-subscription

In this episode of Scaling Postgres, we discuss working with multiranges, missing Postgres metrics, PostGIS for newbies and videos from CitusCon. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/multiranges-in-postgresql-14/ https://coroot.com/blog/pg-missing-metrics https://blog.crunchydata.com/blog/postgis-for-newbies https://www.youtube.com/playlist?list=PLlrxD0HtieHjSzUZYCMvqffEU5jykfPTd#cituscon2022 https://pganalyze.com/blog/5mins-postgres-sql-json-table-postgres15 https://yugonagata-pgsql.blogspot.com/2022/04/pgivm-postgresql-extension-providing.html https://www.postgresql.org/about/news/update-on-the-trademark-actions-against-the-postgresql-community-2437/ https://pgstef.github.io/2022/04/15/pgbackrest_multi-repositories_tips_and_tricks.html https://postgresql.life/post/jehan-guillaume_de_rorthais/ https://www.rubberduckdevshow.com/episodes/38-small-projects/  

In this episode of Scaling Postgres, we discuss how to bin date times, search every field of a table, use JSON_TABLE and methods for a client to pick the primary database. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-mapping-timestamps-date_bin/ https://www.depesz.com/2022/04/05/dba-tips-n-tricks-search-in-every-field-in-a-table/ https://www.depesz.com/2022/04/06/waiting-for-postgresql-15-json_table/ https://www.postgresql.org/docs/14/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS https://www.cybertec-postgresql.com/en/new-target_session_attrs-settings-for-high-availability-and-scaling-in-postgresql-v14/ https://www.postgresql.fastware.com/blog/enhancement-to-target-session-attrs https://pganalyze.com/blog/5mins-postgres-tuning-vacuum-autovacuum https://www.endpointdev.com/blog/2022/04/shapefiles-postgis/ https://dev.to/yugabyte/how-to-enable-ssl-for-postgres-connections-5321 https://pos

In this episode of Scaling Postgres, we discuss how to optimize vacuum, how to drop roles, the new merge command in PG15 and a pgbouncer tutorial. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.enterprisedb.com/blog/pg-phriday-tidying-vacuum https://www.cybertec-postgresql.com/en/drop-role-or-drop-user-postgresql/ https://www.depesz.com/2022/03/31/waiting-for-postgresql-15-add-support-for-merge-sql-command/ https://www.postgresql.org/docs/devel/sql-merge.html https://www.youtube.com/watch?v=ddKm7a7xOpk https://www.depesz.com/2022/03/31/waiting-for-postgresql-15-sql-json-constructors/ https://www.depesz.com/2022/04/01/waiting-for-postgresql-15-is-json-predicate/ https://www.depesz.com/2022/04/01/waiting-for-postgresql-15-sql-json-query-functions/ https://www.highgo.ca/2022/03/31/parallel-commit-in-postgres-fdw/ https://pganalyze.com/blog/how-postgres-chooses-index https://postgrespro.com/blog/pgsql/5969403 https://www.citusdata.com/blog

In this episode of Scaling Postgres, we discuss a new version of pgbouncer, query parameter types, a performance improvement from a lateral join and security invoker views. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: http://www.pgbouncer.org/2022/03/pgbouncer-1-17-0 https://www.cybertec-postgresql.com/en/query-parameter-data-types-performance/ https://www.depesz.com/2022/03/22/waiting-for-postgresql-15-add-support-for-security-invoker-views/ https://pganalyze.com/blog/5mins-postgres-max-wal-size-full-page-writes-uuid-bigint https://b-peng.blogspot.com/2022/03/pgpool-debian.html https://postgresql.life/post/michael_christofides/ https://www.rubberduckdevshow.com/episodes/35-organizing-your-tests/  

In this episode of Scaling Postgres, we discuss how best to configure autovacuum, the dangers of duplicate key violations, how to set up pipelining, and how to tune max_wal_size. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2022/03/13/is-my-autovacuum-configured-properly/ https://aws.amazon.com/blogs/database/hidden-dangers-of-duplicate-key-violations-in-postgresql-and-how-to-avoid-them/ https://www.cybertec-postgresql.com/en/pipeline-mode-better-performance-on-slow-network/ https://www.enterprisedb.com/blog/tuning-maxwalsize-postgresql https://www.enterprisedb.com/blog/pg-phriday-wrapping-transactions-right-round https://www.endpointdev.com/blog/2022/03/using-pgtap-automate-database-testing/ https://ctodive.com/hooks-the-secret-feature-powering-the-postgres-ecosystem-f05b3b82e0ba https://www.citusdata.com/blog/2022/03/12/how-to-benchmark-performance-of-citus-and-postgres-with-hammerdb/ https://pganalyze.com/blog/5mins-po

In this episode of Scaling Postgres, we discuss 100x optimizations, how to audit table activity, designing schemas and how to set up table checksums. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://towardsdatascience.com/how-we-optimized-postgresql-queries-100x-ff52555eabe https://supabase.com/blog/2022/03/08/audit https://pganalyze.com/blog/5mins-postgres-auditing-pgaudit-supabase-supa-audit https://www.endpointdev.com/blog/2022/03/database-design-with-documents/ https://www.cybertec-postgresql.com/en/postgresql-creating-checksums-for-tables/ https://blog.crunchydata.com/blog/performance-and-spatial-joins https://blog.crunchydata.com/blog/cql-filtering-in-pg_featureserv https://postgrespro.com/blog/pgsql/5969296 https://postgresql.life/post/ibrar_ahmed/ https://www.rubberduckdevshow.com/episodes/33-reviewing-owasp-6-10/  

In this episode of Scaling Postgres, we discuss the best ways to index with tsearch, what to check if tuples are not yet removable, wicked problems with Postgres and different pattern matching techniques. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2022/03/01/how-to-index-data-with-tsearch/ https://pganalyze.com/blog/5mins-postgres-autovacuum-dead-tuples-not-yet-removable-postgres-xmin-horizon https://www.slideshare.net/AlexanderKorotkov/solving-postgresql-wicked-problems https://github.com/orioledb/orioledb https://blog.crunchydata.com/blog/how-to-cheat-at-wordle-with-postgresql https://databaseci.com/docs/migra https://b-peng.blogspot.com/2022/02/auto-failback.html https://blog.crunchydata.com/blog/cloud-data-sources-for-the-postgres-operator-for-kubernetes https://www.highgo.ca/2022/02/28/a-snippet-to-acquire-a-lightweight-lock/ https://postgresql.life/post/gianni_ciolli/ https://www.rubberduckdevshow.com/episodes/32-r

In this episode of Scaling Postgres, we discuss optimizing trigram searches, a review of Postgres replication, improvements to logical replication and a significant Timescale investment. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://alexklibisz.com/2022/02/18/optimizing-postgres-trigram-search.html https://pganalyze.com/blog/5mins-postgres-optimizing-postgres-text-search-trigrams-gist-indexes https://www.enterprisedb.com/blog/replication-revue https://www.percona.com/blog/logical-replication-decoding-improvements-in-postgresql-13-and-14/ https://www.timescale.com/blog/year-of-the-tiger-110-million-to-build-the-future-of-data-for-developers-worldwide/ https://www.shayon.dev/post/2022/47/pg-osc-zero-downtime-schema-changes-in-postgresql/ https://www.cybertec-postgresql.com/en/monitoring-google-cloud-postgresql-with-pgwatch2/ https://postgresql.verite.pro/blog/2022/02/21/psql-hack-select-except.html https://postgresql.life/post/julia_gugel/

In this episode of Scaling Postgres, we discuss the new releases of Postgres, a hairy upgrade incident, why slowing down can make you go faster and using loadable modules for WAL archiving. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2022/02/13/how-much-disk-space-you-can-save-by-using-int4-int-instead-of-int8-bigint/ https://blog.anayrat.info/en/2022/02/14/postgresql-toast-compression-and-toast_tuple_target/ https://blog.crunchydata.com/blog/safer-application-users-in-postgres https://pganalyze.com/blog/5mins-postgres-high-cpu-utilization-explain-plans-linux-perf https://www.highgo.ca/2022/02/18/the-significance-of-lsn-in-postgresql-system/ https://dev.to/yugabyte/sqlcl-to-transfer-data-from-oracle-to-postgresql-or-yugabytedb-lha https://postgrespro.com/blog/pgsql/5969262 https://www.cybertec-postgresql.com/en/pg_timetable-v4-4-is-available-immediately/ https://postgresql.life/post/henrietta_dombrovskaya/ https://www.ru

In this episode of Scaling Postgres, we discuss the new releases of Postgres, a hairy upgrade incident, why slowing down can make you go faster and using loadable modules for WAL archiving. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-142-136-1210-1115-and-1020-released-2402/ https://ardentperf.com/2022/02/10/a-hairy-postgresql-incident/ https://vyruss.org/blog/slow-things-down-to-make-them-go-faster-fosdem-2022.html https://www.depesz.com/2022/02/06/waiting-for-postgresql-15-allow-archiving-via-loadable-modules/ http://rhaas.blogspot.com/2022/02/server-side-lz4-backup-compression.html https://blog.crunchydata.com/blog/postgres-constraints-for-newbies https://pganalyze.com/blog/5mins-postgres-custom-aggregates-rust-sql-pgx https://www.youtube.com/channel/UCqpEeVhxpsj7DbIh8K8npPQ/videos https://www.depesz.com/2022/02/06/waiting-for-postgresql-15-add-unique-null-treatment-option/ https://www.postgr

In this episode of Scaling Postgres, we discuss JSONB performance, bidirectional replication, getting the most recent record and using JSON in Postgres 14. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.evanjones.ca/postgres-large-json-performance.html https://pganalyze.com/blog/5mins-postgres-jsonb-toast https://www.enterprisedb.com/blog/pg-phriday-nominally-bidirectional https://blog.timescale.com/blog/select-the-most-recent-record-of-many-items-with-postgresql/ https://aaronbos.dev/posts/postgres-14-json https://www.postgresql.fastware.com/blog/how-to-gain-insight-into-the-pg-stat-replication-slots-view-by-examining-logical-replication https://www.cybertec-postgresql.com/en/automatic-partition-creation-in-postgresql/ https://blog.crunchydata.com/blog/secure-permissions-for-pgbackrest https://blog.yugabyte.com/postgresql-timestamps-timezones/ https://blog.yugabyte.com/postgresql-timestamps-timezones-interval-minefield/ https://www.do

In this episode of Scaling Postgres, we discuss the new series 5 minutes of Postgres, using unnest, switching to SCRAM authentication and reviewing multi-master solutions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: 5mins of Postgres E1: Using Postgres statistics to improve bad query plans, pg_hint_plan extension 5mins of Postgres E2: Using unnest(..), generate_series(), and PostGIS Postgres UNNEST cheat sheet for bulk operations PostgreSQL 14 and Recent SCRAM Authentication Changes – Should I Migrate to SCRAM? PG Phriday: Replication Engine Potpourri Accelerating VACUUM in Postgres Postgres SQL Lessons From Advent of Code Challenges Not so equal texts in PostgreSQL - and how to compare texts in a more elaborate way "Read-Only" Mode for PostgreSQL Waiting for PostgreSQL 15 – Add HEADER support to COPY text format Elevation Profiles and Flightlines with PostGIS Transaction ID and Snapshot information functions Flavio Gurgel Rubber Duck Dev S

In this episode of Scaling Postgres, we discuss how to generate shaped sample data, track changes between Postgres versions, identify missed bottlenecks and use different index types. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.timescale.com/blog/how-to-shape-sample-data-with-postgresql-generate_series-and-sql/ https://www.depesz.com/2022/01/17/configuration-changes-across-pg-versions/ https://pawelurbanek.com/postgresql-query-bottleneck https://blog.crunchydata.com/blog/postgres-indexes-for-newbies https://www.depesz.com/2022/01/17/waiting-for-postgresql-15-introduce-log_destinationjsonlog/ http://rhaas.blogspot.com/2022/01/surviving-without-superuser-part-two.html https://www.shayon.dev/post/2022/17/why-i-enjoy-postgresql-infrastructure-engineers-perspective/ https://aws.amazon.com/blogs/database/postgresql-bi-directional-replication-using-pglogical/ https://www.cybertec-postgresql.com/en/data-normalization-in-post

In this episode of Scaling Postgres, we discuss how to monitor DML & DDL progress, using more SQL, one way to handle replication slot fail-over, and recent Postgres contributors. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20220114-progress-bar-for-postgres-queries-lets-dive-deeper https://dev.to/yugabyte/quick-on-active-sql-from-pgstatactivity-27hi https://pgdash.io/blog/more-sql-less-code-with-postgresql.html http://rhaas.blogspot.com/2022/01/who-contributed-to-postgresql.html https://blog.crunchydata.com/blog/a-postgres-primer-for-oracle-dbas https://blog.crunchydata.com/blog/postgis-3.2-new-and-improved https://www.percona.com/blog/creating-a-standby-cluster-with-the-percona-distribution-for-postgresql-operator/ https://postgresql.life/post/emil_shkolnik/ https://www.rubberduckdevshow.com/episodes/25-javascript-options-in-rails-7/

In this episode of Scaling Postgres, we discuss including buffers with explain analyze, having a healthier Postgres DB, the events of 2021 and aspiring for greater high availability solutions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20220106-explain-analyze-needs-buffers-to-improve-the-postgres-query-optimization-process https://blog.crunchydata.com/blog/five-tips-for-a-healthier-postgres-database-in-the-new-year https://www.migops.com/blog/2022/01/07/a-short-summary-of-the-year-2021-in-the-postgresql-ecosystem/ https://pganalyze.com/blog/postgres-2021-year-in-review https://www.enterprisedb.com/blog/pg-phriday-defining-high-availability-postgres-world https://blog.rustprooflabs.com/2022/01/pgdd-for-everyone https://www.youtube.com/watch?v=LCowYnqSdbo https://www.endpointdev.com/blog/2022/01/postgresql-no-superuser-fix/ https://www.endpointdev.com/blog/2021/12/ecommerce-customer-names-interesting-unico