Scaling Postgres

In this episode of Scaling Postgres, we discuss the new releases of Postgres, a hairy upgrade incident, why slowing down can make you go faster and using loadable modules for WAL archiving. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2022/02/13/how-much-disk-space-you-can-save-by-using-int4-int-instead-of-int8-bigint/ https://blog.anayrat.info/en/2022/02/14/postgresql-toast-compression-and-toast_tuple_target/ https://blog.crunchydata.com/blog/safer-application-users-in-postgres https://pganalyze.com/blog/5mins-postgres-high-cpu-utilization-explain-plans-linux-perf https://www.highgo.ca/2022/02/18/the-significance-of-lsn-in-postgresql-system/ https://dev.to/yugabyte/sqlcl-to-transfer-data-from-oracle-to-postgresql-or-yugabytedb-lha https://postgrespro.com/blog/pgsql/5969262 https://www.cybertec-postgresql.com/en/pg_timetable-v4-4-is-available-immediately/ https://postgresql.life/post/henrietta_dombrovskaya/ https://www.ru

In this episode of Scaling Postgres, we discuss the new releases of Postgres, a hairy upgrade incident, why slowing down can make you go faster and using loadable modules for WAL archiving. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-142-136-1210-1115-and-1020-released-2402/ https://ardentperf.com/2022/02/10/a-hairy-postgresql-incident/ https://vyruss.org/blog/slow-things-down-to-make-them-go-faster-fosdem-2022.html https://www.depesz.com/2022/02/06/waiting-for-postgresql-15-allow-archiving-via-loadable-modules/ http://rhaas.blogspot.com/2022/02/server-side-lz4-backup-compression.html https://blog.crunchydata.com/blog/postgres-constraints-for-newbies https://pganalyze.com/blog/5mins-postgres-custom-aggregates-rust-sql-pgx https://www.youtube.com/channel/UCqpEeVhxpsj7DbIh8K8npPQ/videos https://www.depesz.com/2022/02/06/waiting-for-postgresql-15-add-unique-null-treatment-option/ https://www.postgr

In this episode of Scaling Postgres, we discuss JSONB performance, bidirectional replication, getting the most recent record and using JSON in Postgres 14. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.evanjones.ca/postgres-large-json-performance.html https://pganalyze.com/blog/5mins-postgres-jsonb-toast https://www.enterprisedb.com/blog/pg-phriday-nominally-bidirectional https://blog.timescale.com/blog/select-the-most-recent-record-of-many-items-with-postgresql/ https://aaronbos.dev/posts/postgres-14-json https://www.postgresql.fastware.com/blog/how-to-gain-insight-into-the-pg-stat-replication-slots-view-by-examining-logical-replication https://www.cybertec-postgresql.com/en/automatic-partition-creation-in-postgresql/ https://blog.crunchydata.com/blog/secure-permissions-for-pgbackrest https://blog.yugabyte.com/postgresql-timestamps-timezones/ https://blog.yugabyte.com/postgresql-timestamps-timezones-interval-minefield/ https://www.do

In this episode of Scaling Postgres, we discuss the new series 5 minutes of Postgres, using unnest, switching to SCRAM authentication and reviewing multi-master solutions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: 5mins of Postgres E1: Using Postgres statistics to improve bad query plans, pg_hint_plan extension 5mins of Postgres E2: Using unnest(..), generate_series(), and PostGIS Postgres UNNEST cheat sheet for bulk operations PostgreSQL 14 and Recent SCRAM Authentication Changes – Should I Migrate to SCRAM? PG Phriday: Replication Engine Potpourri Accelerating VACUUM in Postgres Postgres SQL Lessons From Advent of Code Challenges Not so equal texts in PostgreSQL - and how to compare texts in a more elaborate way "Read-Only" Mode for PostgreSQL Waiting for PostgreSQL 15 – Add HEADER support to COPY text format Elevation Profiles and Flightlines with PostGIS Transaction ID and Snapshot information functions Flavio Gurgel Rubber Duck Dev S

In this episode of Scaling Postgres, we discuss how to generate shaped sample data, track changes between Postgres versions, identify missed bottlenecks and use different index types. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.timescale.com/blog/how-to-shape-sample-data-with-postgresql-generate_series-and-sql/ https://www.depesz.com/2022/01/17/configuration-changes-across-pg-versions/ https://pawelurbanek.com/postgresql-query-bottleneck https://blog.crunchydata.com/blog/postgres-indexes-for-newbies https://www.depesz.com/2022/01/17/waiting-for-postgresql-15-introduce-log_destinationjsonlog/ http://rhaas.blogspot.com/2022/01/surviving-without-superuser-part-two.html https://www.shayon.dev/post/2022/17/why-i-enjoy-postgresql-infrastructure-engineers-perspective/ https://aws.amazon.com/blogs/database/postgresql-bi-directional-replication-using-pglogical/ https://www.cybertec-postgresql.com/en/data-normalization-in-post

In this episode of Scaling Postgres, we discuss how to monitor DML & DDL progress, using more SQL, one way to handle replication slot fail-over, and recent Postgres contributors. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20220114-progress-bar-for-postgres-queries-lets-dive-deeper https://dev.to/yugabyte/quick-on-active-sql-from-pgstatactivity-27hi https://pgdash.io/blog/more-sql-less-code-with-postgresql.html http://rhaas.blogspot.com/2022/01/who-contributed-to-postgresql.html https://blog.crunchydata.com/blog/a-postgres-primer-for-oracle-dbas https://blog.crunchydata.com/blog/postgis-3.2-new-and-improved https://www.percona.com/blog/creating-a-standby-cluster-with-the-percona-distribution-for-postgresql-operator/ https://postgresql.life/post/emil_shkolnik/ https://www.rubberduckdevshow.com/episodes/25-javascript-options-in-rails-7/

In this episode of Scaling Postgres, we discuss including buffers with explain analyze, having a healthier Postgres DB, the events of 2021 and aspiring for greater high availability solutions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20220106-explain-analyze-needs-buffers-to-improve-the-postgres-query-optimization-process https://blog.crunchydata.com/blog/five-tips-for-a-healthier-postgres-database-in-the-new-year https://www.migops.com/blog/2022/01/07/a-short-summary-of-the-year-2021-in-the-postgresql-ecosystem/ https://pganalyze.com/blog/postgres-2021-year-in-review https://www.enterprisedb.com/blog/pg-phriday-defining-high-availability-postgres-world https://blog.rustprooflabs.com/2022/01/pgdd-for-everyone https://www.youtube.com/watch?v=LCowYnqSdbo https://www.endpointdev.com/blog/2022/01/postgresql-no-superuser-fix/ https://www.endpointdev.com/blog/2021/12/ecommerce-customer-names-interesting-unico

In this episode of Scaling Postgres, we discuss easily writing a recursive CTE, the zheap undo capability, high availability considerations and fast ways to load data. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://dev.to/yugabyte/learn-how-to-write-sql-recursive-cte-in-5-steps-3n88 https://www.cybertec-postgresql.com/en/zheap-undo-logs-discarding-in-postgresql/ https://tapoueh.org/blog/2021/12/postgres-ha-roles-are-dynamic/ https://pganalyze.com/blog/fastest-way-importing-data-into-postgres-with-ruby-rails https://aiven.io/blog/explore-the-new-search-and-cycle-features-in-postgresql-14 http://blog.cleverelephant.ca/2021/12/knn-syntax.html https://www.citusdata.com/blog/2021/12/11/uk-covid-19-dashboard-built-using-postgres-and-citus/ https://www.cybertec-postgresql.com/en/manage-encryption-keys-with-postgresql-tde/ https://www.percona.com/blog/postgresql-high-availability-and-disaster-recovery-on-kubernetes https://postgre

In this episode of Scaling Postgres, we discuss Aurora vs. Postgres, surprising transaction behavior, write-only & read-only database connections and indexing best practices. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.migops.com/blog/2021/11/26/is-aurora-postgresql-really-faster-and-cheaper-than-rds-postgresql-benchmarking/ https://blog.lawrencejones.dev/isolation-levels/ https://www.enterprisedb.com/blog/taking-advantage-write-only-and-read-only-connections-pgbouncer-django https://www.pgmustard.com/blog/indexing-best-practices-postgresql http://aklaver.org/wordpress/2021/12/07/postgres-and-json/ http://rhaas.blogspot.com/2021/12/surviving-without-superuser-part-one.html https://blog.crunchydata.com/blog/extracting-and-substituting-text-with-regular-expressions-in-postgresql https://fluca1978.github.io/2021/12/06/pgterminatebackend.html https://www.highgo.ca/2021/12/10/a-look-inside-postgresqls-extended-query-pro

In this episode of Scaling Postgres, we discuss going slower to go faster, GIN indexes, collation stability and features of PG14 and beyond. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pganalyze.com/blog/gin-index http://rhaas.blogspot.com/2021/11/collation-stability.html https://vyruss.org/blog/slow-things-down-to-make-them-go-faster-postgres-build-2021.html https://www.postgresql.fastware.com/blog/postgresql-14-and-beyond https://blog.timescale.com/blog/postgresql-vs-python-for-data-cleaning-a-guide/ https://www.cybertec-postgresql.com/en/primary-keys-vs-unique-constraints-in-postgresql/ https://www.percona.com/blog/postgresql-14-database-monitoring-and-logging-enhancements/ https://yugonagata-pgsql.blogspot.com/2021/11/transition-tables-in-incremental-view.html https://blog.crunchydata.com/blog/postgis-day-2021 https://blog.crunchydata.com/blog/tricks-for-faster-spatial-indexes https://blog.rustprooflabs.com/2021/

In this episode of Scaling Postgres, we discuss using entity-attribute-value designs, new JSON subscripting capabilities, the mysql-fdw support for aggregate pushdowns and new regex functions in PG 15. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/entity-attribute-value-eav-design-in-postgresql-dont-do-it/ https://blog.logrocket.com/whats-new-json-postgresql-v14/ https://www.enterprisedb.com/blog/aggregate-push-down-mysqlfdw https://www.depesz.com/2021/11/26/waiting-for-postgresql-15-add-assorted-new-regexp_xxx-sql-functions/ https://blog.crunchydata.com/blog/using-timescaledb-extension-with-the-pgo-the-postgres-operator https://www.highgo.ca/2021/11/27/how-to-run-a-specific-regression-test/ https://postgresql.life/post/pavel_luzanov/

In this episode of Scaling Postgres, we discuss if foreign keys should have indexes, how to run graph queries, how to configure Linux huge pages and the benefits as well as text size similarities. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.percona.com/blog/should-i-create-an-index-on-foreign-keys-in-postgresql/ https://www.youtube.com/watch?v=_RXfnnqsLlw https://www.enterprisedb.com/blog/improving-postgresql-performance-without-making-changes-postgresql https://www.depesz.com/2021/11/19/does-varcharn-use-less-disk-space-than-varchar-or-text/ https://pganalyze.com/blog/postgresql-views-django-python https://www.enterprisedb.com/blog/pg-phriday-isolating-postgres-repmgr https://www.cybertec-postgresql.com/en/postgresql-on-wsl2-for-windows-install-and-setup/ https://www.depesz.com/2021/11/16/waiting-for-postgresql-15-allow-publishing-the-tables-of-schema/ https://www.migops.com/blog/2021/11/15/postgresql-15-will-include-

In this episode of Scaling Postgres, we discuss new releases of Postgres due to a man-in-the-middle vulnerability, the high availability solution pg_auto_failover, lesser known Postgres features and LZ4 compression. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-141-135-129-1114-1019-and-9624-released-2349/ https://www.enterprisedb.com/blog/postgres-mitm21-vulnerabilities https://www.youtube.com/watch?v=fuYwlVw0e_o https://www.pgbouncer.org/2021/11/pgbouncer-1-16-1 https://tapoueh.org/blog/2021/11/an-introduction-to-the-pg_auto_failover-project/ https://hakibenita.com/postgresql-unknown-features https://www.postgresql.fastware.com/blog/what-is-the-new-lz4-toast-compression-in-postgresql-14 https://blog.jooq.org/postgresql-14s-enable_memoize-for-improved-performance-of-nested-loop-joins/ https://blog.timescale.com/blog/generating-more-realistic-sample-time-series-data-with-postgresql-ge

In this episode of Scaling Postgres, we discuss the open sourcing of Babelfish, deconstructing the Postgres planner, when to avoid exist & not exist and having fun with SQL. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://babelfishpg.org/blog/releases/2021/10/babelfish-launch/ https://aws.amazon.com/blogs/aws/goodbye-microsoft-sql-server-hello-babelfish/ https://pganalyze.com/blog/deconstructing-the-postgres-planner https://postgres.ai/blog/20211103-three-cases-against-if-not-exists-and-if-exists-in-postgresql-ddl https://blog.crunchydata.com/blog/fun-with-sql-in-postgres-finding-revenue-accrued-per-day https://notes.eatonphil.com/exploring-plpgsql-forth-like.html https://www.highgo.ca/2021/11/01/the-postgresql-timeline-concept/ https://blog.crunchydata.com/blog/patroni-etcd-in-high-availability-environments https://blog.crunchydata.com/blog/resize-postgres-kubernetes-volume-instance-sets https://aws.amazon.com/blogs/database/ve

In this episode of Scaling Postgres, we discuss the downsides of indexes, TCP keep alive options, developing with Postgres as your DB and learning PL/pgSQL. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20211029-how-partial-and-covering-indexes-affect-update-performance-in-postgresql https://www.cybertec-postgresql.com/en/tcp-keepalive-for-a-better-postgresql-experience/ https://www.youtube.com/watch?v=mliO5_XhAKE https://notes.eatonphil.com/exploring-plpgsql.html https://blog.crunchydata.com/blog/secure-postgresql-14-with-the-cis-benchmark https://blog.crunchydata.com/blog/database-security-best-practices-on-kubernetes https://blog.crunchydata.com/blog/encrypting-postgres-data-at-rest-in-kubernetes https://postgresql.life/post/alexander_kukushkin/ https://www.rubberduckdevshow.com/episodes/18-how-many-3rd-party-libraries-should-you-use/

In this episode of Scaling Postgres, we discuss automatic indexing, function pipelines, with hold cursors and the different query scans. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2021/10/22/why-is-it-hard-to-automatically-suggest-what-index-to-create/ https://blog.timescale.com/blog/function-pipelines-building-functional-programming-into-postgresql-using-custom-operators/ https://www.cybertec-postgresql.com/en/with-hold-cursors-and-transactions-in-postgresql/ https://arctype.com/blog/postgres-explain-analyze-scan/ https://postgres.ai/blog/20211018-postgresql-lock-trees https://blog.crunchydata.com/blog/postgres-queries-for-projecting-monthly-revenue-run-rate https://about.sourcegraph.com/blog/postgres-text-search-balancing-query-time-and-relevancy/ https://elephanttamer.net/?p=68 https://aws.amazon.com/blogs/database/upgrade-amazon-rds-and-amazon-aurora-postgresql-version-9-6/ https://postgresql.life/pos

In this episode of Scaling Postgres, we discuss one method to shard a database, federating your database, the benefits of configuring linux HugePages and performance tuning. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.notion.so/blog/sharding-postgres-at-notion https://blog.crunchydata.com/blog/multi-cluster-postgres-federation-on-kubernetes-with-postgres-14-using-gitops https://www.percona.com/blog/why-linux-hugepages-are-super-important-for-database-servers-a-case-with-postgresql/ https://www.enterprisedb.com/blog/webinar-recap-tuning-tips-maximize-postgres-performance https://aws.amazon.com/blogs/database/tune-sorting-operations-in-postgresql-with-work_mem/ https://www.cybertec-postgresql.com/en/show-tables-in-postgresql-whats-wrong-with-it/ https://blog.rustprooflabs.com/2021/10/postgres-14-query-id https://www.highgo.ca/2021/10/15/backup-label-in-postgresql/ https://www.postgresonline.com/journal/index.php?/archives/401-Multirange

In this episode of Scaling Postgres, we discuss select for update, the pgx framework to generate extensions, cool additions to Postgres 14 and full-text search. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.migops.com/blog/2021/10/05/select-for-update-and-its-behavior-with-foreign-keys-in-postgresql/ https://blog.rustprooflabs.com/2021/10/pgdd-extension-using-pgx-rust https://www.enterprisedb.com/blog/cool-new-contributions-postgresql-14 https://arctype.com/blog/postgres-full-text-search/ https://www.enterprisedb.com/postgresql-database-backup-recovery-what-works-wal-pitr https://yugonagata-pgsql.blogspot.com/2021/10/transition-tables-in-imcremental-view.html https://www.enterprisedb.com/blog/how-set-kerberos-authentication-using-active-directory-postgresql-database https://www.cybertec-postgresql.com/en/openstreetmap-service-by-cybertec/ https://blog.crunchydata.com/blog/postgresql-14-on-kubernetes https://postgresql

In this episode of Scaling Postgres, we discuss the release of Postgres 14, how best to use JSON, why are your indexes not being used and sequence gaps. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-released-2318/ https://www.citusdata.com/blog/2021/10/01/how-we-shipped-postgres-14-on-azure-within-a-day/ https://www.instaclustr.com/postgresql-14/ https://www.migops.com/blog/2021/09/29/upgrading-postgresql-9-6-to-postgresql-13/ https://www.cybertec-postgresql.com/en/json-postgresql-how-to-use-it-right/ https://www.depesz.com/2021/09/27/using-recursive-queries-to-get-distinct-elements-from-table/ https://www.pgmustard.com/blog/why-isnt-postgres-using-my-index https://postgres.ai/blog/20210923-zero-downtime-postgres-schema-migrations-lock-timeout-and-retries https://www.cybertec-postgresql.com/en/gaps-in-sequences-postgresql/ https://mydbanotebook.org/post/auto-increment/ https:/

In this episode of Scaling Postgres, we discuss Postgres 14 RC1, little improvements in PG14, an index advisor and how to handle 1 million insert statements. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-rc-1-released-2309/ https://blog.crunchydata.com/blog/postgres-14-its-the-little-things https://pganalyze.com/blog/introducing-pganalyze-index-advisor https://dev.to/yugabyte/when-you-have-millions-of-insert-statements-in-a-file-3nck https://fluca1978.github.io/2021/09/23/OracleAlterSequence.html https://www.cybertec-postgresql.com/en/building-an-oracle-to-postgresql-migrator-lessons-learned/ https://b-peng.blogspot.com/2021/09/reserved-connections.html https://b-peng.blogspot.com/2021/09/enable-shared-relcache.html https://postgresql.life/post/michael_goldberg/ https://www.rubberduckdevshow.com/episodes/14-background-job-processing/