Scaling Postgres

In this episode of Scaling Postgres, we discuss stat tracking, temporary tables, a new utility called pg_squeeze and an update to pg_auto_failover. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pgcraftsman.io/2020/10/15/tracking-postgres-stats/ https://www.cybertec-postgresql.com/en/postgresql-sophisticating-temporary-tables/ https://www.cybertec-postgresql.com/en/pg_squeeze-optimizing-postgresql-storage/ https://www.citusdata.com/blog/2020/10/10/whats-new-in-pg-auto-failover/ https://blog.arkency.com/multitenancy-with-postgres-schemas-key-concepts-explained/ https://info.crunchydata.com/blog/postgresql-monitoring-for-app-developers-alerts-troubleshooting https://info.crunchydata.com/blog/tuning-your-postgres-database-for-high-write-loads https://dataegret.com/2020/10/postgres-13-observability-updates/ http://blog.cleverelephant.ca/2020/10/podcasts.html https://postgresql.life/post/greg_sabino_mullane/  

In this episode of Scaling Postgres, we discuss connection scalability, benchmarking Postgres performance, the community impact of an acquisition and the progress of zheap. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.citusdata.com/blog/2020/10/08/analyzing-connection-scalability/ https://blog.anarazel.de/2020/10/07/measuring-the-memory-overhead-of-a-postgres-connection/ https://www.postgresql.org/message-id/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de https://www.2ndquadrant.com/en/blog/oltp-performance-since-postgresql-8-3/ https://www.enterprisedb.com/blog/postgresql-tpc-c-benchmarks-postgresql-12-vs-postgresql-13-performance https://momjian.us/main/blogs/pgblog/2020.html#October_7_2020 https://www.enterprisedb.com/blog/edb-community-acquisition-statement https://www.cybertec-postgresql.com/en/zheap-reinvented-postgresql-storage/ https://pawelurbanek.com/postgresql-fix-performance https://www.enterprisedb.co

In this episode of Scaling Postgres, we discuss a company acquisition within the Postgres community, how to save terabytes of space, setting up WAL archiving and vacuum/analyze tips. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.enterprisedb.com/news/edb-completes-acquisition-2ndquadrant-becomes-largest-dedicated-provider-postgresql-products https://www.enterprisedb.com/blog/how-edb-became-leader-postgres-market https://medium.com/braintree-product-technology/postgresql-at-scale-saving-space-basically-for-free-d94483d9ed9a https://www.highgo.ca/2020/10/01/postgresql-wal-archiving-and-point-in-time-recovery/ https://www.2ndquadrant.com/en/blog/postgresql-vacuum-and-analyze-best-practice-tips/ https://www.percona.com/blog/2020/09/30/postgresql_fdw-authentication-changes-in-postgresql-13/ https://www.percona.com/blog/2020/10/01/postgresql-13-new-feature-dropdb-force/ https://www.cybertec-postgresql.com/en/a-quick-look-at-po

In this episode of Scaling Postgres, we discuss the release of Postgres 13, what is new in Postgres 13, how to debug PL/PGSQL and Postgres monitoring best practices. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-13-released-2077/ https://pganalyze.com/blog/postgres13-better-performance-monitoring-usability https://www.enterprisedb.com/blog/whats-new-postgresql-13 https://www.2ndquadrant.com/en/blog/new-features-in-postgresql-13-full-webinar-video/ https://www.youtube.com/watch?v=wIRbpkQbioY https://www.cybertec-postgresql.com/en/debugging-pl-pgsql-get-stacked-diagnostics/ https://www.youtube.com/watch?v=wulQlXaXOxc https://hakibenita.com/sql-anomaly-detection https://www.highgo.ca/2020/09/25/in-memory-table-with-pluggable-storage-api/ https://info.crunchydata.com/blog/announcing-crunchy-bridge-a-modern-postgres-as-a-service https://info.crunchydata.com/blog/mapping-data-types-in-

In this episode of Scaling Postgres, we discuss the release of Postgres 13 RC1, how to upgrade Postgres at scale, using limit with ties and understanding Heap Only Tuple (HOT) updates. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/2072/ https://about.gitlab.com/blog/2020/09/11/gitlab-pg-upgrade/ https://www.2ndquadrant.com/en/blog/postgresql-13-limit-with-ties/ https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/ https://www.2ndquadrant.com/en/blog/7-best-practice-tips-for-postgresql-bulk-data-loading/ https://info.crunchydata.com/blog/postgresql-13-hidden-gems https://info.crunchydata.com/blog/exploring-plpy-turn-postgres-table-data-into-a-numpy-array https://momjian.us/main/writings/pgsql/AI.pdf https://www.citusdata.com/blog/2020/09/17/citus-and-postgres-at-any-scale/ https://postgresql.life/post/anthony_nowocien/

In this episode of Scaling Postgres, we discuss slow WAL archival, index improvements coming for Postgres 13, accessing node metrics via SQL and deciding between a join or subquery. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.percona.com/blog/2020/09/09/why-postgresql-wal-archival-is-slow/ https://www.percona.com/blog/2020/09/10/index-improvements-in-postgresql-13/ https://blog.rustprooflabs.com/2020/09/postgres-beta3-btree-dedup https://info.crunchydata.com/blog/postgresql-node-metrics-pgnodemx https://info.crunchydata.com/blog/joins-or-subquery-in-postgresql-lessons-learned https://www.cybertec-postgresql.com/en/generating-a-normal-distribution-in-sql/ https://www.2ndquadrant.com/en/blog/which-partition-contains-a-specific-row-in-my-postgresql-database/ https://blog.hagander.net/fdws-curl-and-limit-247/ https://www.2ndquadrant.com/en/blog/pg-phriday-10-things-postgres-could-improve-part-3/ https://momjian.us/main

In this episode of Scaling Postgres, we discuss external compression, vacuuming in parallel, working with JSON Aggregates and the JSON & Array datatypes. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://amitdkhan-pg.blogspot.com/2020/08/need-for-external-compression-methods.html https://www.2ndquadrant.com/en/blog/parallelism-comes-to-vacuum/ https://blog.k-nut.eu/postgres-json-aggregate-functions https://www.2ndquadrant.com/en/blog/webinar-json-array-contemporary-postgresql-data-types-follow-up/ https://www.2ndquadrant.com/en/blog/how-to-get-the-best-out-of-postgresql-logs/ https://www.cybertec-postgresql.com/en/postgresql-vs-postgresql-vs-postgre-vs-oracle-vs-all-the-rest/ https://www.2ndquadrant.com/en/blog/oracle-to-postgresql-cursors-and-ltrees/ https://info.crunchydata.com/blog/three-easy-things-to-remember-about-postgres-indexes https://severalnines.com/database-blog/overview-trusted-extensions-postgresql-13 https

In this episode of Scaling Postgres, we discuss 28 tips & tricks, Postgres 13 is lucky, autovacuum tuning and logical replication pitfalls. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pgdash.io/blog/postgres-tips-and-tricks.html https://info.crunchydata.com/blog/why-postgresql-13-is-a-lucky-release https://www.cybertec-postgresql.com/en/tuning-autovacuum-postgresql/ https://elephanttamer.net/?p=35 https://www.youtube.com/watch?v=eDkwqAZ-rpk https://b-peng.blogspot.com/2020/08/authentication-in-pgpool.html https://info.crunchydata.com/blog/using-postgres-to-back-prometheus-for-your-postgresql-monitoring-1 https://www.highgo.ca/2020/08/28/who-is-spending-wal-crazily/ https://www.2ndquadrant.com/en/blog/advanced-partition-matching-for-partition-wise-join/ https://www.depesz.com/2020/08/24/waiting-for-postgresql-14-pg_stat_statements-track-number-of-rows-processed-by-some-utility-commands/ https://info.crunchydata.com/b

In this episode of Scaling Postgres, we discuss speeding up performance with statistics, setting synchronous_commit, updates to TLS settings and how to avoid cursors. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://build.affinity.co/how-we-used-postgres-extended-statistics-to-achieve-a-3000x-speedup-ea93d3dcdc61 https://www.percona.com/blog/2020/08/21/postgresql-synchronous_commit-options-and-synchronous-standby-replication/ https://www.highgo.ca/2020/08/21/tls-related-updates-in-postgresql-13/ https://www.2ndquadrant.com/en/blog/oracle-to-postgresql-cursors-and-common-table-expressions/ https://www.enterprisedb.com/blog/8-cool-interesting-facts-things-postgresql-can-do https://www.2ndquadrant.com/en/blog/updating-the-postgresql-root-crt-file/ https://www.cybertec-postgresql.com/en/postgresql-group-by-expression/ https://pganalyze.com/blog/postgres-row-level-security-django-python https://www.highgo.ca/2020/08/21/how-to-set

In this episode of Scaling Postgres, we discuss recursive queries, using lateral joins, paging through a data sets efficiently and working with JSONB arrays. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/2060/ https://www.cybertec-postgresql.com/en/recursive-queries-postgresql/ https://info.crunchydata.com/blog/iterators-in-postgresql-with-lateral-joins https://momjian.us/main/blogs/pgblog/2020.html#August_12_2020 https://engineering.shopify.com/blogs/engineering/pagination-relative-cursors https://use-the-index-luke.com/sql/partial-results/fetch-next-page https://levelup.gitconnected.com/working-with-a-jsonb-array-of-objects-in-postgresql-d2b7e7f4db87 https://www.2ndquadrant.com/en/blog/running-multiple-pgbouncer-instances-with-systemd/ https://www.highgo.ca/2020/08/08/partitioning-improvements-in-postgresql-13/ https://www.highgo.ca/2020/08/10/postgresql-high-availability-the-considerations-and-candidates/

In this episode of Scaling Postgres, we discuss BRIN pitfalls, using pg_stat_statements for troubleshooting, natural joins to detect duplicates and the geography type. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://info.crunchydata.com/blog/avoiding-the-pitfalls-of-brin-indexes-in-postgres https://www.cybertec-postgresql.com/en/a-quick-pg_stat_statements-troubleshooting-hack/ https://blog.jooq.org/2020/08/05/use-natural-full-join-to-compare-two-tables-in-sql/ https://info.crunchydata.com/blog/postgis-and-the-geography-type https://www.2ndquadrant.com/en/blog/webinar-being-committed-a-review-of-transaction-control-statements-3-3-follow-up/ https://www.ongres.com/blog/a_generalized_unstructured_data_type_for_postgres/ https://supabase.io/blog/2020/08/02/continuous-postgresql-backup-walg/ https://www.2ndquadrant.com/en/blog/nss-on-windows-for-postgresql-development/ https://postgresql.life/post/tatsuo_ishii/  

In this episode of Scaling Postgres, we discuss suggestions for scaling postgres, replication slots safety, implementing scram password authentication with channel binding and setting effective_cache_size. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://hakibenita.com/sql-tricks-application-dba https://stribny.name/blog/2020/07/scaling-relational-sql-databases https://www.2ndquadrant.com/en/blog/pg13-slot-size-limit/ https://www.depesz.com/2020/07/27/waiting-for-postgresql-14-rename-wal_keep_segments-to-wal_keep_size/ https://www.citusdata.com/blog/2020/07/28/securely-authenticate-with-scram-in-postgres-13/ https://www.percona.com/blog/2020/07/28/migrating-to-postgresql-version-13-incompatibilities-you-should-be-aware-of/ https://www.cybertec-postgresql.com/en/effective_cache_size-a-practical-example/ https://www.2ndquadrant.com/en/blog/webinar-being-committed-a-review-of-transaction-control-statements-1-3-follow-up-2/ https://www.sqlstyle.gu

In this episode of Scaling Postgres, we discuss how best to migrate to using partitions, like & ilike performance, determining the best fillfactor and the work towards in-memory tables. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.2ndquadrant.com/en/blog/partitioning-a-large-table-without-a-long-running-lock/ https://www.cybertec-postgresql.com/en/postgresql-more-performance-for-like-and-ilike-statements/ https://www.cybertec-postgresql.com/en/what-is-fillfactor-and-how-does-it-affect-postgresql-performance/ https://www.highgo.ca/2020/07/24/approaches-to-achieve-in-memory-table-storage-with-postgresql-pluggable-api/ https://www.2ndquadrant.com/en/blog/webinar-being-committed-a-review-of-transaction-control-statements-1-3-follow-up/ https://www.enterprisedb.com/blog/monitoring-postgresql-database-system-activities-performance-system-stats-extension https://www.enterprisedb.com/blog/maintaining-postgresql-for-high-performance-what-i

In this episode of Scaling Postgres, we discuss working with JSONB types, calculating earth distance, utilizing dates, times & intervals and Postgres authentication. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://momjian.us/main/blogs/pgblog/2020.html#July_13_2020 https://adamfallon.com/2020/07/10/recreating-yikyak-with-postgres/ https://pgdash.io/blog/date-time-interval-postgres.html https://www.enterprisedb.com/blog/how-to-secure-postgresql-database-best-practice-tips-securing-authentication-methods https://info.crunchydata.com/blog/pgbouncer-scram-authentication-postgresql https://supabase.io/blog/2020/07/09/postgresql-templates/ https://www.cybertec-postgresql.com/en/pgwatch2-v1-8-0-released/ https://www.2ndquadrant.com/en/blog/barman-cloud-part-2-cloud-backup/ https://postgresql.life/post/stacey_haysler/  

In this episode of Scaling Postgres, we discuss determining if Postgres is fine, generated columns vs. triggers, insert features and synchronous replication usage. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.2ndquadrant.com/en/blog/its-the-end-of-the-world-as-we-know-it-and-postgres-is-fine/ https://www.ongres.com/blog/generate_columns_vs_triggers/ https://info.crunchydata.com/blog/a-walk-through-insert https://www.youtube.com/watch?v=wLJvw5blCo8 https://www.highgo.ca/2020/07/06/features-in-pg13-deduplication-in-b-tree-indexes/ https://www.endpoint.com/blog/2020/07/02/random-strings-and-integers-that-actually-arent https://info.crunchydata.com/blog/getting-started-with-postgresql-operator-4.3-in-openshift https://www.highgo.ca/2020/07/11/a-simple-way-to-trace-a-bug/ https://www.cybertec-postgresql.com/en/pg_timetable-start-up-improvements/ https://postgresql.life/post/kohei_kaigai/  

In this episode of Scaling Postgres, we discuss fantastic data types, custom domain data types, improving insert performance and how to query row differences. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.2ndquadrant.com/en/blog/webinar-fantastic-data-types-and-where-to-use-them-follow-up/ https://www.cybertec-postgresql.com/en/postgresql-useful-new-data-types/ https://blog.timescale.com/blog/13-tips-to-improve-postgresql-insert-performance/ https://www.cybertec-postgresql.com/en/calculating-differences-between-rows-in-sql/ https://vsevolod.net/postgresql-jsonb-index/ https://www.enterprisedb.com/blog/why-use-synchronous-replication-in-postgresql-configure-streaming-replication-wal https://www.2ndquadrant.com/en/blog/oracle-to-postgresql-binary-objects/ https://www.youtube.com/watch?v=ykmqp_ETniE https://www.2ndquadrant.com/en/blog/webinar-postgresql-partitioning-roadmap-follow-up/ https://www.enterprisedb.com/blog/how-to-secure-postg

In this episode of Scaling Postgres, we discuss Postgres 13 Beta 2, a system stats extension, tuning your operating systems for Postgres and different join strategies. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/2047/ https://www.youtube.com/watch?v=G5f2dfIB4GY https://pgsnake.blogspot.com/2020/06/systemstats-extension-for-postgresql.html https://hunleyd.github.io/posts/tuned-PG-and-you/ https://www.cybertec-postgresql.com/en/join-strategies-and-performance-in-postgresql/ http://www.sai.msu.su/~megera/postgres/talks/json-unification-database-meetup-2020.pdf https://www.cybertec-postgresql.com/en/postgresql-ltree-vs-with-recursive/ https://pganalyze.com/blog/active-record-subqueries-rails https://info.crunchydata.com/blog/composite-primary-keys-postgresql-and-django https://www.highgo.ca/2020/06/22/types-of-indexes-in-postgresql/ https://www.2ndquadrant.com/en/blog/oracle-to-postgresql-reasons-to-migrate/ htt

In this episode of Scaling Postgres, we discuss replication complications, adding one word to boost performance, handling pgbouncer users and serializable isolation testing. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.2ndquadrant.com/en/blog/pg-phriday-10-things-postgres-could-improve-part-2/ https://jlongster.com/how-one-word-postgresql-performance https://www.2ndquadrant.com/en/blog/understanding-user-management-in-pgbouncer/ http://jepsen.io/analyses/postgresql-12.3 https://www.2ndquadrant.com/en/blog/webinar-understanding-the-postgresql-table-page-layout-follow-up/ https://www.cybertec-postgresql.com/en/sql-trickery-hypothetical-aggregates/ https://www.cybertec-postgresql.com/en/wrapping-db2-with-postgresql/ https://www.2ndquadrant.com/en/blog/oracle-to-postgresql-basic-architecture/ https://www.citusdata.com/blog/2020/06/13/citus-9-3-release-notes/ https://postgresql.life/post/tomas_vondra/  

In this episode of Scaling Postgres, we discuss safety systems, the purpose and existence of failover slots, complications with transaction IDs and how to repartition without downtime. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://momjian.us/main/blogs/pgblog/2020.html#June_8_2020 https://www.2ndquadrant.com/en/blog/failover-slots-postgresql/ https://www.2ndquadrant.com/en/blog/10-things-postgres-could-improve-part-1/ https://info.crunchydata.com/blog/control-runaway-postgres-queries-with-statement-timeout https://blog.hagander.net/repartitioning-with-logical-replication-in-postgresql-13-246/ https://www.percona.com/blog/2020/06/09/multi-master-replication-solutions-for-postgresql/ https://www.cybertec-postgresql.com/en/composite-type-performance-issues-in-postgresql/ https://www.highgo.ca/2020/06/12/transactions-in-postgresql-and-their-mechanism/ http://blog.cleverelephant.ca/2020/04/developer-life.html http://blog.cleverelephant.ca/2020/0

In this episode of Scaling Postgres, we discuss index deduplication in Postgres 13, fast hierarchical access, more essential monitoring and cloud native Postgres. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/b-tree-index-deduplication/ https://www.cybertec-postgresql.com/en/postgresql-speeding-up-recursive-queries-and-hierarchic-data/ https://pgdash.io/blog/essential-postgres-monitoring-part2.html https://pgdash.io/blog/essential-postgres-monitoring-part3.html https://www.2ndquadrant.com/en/blog/webinar-cloud-native-bdr-and-postgresql-follow-up/ https://www.highgo.ca/2020/06/01/optimizing-sql-simplifying-queries-with-window-functions/ https://www.percona.com/blog/2020/05/29/removing-postgresql-bottlenecks-caused-by-high-traffic/ https://www.enterprisedb.com/blog/postgresql-wal-write-ahead-logging-management-strategy-tradeoffs https://habr.com/en/company/postgrespro/blog/504498/ https://postgresql.life/post/ma