Scaling Postgres

In this episode of Scaling Postgres, we discuss one cause of broken indexes, Postgres trademark issues, percentiles vs. averages and logical replication improvements. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://elephanttamer.net/?p=61 https://www.postgresql.org/about/news/trademark-actions-against-the-postgresql-community-2302/ https://postgresql.fund/blog/postgres-core-team-attacks-postgres-community/ https://postgresql.fund/blog/respecting-majority-questioning-status-quo-as-a-minority/ https://blog.timescale.com/blog/how-percentile-approximation-works-and-why-its-more-useful-than-averages/ http://amitkapila16.blogspot.com/2021/09/logical-replication-improvements-in.html https://www.cybertec-postgresql.com/en/postgresql-create-indexes-after-bulk-loading/ https://www.highgo.ca/2021/09/17/a-quick-test-for-postgres_fdw-batch-insertion/ https://www.cybertec-postgresql.com/en/postgresql-schedulers-comparison-table/ https:/

In this episode of Scaling Postgres, we discuss having boundless text fields, revoking public schema permissions, less index bloat in PG14 and comparing hardware performance. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://brandur.org/text https://www.depesz.com/2021/09/10/waiting-for-postgresql-15-revoke-public-create-from-public-schema-now-owned-by-pg_database_owner/ https://www.cybertec-postgresql.com/en/index-bloat-reduced-in-postgresql-v14/ https://dev.to/aws-heroes/aurora-postgresql-db-r6g-compared-to-db-r5-with-ybio-1bnd https://brandur.org/sqlc https://blog.rustprooflabs.com/2021/09/psycopg3-initial-review https://www.varrazzo.com/blog/2020/03/06/thinking-psycopg3/ https://fluca1978.github.io/2021/09/10/pgbackrest-jq.html https://postgresql.life/post/akshay_joshi/ https://www.rubberduckdevshow.com/episodes/12-programming-with-dates-times-zones

In this episode of Scaling Postgres, we discuss connection scaling, how to cache your sequences, a message queuing implementation and the harms of subtransactions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://richyen.com/postgres/2021/09/03/less-is-more-max-connections.html https://dev.to/yugabyte/uuid-or-cached-sequences-42fi https://blog.crunchydata.com/blog/message-queuing-using-native-postgresql https://postgres.ai/blog/20210831-postgresql-subtransactions-considered-harmful https://blog.timescale.com/blog/how-to-create-lots-of-sample-time-series-data-with-postgresql-generate_series/ https://blog.anayrat.info/en/2021/09/01/partitioning-use-cases-with-postgresql/ https://www.postgresql.fastware.com/blog/logical-replication-tablesync-workers https://dev.to/ftisiot/solving-the-knapsack-problem-in-postgresql-40cc https://www.cybertec-postgresql.com/en/postgresql-the-power-of-a-single-missing-index/ http://postgis.net/202

In this episode of Scaling Postgres, we discuss how to use aggregate filters, the importance of vacuum with insert-only tables, how to fix out-of-sync sequences and faster functional index options. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.crunchydata.com/blog/fast-flexible-summaries-with-aggregate-filters-and-windows https://blog.crunchydata.com/blog/insert-only-tables-and-autovacuum-issues-prior-to-postgresql-13 https://www.cybertec-postgresql.com/en/fixing-out-of-sync-sequences-in-postgresql/ https://dev.to/aws-heroes/index-only-scan-on-functional-indexes-9o2 https://www.percona.com/blog/postgresql-database-security-os-authentication/ https://www.depesz.com/2021/08/26/how-to-get-advisory-lock-in-shell/ https://www.migops.com/blog/2021/08/27/announcing-pg_dbms_job-in-postgresql-for-oracle-dbms_job-compatibility/ https://www.highgo.ca/2021/08/23/tuning-the-pgpool-ii-load-balancer/ https://arctype.com/blog/postgre

In this episode of Scaling Postgres, we discuss how to use foreign data wrappers, compression available in timescaledb, working with fuzzy search and describing Postgres query execution. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.crunchydata.com/blog/understanding-postgres_fdw https://blog.rustprooflabs.com/2021/08/timescale-compression-openstreetmap-tags https://arctype.com/blog/postgres-fuzzy-search/ https://dev.to/yugabyte/postgres-query-execution-simple-query-2i71 https://www.highgo.ca/2021/08/20/how-batch-insertion-was-done-in-postgres_fdw/ https://dev.to/yugabyte/pgbench-client-jobs-68g https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-secure-cloud-raster-access https://www.migops.com/blog/2021/08/20/validate-postgresql-extension-upgrade-scripts-using-pg_validate_extupgrade/ https://www.cybertec-postgresql.com/en/pg_timetable_v4-is-out/ https://postgresql.life/post/andrey_borodin/ https://www.rubber

In this episode of Scaling Postgres, we discuss new releases for Postgres & pgbouncer, have a performance discussion, describe the different index types and how to use them. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-134-128-1113-1018-9623-and-14-beta-3-released-2277/ https://www.pgbouncer.org/2021/08/pgbouncer-1-16-0 https://www.rubberduckdevshow.com/episodes/8-database-performance/ https://pganalyze.com/blog/postgres-create-index https://www.youtube.com/watch?v=gBGUuMrO6MI https://www.cybertec-postgresql.com/en/postgresql-vs-redis-vs-memcached-performance/ https://www.percona.com/blog/impact-of-network-and-cursor-on-query-performance-of-postgresql/ https://www.ongres.com/blog/noset-postgresql-extension-for-blocking-set-reset-execution/ https://www.percona.com/blog/migrating-postgresql-to-kubernetes https://www.highgo.ca/2021/08/09/horizontal-scalability-options-in-postgresq

In this episode of Scaling Postgres, we discuss ways to work around the planner, working with grouping sets, using a lateral join and not-so-easy Postgres issues. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://heap.io/blog/when-the-postgres-planner-is-not-very-smart https://www.cybertec-postgresql.com/en/postgresql-grouping-sets-rollup-cube/ https://fluca1978.github.io/2021/08/07/PostgreSQLLateralJoin.html https://pgdash.io/blog/postgresql-six-not-so-easy-pieces.html https://www.postgresql.fastware.com/blog/logical-decoding-of-two-phase-commits https://blog.crunchydata.com/blog/using-cert-manager-to-deploy-tls-for-postgres-on-kubernetes https://postgresql.life/post/bryn_llewellyn/ https://www.rubberduckdevshow.com/episodes/7-writing-secure_code/

In this episode of Scaling Postgres, we discuss JSONB vs. window functions, a Postgres security checklist, implementing full-text search and using NFL timeseries data. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://swarm64.com/post/postgresql-jsonb-vs-window-functions/ https://www.cybertec-postgresql.com/en/postgresql-security-things-to-avoid-in-real-life/ https://blog.crunchydata.com/blog/postgres-full-text-search-a-search-engine-in-a-database https://blog.timescale.com/blog/hacking-nfl-data-with-postgresql-timescaledb-and-sql/ https://ardentperf.com/2021/07/26/postgresql-logical-replicas-and-snapshots-proceed-carefully/ https://www.pgmustard.com/blog/max-parallel-workers-per-gather https://fluca1978.github.io/2021/07/27/pgbackrestAsync.html https://b-peng.blogspot.com/2021/07/deploying-pgpool2-exporter-with-docker.html https://www.percona.com/blog/improve-postgresql-query-performance-insights-with-pg_stat_monitor/ https:/

In this episode of Scaling Postgres, we discuss the current status of zheap, the gaps in sequences, how to efficiently paginate and how to set up patroni with etcd and haproxy. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-zheap-current-status/ https://incident.io/blog/one-two-skip-a-few https://pganalyze.com/blog/pagination-django-postgres https://arctype.com/blog/postgres-patroni/ https://fluca1978.github.io/2021/07/20/PostgreSQLExtensions.html https://www.highgo.ca/2021/07/23/the-amazing-buffer-tag-in-postgresql/ https://www.paulox.net/2021/07/19/maps-with-django-part-2-geodjango-postgis-and-leaflet/ https://blog.crunchydata.com/blog/using-postgis-and-pg_featureserv-with-qgis https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-st_makevalid https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-st_contour-and-st_setz https://postgresql.life/post/rafia_sabih/ https:/

In this episode of Scaling Postgres, we discuss understanding lateral joins, generating JSON with Postgres, the dbcritic utility, and fetch first rows vs. limit. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/understanding-lateral-joins-in-postgresql/ https://blog.crunchydata.com/blog/generating-json-directly-from-postgres https://www.channable.com/tech/dbcritic-constructively-criticizing-your-postgres-schema https://www.cybertec-postgresql.com/en/postgresql-limit-vs-fetch-first-rows-with-ties/ https://elephanttamer.net/?p=58 https://www.youtube.com/watch?v=znPL6iDDou8 https://fluca1978.github.io/2021/07/13/PostgreSQLWalTraffic.html https://www.postgresql.fastware.com/blog/improved-logging-by-libpq-in-postgresql-14 https://www.highgo.ca/2021/07/14/first-contact-with-the-pg_filedump/ http://amitkapila16.blogspot.com/2021/07/logical-replication-of-in-progress.html https://postgresql.life/post

In this episode of Scaling Postgres, we discuss how to delay replication, working with materialized view permissions, paranoid configuration options and addressing memory overcommit. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/simulating-temporal-tables-with-dblink-and-replication-delay/ https://blog.rustprooflabs.com/2021/07/postgres-permission-mat-view https://ardentperf.com/2021/07/06/paranoid-sql-execution-on-postgresql/ https://sosna.de/posts/pgaas-memory-overcommit/ https://pganalyze.com/blog/postgresql-partitioning-django https://www.migops.com/blog/2021/07/02/credcheck-extension-to-enforce-username-and-password-checks-in-postgresql/ https://blog.crunchydata.com/blog/the-next-generation-of-kubernetes-native-postgres https://blog.crunchydata.com/blog/getting-started-with-pgo-postgres-operator-5.0 https://b-peng.blogspot.com/2021/07/postgres-disaster-recovery-on-k8s-zalando.html http

In this episode of Scaling Postgres, we discuss tracking per-operation explain analyze, Postgres 14 contribution stats, parallel execution on foreign servers and incremental view maintenance. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.pgmustard.com/blog/calculating-per-operation-times-in-postgres-explain-analyze https://www.highgo.ca/2021/06/28/some-interesting-statistics-about-pg-14-contributions/ https://www.highgo.ca/2021/06/28/parallel-execution-of-postgres_fdw-scans-in-pg-14-important-step-forward-for-horizontal-scaling/ https://yugonagata-pgsql.blogspot.com/2021/06/implementing-incremental-view.html https://lesovsky.medium.com/updates-of-postgresql-observability-diagram-66f0bbbfe021 https://www.migops.com/blog/2021/07/01/ora2pg-now-supports-oracle_fdw-to-increase-the-data-migration-speed/ http://postgis.net/2021/07/02/postgis-3.1.3/ https://www.cybertec-postgresql.com/en/postgis-setup-with-ubuntu-20-04-2/ https

In this episode of Scaling Postgres, we discuss practical partitioning, understanding pg_repack, explaining buffers and enforcing multi-row constraints. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-beta-2-released-2249/ https://www.postgresvision.com/ https://vyruss.org/blog/practical-partitioning-in-production-with-postgres-postgres-vision-2021.html https://www.percona.com/blog/2021/06/24/understanding-pg_repack-what-can-go-wrong-and-how-to-avoid-it/ https://www.depesz.com/2021/06/20/explaining-the-unexplainable-part-6-buffers/ https://www.cybertec-postgresql.com/en/postgresql-constraints-over-multiple-rows/ https://www.cybertec-postgresql.com/en/from-md5-to-scram-sha-256-in-postgresql/ https://www.depesz.com/2021/06/23/a-tale-of-making-company-wide-standard-psqlrc/ https://fabianlindfors.se/blog/schema-migrations-in-postgres/ https://pganalyze.com/blog/geodjango-postgis htt

In this episode of Scaling Postgres, we discuss Redis vs. Postgres, working in the hard quadrant, how Postgres optimizes joins and working with materialized views. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://spin.atomicobject.com/2021/02/04/redis-postgresql/ https://momjian.us/main/blogs/pgblog/2021.html#June_16_2021 https://www.youtube.com/watch?v=XA3SBgcZwtE https://www.cybertec-postgresql.com/en/creating-and-refreshing-materialized-views-in-postgresql/ https://blog.crunchydata.com/blog/fun-with-pg_checksums https://www.cybertec-postgresql.com/en/cross-join-in-postgresql/ https://paquier.xyz/postgresql-2/postgres-14-monitoring-copy/ https://www.youtube.com/c/percona/videos https://pgsqlpgpool.blogspot.com/2021/06/promoting-specied-node-in-pgpool-ii.html https://blog.timescale.com/blog/how-and-why-to-become-a-postgresql-contributor/ https://postgresql.life/post/josh_berkus/ https://www.rubberduckdevshow.com/

In this episode of Scaling Postgres, we discuss only using bigint, high availability, Postgres autocommit and synchronized sequential scans. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.rustprooflabs.com/2021/06/postgres-bigint-by-default https://www.percona.com/blog/2021/06/11/postgresql-ha-with-patroni-your-turn-to-test-failure-scenarios/ https://www.cybertec-postgresql.com/en/disabling-autocommit-in-postgresql-can-damage-your-health/ https://www.cybertec-postgresql.com/en/data-warehousing-making-use-of-synchronized-seq-scans/ https://blog.crunchydata.com/blog/better-range-types-in-postgres-14-turning-100-lines-of-sql-into-3 https://www.youtube.com/watch?v=gDFIc2uaMc8 https://tapoueh.org/blog/2021/06/postgresql-as-a-microservice/ https://patternmatchers.wordpress.com/2021/06/11/ignore-nulls-in-postgres/ https://paquier.xyz/postgresql-2/postgres-14-memory-dumps/ https://blog.crunchydata.com/blog/postgresql-on-linux

In this episode of Scaling Postgres, we discuss managing autovacuum, better JSON in Postgres 14, how to avoid redundant updates and operating system tuning. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/enabling-and-disabling-autovacuum-in-postgresql/ https://blog.crunchydata.com/blog/better-json-in-postgres-with-postgresql-14 https://fluca1978.github.io/2021/06/03/PostgreSQLUpdateTrigger.html https://www.enterprisedb.com/blog/tuning-debian-ubuntu-postgresql https://arctype.com/blog/postgres-performance-monitoring/ https://www.percona.com/blog/2021/06/02/postgres_fdw-enhancement-in-postgresql-14/ https://paquier.xyz/postgresql-2/postgres-14-reindex-tbspace/ https://pgsqlpgpool.blogspot.com/2021/05/dropping-support-for-version-2-protocol.html https://yugonagata-pgsql.blogspot.com/2021/05/implementing-incremental-view.html https://www.highgo.ca/2021/05/31/getting-started-with-pg_bulkload/

In this episode of Scaling Postgres, we discuss how to interpret explain analyze, the best ways to load data in bulk, how to limit rows affected by update or delete and using pg_buffercache. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/how-to-interpret-postgresql-explain-analyze-output/ https://www.cybertec-postgresql.com/en/postgresql-bulk-loading-huge-amounts-of-data/ https://blog.crunchydata.com/blog/simulating-update-or-delete-with-limit-in-postgres-ctes-to-the-rescue https://fluca1978.github.io/2021/05/28/PostgreSQLMemoryFuntions.html https://www.percona.com/blog/2021/05/27/new-features-in-postgresql-14-bulk-inserts-for-foreign-data-wrappers/ https://www.postgresql.fastware.com/blog/enhancement-to-target-session-attrs https://www.ongres.com/blog/tracking-materialized-views-refresh-performance-with-mvstat-in-postgresql/ https://seiler.us/2021-05-23-speeding-up-pgbackrest-recovery/ https:

In this episode of Scaling Postgres, we discuss the release of Postgres 14 Beta 1, different PG 14 improvements, choosing the best type of primary key and using composite keys. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-beta-1-released-2213/ https://pganalyze.com/blog/postgres-14-performance-monitoring https://www.cybertec-postgresql.com/en/uuid-serial-or-identity-columns-for-postgresql-auto-generated-primary-keys/ https://www.endpoint.com/blog/2021/05/20/database-design-using-composite-keys https://www.youtube.com/playlist?list=PLownlFUq_rL6txMapX6UUuquqGdUINmkd https://akorotkov.github.io/blog/2021/05/22/pg-14-query-parsing/ https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-st_interpolateraster https://www.migops.com/blog/2021/05/19/porting-array-length-from-oracle-plsql-to-postgresql-plpgsql/ https://blog.crunchydata.com/blog/pgo-4.7-the-crunchy-postgres-operator-now

In this episode of Scaling Postgres, we discuss the newest Postgres releases, implementing a data warehouse, using exclusion operators and setting up read-only users. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-133-127-1112-1017-and-9622-released-2210/ https://www.narrator.ai/blog/using-postgresql-as-a-data-warehouse/ https://www.youtube.com/watch?v=M58MR1vqnAk https://blog.crunchydata.com/blog/creating-a-read-only-postgres-user https://paquier.xyz/postgresql-2/postgres-14-table-compression/ https://momjian.us/main/blogs/pgblog/2021.html#May_10_2021 https://www.cybertec-postgresql.com/en/setting-up-postgresql-streaming-replication/ https://blog.crunchydata.com/blog/tls-postgres-kubernetes-openssl https://www.enterprisedb.com/blog/regression-analysis-postgresql-tensorflow-part-3-data-analysis https://postgresql.life/post/laurenz_albe/

In this episode of Scaling Postgres, we discuss selecting random rows quickly, faster distinct queries, how replication handles temp and unlogged tables and how to terminate connections. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.hagander.net/getting-random-rows-faster-very-much-faster-249/ https://blog.timescale.com/blog/how-we-made-distinct-queries-up-to-8000x-faster-on-postgresql/ https://fluca1978.github.io/2021/05/10/PostgreSQLUnloggedTablesReplication.html https://www.cybertec-postgresql.com/en/terminating-database-connections-in-postgresql/ https://www.tangramvision.com/blog/creating-postgresql-test-data-with-sql-pl-pgsql-and-python https://momjian.us/main/blogs/pgblog/2021.html#May_3_2021 https://akorotkov.github.io/blog/2021/05/06/jsonpath-double-asterisk-lax/ https://blog.crunchydata.com/blog/using-kubernetes-chances-are-you-need-a-database https://blog.crunchydata.com/blog/the-many-spatial-indexes-of-pos