Scaling Postgres

In this episode of Scaling Postgres, we discuss easily writing a recursive CTE, the zheap undo capability, high availability considerations and fast ways to load data. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://dev.to/yugabyte/learn-how-to-write-sql-recursive-cte-in-5-steps-3n88 https://www.cybertec-postgresql.com/en/zheap-undo-logs-discarding-in-postgresql/ https://tapoueh.org/blog/2021/12/postgres-ha-roles-are-dynamic/ https://pganalyze.com/blog/fastest-way-importing-data-into-postgres-with-ruby-rails https://aiven.io/blog/explore-the-new-search-and-cycle-features-in-postgresql-14 http://blog.cleverelephant.ca/2021/12/knn-syntax.html https://www.citusdata.com/blog/2021/12/11/uk-covid-19-dashboard-built-using-postgres-and-citus/ https://www.cybertec-postgresql.com/en/manage-encryption-keys-with-postgresql-tde/ https://www.percona.com/blog/postgresql-high-availability-and-disaster-recovery-on-kubernetes https://postgre

In this episode of Scaling Postgres, we discuss Aurora vs. Postgres, surprising transaction behavior, write-only & read-only database connections and indexing best practices. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.migops.com/blog/2021/11/26/is-aurora-postgresql-really-faster-and-cheaper-than-rds-postgresql-benchmarking/ https://blog.lawrencejones.dev/isolation-levels/ https://www.enterprisedb.com/blog/taking-advantage-write-only-and-read-only-connections-pgbouncer-django https://www.pgmustard.com/blog/indexing-best-practices-postgresql http://aklaver.org/wordpress/2021/12/07/postgres-and-json/ http://rhaas.blogspot.com/2021/12/surviving-without-superuser-part-one.html https://blog.crunchydata.com/blog/extracting-and-substituting-text-with-regular-expressions-in-postgresql https://fluca1978.github.io/2021/12/06/pgterminatebackend.html https://www.highgo.ca/2021/12/10/a-look-inside-postgresqls-extended-query-pro

In this episode of Scaling Postgres, we discuss going slower to go faster, GIN indexes, collation stability and features of PG14 and beyond. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pganalyze.com/blog/gin-index http://rhaas.blogspot.com/2021/11/collation-stability.html https://vyruss.org/blog/slow-things-down-to-make-them-go-faster-postgres-build-2021.html https://www.postgresql.fastware.com/blog/postgresql-14-and-beyond https://blog.timescale.com/blog/postgresql-vs-python-for-data-cleaning-a-guide/ https://www.cybertec-postgresql.com/en/primary-keys-vs-unique-constraints-in-postgresql/ https://www.percona.com/blog/postgresql-14-database-monitoring-and-logging-enhancements/ https://yugonagata-pgsql.blogspot.com/2021/11/transition-tables-in-incremental-view.html https://blog.crunchydata.com/blog/postgis-day-2021 https://blog.crunchydata.com/blog/tricks-for-faster-spatial-indexes https://blog.rustprooflabs.com/2021/

In this episode of Scaling Postgres, we discuss using entity-attribute-value designs, new JSON subscripting capabilities, the mysql-fdw support for aggregate pushdowns and new regex functions in PG 15. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/entity-attribute-value-eav-design-in-postgresql-dont-do-it/ https://blog.logrocket.com/whats-new-json-postgresql-v14/ https://www.enterprisedb.com/blog/aggregate-push-down-mysqlfdw https://www.depesz.com/2021/11/26/waiting-for-postgresql-15-add-assorted-new-regexp_xxx-sql-functions/ https://blog.crunchydata.com/blog/using-timescaledb-extension-with-the-pgo-the-postgres-operator https://www.highgo.ca/2021/11/27/how-to-run-a-specific-regression-test/ https://postgresql.life/post/pavel_luzanov/

In this episode of Scaling Postgres, we discuss if foreign keys should have indexes, how to run graph queries, how to configure Linux huge pages and the benefits as well as text size similarities. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.percona.com/blog/should-i-create-an-index-on-foreign-keys-in-postgresql/ https://www.youtube.com/watch?v=_RXfnnqsLlw https://www.enterprisedb.com/blog/improving-postgresql-performance-without-making-changes-postgresql https://www.depesz.com/2021/11/19/does-varcharn-use-less-disk-space-than-varchar-or-text/ https://pganalyze.com/blog/postgresql-views-django-python https://www.enterprisedb.com/blog/pg-phriday-isolating-postgres-repmgr https://www.cybertec-postgresql.com/en/postgresql-on-wsl2-for-windows-install-and-setup/ https://www.depesz.com/2021/11/16/waiting-for-postgresql-15-allow-publishing-the-tables-of-schema/ https://www.migops.com/blog/2021/11/15/postgresql-15-will-include-

In this episode of Scaling Postgres, we discuss new releases of Postgres due to a man-in-the-middle vulnerability, the high availability solution pg_auto_failover, lesser known Postgres features and LZ4 compression. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-141-135-129-1114-1019-and-9624-released-2349/ https://www.enterprisedb.com/blog/postgres-mitm21-vulnerabilities https://www.youtube.com/watch?v=fuYwlVw0e_o https://www.pgbouncer.org/2021/11/pgbouncer-1-16-1 https://tapoueh.org/blog/2021/11/an-introduction-to-the-pg_auto_failover-project/ https://hakibenita.com/postgresql-unknown-features https://www.postgresql.fastware.com/blog/what-is-the-new-lz4-toast-compression-in-postgresql-14 https://blog.jooq.org/postgresql-14s-enable_memoize-for-improved-performance-of-nested-loop-joins/ https://blog.timescale.com/blog/generating-more-realistic-sample-time-series-data-with-postgresql-ge

In this episode of Scaling Postgres, we discuss the open sourcing of Babelfish, deconstructing the Postgres planner, when to avoid exist & not exist and having fun with SQL. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://babelfishpg.org/blog/releases/2021/10/babelfish-launch/ https://aws.amazon.com/blogs/aws/goodbye-microsoft-sql-server-hello-babelfish/ https://pganalyze.com/blog/deconstructing-the-postgres-planner https://postgres.ai/blog/20211103-three-cases-against-if-not-exists-and-if-exists-in-postgresql-ddl https://blog.crunchydata.com/blog/fun-with-sql-in-postgres-finding-revenue-accrued-per-day https://notes.eatonphil.com/exploring-plpgsql-forth-like.html https://www.highgo.ca/2021/11/01/the-postgresql-timeline-concept/ https://blog.crunchydata.com/blog/patroni-etcd-in-high-availability-environments https://blog.crunchydata.com/blog/resize-postgres-kubernetes-volume-instance-sets https://aws.amazon.com/blogs/database/ve

In this episode of Scaling Postgres, we discuss the downsides of indexes, TCP keep alive options, developing with Postgres as your DB and learning PL/pgSQL. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20211029-how-partial-and-covering-indexes-affect-update-performance-in-postgresql https://www.cybertec-postgresql.com/en/tcp-keepalive-for-a-better-postgresql-experience/ https://www.youtube.com/watch?v=mliO5_XhAKE https://notes.eatonphil.com/exploring-plpgsql.html https://blog.crunchydata.com/blog/secure-postgresql-14-with-the-cis-benchmark https://blog.crunchydata.com/blog/database-security-best-practices-on-kubernetes https://blog.crunchydata.com/blog/encrypting-postgres-data-at-rest-in-kubernetes https://postgresql.life/post/alexander_kukushkin/ https://www.rubberduckdevshow.com/episodes/18-how-many-3rd-party-libraries-should-you-use/

In this episode of Scaling Postgres, we discuss automatic indexing, function pipelines, with hold cursors and the different query scans. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2021/10/22/why-is-it-hard-to-automatically-suggest-what-index-to-create/ https://blog.timescale.com/blog/function-pipelines-building-functional-programming-into-postgresql-using-custom-operators/ https://www.cybertec-postgresql.com/en/with-hold-cursors-and-transactions-in-postgresql/ https://arctype.com/blog/postgres-explain-analyze-scan/ https://postgres.ai/blog/20211018-postgresql-lock-trees https://blog.crunchydata.com/blog/postgres-queries-for-projecting-monthly-revenue-run-rate https://about.sourcegraph.com/blog/postgres-text-search-balancing-query-time-and-relevancy/ https://elephanttamer.net/?p=68 https://aws.amazon.com/blogs/database/upgrade-amazon-rds-and-amazon-aurora-postgresql-version-9-6/ https://postgresql.life/pos

In this episode of Scaling Postgres, we discuss one method to shard a database, federating your database, the benefits of configuring linux HugePages and performance tuning. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.notion.so/blog/sharding-postgres-at-notion https://blog.crunchydata.com/blog/multi-cluster-postgres-federation-on-kubernetes-with-postgres-14-using-gitops https://www.percona.com/blog/why-linux-hugepages-are-super-important-for-database-servers-a-case-with-postgresql/ https://www.enterprisedb.com/blog/webinar-recap-tuning-tips-maximize-postgres-performance https://aws.amazon.com/blogs/database/tune-sorting-operations-in-postgresql-with-work_mem/ https://www.cybertec-postgresql.com/en/show-tables-in-postgresql-whats-wrong-with-it/ https://blog.rustprooflabs.com/2021/10/postgres-14-query-id https://www.highgo.ca/2021/10/15/backup-label-in-postgresql/ https://www.postgresonline.com/journal/index.php?/archives/401-Multirange

In this episode of Scaling Postgres, we discuss select for update, the pgx framework to generate extensions, cool additions to Postgres 14 and full-text search. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.migops.com/blog/2021/10/05/select-for-update-and-its-behavior-with-foreign-keys-in-postgresql/ https://blog.rustprooflabs.com/2021/10/pgdd-extension-using-pgx-rust https://www.enterprisedb.com/blog/cool-new-contributions-postgresql-14 https://arctype.com/blog/postgres-full-text-search/ https://www.enterprisedb.com/postgresql-database-backup-recovery-what-works-wal-pitr https://yugonagata-pgsql.blogspot.com/2021/10/transition-tables-in-imcremental-view.html https://www.enterprisedb.com/blog/how-set-kerberos-authentication-using-active-directory-postgresql-database https://www.cybertec-postgresql.com/en/openstreetmap-service-by-cybertec/ https://blog.crunchydata.com/blog/postgresql-14-on-kubernetes https://postgresql

In this episode of Scaling Postgres, we discuss the release of Postgres 14, how best to use JSON, why are your indexes not being used and sequence gaps. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-released-2318/ https://www.citusdata.com/blog/2021/10/01/how-we-shipped-postgres-14-on-azure-within-a-day/ https://www.instaclustr.com/postgresql-14/ https://www.migops.com/blog/2021/09/29/upgrading-postgresql-9-6-to-postgresql-13/ https://www.cybertec-postgresql.com/en/json-postgresql-how-to-use-it-right/ https://www.depesz.com/2021/09/27/using-recursive-queries-to-get-distinct-elements-from-table/ https://www.pgmustard.com/blog/why-isnt-postgres-using-my-index https://postgres.ai/blog/20210923-zero-downtime-postgres-schema-migrations-lock-timeout-and-retries https://www.cybertec-postgresql.com/en/gaps-in-sequences-postgresql/ https://mydbanotebook.org/post/auto-increment/ https:/

In this episode of Scaling Postgres, we discuss Postgres 14 RC1, little improvements in PG14, an index advisor and how to handle 1 million insert statements. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-14-rc-1-released-2309/ https://blog.crunchydata.com/blog/postgres-14-its-the-little-things https://pganalyze.com/blog/introducing-pganalyze-index-advisor https://dev.to/yugabyte/when-you-have-millions-of-insert-statements-in-a-file-3nck https://fluca1978.github.io/2021/09/23/OracleAlterSequence.html https://www.cybertec-postgresql.com/en/building-an-oracle-to-postgresql-migrator-lessons-learned/ https://b-peng.blogspot.com/2021/09/reserved-connections.html https://b-peng.blogspot.com/2021/09/enable-shared-relcache.html https://postgresql.life/post/michael_goldberg/ https://www.rubberduckdevshow.com/episodes/14-background-job-processing/

In this episode of Scaling Postgres, we discuss one cause of broken indexes, Postgres trademark issues, percentiles vs. averages and logical replication improvements. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://elephanttamer.net/?p=61 https://www.postgresql.org/about/news/trademark-actions-against-the-postgresql-community-2302/ https://postgresql.fund/blog/postgres-core-team-attacks-postgres-community/ https://postgresql.fund/blog/respecting-majority-questioning-status-quo-as-a-minority/ https://blog.timescale.com/blog/how-percentile-approximation-works-and-why-its-more-useful-than-averages/ http://amitkapila16.blogspot.com/2021/09/logical-replication-improvements-in.html https://www.cybertec-postgresql.com/en/postgresql-create-indexes-after-bulk-loading/ https://www.highgo.ca/2021/09/17/a-quick-test-for-postgres_fdw-batch-insertion/ https://www.cybertec-postgresql.com/en/postgresql-schedulers-comparison-table/ https:/

In this episode of Scaling Postgres, we discuss having boundless text fields, revoking public schema permissions, less index bloat in PG14 and comparing hardware performance. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://brandur.org/text https://www.depesz.com/2021/09/10/waiting-for-postgresql-15-revoke-public-create-from-public-schema-now-owned-by-pg_database_owner/ https://www.cybertec-postgresql.com/en/index-bloat-reduced-in-postgresql-v14/ https://dev.to/aws-heroes/aurora-postgresql-db-r6g-compared-to-db-r5-with-ybio-1bnd https://brandur.org/sqlc https://blog.rustprooflabs.com/2021/09/psycopg3-initial-review https://www.varrazzo.com/blog/2020/03/06/thinking-psycopg3/ https://fluca1978.github.io/2021/09/10/pgbackrest-jq.html https://postgresql.life/post/akshay_joshi/ https://www.rubberduckdevshow.com/episodes/12-programming-with-dates-times-zones

In this episode of Scaling Postgres, we discuss connection scaling, how to cache your sequences, a message queuing implementation and the harms of subtransactions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://richyen.com/postgres/2021/09/03/less-is-more-max-connections.html https://dev.to/yugabyte/uuid-or-cached-sequences-42fi https://blog.crunchydata.com/blog/message-queuing-using-native-postgresql https://postgres.ai/blog/20210831-postgresql-subtransactions-considered-harmful https://blog.timescale.com/blog/how-to-create-lots-of-sample-time-series-data-with-postgresql-generate_series/ https://blog.anayrat.info/en/2021/09/01/partitioning-use-cases-with-postgresql/ https://www.postgresql.fastware.com/blog/logical-replication-tablesync-workers https://dev.to/ftisiot/solving-the-knapsack-problem-in-postgresql-40cc https://www.cybertec-postgresql.com/en/postgresql-the-power-of-a-single-missing-index/ http://postgis.net/202

In this episode of Scaling Postgres, we discuss how to use aggregate filters, the importance of vacuum with insert-only tables, how to fix out-of-sync sequences and faster functional index options. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.crunchydata.com/blog/fast-flexible-summaries-with-aggregate-filters-and-windows https://blog.crunchydata.com/blog/insert-only-tables-and-autovacuum-issues-prior-to-postgresql-13 https://www.cybertec-postgresql.com/en/fixing-out-of-sync-sequences-in-postgresql/ https://dev.to/aws-heroes/index-only-scan-on-functional-indexes-9o2 https://www.percona.com/blog/postgresql-database-security-os-authentication/ https://www.depesz.com/2021/08/26/how-to-get-advisory-lock-in-shell/ https://www.migops.com/blog/2021/08/27/announcing-pg_dbms_job-in-postgresql-for-oracle-dbms_job-compatibility/ https://www.highgo.ca/2021/08/23/tuning-the-pgpool-ii-load-balancer/ https://arctype.com/blog/postgre

In this episode of Scaling Postgres, we discuss how to use foreign data wrappers, compression available in timescaledb, working with fuzzy search and describing Postgres query execution. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.crunchydata.com/blog/understanding-postgres_fdw https://blog.rustprooflabs.com/2021/08/timescale-compression-openstreetmap-tags https://arctype.com/blog/postgres-fuzzy-search/ https://dev.to/yugabyte/postgres-query-execution-simple-query-2i71 https://www.highgo.ca/2021/08/20/how-batch-insertion-was-done-in-postgres_fdw/ https://dev.to/yugabyte/pgbench-client-jobs-68g https://blog.crunchydata.com/blog/waiting-for-postgis-3.2-secure-cloud-raster-access https://www.migops.com/blog/2021/08/20/validate-postgresql-extension-upgrade-scripts-using-pg_validate_extupgrade/ https://www.cybertec-postgresql.com/en/pg_timetable_v4-is-out/ https://postgresql.life/post/andrey_borodin/ https://www.rubber

In this episode of Scaling Postgres, we discuss new releases for Postgres & pgbouncer, have a performance discussion, describe the different index types and how to use them. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-134-128-1113-1018-9623-and-14-beta-3-released-2277/ https://www.pgbouncer.org/2021/08/pgbouncer-1-16-0 https://www.rubberduckdevshow.com/episodes/8-database-performance/ https://pganalyze.com/blog/postgres-create-index https://www.youtube.com/watch?v=gBGUuMrO6MI https://www.cybertec-postgresql.com/en/postgresql-vs-redis-vs-memcached-performance/ https://www.percona.com/blog/impact-of-network-and-cursor-on-query-performance-of-postgresql/ https://www.ongres.com/blog/noset-postgresql-extension-for-blocking-set-reset-execution/ https://www.percona.com/blog/migrating-postgresql-to-kubernetes https://www.highgo.ca/2021/08/09/horizontal-scalability-options-in-postgresq

In this episode of Scaling Postgres, we discuss ways to work around the planner, working with grouping sets, using a lateral join and not-so-easy Postgres issues. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://heap.io/blog/when-the-postgres-planner-is-not-very-smart https://www.cybertec-postgresql.com/en/postgresql-grouping-sets-rollup-cube/ https://fluca1978.github.io/2021/08/07/PostgreSQLLateralJoin.html https://pgdash.io/blog/postgresql-six-not-so-easy-pieces.html https://www.postgresql.fastware.com/blog/logical-decoding-of-two-phase-commits https://blog.crunchydata.com/blog/using-cert-manager-to-deploy-tls-for-postgres-on-kubernetes https://postgresql.life/post/bryn_llewellyn/ https://www.rubberduckdevshow.com/episodes/7-writing-secure_code/