Scaling Postgres

In this episode of Scaling Postgres, we discuss how to scale out, how the planner estimates, uses of create statistics and investigating PostgreSQL run time environments. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://dev.to/heroku/preparing-your-postgres-data-for-scale-out-km https://rjuju.github.io/postgresql/2020/02/28/pg_qualstats-2-selectivity-error.html https://powa.readthedocs.io/en/latest/components/stats_extensions/pg_qualstats.html https://www.2ndquadrant.com/en/blog/webinar-create-statistics-command-postgresql/ https://www.youtube.com/watch?v=rTIvco34REM https://www.2ndquadrant.com/en/blog/postgis-perf-issue-2019/ https://www.thatguyfromdelhi.com/2020/02/optimizations-in-group-by-vs-select.html https://www.2ndquadrant.com/en/blog/creating-a-postgresql-procedural-language-part-3-executing-user-code/ https://postgresql.life/ https://www.highgo.ca/2020/02/28/parallel-vacuum-in-upcoming-postgresql-13/ https://habr.com/en/company/post

In this episode of Scaling Postgres, we discuss the use cases for UUIDs, using pg_settings, setting up an automated restore and the future of parallelism. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.depesz.com/2020/02/19/why-im-not-fan-of-uuid-datatype/ https://medium.com/yugabyte/natural-versus-surrogate-primary-keys-in-a-distributed-sql-database-the-distributed-sql-blog-886a6d5f8fa2 https://medium.com/@shawnoy/generate-a-uuid-as-postgresql-default-value-fc0ae41a37d2 https://fluca1978.github.io/2020/02/13/pgsettings.html https://pgdash.io/blog/testing-postgres-backups.html http://amitkapila16.blogspot.com/2020/02/this-blog-post-is-about-journey-of.html https://www.cybertec-postgresql.com/en/shared_buffers-looking-into-the-postgresql-i-o-cache/ https://www.depesz.com/2020/02/18/which-tables-should-be-auto-vacuumed-or-auto-analyzed-update/ https://www.depesz.com/2020/02/17/waiting-for-postgresql-13-add-x-to-default-prompt1-and-prompt2

In this episode of Scaling Postgres, we discuss new postgres releases, useless vacuuming, isolation differences between databases, and different ways to compress WAL files. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/2011/ https://www.postgresql.org/developer/roadmap/ http://rhaas.blogspot.com/2020/02/useless-vacuuming.html https://postgresql.verite.pro/blog/2020/02/14/isolation-repeatable-read-postgresql-mysql.html https://www.percona.com/blog/2020/02/13/compression-of-postgresql-wal-archives-becoming-more-important/ https://www.pgmustard.com/blog/work-mem https://medium.com/@apoor/quickly-load-csvs-into-postgresql-using-python-and-pandas-9101c274a92f https://fluca1978.github.io/2020/02/09/PostgreSQLDROPCOlumn.html https://www.cybertec-postgresql.com/en/migrating-from-ms-sql-to-postgresql-uppercase-vs-lowercase/ https://www.2ndquadrant.com/en/blog/how-to-automate-postgresql-12-replication-and-failover-with-re

In this episode of Scaling Postgres, we discuss full text search, a process for query optimization, caution with exception blocks and adding a procedural language. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.youtube.com/watch?v=c8IrUHV70KQ https://fosdem.org/2020/schedule/event/postgresql_the_state_of_full_text_search_in_postgresql_12/attachments/slides/4138/export/events/attachments/postgresql_the_state_of_full_text_search_in_postgresql_12/slides/4138/FTS.pdf https://medium.com/scopedev/introduction-to-profiling-and-optimizing-sql-queries-for-software-engineers-3cf376ecc712 https://pgdba.org/post/2020/02/exception_block/ https://fluca1978.github.io/2020/02/05/PLPGSQLExceptions.html https://www.2ndquadrant.com/en/blog/creating-a-postgresql-procedural-language-part-1-setup/ https://severalnines.com/database-blog/overview-job-scheduling-tools-postgresql https://www.enterprisedb.com/blog/can-pgbouncer-session-survive-everything-we-throw-i

In this episode of Scaling Postgres, we discuss the "Ballad of Bloat", options for auditing, a configuration help resource and determining when a vacuum is needed. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.youtube.com/watch?v=2NxIngqq1y0 https://www.cybertec-postgresql.com/en/row-change-auditing-options-for-postgresql/ https://www.ongres.com/blog/postgresqlconf-configuration-for-humans/ https://postgresqlco.nf/en/doc/param/ https://richyen.com/postgres/2020/01/29/underestimating_log.html https://www.depesz.com/2020/01/29/which-tables-should-be-auto-vacuumed-or-auto-analyzed/ https://www.depesz.com/2020/01/28/dont-do-these-things-in-postgresql/ https://pgdash.io/blog/pgctl-tips-tricks.html https://www.credativ.com/blog/debian-integration-patroni-and-vip-manager https://info.crunchydata.com/blog/schedule-postgresql-backups-and-retention-with-kubernetes https://info.crunchydata.com/blog/migrating-from-oracle-to-postgresql-questions-an

In this episode of Scaling Postgres, we discuss storing non-relational data, recognizing contributors, Postgres security features and implementing high availability. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.enterprisedb.com/blog/8-ways-rdbms-postgres-handles-non-relational-data https://momjian.us/main/writings/pgsql/non-relational.pdf http://rhaas.blogspot.com/2020/01/contributors-team-redux.html https://www.highgo.ca/2020/01/22/understanding-security-features-in-postgresql-part-1/ https://www.highgo.ca/2020/01/22/understanding-security-features-in-postgresql-part-2/ https://www.highgo.ca/2020/01/22/understanding-security-features-in-postgresql-part-3/ https://www.2ndquadrant.com/en/blog/how-to-automate-postgresql-12-replication-and-failover-with-repmgr-part-1/ https://info.crunchydata.com/blog/deploy-high-availability-postgresql-on-kubernetes https://www.2ndquadrant.com/en/blog/jsonb_set_lax/ https://www.depesz.com/2020/01/21/wai

In this episode of Scaling Postgres, we discuss tips & tricks, how to get faster APIs, full text search considerations and pgbouncer_fdw. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/tips-and-tricks-to-kick-start-postgres-year-2020/ https://goiabada.blog/faster-ruby-apis-with-postgres-238c2f4a272c https://jetrockets.pro/blog/two-edge-cases-in-postgresql-full-text-search https://info.crunchydata.com/blog/making-pgbouncer-easier-to-monitor https://pganalyze.com/blog/materialized-views-ruby-rails https://www.youtube.com/watch?v=Vr6OiuIL8W8 https://www.2ndquadrant.com/en/blog/webinar-bdr-latest-features-updates-follow-up/ https://www.depesz.com/2020/01/14/waiting-for-postgresql-13-alter-table-alter-column-drop-expression/  

In this episode of Scaling Postgres, we discuss user account configuration, pricing, the system identifier and the HyperLogLog extension. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://tightlycoupled.io/my-goto-postgres-configuration-for-web-services/ https://medium.com/@rbranson/rds-pricing-has-more-than-doubled-ef8c3b7e5218 https://www.2ndquadrant.com/en/blog/support-for-postgresqls-system-identifier-in-barman/ https://eng.getwisdom.io/postgres-hyperloglog/ https://www.highgo.ca/2020/01/10/how-to-create-test-and-debug-an-extension-written-in-c-for-postgresql/ https://www.highgo.ca/2020/01/10/how-to-build-and-debug-postgresql-12-using-latest-eclipse-ide-on-ubuntu-18-04/ https://rjuju.github.io/postgresql/2020/01/06/pg_qualstats-2-global-index-advisor.html https://www.depesz.com/2020/01/07/finally-fixed-time-calculations-for-parallel-plans-in-explain-depesz-com/ https://www.cybertec-postgresql.com/en/psql_editor-fighting-with-sublime-text-

In this episode of Scaling Postgres, we discuss max_locks_per_transaction, logical replication, managing backup & recovery and pgbouncer connection pooling. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-you-might-need-to-increase-max_locks_per_transaction/ https://www.cybertec-postgresql.com/en/recovery-conf-is-gone-in-postgresql-v12/ https://www.youtube.com/channel/UCiOSEHQe91goxYNJDjj2rLw/videos https://www.youtube.com/watch?v=nGjLUk2RQ5U https://www.2ndquadrant.com/en/blog/webinar-you-forgot-to-put-the-where-in-delete-follow-up/ https://medium.com/@kristi.anderson/postgresql-connection-pooling-part-2-pgbouncer-2209f26fc008 https://www.enterprisedb.com/blog/can-pgbouncer-handle-failover-new-machine https://medium.com/netflix-techblog/dblog-a-generic-change-data-capture-framework-69351fb9099b https://medium.com/yugabyte/an-introduction-to-postgresql-table-functions-in-yugabytedb-the-distribute

In this episode of Scaling Postgres, we discuss the State of Postgres, how best to dump your database, setting configuration parameters and managing partial WAL files. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://stateofpostgres.com/ https://xzilla.net//blog/2019/Dec/5-Quick-Thoughts-On-The-State-Of-Postgres-Survey.html https://www.depesz.com/2019/12/10/how-to-effectively-dump-postgresql-databases/ https://www.cybertec-postgresql.com/en/setting-postgresql-configuration-parameters/ https://www.2ndquadrant.com/en/blog/barman-2-10-recovery-of-partial-wal-files/ https://info.crunchydata.com/blog/how-to-perform-a-major-version-upgrade-using-pg_upgrade-in-postgresql https://www.optimadata.nl/blogs/3/nlm8ci-how-to-run-postgres-on-docker-part-3 https://rjuju.github.io/postgresql/2019/12/10/powa-4-new-powa-collector.html https://github.com/yandex/odyssey  

In this episode of Scaling Postgres, we discuss logical replication aspects, how to monitor queries, investigating a SQL beautifier and removing incomplete startup packet. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/1998/ https://rafiasabih.blogspot.com/2019/12/interesting-aspects-of-logical.html https://pgdash.io/blog/understanding-postgres-query-performance.html https://www.cybertec-postgresql.com/en/abusing-postgresql-as-an-sql-beautifier/ https://www.percona.com/blog/2019/12/03/postgresql-12-improvement-benign-log-entries-incomplete-startup-packet/ https://fluca1978.github.io/2019/12/05/Explain_settings.html    

In this episode of Scaling Postgres, we discuss book sales, boosts to b-tree indexes, more Postgres 12 features and how to setup and use pgBackRest. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://sql-performance-explained.com/ https://theartofpostgresql.com/blog/2019-11-black-friday/ https://www.cybertec-postgresql.com/en/b-tree-index-improvements-in-postgresql-v12/ https://www.cybertec-postgresql.com/en/discovering-less-known-postgresql-12-features/ https://www.2ndquadrant.com/en/blog/setting-ssl-tls-protocol-versions-with-postgresql-12/ https://info.crunchydata.com/blog/hope-is-not-a-strategy-the-hpe-ssd-issues-and-protecting-your-postgresql-data https://info.crunchydata.com/blog/how-to-get-started-with-pgbackrest-and-postgresql-12 https://www.youtube.com/channel/UC6gJN4hEmnPHZ53HP2r7zhA/videos  

In this episode of Scaling Postgres, we discuss global indexes, ways to cache aggregates, how vacuum processing works and the purpose of effective cache size. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.percona.com/blog/2019/11/20/proposal-for-global-indexes-in-postgresql/ https://evilmartians.com/chronicles/pulling-the-trigger-how-to-update-counter-caches-in-you-rails-app-without-active-record-callbacks https://severalnines.com/database-blog/overview-vacuum-processing-postgresql https://www.cybertec-postgresql.com/en/effective_cache_size-what-it-means-in-postgresql/ https://www.youtube.com/channel/UCLjZee4-RXhkOxz-p-h5hBg/videos https://rafiasabih.blogspot.com/2019/11/which-size-fits-me.html https://hakibenita.com/sql-dos-and-donts https://pganalyze.com/blog/similarity-in-postgres-and-ruby-on-rails-using-trigrams https://info.crunchydata.com/blog/enhancing-your-postgresql-12-security-with-the-cis-benchmark https://severalnines.com/d

In this episode of Scaling Postgres, we discuss chaos order, pub sub, petabyte scale analytics and PgBouncer connections. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/1994/ https://www.simplethread.com/chaos-order-randomizing-queries-uncover-order-dependency/ https://layerci.com/blog/postgres-is-the-answer/ https://techcommunity.microsoft.com/t5/Azure-Database-for-PostgreSQL/Architecting-petabyte-scale-analytics-by-scaling-out-Postgres-on/ba-p/969685 https://www.youtube.com/watch?v=lwG-ZIUy35E https://www.enterprisedb.com/blog/pgbouncer-connection-pooling-what-do-when-persistent-connectivity-lost https://www.2ndquadrant.com/en/blog/webinar-using-ssl-with-postgresql-pgbouncer/ https://www.depesz.com/2019/11/16/waiting-for-postgresql-13-introduce-the-force-option-for-the-drop-database-command/ https://www.depesz.com/2019/11/16/waiting-for-postgresql-13-allow-sampling-of-statements-depending-on-duration/ https:/

In this episode of Scaling Postgres, we discuss Postgres 12 performance impressions, mystery solving, GiST and SP-GiST performance and k-anonymity. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-v12-initial-query-performance-impressions/ http://richyen.com/postgres/2019/11/06/auto_explain.html https://www.2ndquadrant.com/en/blog/postgresql-12-implementing-k-nearest-neighbor-space-partitioned-generalized-search-tree-indexes/ https://blog.taadeem.net///english/2019/11/07/achieving-k-anonymity-with-postgresql http://blog.cleverelephant.ca/2019/11/pgsql-gzip.html https://theartofpostgresql.com/explain-plan-visualizer/ https://www.highgo.ca/2019/11/07/streaming-replication-setup-in-pg12-how-to-do-it-right/ https://www.highgo.ca/2019/11/07/replication-failover-with-pg_rewind-in-pg12/ https://www.percona.com/blog/2019/11/08/configure-haproxy-with-postgresql-using-built-in-pgsql-check/ https://www.cybertec-

In this episode of Scaling Postgres, we discuss partitioning, logical replication upgrades, columnar compression and HAProxy connections. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.2ndquadrant.com/en/blog/webinar-postgresql-partitioning-follow-up/ https://www.cybertec-postgresql.com/en/upgrading-postgres-major-versions-using-logical-replication/ https://blog.timescale.com/blog/building-columnar-compression-in-a-row-oriented-database/ https://www.percona.com/blog/2019/10/31/postgresql-application-connection-failover-using-haproxy-with-xinetd/ https://rob.conery.io/2019/10/24/virtual-computed-columns-in-postgresql-12/ https://rob.conery.io/2019/10/29/fine-tuning-full-text-search-with-postgresql-12/ https://www.percona.com/blog/2019/10/29/monitoring-postgresql-databases-using-percona-monitoring-management/ https://mbeena-learnings.blogspot.com/2019/10/benchmark-partition-table-1.html https://blog.panoply.io/postgres-case-statement-ba

In this episode of Scaling Postgres, we discuss the benefits of pg_receivewal, how to setup simple application failover, new pg_checksums and setting up pgBouncer. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/never-lose-a-postgresql-transaction-with-pg_receivewal/ https://www.percona.com/blog/2019/10/23/seamless-application-failover-using-libpq-features-in-postgresql/ https://techcommunity.microsoft.com/t5/Azure-Database-for-PostgreSQL/Auto-Failover-with-PostgreSQL-12/ba-p/917637 https://www.credativ.com/blog/pgchecksums-10-released https://www.enterprisedb.com/blog/pgbouncer-tutorial-installing-configuring-and-testing-persistent-postgresql-connection-pooling https://info.crunchydata.com/blog/monitoring-postgresql-clusters-in-kubernetes https://blog.panoply.io/a-beginners-guide-to-formatting-dates-in-sql http://postgis.net/2019/10/20/postgis-3.0.0/ https://www.alibabacloud.com/blog/using-postgresql-to-create

In this episode of Scaling Postgres, we discuss prewarming your cache, working with nondeterministic collations, generated column performance and foreign keys with partitions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/prewarming-postgresql-i-o-caches/ https://postgresql.verite.pro/blog/2019/10/14/nondeterministic-collations.html https://pgdba.org/post/2019/10/generated_columns/ https://www.2ndquadrant.com/en/blog/postgresql-12-foreign-keys-and-partitioned-tables/ https://www.2ndquadrant.com/en/blog/postgres-bdr-it-is-also-about-fast-safe-upgrades/ https://www.2ndquadrant.com/en/blog/managing-another-postgresql-commitfest/ http://rhaas.blogspot.com/2019/10/braces-are-too-expensive.html http://evol-monkey.blogspot.com/2019/10/tuning-checkpoints.html https://www.pgbouncer.org/2019/10/pgbouncer-1-12-0 https://medium.com/@kristi.anderson/postgresql-connection-pooling-part-1-pros-cons-43bd4b502279 http://postgis

In this episode of Scaling Postgres, we discuss changes to recovery configuration, the alter system command, transaction isolation and vacuum for temp tables. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.2ndquadrant.com/en/blog/replication-configuration-changes-in-postgresql-12/ https://www.percona.com/blog/2019/10/11/how-to-set-up-streaming-replication-in-postgresql-12/ https://postgresrocks.enterprisedb.com/t5/Postgres-Gems/Where-is-my-recovery-conf-file-in-PostgreSQL-v12/ba-p/4360 https://www.postgresql.org/docs/12/sql-altersystem.html https://www.postgresql.org/docs/12/app-pgbasebackup.html https://pgdba.org/post/2019/10/transactions/ https://www.cybertec-postgresql.com/en/what-is-autovacuum-doing-to-my-temporary-tables/ https://www.highgo.ca/2019/10/08/newbie-to-postgresql-where-to-start/  

In this episode of Scaling Postgres, we discuss the release features of Postgres 12, generated columns, parallel estimates and change data capture. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/1976/ https://www.enterprisedb.com/blog/8-major-improvements-postgresql-12 https://pgdash.io/blog/postgres-12-generated-columns.html https://blog.dataegret.com/2019/10/freshly-baked-postgresql-12-and-changes.html https://www.cybertec-postgresql.com/en/how-postgresql-estimates-parallel-queries/ https://www.cybertec-postgresql.com/en/fixing-track_activity_query_size-in-postgresql-conf/ https://info.crunchydata.com/blog/postgresql-change-data-capture-with-debezium https://medium.com/avitotech/how-to-work-with-postgres-in-go-bad2dabd13e4 https://www.highgo.ca/2019/09/30/the-transparent-data-encryption-in-postgresql/ https://www.highgo.ca/2019/10/01/a-guide-to-create-user-defined-extension-modules-to-postgres/ https://www.