Scaling Postgres

In this episode of Scaling Postgres, we review articles covering methods to replicate data, SQL functions, tools for high availability and locking issues. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://severalnines.com/blog/how-replicate-postgresql-data-remote-sites https://www.cybertec-postgresql.com/en/unaccent-getting-rid-of-umlauts-accents-and-special-characters/ https://www.citusdata.com/blog/2019/03/13/fun-with-sql-text-functions/ https://scalegrid.io/blog/managing-high-availability-in-postgresql-part-2/ https://medium.com/innovation-and-technology/part-1-postgis-at-the-city-of-boston-9476293d71c2 https://www.youtube.com/watch?v=OUHsBuXzdwk  

In this episode of Scaling Postgres, we review articles covering index-only scans, locking tables, hot standby feedback and handling very large backups. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://scalegrid.io/blog/2019-database-trends-sql-vs-nosql-top-databases-single-vs-multiple-database-use/ https://medium.com/pgmustard/index-only-scans-in-postgres-a1603f551ee https://www.cybertec-postgresql.com/en/lock-table-can-harm-your-database/ http://richyen.com/replication/postgres/hot_standby_feedback/2019/03/05/i_fought_the_wal.html https://rob.conery.io/2019/03/04/postgresql-tools-for-the-visually-inclined/ https://medium.com/leboncoin-engineering-blog/managing-postgresql-backups-and-replication-for-very-large-databases-part-2-1ec10d95e443 http://highscalability.com/blog/2018/12/5/how-shiftleft-uses-postgresql-extension-timescaledb.html https://www.percona.com/blog/2019/03/06/settling-the-myth-of-transparent-hugepages-for-databases/ https:

In this episode of Scaling Postgres, we review articles covering the Postgres query planner, BRIN indexes, approximate algorithms, and the uses of delayed replication. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://gocardless.com/blog/debugging-the-postgres-query-planner/ https://info.crunchydata.com/blog/postgresql-brin-indexes-big-data-performance-with-minimal-storage https://www.citusdata.com/blog/2019/02/28/approximation-algorithms-for-your-database/ https://about.gitlab.com/2019/02/13/delayed-replication-for-disaster-recovery-with-postgresql/ https://blog.2ndquadrant.com/achieving-high-availability-postgresql/ https://severalnines.com/blog/current-state-open-source-backup-management-postgresql https://info.crunchydata.com/blog/pgbackrest-performing-backups-on-a-standby-cluster https://www.cybertec-postgresql.com/en/exclusive-backup-deprecated-what-now/ https://rafiasabih.blogspot.com/2019/03/using-parallelism-for-queries-from-pl.html

In this episode of Scaling Postgres, we review articles covering a fsync stopgap, tuning autovacuum_naptime, upcoming CTE / WITH clause changes and the Postgres community. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://rubytree.lpages.co/scaling-postgres-configuration/ https://www.postgresql.org/about/news/1920/ http://rhaas.blogspot.com/2019/02/tuning-autovacuumnaptime.html https://info.crunchydata.com/blog/with-queries-present-future-common-table-expressions https://medium.com/@hakibenita/be-careful-with-cte-in-postgresql-fca5e24d2119 https://blog.timescale.com/postgresql-database-community/ http://blog.cleverelephant.ca/2019/02/proj4-postgis.html https://blog.2ndquadrant.com/geo-redundancy-postgresql-database-backups-barman/ https://pgstef.github.io/2019/02/20/monitor_pgbackrest_backups_with_nagios.html  

In this episode of Scaling Postgres, we review articles covering the value of SQL, window functions, scaling database migrations and efficient data storage. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.youtube.com/watch?v=eZhSUXxfEu0 http://www.craigkerstiens.com/2019/02/12/sql-most-valuable-skill/ https://blog.jooq.org/2019/02/14/how-to-calculate-a-cumulative-percentage-in-sql/ https://modern-sql.com/blog/2019-02/postgresql-11 https://benchling.engineering/move-fast-and-migrate-things-how-we-automated-migrations-in-postgres-d60aba0fc3d4 https://azure.microsoft.com/en-us/blog/performance-best-practices-for-using-azure-database-for-postgresql-connection-pooling/ https://info.crunchydata.com/blog/how-your-postgresql-data-model-affects-storage https://fosdem.org/2019/schedule/event/whats_new_in_postgresql_11/ https://fosdem.org/2019/schedule/track/databases/ https://www.youtube.com/watch?v=lgzXuFQ0Pbk https://www.youtube.com/watch?v=NGcV3

In this episode of Scaling Postgres, we review articles covering an outage from running out of TXIDs, breaking scale, that PG fsync issue and pg_stat_statements. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://mailchi.mp/953cb079837a/important-information-about-an-ongoing-mandrill-outage https://mailchi.mp/436dba04bd62/update-on-the-mandrill-outage-service-restored https://twitter.com/mandrillapp/status/1092926493548834817 https://thebuild.com/blog/2019/02/08/do-not-change-autovacuum-age-settings/ https://andreas.scherbaum.la/blog/archives/970-How-long-will-a-64-bit-Transaction-ID-last-in-PostgreSQL.html https://www.postgresql.org/docs/11/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND https://postgresql.verite.pro/blog/2019/02/08/open-instances.html https://thebuild.com/presentations/2019-fosdem-broken.pdf https://www.citusdata.com/blog/2019/02/08/the-most-useful-postgres-extension-pg-stat-statements/ https://pgdash.io/blog/pgbouncer-connectio

In this episode of Scaling Postgres, we review articles covering over one billion records per month, schema changes with no downtime, the life of a SQL query and throughput tracking. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680 https://medium.com/@gajus/lessons-learned-scaling-postgresql-database-to-1-2bn-records-month-edc5449b3067 https://numeracy.co/blog/life-of-a-sql-query https://www.postgresql.org/developer/backend/ https://blog.2ndquadrant.com/pg-phriday-terrific-throughput-tracking/ https://info.crunchydata.com/blog/managing-transaction-id-wraparound-in-postgresql https://info.crunchydata.com/blog/a-guide-to-building-an-active-active-postgresql-cluster https://blog.2ndquadrant.com/banking-on-postgres-webinar/  

In this episode of Scaling Postgres, we review articles covering Microsoft acquiring Citus Data, split WAL files, maintenance_work_mem and logical replication. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blogs.microsoft.com/blog/2019/01/24/microsoft-acquires-citus-data-re-affirming-its-commitment-to-open-source-and-accelerating-azure-postgresql-performance-and-scale/ https://www.citusdata.com/blog/2019/01/24/microsoft-acquires-citus-data/ https://www.youtube.com/watch?v=AeMaBwd90SI http://richyen.com/replication/postgres/2019/01/22/split_wal_files.html http://rhaas.blogspot.com/2019/01/how-much-maintenanceworkmem-do-i-need.html https://hakibenita.com/how-we-solved-a-storage-problem-in-postgre-sql-without-adding-a-single-bytes-of-storage https://pgdash.io/blog/postgres-replication-gotchas.html https://www.cybertec-postgresql.com/en/pg_permission-inspecting-your-postgresql-security-system/ https://info.crunchydata.com/blog/schedule-pgbackr

In this episode of Scaling Postgres, we review articles covering pgBouncer, Postgres 11 gems, DocumentDB similarities and JSON capabilities. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.2ndquadrant.com/pg-phriday-pgbouncer-bust/ https://pgbouncer.github.io/changelog.html https://pgbouncer.github.io/usage.html https://www.cybertec-postgresql.com/en/unearthing-some-hidden-postgresql-11-gems/ https://www.citusdata.com/blog/2019/01/13/citus-data-top-posts-of-2018/ https://www.enterprisedb.com/blog/documentdb-really-postgresql https://severalnines.com/blog/overview-json-capabilities-within-postgresql https://severalnines.com/blog/one-security-system-application-connection-pooling-and-postgresql-case-ldap https://www.citusdata.com/blog/2019/01/15/contributing-to-postgres/ https://blog.2ndquadrant.com/maintaining-feature-branches-submitting-patches-git/  

In this episode of Scaling Postgres, we review articles covering stuck transaction ids, pgBouncer authentication, sequential UUIDs, and monitoring. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: http://richyen.com/replication/postgres/2019/01/08/zombie_transactions.html https://www.cybertec-postgresql.com/en/pgbouncer-authentication-made-easy/ https://blog.2ndquadrant.com/sequential-uuid-generators-ssd/ https://blog.2ndquadrant.com/webinar-pglogical-postgres-bdr-update/ http://datafool.blogspot.com/2019/01/monitoring-postgres-with-prometheus.html https://www.postgresql.eu/events/pgconfeu2018/sessions/session/2166/slides/147/monitoring.pdf https://arxiv.org/pdf/1901.01973.pdf  

In this episode of Scaling Postgres, we review articles covering Postgres as DBMS of 2018, data anonymization, implementing JIT and reserved keywords. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://db-engines.com/en/blog_post/79 http://blog.taadeem.net/english/2019/01/03/8_anonymization_strategies_with_postgres https://severalnines.com/blog/overview-just-time-compilation-jit-postgresql https://www.cybertec-postgresql.com/en/keyword-mess/ https://www.citusdata.com/blog/2019/01/02/fun-with-sql-self-joins/ https://severalnines.com/blog/postgresql-replication-disaster-recovery  

In this episode of Scaling Postgres, we review articles covering the rise of Postgres, row count estimates, nulls and linux configuration. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://severalnines.com/blog/postgresql-rise-2018-postgres-findings-2019-trends https://medium.com/pgmustard/row-count-estimates-in-postgres-8540087a826e https://www.cybertec-postgresql.com/en/a-postgresql-story-about-null-is-null-null-and-not-null/ https://www.percona.com/blog/2018/12/20/benchmark-postgresql-with-linux-hugepages/  

In this episode of Scaling Postgres, we review articles covering logical replication, prepared transactions, Ansible Postgres deployment Ansible and tidy partitions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://severalnines.com/blog/using-postgresql-logical-replication-maintain-always-date-readwrite-test-server https://www.cybertec-postgresql.com/en/prepared-transactions/ https://blog.2ndquadrant.com/video-ansible-postgresql/ https://blog.dataegret.com/2018/12/keeping-postgres-tidy-with-partitioning.html https://pgdash.io/blog/postgres-monitor-wal-files.html https://info.crunchydata.com/blog/get-started-runnning-postgresql-on-kubernetes  

In this episode of Scaling Postgres, we review articles covering multi-terabyte scaling, building a kubernetes DBaaS, encryption and building high availability. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-affiliate-projects-for-horizontal-multi-terabyte-scaling/ https://www.youtube.com/watch?v=q26U2rQcqMw https://blog.2ndquadrant.com/databases-vs-encryption/ https://www.depesz.com/2018/12/05/waiting-for-postgresql-12-add-log_statement_sample_rate-parameter/ https://blog.timescale.com/high-availability-timescaledb-postgresql-patroni-a4572264a831 https://severalnines.com/blog/how-upgrade-postgresql10-postgresql11-zero-downtime https://arcentry.com/blog/postgres-might-just-be-the-most-advanced-database-ever/ https://www.citusdata.com/blog/2018/11/30/why-rdbms-is-the-future-of-distributed-databases/  

In this episode of Scaling Postgres, we review articles covering sequential UUIDs, running totals with window functions, read committed vs repeatable read and TOAST. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.2ndquadrant.com/sequential-uuid-generators/ http://denisgobo.blogspot.com/2018/11/easy-running-totals-with-windowing.html https://www.cybertec-postgresql.com/en/transactions-in-postgresql-read-committed-vs-repeatable-read/ http://www.craigkerstiens.com/2013/11/18/best-postgres-feature-youre-not-using/ https://postgresrocks.enterprisedb.com/t5/Postgres-Gems/Toasting-Up-Large-Objects-BLOBs-CLOBs/ba-p/2233 https://www.citusdata.com/blog/2018/11/27/postgres-more-than-a-relational-database/ https://pgdash.io/blog/postgres-gui-tools.html https://blog.2ndquadrant.com/7113-2/  

In this episode of Scaling Postgres, we review articles covering materialized views in Rails, pgbouncer auth_user, implicit vs explicit joins and JIT. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://medium.com/@rebo_dood/the-benefits-of-materialized-views-and-how-to-use-them-in-your-ruby-on-rails-project-4ac1b5432881 http://richyen.com/pgbouncer/postgres/2018/11/21/pgbouncer_auth_user.html https://www.cybertec-postgresql.com/en/postgressql-implicit-vs-explicit-joins/ https://www.percona.com/blog/2018/11/19/installing-and-configuring-jit-in-postgresql-11/  

In this episode of Scaling Postgres, we review articles covering bloat vs. zheap, composite indexes, unsafe operations, and being a better developer. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://labs.contactually.com/dealing-with-significant-postgres-database-bloat-what-are-your-options-a6c1814a03a5 https://www.youtube.com/watch?v=cvCLFZbtJxQ https://www.pgcon.org/2018/schedule/attachments/501_zheap-a-new-storage-format-postgresql-5.pdf https://blog.timescale.com/use-composite-indexes-to-speed-up-time-series-queries-sql-8ca2df6b3aaa https://leopard.in.ua/2016/09/20/safe-and-unsafe-operations-postgresql https://www.youtube.com/watch?v=IwahVdNboc8 https://www.youtube.com/watch?v=0cZ4elFNAQQ https://www.youtube.com/watch?v=hWh8QoV8z8k https://www.youtube.com/watch?v=gV5W4AhWBzo https://tapoueh.org/blog/2018/11/preventing-sql-injections/ http://evol-monkey.blogspot.com/2018/11/zero-downtime-upgrade-postgresql-10-to.html https://severalnines.co

In this episode of Scaling Postgres, we review articles covering parallel queries, real time analytics, killed index tuples and unexpected downtime. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://m.signalvnoise.com/update-on-basecamp-3-being-stuck-in-read-only-as-of-nov-8-9-22am-cst-c41df1a58352 https://www.cybertec-postgresql.com/en/killed-index-tuples/ https://drive.google.com/file/d/1ibCP2S0uFjW58MouC_nLT_gsHITv7sHZ/view https://www.youtube.com/watch?v=VyQTVLJgJ60 https://www.youtube.com/watch?v=wJOCaNE2_C0 https://www.youtube.com/watch?v=feTihjJJs3g https://www.youtube.com/channel/UCiOSEHQe91goxYNJDjj2rLw/videos https://www.youtube.com/channel/UCer4R0y7DrLsOXo-bI71O6A/videos  

In this episode of Scaling Postgres, we review articles covering Postgres goodies, materialized vs. rollup, column oriented data and parallel scans. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://blog.2ndquadrant.com/webinar-postgresql-database-followup/ https://slides.com/apatheticmagpie/postgres-modern-pgconfeu#/ https://www.citusdata.com/blog/2018/10/31/materialized-views-vs-rollup-tables/ https://www.brianlikespostgres.com/poor-mans-column-oriented-database.html https://rafiasabih.blogspot.com/2018/10/using-parallel-sequential-scan-in.html http://hacksoclock.blogspot.com/2018/10/its-just-expression.html https://www.cybertec-postgresql.com/en/peeking-at-query-performance-of-the-upcoming-version-11-of-postgresql/ http://www.craigkerstiens.com/2018/10/30/postgres-biggest-mistake/ http://blog.taadeem.net/english/2018/10/29/Introducing-PostgreSQL-Anonymizer https://blog.timescale.com/1-0-enterprise-production-ready-time-series-database-open

In this episode of Scaling Postgres, we review articles covering PG11 sharding, giving back, SCRAM authentication and the evolution of partitioning. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.citusdata.com/blog/2018/10/24/why-citus-data-is-donating-1-percent-equity/ https://pgdash.io/blog/postgres-11-sharding.html http://hacksoclock.blogspot.com/2018/10/how-to-set-up-scram-sha-256.html https://blog.2ndquadrant.com/partitioning-evolution-postgresql-11/ https://www.youtube.com/watch?v=3mz71XOKcyU https://www.youtube.com/watch?v=ntVjVNnilX0 https://www.percona.com/blog/2018/10/24/postgresql-locking-part-2-heavyweight-locks/ https://www.pipelinedb.com/blog/pipelinedb-1-0-0-high-performance-time-series-aggregation-for-postgresql