Secrets Of Data Analytics Leaders

The unbundling of the data ecosystem is causing organizations to “duct tape” products and frameworks together to build their solutions and data delivery processes. Organizations fail to build and deploy end-to-end, automated, repeatable data-driven systems, ignoring data engineering & dataops principles as well as best practices. Published at: https://www.eckerson.com/articles/dataops-in-data-engineering

This blog recommends guiding principles for successful implementation of language models to assist data engineering. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-part-iv-guiding-principles-for-success-with-language-models-and-data-engineering

An emerging approach to generative AI will help data engineering teams achieve much-needed productivity gains while controlling risk. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-part-iii-the-emergence-of-small-language-models-for-data-engineering

Dan O'Brien and Kevin Petrie discuss FinOps, which is a cost governance discipline for cloud-based analytics and operational projects.

An annual assessment of the positioning strategies of the leading 21 BI vendors finds a lack of differentiation that makes it difficult for buyers to compare products. In the BI market’s sea of sameness, Qlik is the only vendor that stands out with this clever, memorable position. Published at: https://www.eckerson.com/articles/independent-study-bi-vendor-messaging-shows-lack-of-differentiation

MDM creates business value in three ways: it streamlines infrastructure, streamlines processes, and reduces risk. Published at: https://www.eckerson.com/articles/driving-roi-with-master-data-management-part-1-build-your-business-case

“Universal” semantic layer tools introduced in recent years promise to standardize business metrics across the data stack, and eliminate silos of metrics trapped in semantic layers that are limited to specific data sources or BI platforms. This post offers considerations for adopting a universal semantic layer. Published at: https://www.eckerson.com/articles/the-universal-semantic-layer-more-than-enough

An Analytics Center of Excellence empowers business teams to meet their own data needs by changing the role of IT from developer to facilitator. The reality, however, is that IT needs be both a facilitator and a developer. Published at: https://www.eckerson.com/articles/analytics-center-of-excellence-part-i-how-to-shape-the-organization

Traditional companies must balance new and old technologies as part of an ever-modernizing data stack. This blog explores how companies strike the right balance to navigate economic uncertainty, AI disruption, and the need for tool consolidation. Published at: https://www.eckerson.com/articles/the-modernizing-data-stack-three-ways-to-balance-new-and-old

LLMs are hugely popular with data engineers because they boost productivity. But companies must adapt their data governance programs to control risks related to data quality, privacy, intellectual property, fai-Datarness, and explainability. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-part-ii-risks-and-governance-approaches-for-data-engineers-to-use-large-language-models

Despite innovations in data architecture, infrastructure, and analytics, most organizations today still struggle to realize the promised value of data. Learn how the data mesh principle of data as a product can help, as part of a data mesh initiative or as a stand-alone strategy. Published at: https://www.eckerson.com/articles/data-products-part-of-a-data-mesh-initiative-or-a-stand-alone-strategy

Data mesh is a new paradigm for fulfilling the promised value of data. It decentralizes both data ownership and the data itself, shifting them toward the functional domains that create and use data to operate. But data mesh is not for everyone. Learn how to assess if you’re ready for data mesh. Published at: https://www.eckerson.com/articles/data-mesh-evaluating-your-organization-s-readiness-for-a-decentralized-data-future

There’s so much hype surrounding data products that you have to wonder if it’s just another buzzword. But there’s more to data products than buzz. In this article, you’ll learn how the concept is a meaningful step forward in the art and science of data management. Published at: https://www.eckerson.com/articles/best-practices-for-developing-and-scaling-data-products

A zone-based data refinery creates an agile, adaptable data environment that supports new and unanticipated business requirements quickly. It turns a monolithic data warehouse into a flexible data environment that gracefully adapts to new and unanticipated business requirements while maximizing reuse and standards. Published at: https://www.eckerson.com/articles/how-zone-based-data-processing-turns-your-monolithic-data-warehouse-into-a-flexible-modern-data-architecture

Many data engineers already use large language models to assist data ingestion, transformation, DataOps, and orchestration. This blog commences a series that explores the emergence of ChatGPT, Bard, and LLM tools from data pipeline vendors, and their implications for the discipline of data engineering. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-examining-the-role-of-chatgpt-and-large-language-models-in-data-engineering

At IAPP Summit, privacy and data governance leaders expressed the importance of a collaborative operating model. Published at: https://www.eckerson.com/articles/the-convergence-of-data-governance-and-privacy-takeaways-from-the-global-privacy-summit

Embeddings are a learned way of representing data in space. Vector databases make it easier to work with embeddings generated from deep learning models. They will become an essential tool in the AI stack because they reduce the time to structure data and train models. Published at: https://www.eckerson.com/articles/the-why-what-who-and-where-of-vector-databases

A robust data workflow testing strategy helps ensure the accuracy and reliability of data processed within a pipeline. Use this checklist to meet your organization’s data quality requirements according to the dimensions of accuracy, completeness, conformity, consistency, integrity, precision, timeliness, and uniqueness. Published at: https://www.eckerson.com/articles/developing-a-robust-data-quality-strategy-for-your-data-pipeline-workflows

An operational data hub (ODH) is a pattern in data architecture that provides a central location and a standard protocol for operational systems to communicate about and share data among themselves. Operational systems post messages about data events (add, change, delete) and subscribe to messages of interest posted by other applications. The hub works to share data among applications without the clutter and chaos of point-to-point data feeds. Published at: https://www.eckerson.com/articles/operational-data-hub-responding-to-data-friction-and-technical-debt

Data mesh is a hot topic in the data world, generating conversations about the benefits and drawbacks of its decentralized approach. Concerns about an explosion of data silos and inconsistent data quality are justified. But to those who feel a bit like Chicken Little, maybe the sky is not falling. Published at: https://www.eckerson.com/articles/data-mesh-the-sky-is-not-falling