DataKitchen IO (@datakitchen_io) header
DataKitchen IO (@datakitchen_io) profile picture

πŸ‘¨β€πŸ³ Our #DataOps Platform simplifies complex data toolchains, environments & teams πŸ‘©β€πŸ³ Get Your πŸ†“ DataOps Certification ➑️ https://teapawt.net/t.co/Vdd3AAgTxI

Cambridge, MA
Joined January 2015
Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)Photo by DataKitchen IO (@datakitchen_io)
Working at 70% today beats perfect someday. Automated tests over hope. Test coverage over lineage diagrams. Measurement over opinion. The DataOps Quality Manifesto, 24 principles. Read it and sign it: hubs.ly/Q04CgVcl0
24
A check fails at 6:14am and finance opens the dashboard at 8. Stop, quarantine, retry, or publish? Halt when data would be silently wrong. Warn when it would be obviously incomplete. The full decision table: hubs.ly/Q04C2TCb0
27
Your Fabric pipeline finished green and the Power BI report is wrong anyway. TestGen 5.92.1 tests OneLake bronze, silver, and gold through the SQL analytics endpoint. Open source, read-only, no rows leave the tenant: hubs.ly/Q04CgTM10
30
One small pharma could reuse at most 40% of its two-year-old warehouse for its second indication. The first brand gets a platform. The second finds out what kind. How to add brand two without a rebuild: hubs.ly/Q04C2Tpp0
27
Your supplier changed a column, and your dashboard broke. A contract on a wiki page tests nothing. Nov 17, 12pm ET: Chris Bergh builds one live in open source TestGen, 80% of the tests in 60 seconds. Free: hubs.ly/Q04CsMxF0
17
Venture-funded open source has a timer. Elementary's went off Oct 19 when Boomi bought it. GX Cloud went to FICO and shut down. Before you build on an open source tool, ask who's waiting for the exit: hubs.ly/Q04CgTf70
28
One customer profiled five tables against a live transactional database and slowed the app for 20 minutes. On Aurora, point TestGen at the reader endpoint and the work runs on replicas. Setup and grants: hubs.ly/Q04C2SCR0
33
The 2026 Data Contracts Software Landscape, open source and commercial. Nine tools round-trip ODCS and enforce it with tests. Few write the tests for you, and 10 tables of 20 columns need at least 430: hubs.ly/Q04CwFY80
28
An LLM is trained to be plausible, not accurate. Chip Bloche's AI 101 for data quality and governance people is on demand: agents, prompts, and why context needs governing like data. 64 minutes, with slides and transcript: hubs.ly/Q04CgT1_0
23
Does a 1-3 person data team need a QA engineer for data quality? For the mechanical checks, no. TestGen installs in about 15 minutes and generates about 80% of the tests. Your analyst writes the business rules. hubs.ly/Q04C2SkY0
19
A data contract is tests, so run it where tests run: at the supplier, on ingest, as a gate before gold ships, and in CI. Nov 17, 12pm ET: Chris Bergh builds one live in open source TestGen. Free: hubs.ly/Q04CgT2D0
33
Two dashboards, two revenue numbers, and both refreshed this morning. Usually neither is broken. Walk the split back one layer at a time. The first layer where the totals part is where the cause lives: hubs.ly/Q04C2RNR0
1
32
Four free books from DataKitchen. Tests pass and the numbers are wrong? Data quality. DataOps new to your team? The Cookbook. Spreading it past one team? Recipes. Building a drug launch data operation? The pharma book: hubs.ly/Q04ByVYD0
25
Stop being the goalie for your supplier's data. Nov 17, 12pm ET: hand them a data contract and open source TestGen, with one rule. Run this before you send us the file. Free webinar, recording to all who register: hubs.ly/Q04Cb_w-0
1
75
Your Databricks job finished green and the number was wrong. A tripwire task between bronze, silver, and gold runs TestGen and exits non-zero on failure, so the next layer never gets written: hubs.ly/Q04ByVKy0
40
$1B of VC went into data observability and quality vendors. Investors need it back, and it comes back through per-table pricing: about $172,500 a year to monitor 1,000 tables. Same free algorithms underneath: hubs.ly/Q04ByV5_0
27
A generated minimum-fare check set its floor at negative $367 and passed. That's why a small team's first 20 data quality checks need business rules too, each with a severity and one owner: hubs.ly/Q04C2RNy0
33
"Data contract" searches are up 300% in a year, and most contracts are documents nobody runs. Nov 17, 12pm ET: Chris Bergh builds one live in open source TestGen, 80% of the tests in 60 seconds. Free: hubs.ly/Q04C4_140
37
Matching row counts prove the rows arrived, not that a single value survived. TestGen's Table Comparison tests turn the source profile into 345 checks on the target. Five seconds a run, every mock load: hubs.ly/Q04ByTXD0
36
75 payments in the 2024 Open Payments file are dated before 1900. Three are dated the year 2. The dates parse, so the file passed CMS validation. The pre-submission checklist to run before you attest on March 31: hubs.ly/Q04C2PCP0
34