AI Will Not Fix Your Broken HR Data. It Will Confidently Scale It.

Former Amazon People Analytics leader Ian O’Keefe on why analytics and AI keep failing HR, and the foundation you cannot skip.

From CultureClub X, the CultureMonkey podcast for CHROs and people leaders. Hosted by Darcy Mehta.

CultureClub X by CultureMonkey — Season 06, Episode 16

There is a number that should stop every HR leader mid-roadmap. A widely cited 2025 MIT study, The GenAI Divide: State of AI in Business, found that about 95% of enterprise generative AI pilots delivered no measurable impact on the business. The researchers were clear that the failures were not about the models. They were about data readiness, workflow integration, and organizational context.

Ian O’Keefe has been saying a version of this for years. He is the Founder and CEO of Ikona Analytics and the former Head of People Analytics at Amazon, where his team built and scaled data products for more than 400,000 corporate employees. Before that he built analytics functions at JPMorgan Chase, Google, and American Express. In this episode of CultureClub X, he explains why most people analytics and AI efforts break long before anyone builds a model, and he does not soften the warning.

“If you’re putting new technology on top of old problems, you’re going to amplify old problems. And AI will not only amplify, but AI will confidently and eloquently be wrong in convincing ways that are hard to look past.”

That is the whole episode in two sentences. AI does not repair a weak foundation. It scales whatever is already there, mistakes included, and it delivers them with a confidence that makes them harder to catch.

The mistake almost everyone makes first

Ask Ian for the single biggest foundational mistake and he does not point at tools or talent. He points at sequence. Teams rush to stand up reports and dashboards, then wonder why the outputs are wrong. But the output is rarely the real problem.

“Fixing the report is not the issue. Fixing everything that’s upstream transactionally is the issue typically.”

The broken headcount report is a symptom. The cause sits upstream, in the production systems, the workflows, and the moment a transaction is created by an HR teammate or a manager. Working only at the point where data is consumed, and never at the point where it is generated, is the trap. His fix is to start from the decision, not the data. Before building anything, ask the leader what they would actually do differently if they had the answer. If the report would not change a single process, policy, or decision, you are measuring the wrong thing. Decisions first, then the metrics and infrastructure that support them. That discipline is what separates real HR analytics from expensive dashboards.

Reporting is not analytics, and the confusion is costly

One reason foundations stay weak is that two very different disciplines get treated as one.

“You can think about reporting as looking back at what happened, and analytics as why it happened or what probably will happen.”

Reporting is an extension of your tools, processes, and transactions: how many candidates entered the system, how many hires closed, how you count people on a headcount report. It sounds basic and it is brutally hard, because a single employee carries data from every corner of HR. They were once a candidate. They are paid. They sit in talent reviews. They may be a leader. Add procurement, contingent workers, and the fact that finance counts humans differently than HR does, and the reconciliation problem explodes. When leaders expect analytics but receive reporting, the credibility of the entire people analytics function quietly erodes. Naming which one you are actually delivering is the first honest step.

New technology, old problems

Ian has watched this pattern repeat for a quarter century. Every few years a new technology arrives and gets pointed at the same unsolved problems. The internet would fix it. Then the cloud. Then big data. Then machine learning. Now AI. Each time, the promise is that the tool will do the work the foundation never did.

He frames the current moment honestly. This is an internet moment, he says, and probably a bigger one than 1999. But the lesson from that era still holds. A powerful new capability does not redefine how work gets done unless the plumbing underneath it is sound. Point AI at a broken data ecosystem and you do not get magic. You get the same errors, faster and more persuasively wrong.

What AI actually needs is not in your data

Here is the counterintuitive part. The thing that makes AI genuinely useful is often not sitting in your systems at all.

“Much of what AI needs, rules, context, judgment, is up here in people’s heads, and we have to unlock that.”

AI is excellent at the minutiae and the drudgery. It creates speed and acceleration. But the rules, the context, and the judgment that turn raw data into good decisions live in the experience of your people, not in a table. Building a foundation is partly a technical job and partly the work of surfacing and encoding what practitioners already know. This is also where GenAI in HR tech either earns its keep or quietly fails, depending on whether that human context was captured first.

Change management is the part everyone underestimates

If people analytics were only about analytics, this would be a technology conversation. It is not.

“People analytics isn’t as much about the analytics as it is about the people that generate the data that you have to conduct analytics on.”

The data you analyze is a byproduct of how humans at a place called work come together to get things done. That is why change management is the most underestimated element of a successful foundation. The definitions, the competing metrics, the historical views you need, all of it depends on people changing how they capture and treat data at the source. Ignore that human layer and the cleanest model in the world will sit on top of numbers that do not mean what you think they mean. Treating change management as an engagement problem, not just a process one, is what makes the foundation hold.

The 90-day non-negotiable for CHROs

So what should a CHRO do before spending another dollar on AI or analytics tools? Ian’s answer is unglamorous and exactly right.

“A non-negotiable step is governance, ownership, and understanding of your data ecosystem. Do not expect AI to fix it for you.”

Not a new platform. Not a bigger model. Ownership of the ecosystem, clarity on who is accountable for what, and a real understanding of where data comes from and where it breaks. Do that first, and every tool you buy afterward stands on something solid. Skip it, and you learn the hard way that AI amplifies whatever it is given.

Where the truth actually lives

Ian closes on a hopeful note about the signal most teams have historically thrown away. For years, open-ended survey comments were the bane of practitioners, because nobody knew what to do with all that unstructured text.

“That unstructured, spoken word is where the truth lies more than a quantified value in a lot of ways.”

The scores tell you what moved. The comments tell you why. Capturing that spoken-word experience, then digitizing, structuring, and analyzing it, used to be out of reach. Now it is table stakes, and it is one of the clearest ways real-time listening strengthens your data foundation instead of becoming one more disconnected source. The organizations that get this right do not just understand their people better. They close the employee feedback loop and act on it.

Watch the full conversation

Ian O’Keefe and Darcy Mehta go deeper on every one of these points in the full episode.

Watch on YouTube: https://www.youtube.com/watch?v=etO1ijT1w2o

Full episode and transcript: Data Without Direction on CultureClub X

Connect with the guest

Ian O’Keefe, Founder and CEO of Ikona Analytics and former Head of People Analytics at Amazon. Connect on LinkedIn or visit ikonaanalytics.com.

CultureMonkey is an AI-powered employee engagement platform that turns real-time employee listening into structured, analyzable signal, so people teams can strengthen their data foundation and act faster. See how it works.

Tags: People Analytics, Artificial Intelligence, Human Resources, Data, Leadership