Shay O'Reilly

Intentional career pause focused on renewal and identifying opportunities to apply product and data expertise toward positive social impact.

Owned strategy and execution for mission-critical, user-facing data products used by large, complex organizations to make operational and policy decisions.

• Led end-to-end product strategy for data collection and onboarding workflows, translating ambiguous customer and stakeholder needs into shipped features used at scale
• Drove product discovery through customer interviews, workflow mapping, and rapid iteration to reduce implementation time from months to minutes
• Launched AI-powered analytics products that enabled non-technical users to understand and act on complex performance data
• Navigated complex stakeholder ecosystems across multiple teams and external partners, building alignment through clarity and empathy

Responsible for all products related to attribution, measurement, and reporting on Advertising Cloud's Display unit.

• Led product discovery and delivery for cross-device attribution and reporting in highly constrained data environments
• Led strategic launch of measurement and reporting for mobile app environments

Led analytics strategy for businesses using Cardlytics' purchase data to improve their marketing.

• Built a platform and content for reporting and insights used by internal sales teams as well as directly by customers
• Drove all aspects of testing and measurement methodology to assess advertising effectiveness

Led a team of analysts with two separate charters: optimization and data science for performance marketing campaigns, and usage analytics across Adobe Marketing Cloud.

• Managed a team of six business analysts, all with advanced degrees; responsible for hiring, development, and compensation
• Promoted three times from Business Analyst to Senior Business Analyst to Manager to Senior Manager
• Created automated analyses and prototypes using SQL and R to develop insights and optimization opportunities at scale

Built an evaluation pipeline to assess how accurately AI models answer Medicare questions — adapting methodology from a published JAMA Network Open mystery-shopper study of human counselors. Tests responses across accuracy, completeness, and safety dimensions.

• Designed a five-agent pipeline (questioner, extractor, verifier, scorer, adjudicator) with strict role separation to reduce evaluation bias
• Supports 100+ AI models via OpenRouter plus direct API access to OpenAI, Anthropic, Google, and xAI — enabling apples-to-apples model comparisons
• Generates SHIP-style accuracy comparison tables and audit-ready documentation

Project details and data → View on GitHub →

An AI-powered tool that automates the creation of Google Analytics 4 tracking plans and Google Tag Manager implementations for websites.

View on GitHub →