AI Beyond the Tech Giants – A Look into the Trenches

By

 

AI for the rest of us

Today, I want to take you behind the scenes and offer a glimpse into the lives of data scientists outside the realm of tech giants like Google or Meta.

For many of us, our reality differs significantly from their data-driven utopias. But, our journey is equally thrilling and impactful. Let's dive in!

Small Teams, Big Impacts 

Unlike the big tech companies, who dedicate enormous resources to exploring the frontiers of AI, most of us work in compact teams. We don't always have specialized teams at our disposal, and our relationship with the cloud is more about cost-effectiveness than ownership. Our world is less about groundbreaking research and more about delivering concrete value, swiftly and effectively.

Our daily grind doesn't always involve a dedicated ML platform team. Quite often, we ARE the team - the builders of the platforms, the explorers navigating the frontier between what is and what could be.

Navigating the Challenges

Amidst the hype surrounding LLMs and the ongoing Generative AI gold rush, the main challenges faced by data teams for decades remain largely unchanged. Let's address a few of them:

 🔍 The Detective Role: tracking down hidden data from numerous sources, questioning its existence, and ensuring its availability for analysis.

 🧪 Validating Data Reliability: verifying the reliability of the data we work with. We painstakingly validate, cleanse, and transform the data. Our mantra: “rubbish in, rubbish out.”

 💎 Managing Data Quality: we navigate through a maze of messy datasets, data silos, incompatible formats, and varying levels of data quality. It's an ongoing battle, but we soldier on!

 🤝 Gaining Stakeholder Buy-in: aligning stakeholders and getting their buy-in is a crucial aspect of our work. We navigate the complexities of different perspectives and expectations, translating data insights into actionable recommendations that resonate with our stakeholders' goals. It's a delicate dance of influence and collaboration.

 ⏰ Beyond the Short-term Horizon: while big tech companies plan years ahead, we often work within shorter timeframes. We develop strategies and roadmaps that balance agility with long-term vision.

 💡 Unleashing Data's Business Potential: we constantly seek to answer the fundamental question: How can data truly drive business success? We strive for a clear understanding of how data can impact our organization's bottom line.


 🔄 Change Management and Adoption: implementing data-driven solutions often requires navigating organizational dynamics, resistance to change, and fostering a data-driven culture. Encouraging adoption, addressing concerns, and driving change management initiatives.

Sound familiar❓ I bet it does!

Is Generative AI the Magic Solution?

While Generative AI holds promise, it may not be the silver bullet solution to all our challenges.

I'd love to hear your thoughts! What other challenges do you face in the daily life of data & AI?

 



Popular posts from this blog

Preparing Australian Companies for the Impending Impact of the EU Artificial Intelligence Act

By
Image
Introduction The recent political agreement on the European Union's Artificial Intelligence Act represents a significant shift in the global AI landscape. This comprehensive set of regulations, which aims to ensure trustworthy and ethical AI, sets a precedent that could influence AI governance worldwide, including in Australia. Over the past week, AC SmartData has been assisting both existing and new Australian clients in understanding what this new Act means for them and their AI products. Australian companies, especially those with global aspirations or connections to the EU market, must now prepare for potential ripple effects and align their AI strategies accordingly. Understanding the EU's AI Act The EU Artificial Intelligence Act is the first of its kind: a holistic regulatory framework for AI. It categorizes AI applications based on risk levels and imposes stringent compliance requirements for high-risk applications. These requirements include mandatory risk assessmen...
Read more

Ever wondered about the differences between traditional Machine Learning (ML) and Artificial Neural Networks (ANNs)?

By
Image
A colleague’s question recently got me thinking 🤔 - how can we break down these complex concepts for everyone to grasp? So, here's my attempt to simplify the answer:   📚 Traditional Machine Learning 1. Algorithmic Simplicity ML algorithms are often relatively straightforward mathematically, based on well-understood statistical and mathematical principles. For example, linear regression is based on finding the line (or hyperplane, in higher dimensions) that minimizes squared residuals.   2. Interpretability  Traditional ML algorithms often provide high interpretability. For instance, in a trained decision tree, you can trace down the tree to see exactly why a particular prediction was made.   3. Data Requirements They usually require less data compared to deep learning models.   4. Feature Engineering Traditional ML algorithms often rely heavily on feature engineering. This involves manually creating features from the data based on domain knowl...
Read more