Holding AI to its promise

Holding AI to its promise – Outcomes that balance Efficiency and Effectiveness 

by Ramanathan Vythilingam 

Background 

The last 5-10 years have seen significant change, with the evolution of Artificial Intelligence and its increasing applications in every field. While there is optimism for an overall net positive business impact and betterment of society, it is counter-balanced by pessimism linked to the impact on the role of Humans and the depreciating value being placed on Human effort. This has been the narrative for a while – at least the last 5+ years, when Machine Learning, Neural networks, and other advancements were gaining ground. With the introduction of ChatGPT in November 2022, the reality is even more accelerated. In this short period of time, the world does seem a very different place – it is as if we could demarcate a BC (Before ChatGPT) and AC (After ChatGPT) eras. While GPTs were not necessarily new and have been around since June 2018, the democratization of access truly happened with the chat interface. Now anyone can enter a few lines of conversational text, to generate outcomes. This chat functionality has then transcended mere text outputs, and we have text-to-image and text-to-video generating tools – whose quality of outputs is improving by leaps and bounds. They might not be there yet, but they are fast getting there. 

We live in interesting times – while the technology has unlocked a world of seemingly excess resources, the business outcomes (however we measure them) are not improving. The headline narratives seem to be all about “crises” – Crisis of Attention, Crisis of Creativity, Crisis of Innovation failure rate. And at a time when Marketing and Research should be an exciting space to help businesses navigate the changing paradigms, surveys with professionals highlight a growing frustration with all that they need to grapple with. (1) 

How did we end up here? What decisions continue to be made that will likely ensure a worsening of the current situation? And what should we be cognizant of and start to embed in what we do, to alleviate the situation?  

How and where have we ended up? 

Artificial Intelligence has been around since the mid-1950s, and we can map its trajectory quite well on the Gartner Hype Cycle curve. (Fig1).  

 

Figure 1 : Gartner Hype Cycle, used to visualize the AI journey 

We can see that every alternate decade was either chockful of optimism and inflated expectations or one of pessimism and disillusionment. The upward trajectory of the last decade (2010s) of Machine Learning and Unsupervised training models was made possible by the decreasing cost of compute and memory storage, as well as the open-source economy, that allowed folks to build on others’ work. The 2020s is the decade of Large Language Models and Generative Pre-Trained Transformers (GPTs) and we will continue to see an accelerated uptick in adoption and innovation in everything we do. 

In a world of increasing complexity, the field of cybernetics, which first became popular in 1948, defines the concept of a “black-box”, that is very pertinent to what we are seeing with the evolution of AI. In short, a black-box is defined as a system where the internal structure and workings are unobservable or “invisible,” allowing researchers and practitioners to concentrate solely on the inputs and outputs of the system. A black-box is also characterized by the process of input, process, output and feedback.  

 

Figure 2 : Concept of a Black-box from Cybernetics 

The nature of AI as it develops is increasingly that of a black-box. In the early days, when AI was based on Supervised Learning, we had some notion of the inner workings of the AI, as it was all rule-based. When AI models moved into Deep Learning and Unsupervised learning approaches, the inner workings of the tools were further obscured. We could see the outputs generated, and feedback could be implemented to improve accuracy. With GenAI, the black-box nature of AI is further solidified – the Large Language Model approach means, we will never know what is driving the models and can only evaluate the outputs, and incorporate feedback, by iterating the prompts. 

One the consequences of increasingly black-boxed systems, is the resultant decline in human accountability. Even taking standard primary market research tools, that are used to validate different parts of the Marketing funnel, we used to see this. Most brands would have a version of the Marketing funnel shown in the Figure 3 for new innovations design, validation and launch. At every stage, Brands would validate critical elements like the Concept, Product, Pack, Communication, using existing research tools. Brands would define action standards for key metrics that matter at each stage – and when a test meets the action standards (example with a Concept we would want to see strong relevance and purchase intent and the willingness to pay the price) – the project would move onto the next stage. Usually, big innovations would go through all the stages of research and would need to meet the individual action standards of each stage. So, everything is designed and optimized with consumer feedback.  

 

Figure 3 : The Marketing funnel and the role of Research at each stage 

But the industry reality is that most of the innovations fail to achieve their intended business results in-market, and the reported statistic is sobering – “95% of new innovations miss their mark” (2). Even if we discount this as hyperbole and assume a 30% lower number, that is still a ton of wastage. The more we put guidelines/systems in place and set action standards for each stage, the more it seems we absolve ourselves from accountability for the net outcomes.  

Just to be clear – it is not a problem of accuracy of the prediction at each stage – that we should consider is a given. The assumption is you are working with tools that are trained on the right data-sets and are sufficiently accurate for the purposes for which they are being used. As with all things, they can get better – and we will assume efforts are on to achieve this. 

How do we guide decision making in this environment, that ensures that the net outcomes are positive? This will be covered in the next section. 

So What?   

There is no denying that AI developments have significantly improved the efficiency of the overall process. If the decade of 2010s was about Data Science, the decade of 2020s will be the decade of Synthetic data. Wikipedia defines Synthetic data as data that is artificially generated, rather than real-world events.(3) Synthetic data today ranges from predictive models that are built on past real-world data, to ChatGPT that can be used to generate survey responses much like real world surveys with people. These tools are a lot cheaper, and the results are available a lot faster than traditional research approaches. This tangible reduction in cost and timelines, is what makes them attractive for adoption and scaling. But the measurement of outcomes of the decisions made with these tools, is not that clear-cut. Most times there are other variables which obscure the in-market impact that is attributable to the use of the AI tools. But the fact remains that we are seeing businesses struggle with the complexity and registering higher failure rates and lower returns overall. 

I will take a practitioner’s view in this – as someone who is using the AI tools in research and driving marketing decisions. There will be separate considerations for the teams who create these tools – but given that vast majority of marketing and research professionals are users, not developers, we will focus on the use cases. In the following sections, we will look at specific AI tools used in Research and Insights as examples – to understand what they do in different parts of the Marketing funnel, what results do they generate and more importantly, what does it mean for Humans to take more accountability in setting the tools up for success. All these tools are “black-boxes” – they are exceedingly complex systems, and while we can have an appreciation of their inner workings, chances are that most of the Research Professionals will be limited to what input we provide and to observe the outcomes. Setting the tools up for success requires considered thought – bringing to bear a strong business focus and understanding of what will improve the quality of decision making. Efficiency is a given, but Effectiveness needs to be designed for.  

In practical terms, taking accountability means two things –  

1. Setting the AI tools up for success, by driving the right decisions 
2. Enhancing the impact, by reimagining the use-cases of the AI tools 

Let us look us at some examples. 

1. Human accountability in setting the AI tools up for success
   This is underpinned by the core idea, that as technology develops, what and how we do things will change, but the why we do them remains the same. Human evolution is millions of years in the making, and the core fundamentals of marketing and brand building remain the same. It is about building and maintaining awareness of the products we offer, reinforcing brand associations with category entry points, sold at a price that people are willing to pay and found in places where people shop for the category. Everything is in service to this core principle. The modalities of brand building have evolved, with the proliferation of media touch-points ad place where people discover our Brands, but the core elements of competitive Brand play remain the same. There are two real-world examples we can deep-dive into. 
   1. Ensuring consistency of Brand expression in GenAI
      Let us consider the challenges Brands face in maintaining consistency in the digital ecosystem. In the old reality of traditional media (TV and print), the number of assets and the media channels they were deployed in were limited, allowing for easy checks on the execution of to ensure consistency of brand personality and tonality. Today we have increasing spectrum of media channels and multiple advertising formats in each channel (Figure 4).   Figure 4 : Media channels and respective formatsA conservative estimate is for new innovations to have 30000 pieces of content, to standout and build awareness. And this number is increasing every year, as the media landscape further de-aggregates.  The only way Brands can be present in this complex ecosystem is to create assets at industrial scale. New age GenAI tools are stepping into creative development, and we will see increased acceleration of adoption, as Brands cope with creating 000s of pieces of content for every campaign.  So, while it will become easier over time to work with GenAI tools to create assets, the increased efficiency is already giving rise to a new challenge – that of a disjointed Brand identity. Lack of consistency in Brand expression across channels, will lead to a lower return on media investment – so, while we gain in efficiency of asset creative, we lose out on effectiveness. The solution is to ensure there is accountability for the Human element in defining clear Brand guard-rails in the GenAI tools. There are a few vectors along which Brand personality and identity can be defined –  
      1. Specific asset design parameters, linked to each media channel
         

         Figure 5 – The “canvas” for digital advertising 

         Each media channel has a unique format of content consumption – Facebook, Instagram and TikTok are scroll based; while YouTube is search based. In additional Facebook/Instagram are sound off while TikTok is sound on. Understanding the content consumption behavior and hence how Brand related advertising content is discovered and viewed in each channel, we can define certain core guidelines that are mandatory to adhere to. Some of these are defined by the media platforms themselves – like Meta would recommend aspect ratio of videos to be (1:1, 2:3, 4:5, 9:16) and YouTube would recommend aspect ratio to be 16:9. In summary, these guidelines can be considered as defining the parameters of the canvas, onto which the creative narrative will be laid out. 

      2. Creative multipliers, specific to each media channel
         Figure 6 – The canvas with creative multiplier considerations Once we have the canvas, we can start to think about the creative choices to be over-laid on it, that will aid in capturing attention and aiding effortless information processing. These elements are linked to neuroscience and human psychology learnings around what catches our attention and how we process information (System1). An example would be the YouTube ABCD creative guidelines (Attention, Branding, Communication, Decision) (5) – that are validated to drive better video performance on YouTube. There are similar guidelines defined by Meta and TikTok, to enhance content performance on their channels. There are common themes – like pacing of the video to ensure it catches attention early and holds attention; number of visual clusters and color contrast to enable information processing; human-product interaction to drive relevance and so on.  The media channel specific multipliers need to be considered as constraints or guard-rails, within which creativity needs to be unleashed. They define the “WHAT” is important, and creativity needs to think of “HOW” they will come to life. Creativity without constraints will lead to less effective advertising and a lower return on media investment. 
      3. Distinctive Brand Assets (DBAs), specific to each Brand
         

          Figure 7 – Examples of Distinctive Brand Assets 

         Distinctive Brand Assets are non-name brand elements, that are recognizable and memorable features that instantly bring a particular brand to mind. Examples would be logos, colors, pack-shapes, slogans, characters etc. There is a lot of research to show the power of DBAs in helping build Brands – they can enable creative expressions and play a critical role in building branded memories. Each Brand needs to define a palette of DBAs, including visual and audio properties, that are then inputted to the GenAI creative tool to be incorporated into new creative assets that are generated. While this sounds simple enough, expressions of DBAs are not a binary choice – it is not “presence of logo – YES/NO”. It is a considered creative expression, that can involve colors, shapes, movement etc – that requires clear definitions and brand guidelines. And much like the Creative Multipliers, they form the guard-rails within which creativity should flourish. 

      4. Other Brand persona/archetype aspects 

      A final element of brand consistency is the embodiment of the brand’s personality in everything that is consumer facing. There are different frameworks, grounded in Jungian psychology, that define brand personalities in different ways, the most popular being archetypes. Each archetype has a certain expression unique to them in the choice of semiotics – colors, emotions, tone of narrative structures etc. There are existing AI tools that can help us audit for presence of elements in advertising, but these are simple models, trained on existing labelled data.   

      

       Figure 8 – example of a vision AI tool decoding tags present in an image.   

      When defining the Brand in a GenAI environment, we are now talking about capturing machine-relevant brand identity parameters. This is a whole other dimension of Human input, that needs considered thought. There are nuances within each archetype that need to be considered to drive consistency. Let’s take an example of Happiness as emotion, that a Brand wants to stand for. All the images below represent expressions of  happiness, but they are all also different. As Humans we can discern subtle differences in each of them and this level of detail is required as input, so that the generated content is on Brand.  

      

      Figure 9 – Expressions of the emotion of Happiness  

      Having seen the four parameters for Brand personality, we can now map them along a continuum, that considers the difficulty of AI and the role of Humans. 

       Figure 10 – Framework for Human Accountability in a GenAI creative tool 

      In summary, GenAI creative tools are only as good as the inputs they get to work with. The more time and thought we spend in defining our Brands in the GenAI tool, the better will be the outcomes in creative quality – as measured in terms of suitability for the media channel and consistency of Brand expression therein. Thereby delivering better to the marketing objectives and maintaining consistent Brand expressions across campaigns, across channels and across time.  

   2. Predicting Attention 

In recent times, there has been an explosion of conversations around Attention and how it needs to be thought of as the missing link to drive Media and Creative effectiveness. I am left wondering why everyone in the industry seems so surprised, by the fact that people do not pay attention to interruption advertising. There is renewed interest around Attention measurement, which is still very limited and reliant on eye-tracking, which in itself has been a research tool available for a long time. 

One of the recent AI innovations in this space is that of Predictive Eye Tracking. This is an AI predictive tool that is built on a foundation of eye-tracking studies that captured where people focused their gaze when presented with different stimulus, mainly static visuals and videos of brand advertising. And this aggregated human behavior data is then used to train a model that can predict the attention heat-map for a new static/video. 

The tool can generate the predicted heat-map within seconds and is significantly cheaper than traditional methods that rely on eye tracking studies of Human respondents. Unlike other tools that predict creative quality or other validation metrics, with predicted attention, there is no clear answer to what good or bad attention looks like. The predicted heat-maps reflect neuroscience and human psychology factors that determine which parts of the ad get more attention – like centrality of gaze (elements in the corners rarely get noticed), color contrast, size and position of text font, size and position of product visuals etc.  

How should we review the outputs in a manner that drives effective decision making? 

We need to view the use of this tool, as one that helps us understand the Intent-Perception gap. Here the “Intent” is the Marketing intent. Every piece of advertising is intended to deliver to a business objective – build awareness of an innovation, communicate the superiority of the product, highlight the ingredients in the product that deliver a superior benefit etc. The creative narrative structure is a conscious choice to deliver to this marketing intent. The Perception is what consumers understand when exposed to the piece of advertising. Everything we do in creative testing is to understand the Intent-Perception gap. When the understanding is broadly in line with the marketing intent, we can be confident that the ad will deliver to the business objectives.  

Human accountability is in asking these key questions in reviewing the predicted attention outputs : 

• Do people notice the creative elements that link directly to the Marketing Intent? 
• Is the position of key elements like text supers, ingredient visuals etc in the predicted focus zone? 
• What key creative elements are likely to NOT be noticed during ad viewing? 

And the answers to these questions, should then guide creative optimization choices, like position, size and color of text supers, position and size of product visuals, number of visual clusters competing for attention in each frame. Thereby reducing the Intent-Perception gap, and strongly driving in-market impact from the piece of advertising. 

Now What? 

Thus far, we have explored different applications of AI tools, that pushed the narrative of balancing efficiency with effectiveness. We will never be fully able to place accountability on the tools themselves – as they are black-boxes – and neither can we expect accountability from complex organizations. It does come down to each individual to find a way to navigate this increased complexity and drive the right decisions as best as they can. And in this, the role for Humans in the equation becomes that much more important. One definition of AI is Augmented Intelligence – which is the right way to look at it. The verb “augment” assumes there is something present that can be augmented/enhanced. This needs to be Human Intelligence – so AI is augmenting our thoughts / hypotheses / ideas. 

There is another reality that is unfolding with the developments of AI – happening at an unprecedented scale than we have ever seen before. This is the removal of constraints that existed mainly in terms of resource and time. I still remember a time when the main challenge we had in Marketing and Insights, was the constraint of time and money, that limited what we could do. At every stage of the funnel, we had to prioritize what we could do – from the learning plans to the access to data, and even what was created for the marketing campaign. Today technology advances, specifically with Generative AI, has removed many of these barriers, allowing us to do so much with just a prompt. In the book, Beautiful Constraint, author Adam Morgan talks about the role of constraints in unlocking creative thinking and business value. The book details many examples, where perceived limitations can be transformed into opportunities for innovation and growth. Not despite the limitation, but because of it. In recent years, with the developments in AI the dominant narrative is the removal of constraints – mainly the constraints linked to resources of money and time. GenAI tools are significantly lower in cost and generate outputs at lightning-fast speed. But the removal of constraints is also resulting in a lack of accountability – being free to now do so much more, is not resulting in net positive business outcomes. And this side of AI is never discussed. The entire selling paradigm for AI tools is improving efficiency, which is also resulting in a significant depreciation in the value of Human effort.   

It is almost as if a Double-Jeopardy scenario is playing out for the very time in Human history – excess resources that allows us to do so much more and a lack of accountability transcending the entire ecosystem. 

In summary –  

There are 3 key takeaways from all the examples we have seen.  

While Machines are binary, as Humans we need to be multi-faceted. Even the most sophisticated AI tool, is ultimately a mathematical model that sees everything as black and white. As Humans, we need to be comfortable in the shades of gray. The binary aspect of AI also means it will always be either Efficiency or Effectiveness. Humans need to unlock value by considering the AND – balance dualities of Efficiency AND Effectiveness, Generative AND Creative. 

To achieve this, Fluid intelligence is more important than Crystallized Intelligence. Crystallized Intelligence involves knowledge that comes from our prior learning and past experiences. This was the default approach in the past, but with a rapidly changing world, what we know needs to be adapted for the reality. Fluid Intelligence is the capacity to reason quickly and think abstractly. This is what will help us unlock new applications for AI tools, where we amplify the strengths of AI in creative ways. 

There is also an aspect of how we engage with the business. Hypotheses-led thinking and a constant beta-mode is the dominant state to be in. In the theory of cybernetics, researchers talk about the presence of “accountability sinks” – parts of a complex system where accountability does not exist, leading to overall sub-optimal outcomes. The positive side to the increasingly complex world we live in, for everything we do there is an abundance of data/feedback. The more we can leverage this data to constantly evaluate, re-align and/or pivot, the better will be the business outcomes. The abundance of data, including synthetic data is a great resource to always have hypotheses for every business question. For example, we can use ChatGPT to derive hypotheses that can feed into the discussion guide, and we use the qualitative research to validate / disprove the hypotheses to generate learnings. 

Sources : 

1. Why are 37% of Marketers concerned about AI in Marketing? (LINK) 
2. Why 95% of new products miss the mark? (LINK) 
3. Wikipedia – Synthetic data (LINK) 
4. YouTube ABCD Creative guidelines (LINK) 

 

About the Authors

Ramanathan Vythilingam
Associate Professor (Practice), College of Business (Nanyang Business School)