Can we, may we, should we?

The tech industry is characterized by endless flow of hype cycles. Every couple of years there is a new promise. Often one of enormous increase of speed or infinite scalability. Or, and this is the one that most frequently reappears, a way to no longer need software developers.

Most of these hypes focus on one part of technology. AI differs because it can touch almost every part of a business at the same time. It merges several previously separate discussions into one: software delivery, product capability, operations, customer interaction, risk, compliance, data and economics.

The difference is that AI can be used to build the product, become part of the product, operate the product and do the interaction with the customer.

Compliance is very different this time

In software and financial services, compliance has long shaped process, architecture, controls and operations. But the systems themselves were generally deterministic. Given the same input and the same software, we expected the same result.

AI is unusual because compliance applies both to the product we are building and to the tools we use to build it. Compliance is no longer only about whether the outcome is acceptable but becomes part of the architecture of how the outcome is produced.

With AI, the behaviour of the system itself becomes something that may need continuous governance. You will have to have answers for a wider scope of questions like:

• What are we building?
• How was it built?
• What data went into it?
• How was that data obtained and governed?
• How did the system arrive at this outcome?
• Can we explain or reproduce it?
• Who is accountable when it is wrong?
• How are employees allowed to use it?
• How do customers interact with it?
• What happens when the model changes?

No, really: what happens when the model changes?

Different perspectives and use cases under one name

When we say “we use AI”, what exactly do we mean?

A marketer uses AI to research a topic. Sales uses it to draft a proposal. An engineer uses it to write production code. Security uses it to analyse incidents. A product team lets it interact directly with a customer.

Another team lets an agent take actions on behalf of that customer We now all have that handy colleague who creates all kinds of agents that can create documents based on data.

So all these actors have a different perspective when we talk about AI. They use it for assistance, creation, decision or action. These can fall under entirely different risk classes. Yet we frequently discuss all of them under one heading: AI adoption or strategy.

Perhaps that is problematic since we often have a very different concept in mind when talking about AI. We shouldn’t really have one AI strategy since it can be misleading. We should understand where AI creates value, what kind of AI we are talking about and what the consequences are for that particular use case.

Product management

In regards to product management we have three main questions:

• Can we do it? — technology
• May we do it? — compliance
• Should we do it? — customer and business value

The first two are often dominating the discussion about AI but the third is where it makes a difference. That one is about value.

Can we replace a claims conversation with an AI interaction? Probably. May we? Possibly, with the right controls. Should we? That is a completely different question. Does it improve trust? Does it reduce friction? Does it remove something customers actually valued? Does the organisation save €2 while creating €20 of downstream frustration?

Many AI pilots are still mainly proving that something can be done but that is very different from proving that it should become a product. It has become much easier to prove technical feasibility, but what it takes to create good and useful products hasn’t changed. To talk to users and find out their real needs and understand what way the market moves. That is what counts.

A hype to stay

AI is a hype, but that does not mean it is temporary. The useful parts will remain. The challenge is to separate value from excitement of the possibilities. Can, may and should we do it?