The Retailer Summer Edition 2022

THE RE TA I L ER

3 2

AI: A SERIOUS ETHICAL CHALLENGE FOR RETAILERS

Elliott Goldstein Managing Partner The MBS Group

A re retailers prepared to take on the chal lenges and seize the opportunities asso ciated with machine learning and artifi cial intelligence?

laced with the unconscious bias inherent to humans. Retailers not mitigating against this bias, either in internal recruitment systems or in customer-facing algorithms, run the risk of damaging their employer brand and losing customer trust. Indeed, in some settings, we’re seeing algorithms price goods differently for different demographics of cus tomers. It surelywon’t be long until ‘variable pricing’ becomes perceived as ‘pricing discrimination’ – especially when groups of customers with similar ethnic and/or socioeconomic backgrounds are faced with more expensive products. Against this backdrop, it is critical that retailers put in place a robust governance framework for the ethical use of data andAI. If AI technology malfunctions or produces unfair results, can the creator be held account able? Is there recourse for consumers in the form of an accountable engineer, an internal governance board, or even external governance? Research from The MBS Group published earlier this year found that most consumer-facing businesses cannot answer these questions. Very few companies have a structure of accountability for the ethical use of AI, leaving them unprepared to take on the challenges and seize the opportunities associated with this technology. In retail, maintaining a business’s ethical values, guaranteeing legal com pliance, and ensuring that new technologies are being used optimally requires leadership from the non-executive Board. Today, many NEDs recognise the importance of data and ethics, but few are equipped with the expertise needed to take final responsibility (indeed, most NEDs will have been executives at a time when data or AI was not as integral to a company as it is now). Across the sector, Boards need to grasp the power and risks associated with data, and include at least one non-executive director who can participate in – and lead – conversations around ethical use of AI. Most Boards aren’t here yet – and Chairs must think carefully about making appointments which can provide sufficient checks and balances to the executive. We recommend four actions for retail Boards to effectively govern AI: • Separate AI ‘doing’ from governance – The team responsible for AI innovation and technological development should not be responsible for governing its ethical use • Make ethical use of data and AI a standing item on risk commit tee – AI should be a regular item on the risk committee agenda. Consumer setting up a standalone ethics committee. • Identify one NED to be responsible AI, data and ethics – Name one non-executive board member, who is appropriately trained and/or qualified, to be responsible for AI, data governance and ethics on the board • Measure and audit – Bring in third-party bodies to independently audit algorithms and policies around the collection and imple mentation of customer data – reporting directly to the Board How equipped is your business to grapple with this new horizon? Do let me know your thoughts – or indeed any best practice you have to share from your own organisation. .

In his 2021 Reith lectures on artificial intelligence, Professor Stuart Russell read an extract from Alan Turing’s 1950 essay on computer machinery. “Once the machine thinking method had started,” he read, “it would not take long to outstrip our feeble powers. At some stage therefore we should have to expect the machines to take control.” This astonishing notion may seem like a topic better suited to phi losophers or AI experts than retailers. But it is critical that any retail business developing or implementing AI solutions carefully considers the relationship between innovation, safety and ethics. For our sector, never has this been more pressing than today. The boom in ecommerce has provided never-before-accessed insight on customer behavior, and the cashierless ‘just walk out’ technology sweeping the sector feels symbolic of a new era, one that is underpinned by machine learning and collecting vast swathes of consumer data. However, while most companies understandAI’s transformative potential, few have real clarity on how to ensure – and on who is responsible for ensuring – its ethical use. Without a coherent strategy and a framework of accountability for the corporate governance of data and machine learning, businesses put themselves and others at risk. And in a highly challenging consumer landscape and a cost-of-living crisis, retailers simply cannot afford to lose customer trust. There are several areas to consider. At a minimum, incomplete governance around data usage means that any company that collects or processes personal information about EU or UK citizens runs the risk of non-compliance with GDPR guidelines. Moreover, as retailers diversify their services into quasi banking and insurance products, they will soon be put under similar microscopes to the financial services sector in terms of AI and data governance. Companies may also need to rethink the algorithms used in pricing. Thinking back to the early days of the pandemic, the ludicrous prices of antibacterial hand gel provided examples of what could happen when competitive pricing algorithms played out without human oversight. There are also plenty of risks associated with hyperpersonalisation. One online retailer, for example, reportedly began recommending the products required to create a homemade bomb in its “frequently bought together” section. Another infamous example is of a supermarket, which sent coupons for baby products to a pregnant teenager based on her shopping habits, much to her unaware father’s alarm. Perhaps most pressingly, the potential for bias inAI presents huge moral and reputational risks. AI and other cognitive systems are trained by historic data sets collected by real people, and so are almost always