• Responsible Investing
  • ESG
  • Solutions

Living in the past

Why are controversy scores so controversial?

Many investors use controversy scores as a filter to avoid firms whose damaging behaviour has hit the headlines. But these scores have serious limitations for investors who want to manage ESG risks and have a positive impact.

Volcanic eruptions are a complex phenomenon and can only be forecast a few days in advance at best, even by experts. The same could be said of corporate controversies.

Yet too many investors seem to rely solely on backward-looking controversy scores to try to capture the extent to which companies are involved in historical or ongoing sustainability-linked scandals. Nor do all scandals necessarily indicate the same level of danger for the people and places harmed by a company’s activities.

“Controversies are so controversial in the research space,” says Dan Neale, social transformation lead at the World Benchmarking Alliance (WBA), an organisation set up to rank 2,000 global companies on their contribution towards delivering the Sustainable Development Goals.

An explainer: How the ESG scoring systems work

MSCI’s ESG Controversies Score uses a 0-10 scale, with 0 representing the most severe controversy. Out of 6,969 companies in its dataset, only 28 companies have a score of 0, which equates to a fail. A further 138 companies have a score of 1, which puts them on the watchlist.

The ISS ESG Controversies Score is also based on a 0-10 scale; however, this time 10 represents the most severe controversy. Out of 3,820 companies, 107 companies have a score of 10 (also flagged as red), 659 have a score between 6 and 9 (flagged as amber) and 3,054 have a score between 1 and 5 (flagged as green).

Too much reliance on too little information

Marte Borhaug, global head of sustainable outcomes at Aviva Investors, believes taking controversy scores at face value is a dangerous shortcut. It could also become an issue for sustainable funds, as the market may lose credibility if some funds are tainted because of poorly researched investments.

Taking controversy scores at face value is a dangerous shortcut

“There is a high risk of clients discovering companies they didn’t expect to be there in the first place. There could be a big gap between perception and portfolio reality,” she says.

First and foremost, the lack of consistency in approach by the companies providing controversy scores is stark. Which begs the question: how can investors tell which is ‘right’ and which is not?

For example, only seven companies are flagged for controversies in both datasets of two of the biggest ratings providers, MSCI and ISS (including Shell and phosphate rock miner OCP). This is because their scoring methodologies measure different things, so names often appear on one list but not the other.

Figure 1: Overlap between MSCI and ISS controversy scores
Overlap between MSCI and ISS controversy scores
Source: MSCI, ISS, Aviva Investors, as of March 8, 2021

Although every approach, from MSCI to ISS, has its merits, the differences make it more important for investors to develop the resources and expertise to complement data providers’ analyses and bring a different perspective.

Part of the issue is that the scores do not account for hidden controversies. Instead, they typically reflect headline-making issues, while ignoring lower-profile violations that have not yet been picked up externally.

The scores do not always account for steps taken by companies to improve and remediate issues

Moreover, the scores do not always account for steps taken by companies to improve and remediate issues in a timely manner, which are crucial to assessing a company’s future performance. For example, the legacy of the 2015 ‘Dieselgate’ scandal that rocked German car giant Volkswagen lives on today; the company still has a zero MSCI controversy score despite having made significant organisational changes to fix the issues uncovered.

The WBA’s Neale explains why. “We are talking about some companies with operations in 100 countries and supply chains in 150 countries; you are never going to be able to get to the whole truth of that, so you have to take an awful lot on trust,” he says.

Quantifying harm and comparing events such as deaths, illnesses or sexual harassment is a complex area in itself. “You don’t want to be the people that sit there and say: ‘300 cases of sexual harassment are equivalent to four fatalities’. But quantifying impacts, controversies or responses to controversies is, in some way, necessary for creating rankings. If you’re going to be bold and step into that area, it helps to be transparent on what approach you’re using,” he says.

Furthermore, remedial action is often hard to measure and unlikely to be satisfactory to victims. 

Digging deeper: Case studies and share price signals

Boeing and Airbus have low scores as measured by ESG Elements, [Aviva Investors’ in-house quantitative ESG score, which is based on several MSCI indicators alongside proprietary ones] – mainly due to past controversies. However, when qualitative analysis is added to the process, differences between the two companies are apparent.

The Airbus controversy stemmed from endemic bribery

The Airbus controversy stemmed from endemic bribery within its commercial aircraft business spanning more than ten years, but the company has since taken significant steps to mitigate against this happening again.

Meanwhile, Boeing’s controversy sprang from the tragic loss of life in two crashes of its 737 MAX aircraft, which uncovered safety issues and deep-rooted cultural problems. While the planes have now been re-certified, Malini Chauhan, ESG sector analyst at Aviva Investors, says cultural issues continue to pose a risk, both to investors and other stakeholders.

Boeing has not acknowledged the 737 MAX shortcomings stemmed from a company-wide issue, pinning them instead on two former employees.1

Boeing’s controversy sprang from the tragic loss of life in two crashes of its 737 MAX aircraft

The stark contrast between the two companies’ responses demonstrates the importance of remedial action to mitigate the risk of reoccurrence of incidents with tremendous negative social impact – and shows why investors need to look beyond scores to assess this for themselves at a qualitative level.

The correlation between share prices and controversies varies and seems dependent on many factors. However, the materiality of a controversy is dependent on the firm’s business model, making it doubly important to look beyond the score and into the nature of the controversy to assess its potential impact on valuation. For example, a failure linked to the core product of the company, such as an oil spill for an oil company, a medical defect for a pharmaceutical company or an engine failure for a car company – is likely to hit harder than a failure at the margin.

According to research by Bank of America Merrill Lynch2, the impact of ESG controversies on share prices of companies in the S&P 500 is stark: 24 ESG controversies wiped $534 billion off the value of large US companies over a five-year period starting in 2014.

Figure 2: The impact of controversies on share prices
The impact of controversies on share prices
Note: Data covers 2014-2019. Source: Bank of America Merrill Lynch

Building a mosaic of insights to glimpse at the truth

When it comes to companies’ impact on people and the planet, information from independent sources, such as research from civil society organisations, is essential. These organisations often operate on the ground, close to companies’ operations, and provide insight into current practices that may yet hit the headlines, providing more depth and granularity to data provider scores.

Information from independent sources is essential

“We will look into what the data providers have told us, whether through talking to the company, desk-based research, or talking to NGOs on the ground, to verify their statements. Using company employees, whistle-blowers, and that kind of bottom-up information is hugely important,” says Borhaug.

Moreover, the information is not just a proxy indicative of future controversies; having the information at hand today can also allow investors push for change. “Using a broader range of information is part of the due diligence necessary to be a responsible investor,” says Borhaug.

Understanding a company’s controversy risk requires a combination of art and science. “In an ideal world, there would be sufficient quantitative data to easily aggregate and analyse, but that doesn’t always exist. Sometimes there is an element of intuition, and you just have to monitor the markets and look for warning signs,” notes Richard Butters, ESG analyst at Aviva Investors.

As well as negative media reports, Twitter sentiment, emerging themes on employee site Glassdoor or online petitions can be useful indicators. Yet to have an intuition in the first place, investors need to look at many other data points, and this is where controversy scores can have a role. “If there is a controversy, you tend to see it in a score. If there is a zero score, that would require due diligence,” says Butters.

Investors can learn to better identify the tremors signalling insight

Another key tool of influence and research comes through engagement. Face-to-face interaction can help investors gauge a firm in subtle ways desk-based research can never achieve. Investors need to understand not just the implications of controversies, but also the change undertaken by companies after they have been hit by a scandal. If the remedies taken and progress made are significant, it can be rewarding to invest in such a company before the slow-moving controversy score begins to reflect the improvement.

Predicting corporate controversies will never be a perfect science. There is no model, algorithm or complete dataset available to help magic up an answer. However, just as vulcanologists searching for eruption clues, investors can learn to better identify the tremors signalling insight from the ones that merely add to the informational noise.

Want more content like this?

Sign up to receive our AIQ thought leadership content.

Please enable javascript in your browser in order to see this content.

I acknowledge that I qualify as a professional client or institutional/qualified investor. By submitting these details, I confirm that I would like to receive thought leadership email updates from Aviva Investors, in addition to any other email subscription I may have with Aviva Investors. You can unsubscribe or tailor your email preferences at any time.

For more information, please visit our privacy notice.

Related views

Important information


Except where stated as otherwise, the source of all information is Aviva Investors Global Services Limited (AIGSL). Unless stated otherwise any views and opinions are those of Aviva Investors. They should not be viewed as indicating any guarantee of return from an investment managed by Aviva Investors nor as advice of any nature. Information contained herein has been obtained from sources believed to be reliable, but has not been independently verified by Aviva Investors and is not guaranteed to be accurate. Past performance is not a guide to the future. The value of an investment and any income from it may go down as well as up and the investor may not get back the original amount invested. Nothing in this material, including any references to specific securities, assets classes and financial markets is intended to or should be construed as advice or recommendations of any nature. Some data shown are hypothetical or projected and may not come to pass as stated due to changes in market conditions and are not guarantees of future outcomes. This material is not a recommendation to sell or purchase any investment.

The information contained herein is for general guidance only. It is the responsibility of any person or persons in possession of this information to inform themselves of, and to observe, all applicable laws and regulations of any relevant jurisdiction. The information contained herein does not constitute an offer or solicitation to any person in any jurisdiction in which such offer or solicitation is not authorised or to any person to whom it would be unlawful to make such offer or solicitation.

In Europe, this document is issued by Aviva Investors Luxembourg S.A. Registered Office: 2 rue du Fort Bourbon, 1st Floor, 1249 Luxembourg. Supervised by Commission de Surveillance du Secteur Financier. An Aviva company. In the UK, this document is by Aviva Investors Global Services Limited. Registered in England No. 1151805. Registered Office: 80 Fenchurch Street, London, EC3M 4AE. Authorised and regulated by the Financial Conduct Authority. Firm Reference No. 119178. In Switzerland, this document is issued by Aviva Investors Schweiz GmbH.

In Singapore, this material is being circulated by way of an arrangement with Aviva Investors Asia Pte. Limited (AIAPL) for distribution to institutional investors only. Please note that AIAPL does not provide any independent research or analysis in the substance or preparation of this material. Recipients of this material are to contact AIAPL in respect of any matters arising from, or in connection with, this material. AIAPL, a company incorporated under the laws of Singapore with registration number 200813519W, holds a valid Capital Markets Services Licence to carry out fund management activities issued under the Securities and Futures Act (Singapore Statute Cap. 289) and Asian Exempt Financial Adviser for the purposes of the Financial Advisers Act (Singapore Statute Cap.110). Registered Office: 138 Market Street, #05-01 CapitaGreen, Singapore 048946.

In Australia, this material is being circulated by way of an arrangement with Aviva Investors Pacific Pty Ltd (AIPPL) for distribution to wholesale investors only. Please note that AIPPL does not provide any independent research or analysis in the substance or preparation of this material. Recipients of this material are to contact AIPPL in respect of any matters arising from, or in connection with, this material. AIPPL, a company incorporated under the laws of Australia with Australian Business No. 87 153 200 278 and Australian Company No. 153 200 278, holds an Australian Financial Services License (AFSL 411458) issued by the Australian Securities and Investments Commission. Business address: Level 27, 101 Collins Street, Melbourne, VIC 3000, Australia.

The name “Aviva Investors” as used in this material refers to the global organization of affiliated asset management businesses operating under the Aviva Investors name. Each Aviva investors’ affiliate is a subsidiary of Aviva plc, a publicly- traded multi-national financial services company headquartered in the United Kingdom.

Aviva Investors Canada, Inc. (“AIC”) is located in Toronto and is based within the North American region of the global organization of affiliated asset management businesses operating under the Aviva Investors name. AIC is registered with the Ontario Securities Commission as a commodity trading manager, exempt market dealer, portfolio manager and investment fund manager. AIC is also registered as an exempt market dealer and portfolio manager in each province of Canada and may also be registered as an investment fund manager in certain other applicable provinces.

Aviva Investors Americas LLC is a federally registered investment advisor with the U.S. Securities and Exchange Commission. Aviva Investors Americas is also a commodity trading advisor (“CTA”) registered with the Commodity Futures Trading Commission (“CFTC”) and is a member of the National Futures Association (“NFA”). AIA’s Form ADV Part 2A, which provides background information about the firm and its business practices, is available upon written request to: Compliance Department, 225 West Wacker Drive, Suite 2250, Chicago, IL 60606.