Many investors use controversy scores as a filter to avoid firms whose damaging behaviour has hit the headlines, from human rights violations to environmental disasters. But these scores have serious limitations, making them an inadequate tool for investors who want to manage ESG risks and have a positive impact.
Fumaroles are a good indicator of whether a volcano is active or dormant. Yet vulcanologists cannot rely on fumaroles alone to assess the probability of an eruption. The scientists need to sound deep under the crust to determine the size and pressure of the magma chamber, and their measures need to be dynamic, as magma can build up or ebb away over time. Volcanic eruptions are a complex phenomenon and can only be forecast a few days in advance at best, even by experts.
The same is true for controversy scores, a popular tool used by investors to try to capture the extent to which companies are involved in historical or ongoing sustainability-linked scandals. But just like fumaroles, which only represent the residue of a recent eruption, controversy scores tend to reflect corporate eruptions that have already hit the newswires, or even long since passed.
Nor do all scandals necessarily indicate the same level of danger for the people and places harmed by a company’s activities. As an indicator of future events, controversy scores cannot explain what is really happening under the surface; how deep or broad the issues are, whether they are getting worse or being resolved. Companies’ executive teams can also be skilled at hiding or refuting the negative impacts, which can make issues even harder to detect. For example, 7,500 people die every day due to poor working conditions, 6,500 due to diseases contracted at their workplace and 1,000 from accidents and injuries.1 But not all these end up in the headlines, and therefore not all are captured in controversy scores for the company in question.
Controversies are so controversial in the research space
“Controversies are so controversial in the research space,” says Dan Neale, social transformation lead at the World Benchmarking Alliance (WBA), an organisation set up to rank 2,000 global companies on their contribution towards delivering the Sustainable Development Goals. This includes the Corporate Human Rights Benchmark (CHRB), which is the first open and public benchmark on the human rights performance of 400 major companies.
A big hurdle when CHRB was set up was deciding how to use controversies in its approach. “In the methodology, controversies are easily the most contentious part, even down to the language we use,” says Neale. “For example, the CHRB looks at ‘allegations’, in part as that immediately puts us into a better legal position. CHRB can publicly say a company is alleged to have done something and assess how the firm responds, rather than assessing the severity of an (argued or unproven) impact. You may think it’s true, they may know it’s true, but you don’t have to make any sort of judgment call on the truth of the controversy by dealing with an allegation.”
Yet for investments and volcanoes alike, although a past eruption can leave desolation in its wake and take years to recover from, the biggest danger lies in the potential for future eruptions. Too many investors seem to rely solely on fumarole-like controversy scores to decide whether to exclude a company, believing it protects them from exposure to firms with poor sustainability behaviours. Instead, their portfolio might exclude companies based on historic events without accounting for progress in remedying negative impacts, while potentially ignoring dangerous magma chambers forming at companies with a clean reputation.
An explainer: How the ESG scoring systems work
MSCI’s ESG Controversies Score uses a 0-10 scale, with 0 representing the most severe controversy. Out of 6,969 companies in its dataset, only 28 companies have a score of 0, which equates to a fail. A further 138 companies have a score of 1, which puts them on the watchlist.
The ISS ESG Controversies Score is also based on a 0-10 scale; however, this time 10 represents the most severe controversy. Out of 3,820 companies, 107 companies have a score of 10 (also flagged as red), 659 have a score between 6 and 9 (flagged as amber) and 3,054 have a score between 1 and 5 (flagged as green).
Too much reliance on too little information
Marte Borhaug, global head of sustainable outcomes at Aviva Investors, believes taking controversy scores at face value is a dangerous shortcut. It could also become an issue for sustainable funds, as the market may lose credibility if some funds are tainted because of poorly researched investments.
Taking controversy scores at face value is a dangerous shortcut
“As a customer, when you are looking to buy a fund that markets itself as sustainable – for example a climate transition fund or a social impact fund – you might think it has conducted in-depth analysis and presume your pension or savings are not invested in companies that are contributing negatively to these issues. This might not be the case if the fund relies purely on numerical data such as controversy scores,” she says.
Borhaug sees risks in using a methodology that only relies on one or a few datasets to exclude or include companies. “There is a high risk of clients discovering companies they didn’t expect to be there in the first place. There could be a big gap between perception and portfolio reality,” she says.
Lack of consistency
First and foremost, the lack of consistency in approach by the companies providing controversy scores is stark. Which begs the question: how can investors tell which is ‘right’ and which is not?
For example, only seven companies are flagged for controversies in both datasets of two of the biggest ratings providers, MSCI and ISS (including Shell and phosphate rock miner OCP). This is because their scoring methodologies measure different things, so names often appear on one list but not the other.
Figure 1: Overlap between MSCI and ISS controversy scores
Borhaug says the lack of consistency between ESG scores in general, and controversy scores in particular, is a longstanding challenge, in stark contrast to more traditional financial reporting such as a firms’ credit rating.
“We could compare financial information from two companies using different providers and we would speak the same language, since they are analysed in a similar way. But with ESG data, it is a totally different story.”
You can’t just rely on one dataset. That would be washing our hands of our responsibilities
To give an example, one provider’s score might look at a company’s impact on people, not just the risks the company is facing, while other providers only focus on material factors that are likely to hit the share price.
“This makes it hard for an outsider – like a financial adviser or the end client – to rely on a score and make comparisons,” explains Borhaug. “You can’t just rely on one dataset. That would be washing our hands of our responsibilities.”
Although every approach, from MSCI to ISS, has its merits, the differences make it more important for investors to develop the resources and expertise to complement data providers’ analyses and bring a different perspective.
Hiding in plain sight?
In MSCI’s controversy data, only 28 companies are tagged with a ‘fail’, versus 107 companies in the ISS datasets. Considering the amount of companies out there, even that seems small.
Scores are backward looking rather than flagging companies at risk of becoming embroiled in a scandal
Part of the issue is that the scores do not account for hidden controversies. Instead, they typically reflect headline-making issues, while ignoring lower-profile violations that have not yet been picked up externally. As mentioned, the scores are also backward looking, identifying controversies that have happened rather than flagging companies at risk of becoming embroiled in a scandal.
“Controversy scores give investors the comfort they are not invested in a company that’s currently in the headlines. However, we know there are a lot of controversies going on every day in supply chains and companies that haven’t hit the headlines yet,” says Borhaug. “As investors, we shouldn’t just care about what was in the press today or last year, but focus on how we can limit the negative impact companies have on the people and ecosystems affected by their operations.”
Delayed reward for taking action
Moreover, the scores do not always account for steps taken by companies to improve and remediate issues in a timely manner, which are crucial to assessing a company’s future performance. For example, the legacy of the 2015 ‘Dieselgate’ scandal that rocked German car giant Volkswagen lives on today; the company still has a zero MSCI controversy score despite having made significant organisational changes to fix the issues uncovered. Data providers assess how firms deal with the controversies that affect them, but their judgement may differ between one another as well as between fund managers.
Recognition of a company’s improvement can be hindered by the time it takes for cases to come to court or reach a settlement
Richard Butters, ESG analyst at Aviva Investors, explains that for Volkswagen this is because MSCI scores require a company to post a clean slate (zero controversies) for at least two years before resetting its score. When it comes to controversies with a legal dimension, recognition of a company’s improvement can be hindered by the time it takes for cases to come to court or reach a settlement.
Another example is Wells Fargo, a bank that has had a low controversy score for several years due to past mis-selling scandals. “There has been a complete overhaul of the company’s board, management and processes since 2017. There are indications Wells Fargo is on a path to see the Federal Reserve lift the asset cap it had imposed on the company. Yet the focus of ongoing controversy scores relates to headlines on fines targeting former management, on a personal rather than company basis,” says Butters.
Painting a full picture is no easy task, which helps explains why scores are not more comprehensive and consistent.
The complexity in creating scores
The WBA’s Neale explains that researching allegations is resource intensive. “We are talking about some companies with operations in 100 countries and supply chains in 150 countries; you are never going to be able to get to the whole truth of that, so you have to take an awful lot on trust,” he says.
‘David and Goliath’ situations are also common, where a small NGO may hesitate to raise an allegation against an industry behemoth capable of drowning it in lawsuits. “You’ve only got to look at the Texaco Chevron Ecuador case.2 It has been running for years, and Chevron is looking at a $2 billion legal defence with 60 law firms working on it. This is the kind of trouble you don’t want to get into when you are a small not-for-profit, or even a big research provider,” notes Neale.
Quantifying impacts, controversies or responses to controversies is necessary for creating rankings
He says quantifying harm and comparing events such as deaths, illnesses or sexual harassment is a complex area in itself. “You don’t want to be the people that sit there and say: ‘300 cases of sexual harassment are equivalent to four fatalities’. But quantifying impacts, controversies or responses to controversies is, in some way, necessary for creating rankings. If you’re going to be bold and step into that area, it helps to be transparent on what approach you’re using,” he says.
Furthermore, remedial action is often hard to measure and unlikely to be satisfactory to victims. Given the dire consequences of some controversies, even significant financial compensation may not be deemed satisfactory by the people who were hurt or their families.
“Of the controversies we look at, maybe only three per cent show the remedy was satisfactory to the impacted individuals. It’s impossible to provide a remedy when people are dead, or if you have fundamentally destroyed their livelihood or where they used to live,” says Neale.
Looking at controversy scores in the context of existing cases can illustrate these limitations and shed light on questions investors should consider when it comes to protecting portfolios.
Digging deeper: Case studies and share price signals
As mentioned, controversy scores tend to reflect the past and underestimate corrective action. For example, in 2019, the MSCI rating of consumer-credit reporting agency Equifax was affected for some time by the legacy of a prior cybersecurity controversy. Mikhail Zverev, head of global equities at Aviva Investors, notes the rating remained low despite the company significantly changing its cybersecurity practices and building a more robust IT platform than industry peers. “That is where you really need to dig deep to understand a company’s business. The simplistic application of an MSCI ratings overlay doesn’t work,” he says.
Boeing versus Airbus: Similar scores despite different remedial actions
As measured by ESG Elements, Aviva Investors’ in-house quantitative ESG score, which is based on several MSCI indicators alongside proprietary ones, Boeing and Airbus have low scores, mainly due to past controversies. However, when qualitative analysis is added to the process, differences between the two companies are apparent.
“The quantitative score makes it difficult to give you any indication about the direction companies are going in because the low score by itself suggests they are the worst in the sector,” says Malini Chauhan, ESG sector analyst at Aviva Investors.
The Airbus controversy stemmed from endemic bribery
The Airbus controversy stemmed from endemic bribery within its commercial aircraft business spanning more than ten years, but the company has since taken significant steps to mitigate against this happening again.
“Airbus paid the €3.6 billion fine, admitted its wrongdoing and worked with the authorities extensively, but there will always be residual effects in terms of court hearings and the association it has to the issue. So MSCI still sees the controversy as ongoing,” says Chauhan. “Instead, for a financial analyst, once the fine has been paid, that draws the line under matters. What we then need to understand is how the company is going to change and where the score can go after this.”
Boeing’s controversy sprang from the tragic loss of life in two crashes of its 737 MAX aircraft
Meanwhile, Boeing’s controversy sprang from the tragic loss of life in two crashes of its 737 MAX aircraft, which uncovered safety issues and deep-rooted cultural problems. While the planes have now been re-certified, Chauhan says cultural issues continue to pose a risk, both to investors and other stakeholders.
Boeing has not acknowledged the 737 MAX shortcomings stemmed from a company-wide issue, pinning them instead on two former employees. It said in a statement in January this year: “The agreement [with the Department of Justice] is based on the conduct of two former Boeing employees and their intentional failure to inform the FAA Aircraft Evaluation Group about changes to the manoeuvring characteristics augmentation system.”3
Illustrating the difference between the two companies, while Airbus made significant management changes following the bribery scandal, including appointing a new CEO4 and CFO,5 Boeing has largely kept the same management team.
While Airbus made significant management changes, Boeing has largely kept the same management team
“Boeing’s CEO and chairman were on the board of directors during the entire plane development and the board didn’t provide effective oversight – now they are in charge of leading the change. There are also multiple board directors who were there through the process and are still on the board today; I don’t feel they provided adequate oversight to the 737 MAX development,” notes Chauhan.
The stark contrast between the two companies’ responses demonstrates the importance of remedial action to mitigate the risk of reoccurrence of incidents with tremendous negative social impact – and shows why investors need to look beyond scores to assess this for themselves at a qualitative level.
Controversies and share prices
A reasonable question for investors to ask is the extent to which controversies filter through to the share prices of the companies involved. The answer, unfortunately, is not clear cut as the correlation between share prices and controversies varies and seems dependent on many factors.
The materiality of a controversy is dependent on the firm’s business model
However, the materiality of a controversy is dependent on the firm’s business model, making it doubly important to look beyond the score and into the nature of the controversy to assess its potential impact on valuation. For example, a failure linked to the core product of the company, such as an oil spill for an oil company, a medical defect for a pharmaceutical company or an engine failure for a car company are likely to hit harder than a failure at the margin.
According to research by Bank of America Merrill Lynch6, the impact of ESG controversies on share prices of companies in the S&P 500 is stark: 24 ESG controversies wiped $534 billion off the value of large US companies over a five-year period starting in 2014.
Figure 2: The impact of controversies on share prices
Meanwhile, research from Société Générale found that two-thirds of companies hit by a major controversy saw their stock price underperform the MSCI World Index by an average 12 per cent in each of the following two years and stocks typically lagged their regional benchmark by four per cent.7 The research, based on 80 past ESG controversies dating back to 2005 and spanning various regions and sectors, identified 12 companies that experienced a ‘high’ ESG controversy in the last year, all of which have underperformed since.
Market reactions are much more pronounced for bad than good news
Furthermore, a paper conducted by Monash University8 in Melbourne, Australia, highlighted market reactions are much more pronounced for bad than good news, as investors overweight the possibility the impact of the controversy could be prolonged or happen again.
In particular, the study – which focused on the S&P Composite 1500 Index – found a clear negative abnormal return when firms are subject to bad news, but no clear pattern around positive news. The negative returns are substantially larger in magnitude for the smallest stocks and those held by more transient investors.
There is also the possibility of information leakage ahead of bad news events (shown in Figure 3), as cumulative abnormal returns begin occurring several days before the issue becomes public.
Figure 3: Cumulative average abnormal returns of ESG events
In contrast, the share prices or borrowing costs for some companies almost seem immune to controversies. For instance, tech companies have not always made headlines for the right reasons in recent years, yet their share prices hardly dip when a negative story is revealed, and quickly bounce back. Could the tech sector be an exception?
Building a mosaic of insights to glimpse at the truth
Creating a fool-proof system to identify companies and sectors more immune to controversies than others is unrealistic. However, useful insights can be gleaned by patching together different information sources.
Information from independent sources is essential
When it comes to companies’ impact on people and the planet, information from independent sources, such as research from civil society organisations, is essential. These organisations often operate on the ground, close to companies’ operations, and provide insight into current practices that may yet hit the headlines, providing more depth and granularity to controversy scores.
“We will look into what the data providers have told us, whether through talking to the company, desk-based research, or talking to NGOs on the ground, to verify their statements. Using company employees, whistle-blowers, and that kind of bottom-up information is hugely important,” says Borhaug.
She illustrates this with two examples. “In 2020, during the first lockdowns, the Business & Human Rights Resource Centre alerted investors to unethical and exploitative recruitment practices imposed on many migrant workers in the Gulf states,” she says. “Also last year, research from the Zoological Society of London showed that 54 per cent of the biggest tropical timber and pulp companies do not publicly commit to protecting biodiversity and 44 per cent have yet to publicly commit to zero-deforestation.”9
This breadth and depth of information is essential for the credibility of company rankings. For instance, although the CHRB bases all its research on publicly available data, Neale explains it also uses partners to find allegations and screen them for severity, from public resources like the Business & Human Rights Resource Centre’s allegations database to third-party research organisations covering stories in multiple languages like RepRisk.10,11
Having the information at hand today can also allow investors to push for change
Moreover, the information is not just a proxy indicative of future controversies; having the information at hand today can also allow investors to push for change.
“Using a broader range of information is part of the due diligence necessary to be a responsible investor,” says Borhaug. “The information we get can move us away from simply avoiding risk, or managing risk in the aftermath of a scandal, to preventing the issues happening in the first place. We engage with companies to ensure they understand the negative impact they are having and encourage them to take steps to reduce that impact. It’s not enough to simply adjust portfolios to sidestep a potential scandal.”
Neale from the CHRB also encourages investors to step up and hold firms accountable. “We find a lot of investors, even those using data from CHRB, are refusing to ‘put their vote where their money is’. I am constantly surprised by how much voting doesn’t get used. It is seen by some as a nuclear option of last resort rather than one of the tools investors have,” he says.
Intuition grounded in fact
Understanding a company’s controversy risk requires a combination of art and science. “In an ideal world, there would be sufficient quantitative data to easily aggregate and analyse, but that doesn’t always exist. Sometimes there is an element of intuition, and you just have to monitor the markets and look for warning signs,” notes Butters.
Understanding a company’s controversy risk requires a combination of art and science
As well as negative media reports, Twitter sentiment, emerging themes on employee site Glassdoor or online petitions can be useful indicators. “There is also huge potential in crowdsourcing platforms that support workers’ rights. One such platform called Organise was able to identify cultural issues at Ted Baker through its anonymised employee surveys, which led to the departure of the CEO,” says Butters.12
Yet to have an intuition in the first place, investors need to look at many other data points, and this is where controversy scores can have a role. “If there is a controversy, you tend to see it in a score. If there is a zero score, that would require due diligence,” he says.
Adjusting the importance of different factors according to how material they are in different industries can also help highlight relevant risks within companies – including those liable to affect returns. “In pharmaceuticals, you might look at product safety as a key area. Within financials, you might look at systemic risk or access to finance risks, and in industrials you might look at biodiversity risk,” explains Butters.
Investors’ voting records can also be an important indicator of persistent governance weakness at the companies they own. Many investors continue to view voting against management as a last resort, but then deprive themselves of a valuable tool for both analysis and influence.
Understanding the dynamics
Another key tool of influence and research comes through engagement. Face-to-face interaction can help investors gauge a firm in subtle ways desk-based research can never achieve.
When we speak to company executives, we don’t just assess them on strategy and management
“When we speak to company executives, we don’t just assess them on strategy and management. We assess them as individuals to get a sense of how they understand the risks that exist within their business and, more importantly, how dedicated they are to providing remedies,” says Butters.
Investors need to understand not just the implications of controversies, but also the change undertaken by companies after they have been hit by a scandal. If the remedies taken and progress made are significant, it can be rewarding to invest in such a company before the slow-moving controversy score begins to reflect the improvement.
“Understanding the implications of change is central to how we research companies. One thing the ESG boom has achieved is to make ESG factors capable of moving stock prices over shorter time horizons, so understanding an ESG change relative to the anchored MSCI ratings has become more rewarding, again emphasising the value of engagement,” says Zverev.
Challenging the data providers
Over the longer term, however, it is in all investors’ interest to see significant improvement in the data underlying controversy scores, particularly for those with smaller research and stewardship teams. To achieve this, asset managers need to challenge the data providers, encourage companies to report more information publicly and work collectively towards common standards.
“It is like two people rating someone’s attractiveness based on either a smile or a watch. They are both rating attractiveness but using very different criteria. It is the same with controversies,” says Neale.
Greater transparency from data providers is essential
Greater transparency from data providers is essential for investors to fully understand methodologies and more easily cross-check scores.13 “When providers have closed-box methodologies, it is a bit harder, especially as some are very complex. But if investors are going to spend all that money on data, it is also up to them to peek under the hood and see whether the provider’s approach equates to what they want to do,” he adds.
Borhaug compares the issue to that faced by credit-rating agencies after the global financial crisis. Equally, she doesn’t believe the data providers are the only ones to blame. Investors are also responsible.
“Part of the problem is the different investor demands and what they are asking data providers to do. Many investors find it easier to just buy a data set and apply it to their fund. We all play a role in holding everyone in the chain accountable for communicating correctly about sustainability issues,” she says.
Predicting corporate controversies will never be a perfect science. There is no model, algorithm or complete dataset available to help magic up an answer. However, just as vulcanologists searching for fumarole clues, investors can learn to better identify the tremors signalling insight from the ones that merely add to the informational noise.
“All models are wrong, but some models are useful,” as statistician George E. Box once observed. Along with other ESG factors, the requirement for human judgement over controversies can give active investors an edge.