What would an algorithm tell us if we asked it to solve the climate crisis? It is a nice idea to think we could outsource such a gigantic problem to a machine. In fact, artificial intelligence experts have already started to ask this very question.1
Google famously used computing power to improve its energy efficiency.2 However, while incremental progress will be made through endeavours such as this, the quality of the data will likely hamper its success: as the saying goes, “rubbish in, rubbish out”.
In this AIQ article, we look at the potential for all forms of data – big, micro and alternative – to be translated into meaningful information and then collected and presented in a way that encourages tangible, action-driven outcomes. Challenges arise over privacy, ownership and responsibility. While individuals have a clear role to play in shifting consumer behaviour to a more sustainable footing, we expend the bulk of our attention on governments and companies.
Challenges of collecting data
Let’s start with the difficulties in obtaining the right data from companies, and the reasons why there is a growing clamour for mandatory rather than voluntary reporting. According to the Smith School of Enterprise and the Environment at Oxford University, levels of emissions disclosures range from 67 per cent of large companies in developed markets to just 25 per cent of small firms in emerging economies.3 Climate Action 100+, an investor initiative to pressure the world’s largest greenhouse-gas emitters to take action on climate change, states companies in the oil and gas sector are among the least transparent – only 63 per cent report their emissions data to CDP, a climate-change data collection and assessment programme, and just 38 per cent conduct and report on climate scenario planning.4
A company is rationally going to operate within the laws it has
Glen Peters, research director at the Center for International Climate Research (CICERO) in Oslo, believes this gap in reporting needs to be filled by regulation. “At the end of the day, a company is rationally going to operate within the laws it has, unless it sees some competitive advantage in going faster than what regulation may require. So, the political/regulatory framework is pretty important, and one thing is to try and push it towards putting stronger limitations on CO2 emissions and so on, which then forces all companies to act.”
Experience also suggests pressure from investors, civil society and activist groups is effective. Participants at the Smith School of Enterprise and the Environment’s 7th Sustainable Finance Forum argued: “All of these groups could drive progress by asking more of the right questions of companies on topics of sustainable supply chains.” They added the use of sensory data was “potentially transformative for the sustainable finance system, with investors and activists able to use data to drive change in corporate practices, revolutionise corporate reporting, track natural capital and improve supply chain management”.5 (More on this later.) The California Public Employees’ Retirement System (CalPERS), the largest public pension plan in the US, which manages pensions for more than two million California public employees, retirees, and their families, was a founding member of Climate Action 100+, and an early supporter of the Task Force on Climate-related Financial Disclosures (TCFD). Anne Simpson, director of board governance and strategy at CalPERS, agrees the voluntary nature of reporting is failing.
“Our view as a global institutional investor is that these types of sustainability data need to be included in mandatory reporting. The reason for that is to ensure consistency, reliability, integration with the financials, and to allow us to make comparisons between companies, sectors and over time. Right now, we just can’t do that,” she says. “You can treat one company as a proxy for another in theory, but in practice it’s really giving you a very unsatisfactory view. We can put together a fancy model or an algorithm, but really this is not acceptable to us as fiduciaries.”
Paul Lacoursiere, global head of environmental, social and governance (ESG) research at Aviva Investors, has the same view.
Not all companies disclose even basic environmental and social metrics
“The disclosure of ESG metrics is nowhere near the level of disclosure of financial metrics. It’s not a requirement, and without a clear deadline analogous to what is required for financial reporting, ESG metrics tend to be significantly lagged,” he says. For investors trying to analyse relevant ESG information over a one-year period, Lacoursiere estimates only 20 to 30 per cent would be available. “At this point, not all companies disclose even basic environmental and social metrics.”
In fact, ESG ratings provided by external providers are nowhere near as reliable as standard credit ratings, from both an information accuracy and a rating consistency standpoint. “MSCI, for example, rates most of the opportunity set,” explains Lacoursiere. “If a company does not disclose a metric, MSCI applies some form of industry related average estimate – in this scenario a company can choose to disclose only its strongest metrics, because it knows MSCI will give it an average grade for metrics it doesn’t disclose. I am uncomfortable with that.”
Lacoursiere would like to see an ESG data framework develop, “as in financial reporting – where there is a base set of statistics under each of the categories required, and the companies must put resources behind measuring and disclosing those metrics, so that it’s a level playing field from one company to the next”.
The TCFD aims to provide more guidance on emissions reporting through its Greenhouse Gas Protocol’s Scope 1, 2 and 3 framework (with Scope 3 including upstream and downstream emissions). Yet it is still not enough, according to Lacoursiere.
The TCFD guidelines provide a lot of flexibility around how that report is compiled
“We need more, simply because the TCFD guidelines provide a lot of flexibility around how that report is compiled. So, if every company is using a different methodology, it is not the same as having an accounting standard,” he says.
Not everyone has such a dim view of the progress being made. Fiona Reynolds, CEO of the UN-backed Principles for Responsible Investment (PRI), says PRI is making TCFD-reporting mandatory for signatories from 2020. She argues the TCFD is, slowly but surely, leading to increased harmonisation.
“Scenario analysis – TCFD’s forward-looking element – has been critical in providing both investors and companies with a view of the future and an understanding of how they will be impacted by the transition. This has led, and will continue to lead, to more informed decision making and an understanding of how to align with the goals of the Paris Agreement,” she says.
Led by CalPERS, investor group Climate Action 100+ also works extensively with organisations focused on corporate climate performance – the Carbon Tracker Initiative, CDP, InfluenceMap, the Transition Pathway Initiative and the 2° Investing Initiative – having set up a technical advisory group to develop a relevant set of indicators and analytical frameworks, including science-based emissions targets.6
Figure 1: Core elements of recommended climate-related financial disclosures

The difficulties of harmonising reporting
One explanation for the lack of guidance and harmonisation is the difficulty in coming up with measures that are relevant across industries in the way accounting measures can be. The Sustainability Accounting Standards Board (SASB) is working on it, as are the TCFD and other environmental standards bodies. However, Lacoursiere argues that even if developing common metrics for emissions is achievable, measuring risk is more complicated.
The lack of a coherent, compulsory reporting framework may also be stopping the most exposed firms from fully disclosing their risks
“If you extend [the framework] to ‘what’s your Value-at-Risk due to climate change?’, then the underlying analytics become opaque, so the second phase would be more difficult,” he says. “If you went to three different service providers and asked them to give you that statistic, and didn’t let them talk to each other, you would get three very different numbers back.”
In the meantime, the lack of a coherent, compulsory reporting framework may also be stopping the most exposed firms from fully disclosing their risks, as they could put themselves at a competitive disadvantage. CalPERS’ Simpson believes businesses have not improved their reporting because they believe there is a first-mover disadvantage. “If you say, ‘Wow, look at all my risks and exposures’ – why would I do that when it’s not a requirement for all companies?” she says.
In the US and elsewhere, companies have a fear of liability too. “The TCFD is looking at physical risk, transition risk and litigation risk. Obviously, if a company discloses that it has some litigation risk it could be self-fulfilling, so we’ve seen hardly any disclosure in that area,” says Steve Waygood, chief responsible investment officer at Aviva Investors.
Similar differences in disclosure are found at a country level. “Developed countries have to report on a regular basis on their emissions, and there’s a quite detailed and standardised way they do that reporting,” says CICERO’s Peters. “But developing countries don’t have to do that, or they don’t have to do it so often, and that makes our life that much harder as researchers to figure out what’s going on.”
Competing frameworks
To solve these issues, a common, prescriptive regulatory framework is needed, but this still seems some way off. “Even though there are these initiatives, it’s all very unharmonised and un-comparable from a research perspective,” says Peters. “Companies or countries can basically say what they like and still get away with it. You can always find a way to define your emissions so that they’re going down.”
A common, prescriptive regulatory framework is needed, but this still seems some way off
Rhonda Brauer, a securities lawyer, ESG consultant and co-author of the Council of Institutional Investors’ Sustainability Reporting Frameworks: A Guide for CIOs,7 explains: “The reporting frameworks include, among others, CDP, the Global Reporting Initiative (GRI), the SASB and the TCFD.
“In terms of the ongoing efforts by the framework sponsors to align with each other, the most prominent example is the Corporate Reporting Dialogue’s ongoing two-year effort: the Better Alignment Project. While the participating frameworks have agreed on some common principles, we are not yet near a place where one or more of the actors will merge or leave the market,” Brauer adds.
The Financial Times in November noted these organisations are all trying to push their framework to become the one industry standard,8 making life difficult for those trying to report, and undermining the work to improve disclosures.
Rather than consolidating, harmonisation efforts seem to be needlessly multiplying
Rather than consolidating, harmonisation efforts seem to be needlessly multiplying. A group of central banks launched the Network for Greening the Financial System (NGFS) in April 2019 which, among other recommendations, urged for globally consistent disclosure; the World Benchmarking Alliance was set up in 2018 to show up the best and worst performers on the UN Sustainable Development Goals;9 and the European Union is working on a region-wide taxonomy to help companies and investors define relevant metrics for sustainability. Programmes also comprise the Impact Weighted Accounts Initiative and a project by the CFA Institute to establish standardised ESG reporting for asset managers, similar to the CFA’s Global Investment Performance Standards.
Lacoursiere is slightly more optimistic about convergence, particularly in Europe, where regulators have more appetite to force the issue. “Through the European Securities Market Authority, I think we’ll see convergence within the next five years. All the financial regulators within that network are looking at requiring asset owners to disclose those statistics, which will have a trickle-down effect to asset managers and then companies themselves. I’m a bit more sceptical we’ll see that in North America or in Asia in the next five years, [although] the trend is encouraging.”
The TCFD’s recommendations are a good illustration of the types of data companies should be reporting
Brauer agrees the onus is on regulators to force the issue. “If regulators played a more active role in standardising compulsory sustainability reporting, not only would it end the ‘survey fatigue’ about which companies complain, but it would also create a common playing field for the reporting,” she says. “The resulting disclosure could be audited by accounting and engineering firms, several of which are already active participants in the different framework organisations. Such audits should also help to respond to those who criticise some of the current voluntary disclosures as ‘greenwashing’.”10
Although frameworks remain under discussion, the TCFD’s recommendations are a good illustration of the types of data companies should be reporting and investors should be incorporating into their models.
Filling in the blanks: Could sensors and satellites solve the data dearth?
Until the lack of harmonisation is resolved, many are turning to technology to fill the information gap.
Investors are using big data to fill gaps so they have better information to support investment decisions
“Increasingly, investors are using big data to fill gaps so they have better information to support investment decisions,” says Lacoursiere. Yet there is a long way to go. “A lot of the things we’re talking about today are in their infancy, in terms of applying the datasets and machine learning and other analytic approaches to the problem. It’s not a well-developed practice at this point; it’s not something you can buy off the shelf.”
For instance, the US State of New Mexico recently partnered with Descartes Lab to monitor methane emissions to stop leaks and reduce its carbon footprint, but the level of precision needed remains beyond the best technology.11
Although the use of satellites, sensors and big data analytics to measure emissions and inform investment decisions is just beginning, much of the data and computer programmes that make it possible exist and are constantly improving. We also know how to use much of it already, from companies predicting consumer preferences to cities monitoring water quality. This could finally bring data-driven emissions regulation within reach.12
Whether to measure direct emissions from factories, across a company’s supply chain, or at a country level, an increasing number of options are emerging, including mobile data, big data analytics from online sources, satellite measures and data available from the plethora of sensors scattered around the world.
This could finally bring data-driven emissions regulation within reach
Government satellites, like the European Space Agency’s Sentinel 5P, and other observation missions, like the US Landsat Missions, measure greenhouse-gas emissions (among many other things) and make the data publicly available. They are complemented by commercial satellites – particularly the constellations of tiny satellites called “cube sats” – whose measures can be bought to analyse all manner of information and, increasingly, drone observations.13
For instance, air pollution can be measured through a combination of satellite data and local sensors, and AI analyses of satellite imagery of shipping routes can track and report illegal activities.14 Similarly, Planet Labs’ 200 Dove satellites use cameras and sensors to monitor the earth daily, with the aim of one day measuring emissions with precision. California is planning on using the Dove satellites to monitor its wildfires, as part of its plan to track greenhouse gases.15
This type of approach allows analysts and researchers to find and assess relevant data that does not
This type of approach allows analysts and researchers to find and assess relevant data that does not feature in companies’ or countries’ disclosure reports, such as litigation due to environmental damage, partnerships with clean or carbon-intensive energy providers, or speeches stating a firm’s commitment to better practices. These can be found through scraping (compiling information from online and offline sources) and crawling (using programmes to search across online sources).16 Using AI to sort and analyse the mass of information gathered could make sense of it without deploying armies of researchers.
“Each individual piece of data may not be that instructive, but when you pull them all together, then you can get a coherent story.
This is what we do to some extent when we try and estimate what emissions will be in 2019 before the year is complete,” says Peters.
In a recent IPE article, Ben Caldecott, director of the Oxford Sustainable Finance Programme, argued this could be a game changer for investors. “These rapidly growing data mountains can then feed increasingly sophisticated predictive models to generate more and more insights and results.”17
Caldecott participated in the 2019 launch of the Spatial Finance Initiative, which aims to leverage satellite and other geospatial measurements and use AI to feed these into financial decision making.18 “This creates a significant opportunity for the financial services industry, including, but not limited to, the effective implementation of ESG practices.”
However, using the example of 18 countries that he and colleagues found to have reduced their carbon footprint between 2005 and 2015,19 Peters explains why this is not straightforward. “Typically, what the algorithm does is to find countries where emissions have gone down and economic activity has gone up over that ten-year period, but over ten years you can have a few economic ups and downs. Some of the ups and downs will relate to economic challenges. Each country will have something unique about it.”
To make sense of the data, researchers need to complement it with other relevant information
To make sense of the data, researchers need to complement it with other relevant information. Similarly, when using big or alternative data, investors must first determine what to look for, and then how to interpret their findings.
Other gaps remain. Data found online can be biased – for instance, if an issue causes noise on social media without necessarily representing the facts. Physical data gleaned through satellites or sensors still often lacks precision, and raises privacy issues when it comes to exploring a company or country’s activities (including the impact of imported emissions).
Bringing everything together
Whether using reporting, big data, interviewing companies directly or taking a mix of sources, the first step is to formalise the necessary data. Lacoursiere gives the example of financial analysts trying to estimate company earnings ahead of announcements, for instance by tracking a retail company’s mall footfall, or using satellite imagery to measure activity levels at a mining company’s various sites.
Instead of estimating foot traffic and purchases you are estimating carbon outputs or energy consumption
“There’s an analogy to that in the climate area, where instead of estimating foot traffic and purchases you are estimating carbon outputs or energy consumption. The framework exists, but I don’t think it’s been rigorously applied to estimating things that we know influence climate change.”
At a country level, tools like the UN’s Emissions Gap Report are making it possible to track reported emissions against countries’ or companies’ commitments.20 However, understanding those commitments can be challenging because some are – perhaps deliberately – ambiguous.
“There are various ways to manipulate data, in a justifiable way, that can make it easier or harder to get to a target,” says Peters. Until that happens, if big data can help refine and improve the accuracy of estimates where companies don’t disclose, it will make it much easier to see through cases of greenwashing.
“Greenwashing is about transparency: the more granularity we have as it pertains to the datasets, the more we can check that. A common language and better inputs should help,” says Francois de Bruin, sustainable income and growth portfolio manager at Aviva Investors.
Peters concurs, to a degree. “You can probably go quite some way by mining data, but to get the structures in place to collect that data can be extremely difficult. You can mine data that’s out there, but the job would be a hell of a lot easier if the appropriate data was collected and reported,” he says.
To get the structures in place to collect that data can be extremely difficult
He adds it is difficult, although not impossible, to check for data manipulation. “This comes back to data mining. You’re essentially looking for inconsistencies in data; for example, if China says its coal consumption went down while statistics of steel production, cement production, electricity generation show ten per cent growth. I guess accountants and tax officers probably do something similar for companies, essentially trying to find inconsistencies in data streams.”
Lacoursiere thinks machine learning could also improve investors’ and companies’ climate-risk scenarios, as it already does for weather forecasting. “It’s an interesting relationship. Historically, a given set of atmospheric conditions would result in predictable weather patterns, but those relationships are changing. Neural networks are very good at identifying changing relationships, so they are naturally suited to the problem.”
Human input will always remain indispensable
He and de Bruin agree human input will always remain indispensable, however. “It has to be both, because there is no defined set of rules an algorithm can simply follow and then decide on our behalf. Interpretation is required,” says de Bruin.
In 2018, the United Nations’ Exponential Climate Action Roadmap said the question now was, “How do we provide governments, businesses and citizens with shared roadmaps that show the way, which can be defined and redefined as we go?”
The report argued all the data needed to create these roadmaps was already accessible, from government policies to public emission statistics and published research. However, the data remains scattered and needs to be collated, analysed and presented under a common framework. “By methodically presenting open data in this way, we can see exactly where we stand, focus on the right actions, hold stakeholders accountable and spread best practice. In doing so, roadmaps become vital tools that can be used to drive action and guide strategies.”21
Participants at the summit are working on this, and have since published another two roadmaps, the latest presenting 36 solutions to halve emissions by 2030.22 Many show how big data, technology such as sensors and satellites, and AI can radically change carbon emissions:
- Solar, wind, storage and smart grid technology supported by digital solutions will enable electrification, decentralisation and greening of the energy system.
- Digitalisation can improve delivery by optimising shipments, routes and traffic systems.
- Deforestation can now be predicted and detected through digital solutions, which helps form the basis for proactive action through monitoring and improving agriculture, reforestation and peatland restoration.
- Through the use of Internet of Things, AI, 5G and digital-twin technology, the need for more roads and physical infrastructure can be dramatically reduced through optimising existing infrastructure.
Political will is also needed, as well as greater ownership and accountability by companies and investors, among other stakeholders.
Figure 2: Corporate credit and equities warming potential (in°C) for Aviva’s shareholder funds

Another example in the UN Climate Action Summit 2018 report was on methane leaks. It said: “Solutions to reduce a significant portion of this leakage are generally available, and many are profitable, but aren’t applied at a large scale because return on investment is considered too low. Stronger policy and better monitoring techniques can help close the gap. In particular, the technology industry can play a leading role here, through the use of drones and AI to detect leaks, and to help make sense of the large amounts of data already available.”
Beyond mandatory reporting by companies, de Bruin believes “there has to be a framework for opinion and the only way you can get companies to go the extra mile is to create incentives”.
Meanwhile, Simpson is adamant that accountability is key. “What we’re calling for at Climate Action 100+ is: First, companies need to take responsibility and support the energy transition – and that means being accountable; second, they need to set targets to support the goals of the Paris Agreement; third, companies should all report under the TCFD framework,” she says.
If demand for oil doesn't go down, Shell can't plan to put itself out of business
“Let’s take a company like Shell. Thanks to Climate Action 100+, it has agreed to take responsibility for its Scope 3 emissions. […] The big emissions come when Shell passes the refined product onto its customers, that’s Scope 3: utilities, transport, transportation, airlines and so on. So, by Shell taking responsibility for Scope 3, it means that, as an oil company, it needs to be in a dialogue with other sectors of the economy. Because if demand for oil doesn’t go down, Shell can’t plan to put itself out of business.”
Investors will play a key role
“The free lunch in climate change is akin to free fast food for lunch,” says de Bruin. “There is a near-term benefit (low cost) and it meets immediate needs, but longer term you’re better off paying for sustainable food groups. As the benefits of disclosure become clear, it should create a virtuous cycle of continued reinvestment; a bit like paying for a healthy, balanced diet.”
Investors are increasingly realising this, which is a powerful driver in the search for more and better data. Waygood explains how Aviva compared its portfolios’ carbon footprint to that of the London Stock Exchange.23 “You can see the FTSE 100 is about 3.9 degrees. Our portfolio is better, but still about 3.4 degrees, so that gives you a sense of the gap.”
ESG risks are integral to the risk framework
De Bruin explains how this has changed investors’ approach. “As any credit analyst will tell you, ESG risks are integral to the risk framework. These are all material risks that are in your framework already, so it’s just that we are now explicitly putting it into a category. We are now finally shining the spotlight on exactly what it is that needs to be addressed.”
Interestingly, this spotlight is also influencing bondholders’ perceived responsibility to engage with companies. “I do think, historically, shareholders would have been seen as the flag bearers, but now, increasingly, debt investors have a role to play,” says de Bruin. “Bondholders can say, ‘We’re not refinancing this time round because we want to see certain levels of disclosure’. Essentially, they are the gatekeepers of the capital.”
He also believes large investment firms can use their own data resources to share information and engage with smaller companies. “If they are smaller, they might not have the resources to even consider these decisions. As investors, we might be able to say, ‘This is what we are seeing globally, have you considered this and this?’ It’s a conversation.”
Big data can help investors better understand issues and trade offs in emerging markets
Similarly, big data can help investors better understand issues and trade offs in emerging markets, and to then engage with companies locally, which are keen to discuss the climate transition. “From the initial letters that we have sent, we have had responses not from the investor relations teams, but from the CEOs themselves. They want to engage on this topic, they know it’s pertinent,” says de Bruin.
Better data – and particularly ESG data – also creates investment opportunities. By using an open-architecture framework, de Bruin explains, “If I’m dealing with supplier issues in toys in the US and South America, I can learn from those in Asia”.
On the equities side, Trevor Green, UK equities portfolio manager at Aviva Investors, adds: “One of the things we’ve really progressed with this year is looking at the change in the direction of the [ESG] score because that’s the key. It’s not about absolute levels, [it’s about] who’s moving in a particular direction. That brings new investment ideas to light.”
Lacoursiere believes the next steps should be companies analysing their business models and value chains using the same approach asset managers are taking today, albeit as practitioners rather than evaluators. “If a company isn’t using this info to help make its capex investment decisions, more of them will go wrong. That will just become more apparent as time goes by, and the market will make it more apparent which companies are doing well and which aren’t,” he says.
“Ultimately, I would like to see a world in which there is no difference between ESG and non-ESG investing, as more financial market participants come to realise the significant financial impact of so-called ‘non-financial’ information,” adds Brauer.
Figure 3: Integrating climate risk into investment considerations

Telling the truth
Of the three demands of climate activist group Extinction Rebellion, the first is “Tell the truth”. While not everyone endorses all their views, and indeed actions, this maxim has neat parallels with the debate on data and the desperate need for transparency.
“We think that when people understand the emergency that we are in, then they will actually be pushing and supportive of some sort of emergency mobilisation,” says Andrew Medhurst, who leads the UK national finance working group at Extinction Rebellion.
It is crucial for investors to assess climate risk when allocating capital
The effects of climate change will have a growing impact on individuals, companies and the economy. As such, it is crucial for investors to assess climate risk when allocating capital. The investment community is slowly but surely heading towards an agreement on compulsory reporting standards. However, there will always be a need for richer and more up-to-date information to augment it, as there are limits to what companies and countries can, and are willing to, disclose.
According to Simpson at CalPERS, the issue is cut and dried. “What we’ve done on the climate change agenda is to take emissions reduction as our priority. The reason for that is simple: we’re on a timetable. The clock is ticking, and we can’t be fiddling while Rome burns, in this case almost literally.”
It is a complex task. Given how much is at stake for investors – whether they are passionate about saving the planet or simply looking out for their financial interests – the need to optimise environmental data collection and analysis has become one of their most pressing objectives.