Potential unleashed: How companies can leverage new technologies for safe data sharing

In the 1980s, suppliers like Procter & Gamble knew little about their products’ sales at rising retail behemoth Walmart, other than when restock orders arrived. Around this time, however, Walmart began experimenting with sharing its extensive data through Retail Link, a proprietary software, that facilitated electronic data interchange (EDI). Soon, P&G had access to data from the moment it shipped an item to the retailer up until its sale at a Walmart register.

This new data trove, ranging from inventory levels and store-level sales data, transformed P&G’s understanding of its products and, for Walmart, allowed for better forecasting and inventory management. The collaboration led to a $50 million swing in profitabilityfor P&G within the first eight months and prompted Walmart to expand its data sharing offer to other suppliers. And from there, the rest is history.

A handful of other companies have also engaged in data sharing, generating significant value from it–and not just in retail. For example, the PC industry frequently shares data between original equipment manufacturers (OEMs), chip manufacturers, and software companies, as illustrated by Intel and Dell joining forces to build a safer supply chain. Data sharing is not limited to supply chains; it can also involve competitors. For example, in the automotive insurance industry, U.S. companies teamed up to fight fraud with a claim-history information exchange called LexisNexis CLUE Auto. Given the range and scale of industry challenges that data sharing can address, the OECD estimates its value at a whopping 2.5% of global GDP.

Despite that substantial potential value, however, most companies have remained resistant to data collaboration. Why don’t more firms share data with others to tackle big, industry-wide problems that they simply cannot solve alone? The main reason is executives’ lingering sense that sharing data is more trouble than it’s worth, posing operational and regulatory hurdles along with creating new strategic risks.

That thinking, as we have recently argued, is increasingly out of date, as technology has profoundly changed the risk-to-reward ratio involved in data sharing. Understanding what technology now makes possible should lead many executives to reconsider their stance on data sharing.

Easing operational hurdles

Many companies worry that they lack the internal expertise or digital infrastructure to make data sharing efficient and secure. The lack of standardized data formatting or universally accepted protocols for data exchange have made it even harder for companies to overcome these deficits.

Today, however, technology is making up for those shortcomings. Platforms like Databricks, a cloud-based data management company, now allow for secure storage, sharing, and comprehensive analysis of company datasets. While full data standardization has not yet taken place, the proliferation of application programming interfaces (APIs) marks a significant leap forward in interoperability. By bridging disparate platforms, systems, and applications, APIs have enabled a high degree of automation, such that data is now increasingly being used to its fullest potential.

The emergence of generative AI technologies will further facilitate data preparation and cleaning processes, enhancing the accuracy, completeness, and consistency of data while requiring significantly less human effort. By applying advanced algorithms to identify, correct, or remove inaccuracies and inconsistencies in datasets, generative AI can automate mundane and repetitive tasks involved in data management, freeing up valuable resources for companies.

Easing regulatory compliance

The age of AI has led to increasing sensitivity around data privacy. Indeed, strict regulations like the European Union's General Data Protection Regulation (GDPR) or the California Consumer Privacy Act (CCPA) have dampened many organizations’ enthusiasm for data sharing. Technology, however, can help ensure compliance with data privacy rules and restrictions on cross-border data transfers.

New tools—for example, IBM’s Security Guardium—allow companies to pseudonymize or anonymize personal data, a significant step towards safeguarding individual privacy and allowing for the safe use of data in compliance with privacy laws. Sensitive data discovery tools now have the ability to scan and analyze organizations’ data repositories to uncover sensitive or confidential information hidden there. By employing these advanced scanning technologies, organizations can significantly mitigate the risk of unintentional data exposure or compliance violations.

The rise of AI-generated, synthetic data is also reducing the risks associated with handling sensitive data by generating artificial datasets that mimic the statistical properties of real-life datasets without using the actual data points themselves. Synthetic data-generation techniques are also difficult to reverse engineer, allowing companies to share relevant insights without compromising the confidentiality of raw personal data. The use of synthetic data is particularly promising for expanding the pool of data a business may use to train its AI models, even if it has limitations for other applications (such as personalization tailored to real customers).

Increasing reliance on synthetic data, however, will depend on how existing regulatory frameworks adapt to novel forms of AI-created data, as the current rules were designed with raw data in mind. Beyond data regulation, there are also antitrust concerns for companies entering into data-sharing agreements with one another. But past and present cooperation models have shown that data sharing aimed at solving industry-wide challenges can garner strong support from governments. For example, the Catena-X data exchange network used in the automotive industry is enthusiastically backed by the German government.

Redefining trust

Technology’s biggest impact has been the way it redefines the trust required for companies to collaborate and share their data. Modern software, for instance, has transformed data sharing agreements that govern data transactions, by making them easier to set up and to monitor the processes and policies that partners must implement to protect data.

Comprehensive logging systems have also made it possible to record every instance of access and action taken on data, creating a transparent and verifiable audit trail. These logs are integral to robust security frameworks, ensuring data remains both accessible for legitimate use and securely protected against illicit access.

The proliferation of virtual data rooms, like Snowflake’s Global Data Clean Room, allow partner data to be shared with specific permissions and restrictions, as well as tracking of data access and usage. Further, distributed ledger technology, such as blockchain, might also offer a secure and verifiable environment for the exchange of data. This technology should guarantee the integrity and traceability of data transactions, providing transparency and security in data sharing practices.

When put together, these tech advances make it easier to get data sharing off the ground, even among companies with little or no history of sustained cooperation. Trust is still required between companies to engage in data collaboration, but new tech and tools have made data sharing less of a leap of faith for executives worried that their data will be used against the company by competitors. Coupled with the ease of operational challenges and regulatory risks around privacy, technological advances have fundamentally upended companies’ previous risk-reward calculus.

Conclusion

The idea of a company sharing its coveted data—along a supply chain or even with competitors across its industry—is still a non-starter for many executives. But that doesn’t need to be the case. Technology has created new opportunities to collaborate to tackle complex issues, ranging from fraud detection to supply chain optimization and drug discovery— making it easier to capture more of the extraordinary value inherent in such partnerships. That’s why now is the time for executives to reassess their stance on data-sharing.

Read other Fortune columns by François Candelon. 

François Candelon is a managing director and senior partner of Boston Consulting Group and the global director of the BCG Henderson Institute (BHI). 

Riccarda Joas is a consultant at BCG and an Ambassador at the BCG Henderson Institute.

Guillaume Sajust de Bergues is a lead data scientist at BCG and an Ambassador at the BCG Henderson Institute.

Leonid Zhukov is the director of the BCG Global A.I. Institute and is based in BCG’s New York office.

Some of the companies featured in this column are past or current clients of BCG.

This story was originally featured on Fortune.com