What Upstream Oil, Gas Can Learn About Big Data from Social Media

By: Karen Boman

Upstream oil and gas companies can get more from data by learning a lesson from other industries: companies that regard data as a valuable asset are leading the Big Data revolution, according to a paper presented at the Society of Petroleum Engineers’ Annual Technology Conference and Exhibition in Amsterdam in late October.

Historically, the oil and gas industry has generated a lot of data in its operations, and is considered by some to be the original Big Data industry. Increases in processing power, declining computation and storage costs, and the rising number of sensor technologies embedded in devices have already impacted the exploration and production (E&P) sector. Modern seismic data centers can easily contain as much as 20 petabytes of information, enough information that, if put on a single bookshelf, could circle the Earth’s equator about six times.

Breakthroughs in mathematics – most notably geometry – have significantly helped Big Data practitioners see through Big Data digital information, which frequently is noisy, messy, raw, unstructured and dynamic. The development of software platforms such as Google’s MapReduce or its open-source rival Hadoop also have made Big Data possible.

“And while seismic data sets are notoriously large and cumbersome, many other aspects of the E&P industry are also generating significantly more data than they used to,” said paper authors Robert K. Perrons, Queensland University of Technology, and Jesse W. Jensen with Intel Corporation. “What is more, there’s every reason to believe that this trend towards more digital information is just getting warmed up. Current estimates suggest that the total amount of digital data in the world – including things like books, images, emails, music and video – is doubling every two to three years.”

The Big Data approach to analyzing this data is a relatively new concept that could significantly reshape the industry. However, the Big Data revolution is unfolding differently in the oil and gas industry compared with other industries. In industries such as healthcare, financial, retail and media, companies are gaining new and valuable insights by using new techniques to analyze massive data sets in ways that were never possible with smaller ones.

These insights tend not be discovered by testing hypotheses between variables whose relationships are well understood; rather, they are found by applying advanced analytical techniques to massive numbers of variables that, at first blush, might seem to be unrelated. Companies on the cutting edge of Big Data are not just creatively value by monitoring relationships about which they already know, but finding patterns and making predictions based on complex relationships that were previously unknown. While scientists have traditionally sought to understand the causality and mechanisms underlying these relationships, Big Data leaders frequently care about the “what” far more than the “why”.

“Therein lies a fundamental difference of opinion between the E&P sector and other industries that are considered leaders in Big Data,” said the paper authors. “While there’s no denying that the upstream oil and gas industry is swimming in digital information – and indeed, several Big Data technologies have been used for much longer in this industry than in many others – the way we manage data doesn’t actually bear much resemble to how it gets used in companies like Facebook and Amazon.”

“The E&P sector tends to regard data as information that describes the state of an asset; leaders in Big Data, by stark contrast, realize data is a valuable asset in and of itself.”

While many E&P companies today “clearly” do not consider most of their digital information to be mission-critical to their profitability, this outlook will likely change in the years ahead.

The paper authors raised a number of questions, including how E&P companies should reshape their contracting and collaborating strategies to allow for the integration and high-level analysis of data within an organization.

The authors also asked how Big Data could change the E&P industry’s business model, and whether E&P companies should cultivate teams of data scientists to address the rising importance of increasingly multi-disciplinary nature of IT and data management.

“The E&P sector’s digital revolution is unfinished,” the paper authors noted. “The industry has succeeded handsomely in learning how to generate a staggering amount of data, but we’re still collectively wrestling with the question of what to do with it.”

“The industry’s digital revolution will be completed when we come to terms with how to monetize the data that we’re now capable of collecting and use it to create all the value that it can.”


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s