Painting a Portrait of Big Data

Episode 9 ends Source De[Code] host Ben Coffin’s investigation into Big Data. While I am a self-professed data junkie and love diving in to see what nuggets of insight I can mine from raw files, I have never really truly considered how that data is aggregated or the specifics of why I find it so useful. Episode 9 guest Toby Marsden was able to put names to many aspects of what I loved so much about data. When I am trying to make sense of a new topic or, as in this case, am trying to process additional details about a topic I am familiar with, my mind always tries to find a more relatable comparison from a subject area I am intimately familiar with. For this conversation between Ben and Toby, my mind kept returning to painting. Even the title of this episode, “How to Put Big Data to Use”, recalls the excitement that must come when an art student moves from theory to standing in front of their own blank canvas.

The previous two episodes in Source De[Code] ‘s delve into big data looked at the theory. They explored in some depth what big data is, how it is used, and why it’s so useful. Now, Ben and Toby put these elements into practice by looking at the practical methods by which big data is aggregated and used to improve efficiency in product development and optimization. There are as many methods for painting a masterpiece as there are artistic styles, but each of these is rooted in shared guiding principles. The elements of composition—line, shape, color, value, form, texture, and space—are the fixed compass points that guide both the post-modernists and the Renaissance masters. The way these schools refer to these elements and how they use them might differ, but they remain.

The same holds true in the software space. Over the years, there have been numerous software delivery methods, but, like artistic schools, everything comes back to the basic principles that guide both production and quality. On their own, each stroke of paint is meaningless, but together, those broad and short strokes dipped in pigments come together to form an image. Similarly, siloed data points are meaningless until they are connected across channels. When they combine, they create a unified picture that gives you valuable context that transforms into intelligence that augments decision-making and results in meaningful improvements to development lifecycles, product quality, and customer experience.

Bringing a Masterpiece to Life Begins with Materials

For me, stretching a new canvas and laying out my graphite pencils, oils, fresh brushes, and mediums is almost a religious experience. I lay everything out before me and see nothing but raw potential. When I stand in front of the easel, even before a single mark mars its pristine surface, I am planning the steps necessary to transfer the image from my head to canvas. In the same way, corporate entities have immense amounts of raw data at their disposal aggregated from all touchpoints across their value chain. “Once you have your hands around that,” says Toby Marsden, “it’s how you aggregate it together.”

Your materials determine the path ahead. I paint in oils. This means that the timeline to complete a piece is much longer than if I paint with acrylics or watercolor. Oil painting techniques require layering thin and fat layers of paint onto the canvas to build out the composition, and each layer needs to be suitably dry before you can begin working on the next. My brushes can apply thick swaths of pigment or define intricate detail. The mediums I use can speed up drying time, add texture to layers, blend layers to establish ambiance, and more. I can wield these materials to charge the energy of the work with subtle brush strokes and muted shades or vibrant hues peppered in dynamic bursts. The way that I wield my brushes, paints, and elements of composition is determined by what I want to achieve. This helps me to select the right canvas size, the right brushes, and it narrows down the paints I will keep on hand.

Corporations can think of big data in the same way—with some marked complications. Data lakes are fed by springs sourced from touchpoints across the value stream that captures information from every event that takes a product from design to market. In these organizations, there are multiple teams working on individual components of the end product that are collecting data that is fed into the lake. You can think of each team as a unique data source that provides the materials—the paints, brushes, and canvas—that will ultimately become a work of art.

These sources are akin to the plethora of manufacturers that produce art supplies. Each brand gives their materials unique names or identifying codes, so the same brush or paint color can be called something different. Before I can begin planning my painting, I first need to find and organize my paints, brushes, and other materials. For enterprise big data, you start by “looking at how the data—the terminology—relates to each other,” says Toby, from there, you “map those different areas into a data structure that helps you spot changes and issues that are happening.”

The Elements of Composition

With materials organized, the next step is mapping out your canvas. There are as many methods for painting a masterpiece as there are artistic styles, but each of these is rooted in shared guiding principles. The elements of composition—line, shape, color, value, form, texture, and space—are the fixed compass points that guide artistic expression; and while you will not be able to identify them when they are employed, their absence is always obvious. Even as every new painting is an ode to the artist’s evolution, the elements of composition will be adhered to. Habitual observance of these elements influences the creative process and the overall impact of the finished work. As well, they can help an artist put a name to compositional elements of a work that are weak. Through this exercise, you learn from past mistakes and adapt your process as you find better techniques-- all of which culminate in more polished, powerful works of art.

Toby and Ben talk extensively about the importance of strong principles of governance around enterprise data architecture. Like the elements of composition, good data architecture governance is critical for understanding and effectively managing the complexities of technology changes, evolving regulations, and issues arising in the production process at the enterprise level. Good governance preserves production momentum by harmonizing cross-functional efforts by creating a global view down to the project and team levels. Each data source maps the ebbs, flows, and bottlenecks occurring in their facet of the product lifecycles giving other teams visibility of bottlenecks and potential breakdowns. Like artistic observance of the elements of composition, adherence to governance principles at an organizational level augments the quality of each individual team’s contributions and ultimately results in better-performing products brought to market faster for more loyal customers.

The Making of Meaning

Heraclitus once said, “you never step in the same river twice”. Water references abound in the world of big data; descriptors like lake, stream, waterfall, and flow speak to the same fluid and ever-shifting sands of the river referenced by Heraclitus. Change ebbs and flows around us and alters the world around us. Big data allows us to observe the subtle changes and react with urgency to the floods that breach the riverbanks, then assess the damage and prevent it from happening again. Host Ben Coffin explains that big data gives us the ability to “look out over what would otherwise be an incomprehensible amount of data and be able to pull from that some level of unnoticeable intelligence and call the human eyes’ attention to it”.

I only notice the shifts in my own artistic style when I look back on the complete body of work I’ve created to this point. I can reflect back and make assumptions about what events helped to shape those changes outside of my own creative growth. Trends and patterns in the themes and compositional choices I’ve made when I look back over time. However, self-analysis is not the most important aspect of art. Art’s true value is in the meaning it brings to the viewer, and that meaning changes and evolves with every encounter a viewer has with a favorite work.

Interpreting big data is similar to artistic expression and experience in this. Enterprise data documents the experience of those involved in the product lifecycle. The way that data is interpreted to improve the workflows that optimize product performance is only one aspect of its value. The other side of the coin is how it tracks to the end-user experience. “What customers will see is regular innovation,” says Toby. “When they see regular innovation, they tend to stick with products and brands more.” Unless you are using the massive amounts of data you are capturing to derive insights and make meaning, its value is sequestered in the realm of potential.

About the Guest: Toby Marsden

Toby has over 25 years’ experience of understanding client needs, proposing solutions, and helping customers achieve their goals through successful implementations. He has worked across sales, marketing, and professional services roles for multinational software companies and start-ups. For the last 15 years, Toby has been focused on helping customers manage complex software delivery challenges by adopting automated delivery practices across the application lifecycle. He is currently part of Eggplant’s management team, which become part of the Keysight family in 2020 and provides AI Assisted Intelligent Test Automation. He is currently on a mission to is to develop a 1st class partner ecosystem that creates increased value to Keysight’s customers.

What was the ‘aha’ moment that started you down your path or influenced your journey to where you are now?

I joined HP Software in 2008 as a Solution Sales Director, my role was to provide specialised knowledge, help customers adopt new capabilities and mature their use of HP products. After a few years my role expanded to a global one, which gave me access to a vast customer base with lots of systems and untapped data. I realised to be truly agile and utilise automated principals effectively you need to use data as a weapon

If you hadn't chosen your current profession, what would you have pursued instead? Why?

If I hadn’t injured my ACL and medial ligaments at 16, probably playing Professional Rugby

Where can we find you when you're not innovating the future of technology?

Coaching my son’s football team or watching my daughter in show jumping & dressage competitions

Listen to Source De[Code]

You can listen to this and all episodes of Source De[Code] on your favorite streaming apps like Apple Podcasts, Spotify, Google Podcasts, and Amazon Music. To learn more about the show, its host and guests, or to tap additional resources that take you deeper into the trending technologies discussed on the show, visit Source De[Code] online. Subscribe to the Source De[Code] mailing list to get podcast updates, resources, and to be notified of new episodes.