Skip to content

What Is Big Data?

January 2, 2017

(This article originally appeared in Loss Prevention Magazine)

Big data is a very large amount of complex data that can be structured, semi-structured, and unstructured. This can be a collection of traditional data sources, digital data sources from both inside and outside of your organization, social media, or public records, just to name a few. The reality is big data could be all of your data and any data that is available. The data generally is owned by several different departments within your organization.

Big data is a bit of a buzzword and in some cases isn’t that big at all. It really depends on what your organization’s big data strategy is and where the loss prevention department fits in. It’s really not about the amount of data; it’s what you plan to do with it.

Big data can help develop predictive models for risk, shrink reduction, fraud prevention, or refund management; provide process improvement information to the field; help identify dishonest activity; and provide better overall insights into the business. Possibilities at times can feel limitless and often overwhelming.

In my last article, we talked about social media monitoring. We’ll talk about this a little bit, but this article is about what you can do with all of the data available. I’ll be defining some of the big data terms and talking about strategies and plans to handle this wealth of information.

Descriptive and Predictive Analytics versus Prescriptive Analytics

The first step is to understand the difference between big data and traditional data analytics. Data analytics generally consist of smaller data sets from fewer sources with structured or normalized data. In the LP world, exception-based reporting would be a type of traditional analytics where you have structured or normalized data and are attempting to find patterns or specific aberrations. Another example would be descriptive analytics related to audits or shrink results where you take data, create a summary, and visualize the material.

Big data on the other hand will involve multiple data sources (structured, semi-structured, and unstructured) shared from multiple business partners both inside and outside of the organization. At times, this will include other non-traditional data sources such as social media, Google search results, news, weather, and public record data.

An example of big data analytics would be looking into omni-channel data to help identify process-improvement opportunities. You can be looking at what happens to the shipment when it leaves the store to go to a customer, the impact on the store inventory, ship speed, customer contact, defect rate, non-deliveries, weather, staffing, customer settlement, and fraud risk.

Looking at all of the data sets from multiple sources will assist you in developing a prescriptive analytic. Prescriptive analytics takes traditional descriptive and predictive analytics and automatically synthesizes the big data. It makes predictions and then can generally make several recommendations for options to take advantage of those predictions. So in essence, it’s taking that descriptive analysis, which still counts for the majority of business analytics today, and answers the questions of what happened, why it happened, and how it can improve. It also looks for reasons for past failures and successes and then provides options, in addition to showing the implications of those options. All of this information helps improve the customer experience and allows process improvement and shrink reductions.

Now that you have better understand of the difference between descriptive or traditional analytics and big data analytics, let’s move onto a plan.

Planning Your Big Data Strategy

Here are some key points to keep in mind when planning your big data strategy. One important thing to remember about big data is it’s not just about loss prevention because it requires a lot of IT and other department support, including marketing, operations, finance, omni-channel, and other business stakeholders. It has to be about the organization’s profitability protection, early warning or risk, and process improvement. This isn’t your traditional “let’s stop the shoplifter” or “catch the bad guy” strategy; as the business evolves, so do we. When you improve the process, the shrink reduction is a byproduct.

Next up, a big question in big data is whether you will do it within your LP department or use a business intelligence team within your organization. Does it make sense to outsource it in part or totally?

If you’re taking an in-house, in-LP route, you need to take a look at what your staffing looks like and your team overall. If you’d asked me three years ago if I ever thought I would hire a data scientist, statistician, or an actual IT developer to work in an LP role, I’m not sure that I would’ve said yes. Today, more than ever, LP teams need experts to handle the amount of data collected. The experts are either data scientists or statisticians. You can teach a data scientist or statistician LP, but it’s not so easy to do it the other way around. This changes the philosophy and is difficult for some of us. I know personally I want high-potential loss prevention people and to move them to the best role as the business evolves.

Key Steps

Let’s talk about some key steps.

  • IT partnership is crucial. Having a strong understanding of the organization’s big data program is imperative. What does the company’s data warehousing look like? You don’t need to be the expert or hire an expert in this area, you just need to understand how you’re going to get all the data. You will have to work with IT to ensure your software works with theirs. Your data scientist or statistician will know what to do with the data once it’s available.
  • We need to understand the business before we start solving problems outside of traditional LP. We need to understand what customer-level detail is available or what business strategies are going on in-store or online. Are you answering a question, solving a problem, or just reacting to a trend? Get to the root cause with your data. Also know what the questions are before you start.
  • Break down the silos—think bigger. Work directly with the partners that you haven’t before, constantly keep engaged with business partners to ensure that everyone is looking at the same KPIs. This doesn’t mean work on the same thing. It’s about having a common vision and goal.
  • Put together data narratives that speak everyone’s language. It’s not just about loss prevention; it’s about the process improvement, customer service, and profit protection.
    Have a dynamic plan and make sure that there’s constant visibility with the business stakeholders and understanding on how to proceed.
    Choose the right software.
  • Make sure that you have a plan to visualize, model, data mine, and present. Always remember that you must have a plan.


Sometimes saying no to all the data is the right answer. Sometimes asking for it all is the answer. Challenge yourself to think bigger. Constantly think outside of the traditional comfort zones of loss prevention. How can you take all of that information that you had for many years and combine it with all the other information on the horizon to come up with impactful and insightful solutions? As our business evolves, so must we.