Manufacturers often face a problem to which there is no readily apparent solution. When combining parts from different suppliers, both internal and external, how can they maintain the highest level of quality at all stages? Better yet, when quality issues do arise, how can they report them both upstream and downstream? Can they see the manufacturing performance in real time? Or is the data available for analysis only later on in the production process?
For each of these questions, companies need a clear answer. They need highly effective tools geared towards precise predictions and analysis in order to bring about a real increase in ROI. Ideally, they need to be able to gather multiple forms of data at each stage of a manufacturing process, process this data in real time, and spot trends that can predict various outcomes. And when a possible outcome indicates a potential failure or low-quality component, a company needs to know the exact stage at which such a flaw was introduced so that it can be stopped and fixed.
Performing a Quality Analysis after a product is manufactured can lead to potentially huge losses. Even worse, if a fault is discovered only after a product has been purchased by the end-customer, damages to the brand may have an even more severe, and lasting, financial impact. But when you identify and stop production issues at the earliest stage possible so that a potential flaw is never introduced in the first place, you can save on costs as well as rest assured that the quality of your end product will be high and your company’s image will remain intact.
This is the beauty of Big Data.
Your manufacturing data is precious. And mining this data to gather knowledge from it is equivalent to digging up gold. After all, what you uncover will be realized as a solid increase in profits and a substantially improved bottom line.
Basic Concepts: Big Data Tools
If you take a look at a typical factory, you will see plenty of electronic equipment, each of which has several measurable properties such as power consumption, operating temperature, localization, MTBF, or error rate. By looking at some of these properties and comparing them to the requirements, you know whether to trust a specific device or not.
This is a good solution for small operations but doesn't scale well for larger needs. But what if you could gather all of these measurements in a massive database? And what if you could further use that information to help prevent some malfunctions before they ever appeared? This is what we call Big Data solutions—spotting patterns in huge amounts of data in order to continuously improve production and prevent potential problems as well as any corresponding losses in revenue.
Analyzing these data sets is a difficult and complex task but also a profitable one. Consider that every type of equipment in a factory is a potential data source storing a large amount of information that you would like to compare and correlate. You’re not looking to guess; you want to make clear and informed decisions. And you need to divide the process of achieving such conclusions into three steps:
- Extract
- Transform
- Load
First of all, data needs to be extracted and stored for further processing. At this point, your data is unstructured, or raw. The second step entails transforming the acquired data so that they are unified, pre-processed, ordered and shaped. Once structured, the data can then be loaded into a store for further analysis. This process is known as ETL (Extract, Transform, and Load).
Data Stores
As you can see, there can be more than one database in which to store your data. That's why there are different terms to refer to them, depending on how they are used.
Data Lakes
A data lake is a mass storage of mixed data. This doesn't mean it is completely chaotic, but the general idea behind this type of store is to enable high availability of data at a low cost. Unfortunately, despite their clear need for this type of feature, many companies have also found that building data lakes is both expensive and long-running, indicating a clear need for systems which, out-of-the-box and with minimal effort, enable companies to benefit from the tremendous volume and variety of data they can collect.
Data Warehouse
Data lakes store both structured and unstructured data and are easy to write to, but reading usually requires further processing before the data can be consumed. A data warehouse, on the other hand, stores every bit of information in a tidy manner, just like ordered products awaiting shipment in real-life warehouses. Here, most of the processing is done when writing, allowing for quick and easy access to any higher-level information required.
Data Mart
A related concept is the data mart, typically the presentation layer of a data warehouse. This layer does not store data in any different way but rather focuses on a specific slice or view of the data in question. So, your data warehouse may contain everything you can measure, while your data mart may narrow it down to specific fields for further analysis, such as power consumption, production yield, or even marketing.
How Much Data Do You Produce?
Imagine all of the useful data that surrounds you. Starting from the supply chain and going through assembly, QA, and all the way up to customer satisfaction, each stage can provide you with helpful insights.
You already store information related to subparts purchased. And you most probably also store the results of QA. It’s also relatively easy to measure power usage, operating temperature, or time necessary for each stage of the production process. All of these types of data might help you prevent issues that can potentially occur during the manufacturing process or enable you to better lower your costs of production.
But using them in a raw form may often not be enough to get the results you need. In order to fully manipulate this data and deliver analysis, predictions, and fault-prevention, Big Data solutions are a must.
The range of data available for analysis and performance optimization goes well beyond the manufacturing process. Purchase information, conversion analytics of your marketing campaigns, or geographical location of your customers are all highly relevant data and also subject to the same principles of Big Data. It is important to understand that proper and effective data analysis across an entire organization is critical for it to reach optimal performance and results.
Data is everywhere—make sure you are accessing it, storing it, and gleaning information from it for the greatest reliability, visibility and ROI possible.
Leveraging Big Data: A Complex Task Made Simple
Nowadays, more and more products are developed via multiple departments in one large enterprise or even by various external contractors. Managing such production can be very challenging as each production tier has lower-level tasks that demand specific solutions in order to avoid issues across the entire process. In order to get a proper picture of the whole manufacturing process, you need to gather and process data from many different sources as well as transform this data and make it easy to understand.
This processing can occur in real time so that the data streaming and analysis take place simultaneously. When manufacturing electronic devices, it may be necessary to collect and compare data regarding all critical semiconductor components together with their Lot Code, Date Code, etc. Another record may contain test results of delivered components, documentation of failure analysis down to its roots, customer complaints, and returns.
This is a lot of data, yes. And in addition to all of this, remember that it all comes from different sources varying in structure. It is almost impossible to make high-level decisions without unifying available data and conducting a proper analysis. You need the tools to perform these functions and spot any possible inefficiencies and failings within your manufacturing process early on in order to prevent unwanted product defects down the line.
You need to leverage your Big Data to uncover the golden nuggets of knowledge necessary to optimize your production process and bottom line.
Michael Schuldenfrei is the Chief Technology Fellow at OptimalPlus.
