Rapid business insights using the Elastic Stack
Milestone Digital Media Consultants (MDMC) is a communications and new media consulting firm that helps not-for-profit organizations use data to measure the effectiveness of their televised public service announcements.
This process involves analyzing television programming logs containing millions of records and using data visualization tools to show the amount of airplay each PSA receives. MDMC then combines this data with its own data to produce summary reports for its clients. These “PSA Airplay Reports” provide important insights into the monetary value and audience reach of each campaign.
Moving beyond conventional data tools
A key challenge for MDMC was the amount of time and effort required to normalize and extract the relevant data from programming logs. The conventional data management and reporting tools MDMC was using were not up to the task.
MDMC engaged NetFore to design a solution to help them identify the right data quickly to speed up the process of analyzing it and providing insights to its clients.
Challenges
“We work with large data sets and conventional data management tools were not meeting our needs.
NetFore provided a solution based on the Elastic Stack with custom tools to help us uncover the
relevant data quickly and use it to provide valuable insights to our clients.”
Tom Pepin, Principal,
Milestone Digital Media Consultants
Leveraging the Elastic Stack
NetFore designed a solution for MDMC that leverages the Elastic Stack. This open source stack helps you take data from any source, any format and search, analyze, and visualize it in real time. For this project, we used three essential data analytics tools that are part of the Elastic Stack:
- Elasticsearch for the back-end data store
- Logstash for importing television programming log data into Elasticsearch
- Kibana for creating data visualizations using data stored in Elasticsearch
Netfore custom applications
NetFore developed new custom applications for MDMC to help them quickly normalize non-standard and inaccurate programming data and export it for customer use.
Fast, efficient data cleaning and extraction
The programming data contained in the logs is often inaccurate and inconsistent, making data extraction difficult. To simplify this process, NetFore developed a custom web application for identifying and cleaning programming data faster and more efficiently.
The application allows users to create and refine Elasticsearch queries and filters to quickly pinpoint all records that correspond to a specific PSA campaign. Users can then assign a label to all of the matching log records. These labels can then be used for reporting and analytics purposes.
The application also allows users to generate a CSV file containing the cleaned programming log data along with the proprietary broadcaster data used to create airplay reports. Users can specify a client name, a time range, and a list of PSA campaign names.
Uncovering insights with data visualizations
As part of the solution, NetFore implemented Kibana to enable MDMC to create sophisticated data visualizations to provide clients with insights about their PSA campaigns. Kibana’s data visualization capabilities also helped MDMC analyze programming data in greater detail to look for additional value that could be provided to clients.
Conclusion
Using the Elastic Stack and custom applications built by NetFore, MDMC was able to significantly reduce the time and effort required to extract pertinent data from programming logs and provide timely business insights to its clients.