Fraunhofer Center for Assistive Information and Communication Solutions – AICOSDescription:
In retail, the price strategy is characterized by daily competition and constant analysis of its competitors, which is done manually by a team of certified spotters who visit the physical stores and read their promotion leaflets.
Promotion leaflets are designed to draw the consumer’s attention, therefore they are packed with a considerable amount of condensed information. Even with the existence of leaflets in digital form, they are not easily interpreted by computers making this process very time-consuming and prone to human error.
Currently there is no similar solution for automatic interpretation of leaflets that deals with some critical challenges such as: non-standardized design (different types of lettering that difficult the recognition in images), abbreviations or sentences not following usual grammar rules, the need to find association between information contained in text and images or even ignore images that don’t include branding and the indispensability of knowledge of the domain.
In view of the problem described above, the primary objective of this MSc Thesis is to develop a tool for automatic interpretation of promotional leaflets to support the retail pricing strategy, which will apply visual inspection techniques based on optical character recognition, image processing, text location/recognition, semantic segmentation and machine learning to leaflet images.
The expected output will be a list of product-promotion pairs (types of promotions and respective promotional prices, if present). To evaluate the performance of the implemented method, the output will be compared against retrospective databases of leaflets previously manually annotated (ground truth), provided by a retailer. Also, a web tool will be developed to allow the visualization and processing of the extracted information.
This solution proposes to integrate multiple types of data and domain knowledge to create a cognitive model that enables the understanding and extraction of meaningful information from the promotional leaflets.
The methods developed in this thesis, show that is possible to extract information out of the leaflets with some precision. Results show that the use of domain logic information, can have significant impact on the results extracted and also that some techniques and strategies work better for different types of layouts created by the companies. Also that, the use of machine learning to identify specific objects that are not easily detected by an OCR engine, can help improve the results of segmentation and information extraction.
Author: António Melo
Type: MSc thesis
Partners: Faculdade de Engenharia da Universidade do Porto; Sonae Modelo Continente SA
Year: 2019