Introduction

Today’s newspapers have the power to shape one’s entire perspective on the world. For that reason, it is very important to be able to differentiate between newspapers that keep one in a rather pessimistic state of being, or newspapers that offer a more optimistic outlook on the world. The idea is to create, train, adjust and improve a model so that it is fit to analyze various Swiss newspapers and to determine which papers are written with the most negative and positive attitude.

Goal

The core of this project is, with the help of different tools like Tensorflow, Keras, BERT, and Ktrain to create a model that takes text as input and classifies it as “positive” or “negative”. After having evaluated different data sets to train models, the best model for our use case could be identified. This model is now available as a single component that can be integrated into arbitrary applications.

Models

In the process of the work, I created several models, each model being an improvement of the previous one. The first two models are based on the English language, then I trained the later models for the German language:

Model          Dataset                        Accuracy

K1                 Hotel Reviews(EN)      85%

K1 improv    Hotel Reviews(EN)      96%

K2 auto        Filmstarts(DE)             90%

K3 manual   Filmstarts(DE)             93%

Sentiment Analysis

With the last model, I was then able to do sentiment analysis on the news.

I created a Python script to make a GET request to two news APIs, clean the received data and concatenate it into a single CSV file. This process was automated with the scheduler on the Virtual Machine. What I did next was to do the sentiment prediction for each article. Having the results, I was able to make several plots to visualize the calculated sentiment.

Conclusion

A lot of work and hours were spent creating and optimizing all the processes. These processes include gathering data, extracting details, and display them in a user-friendly way. This work has opened my eyes to the world of data. I had never considered sentiment analysis and using it in newspapers made me realize the importance of data in everyday life. This work could very well be part of studying a psychological thesis on the masses, to see how the polarity of the news can affect the masses. Especially today with the pandemic, see the influence it has on the individual person. Personally, I find it very important how the news is given to us, whether it is true or not, and whether it is possible to improve this situation. Perhaps through this work, people can realize the negativity of certain news outlets and become aware that the world is not as bad as they portray it.

Sentiment Analysis on "20 Minuten" newspaper