Evaluating Urgency of Typhoon-Related Tweets Through Sentiment Analysis Using Artificial Neural Networks

Abstract

Purpose – The main objective of this project was to evaluate typhoon-related Tweets’ urgency using sentiment analysis with supervised learning over an artificial neural network.

Method – The researchers implemented artificial neural network and natural language processing techniques for sentiment analysis and evaluation of the urgency score of typhoon-related Tweets. The model’s accuracy on training and validation was evaluated simultaneously. A separate validation using 100 data was done using confusion matrix analysis.

Results – The accuracy of the model in training was at 99.87% and the loss was 0.0074. Validation was conducted simultaneously with the training. It was found that the accuracy of the model was at 99.17% and the loss was 0.0680. The confusion matrix analysis showed that the sensitivity was 97.67% and the specificity was 100%. The positive predictive value was 100% and the negative predicted value was 98.28%. Both false positive and false discovery rates are at 0% while the false-negative rate was at 2.33%. Finally, the F1 score was 98.82% and accuracy was 99%.

Conclusion – The implementation of the architecture of the model was successful; the researchers concluded that the training produced successful results by looking at the high accuracy prediction of the model and the low loss during the simultaneous training and validation, and confusion matrix analysis for the separate validation.

Recommendations – The researchers recommend expanding the vocabulary of the model by adding more diverse data to the dataset when training. The model produced by this study can be used in incident reporting systems that will be helpful during times of typhoon-related disasters.

Research Implications – Using the model produced by the study in incident reporting applications of the government and NGOs will be more efficient than manually looking at typhoon-related posts on Twitter.