Context: Accuracy and fairness are two different and important concepts. A model can be accurate but unfair. Machine learning algorithms can produce prejudiced results caused by some erroneous or faulty assumptions in the machine learning process. That means Machine Learning models are susceptible to bias. Therefore, this unfairness can be easily reflected in machine learning predictions. Fair machine learning model predictions are independent of variables that are considered sensitive like ethnicity, age, gender, religion, marital status, race, and socio-economic status just to mention a few.
Aim: This thesis provides a review of how machine learning methods have been applied to software-related problems. Specifically, software merge-conflict prediction. Furthermore, it assesses the accuracy to verify how well the models used are performing, and the discrimination to measure the unfairness with respect to the sensitive features, and consistency of the models.
Methods: This study uses M.Owhadi-Kareshk et al. [2019] dataset that collected information from 147, 967 merges in 105 GitHub repositories in seven programming languages. To evaluate the fairness of the prediction models, the study trains the dataset using RandomForestClassifier, ExtraTreesClassifier, AdaBoostClassifier, and DecisionTreeClassifier, then analyzes the prediction models with fairness metrics to identify any kind of discrimination in the prediction process.
Results: Despite high accuracy levels, bias can still be encountered in machine learning prediction models aimed at predicting software-related problems like merge conflicts.
Conclusion: There is a need to strike the right balance between fairness and accuracy in machine learning predictions that tend to solve software-related problems.