Domain Tailored Large Language Models for Log Mask Prediction in Cellular Network Diagnostics

Abstract

Software logs generated by dedicated network testing hardware are often complex and bear minimal similarity to natural language, requiring the expertise of engineers to understand and capture defects recorded in these logs. This manual process is inefficient and expensive for both service providers and their clients. In this study, we demonstrate the transformative potential of Artificial Intelligence (AI), specifically through domain-tailoring of Large Language Models (LLMs) like RoBERTa, BigBird, and Flan-T5, to streamline the process of defect diagnostics. Particularly, we pre-train these models ground up on a real industrial telecommunications log corpus, and perform finetuning on a multi-label classification objective. This facilitates identifying a correct set of log points to be enabled for rapid detection of defects that arise during network testing. Despite encountering several challenges such as intricate text structures, heavily skewed label distribution, and inconsistencies in historical data labelling, our tailored LLMs achieve commendable performance on previously unseen defect cases, significantly reducing the turnaround times. This research not only serves as an exemplar for adapting LLMs in telecommunications industry for automated defect diagnostics, but also has wide implications for software log analysis across various industries.