Telco LLM : Understanding and Electrifying the Language of Telecommunications
Introduction
We’ve created a foundation model for the telecommunications domain we’ve deemed the Telco LLM. This large language model is the culmination of our accumulated experience in developing, tuning, and deploying large language models. The Telco LLM is the latest milestone in advancing AI technology and utilizing it in customer-facing products and services.
Motivation
Telecommunication companies (Telcos) worldwide are on an AI transformation journey. While general domain LLMs like GPT have achieved impressive performance across various NLP tasks, they lack the domain-specific knowledge required for telco applications. These models have been trained on general datasets that lack authoritative telco-related data, limiting their adequacy in addressing the unique challenges posed by the telco domain. Therefore, tailoring a general, larger LLM for the telco domain with a systematically designed dataset and fine-tuning pipeline is imperative in this era of LLMs.
The motivation is simple: to build outstanding products and services, Telcos need an LLM that can understand Telco jargon, Telco tasks, and follow Telco business logic. While prompt engineering and in-context learning can improve performance, even with such techniques, proprietary base models are not ready for domain-specific use cases. To resolve this issue, there is a need to tune base Foundation Models with multi-task telco-specific data. As can be seen by the below image, such tuning enables the Telco LLM to understand telco entities, recognize telco products, detect relevant intents, as well as other tasks that are relevant and necessary to downstream business use cases.
Process
The fine-tuning process starts off with a base Foundation Model, e.g., Claude or GPT, that has been trained on trillions of tokens of text. The base Foundation Model is then fine-tuned with telco task data and telco instruction data, leading to the Telco LLM. The Telco LLM is highly knowledgeable about telco domain and serves as a strong domain-specific Foundation Model. This model can then be fine-tuned with use case specific data (and human feedback data), leading to easier and faster adoption in products and services. In principle, the Telco LLM is designed to achieve three goals:
Telco knowledge and Capabilities from day-one
Better alignment with Telco business logic
Reduced latency and serving cost
Benefits
Telco Knowledge & Capabilities From Day-One
Enterprise transformation is a long journey. It’s also important, however, to bring value early on in this journey. That’s why the Telco LLM derives value as early and often as possible by providing a set of out-of-the-box capabilities relevant to the telco domain.
Example
|
System message Please look at the conversation between the customer and the Telco company counselor and create a list of topics that can represent the entire conversation. Create a list of at least 3 topics, and each topic should be a noun (or compound noun) |
|---|
|
Conversation [Agent] Nice to meet you, this is Yoo Hee-jeong. [Agent] Is 010-9876-5411 the correct number? [Customer] Yes. [Agent] Please verify the name on the account. [Customer] This is Jiwon Kim. [Agent] Thank you. I've verified that you are the account owner. [Customer] Great. How can I set up call waiting service? [Agent] You can set up the call waiting service for free without any additional charge. I will be applying so that you can use it right away. And I checked and you are using SK Telecom, right? Do you plan to continue using SK Telecom in the future? [Customer] Yes. [Agent] Right now, your fee is only 59,050 won and you get 11GB of data. If you plan to continue using it in the future, we recommend signing a year-long contract. If you do so, rates will be cheaper, you get more data, and network speeds are faster. [Customer] Okay, I understand. [Agent] Then, I will sign up for a one-year contract right away. Your registration has been completed. From now on, you will be able to use a lot of data quickly at a cheaper price. [Customer] Thank you. [Agent] Thank you for calling. My name is Yoo Hee-jeong. |
Better Alignment with Telco Business Logic
Although it is possible to prompt the Telco LLM and attempt to inject some business logic, this process is brittle, and the effort required to productionalize such a model is monumental. To improve the Telco LLM’s ability to reflect business logic and follow instructions, we tune the model on a variety of instruction data, such as reasoning tasks, as well as feedback data. This mix of data gives us the best of of many worlds: the Telco LLM is well-versed at tasks relevant to the telco domain, it can follow instructions, and the model reflects values and logic of domain experts. As such, generated answers are more satisfying, more factual, and more relevant to the telco domain.
Faster and Cheaper Generation
Our model is optimized for the telco domain and thus can be served more efficiently. The optimization is incredibly important for latency-sensitive applications, such as Agent Assist, and it’s one of the key benefits of the Telco LLM. Additionally, since business logic and telco knowledge has been baked into the fine-tuning, there is less need for long, hard to maintain prompts; this leads to reduction in costs as well.
The Big Picture
We’re working our way up the “LLM Pyramid” seen in the picture above. Base foundation models, such as GPT-4, are great, but to go from LLM to an LLM-powered service is still a gargantuan task. The ultimate goal is for the Telco LLM to include all levels in the “LLM Pyramid” and make Going to Market as quick and seamless as possible. Ideally, time-to-Market can be minimized, as person-hours, data, know-how, blood, sweat, and tears are “built in” to the ready-made Telco LLM.