최근 생성형 AI를 도입해 업무 효율성 제고 및 생산성 확대를 꾀하는 기업이 확대되고 있다. 생성형 AI 모델을 보유하고 있는 기업들은 기업 대상 신규 서비스를 출시하며 적극적으로 성과 창출에 나서고 있다. 기업의 생성형 AI 도입을 위해서는 고품질 데이터의 신뢰성과 전문성이 강조되고 있다.
▲ SKT employees operating the AI contact center service 'SKT AI CCaaS' (Photo = SKT)
SKT Introduces Corporate LLM… Increases Employee Productivity
Sapion, LLM Inference AI Semiconductor X330 to be Mass-produced in Q3
Introducing Enterprise-Generated AI… Ensuring Data Reliability is Important
Recently, the number of companies that are introducing generative AI to improve work efficiency and increase productivity is increasing. Companies that have generative AI models are actively creating results by launching new services for companies. In order for companies to introduce generative AI, the reliability and expertise of high-quality data are emphasized.
At the '2024 AX Era - Corporate Response Strategies and AI New Technology and Innovation Case Seminar' held by the KIEI Industrial Education Research Institute in Guro-dong on the 22nd, SKT Manager Kang Hyang-hong presented on SKT's Enterprise LLM (Large Language Model) introduction case and considerations.
SKT announced on the 20th that it will actively expand its AI-based B2B business by launching various new B2B services and securing many corporate customers. Key services include the all-in-one subscription-based AI Contact Center (AICC) service 'SKT AI CCaaS' and the 'AI Copywriter' that automatically generates advertising text in a matter of seconds.
Manager Kang said, “SKT is expanding the introduction of enterprise LLM based on the model ‘AX’ trained with about 1 trillion tokens for lightweighting and performance improvement on-premises to resolve customer security issues,” and predicted, “In order to minimize future costs, X330, developed with AI semiconductor company Sapion to enable LLM inference, will be mass-produced starting in the third quarter, and GPU costs will be reduced by half.” Sapion launched X330 in November last year and is developing X340, which supports AI inference.
SKT's LLM 'AX' has 7 billion parameters. Generally, the larger the number of parameters, the better the general performance, but SKT explained that it was made lightweight to specialize for corporate use. For example, OpenAI's ChatGPT 3.5 has 175 billion parameters, Naver's Hyperclova has 204 billion, and Meta's LLaMA2 has 1.2 billion parameters. The larger the number of parameters, the better the performance, but it is difficult to operate in an on-premise environment.
SKT has various LLMs in collaboration with Anthropic, Open AI, Organize, and Conan Technologies, supporting document creation, image creation, and search in cloud and on-premise form. In particular, AICC integrates AI technology into existing contact centers to perform tasks such as voice recognition technology that converts customer voices into text, AI chatbots and callbots that automatically respond to simple requests, summarizing and answering customer inquiries, and organizing and analyzing consultation content.
In areas such as finance, public, manufacturing, and distribution, companies have various DBs, but most of them are underused. In such cases, applying LLM can speed up data retrieval. LLM also supports query operations to utilize insights accumulated in the DB. Manager Kang said, “Generative AI is being used in various fields, but since it is in the early stages of introduction, there is a lot of need to apply it to internal employees to improve employee productivity and then use it for consumer purposes.”
Manager Kang cited hallucination, security, and cost as key considerations when introducing LLM. To overcome the hallucination phenomenon where chatbots answer falsely as if they were real, SKT uses augmented search generation (RAG) to generate text using contextual data similar to the query. In addition, it secures security in an on-premise environment and solves cost issues by applying a lightweight specialized model with 5 to 40 billion parameters.
In addition, to resolve the issue of it currently taking more than 30 weeks to order Nvidia's GPU 'H100' for generative AI learning and the monopoly of foreign GPUs, the NPU of Sapion, a domestic AI semiconductor, was adopted.
Finally, Manager Kang emphasized corporate feedback in the domain area, saying, “Human feedback is the most important to raise corporate-specific LLM performance to over 90%,” and added, “Just as we continuously train new employees, LLMs also need to accumulate and train high-quality, reliable, and professional data in the long term.”