Performance Evaluation and Comparative Analysis of Lao-Chinese Machine Translation using Few-shot Prompting between SEA-LION and Qwen Models in Tourism-Railway Domains

Abstract

The growth of the Laos-China Railway project has rapidly boosted both the economy and tourism, making cross-lingual communication between Laos and China increasingly important. However, Lao is still considered a low-resource language, which limits the performance of machine translation tools. This study aims to evaluate and compare the translation efficiency between two open-weight Large Language Models (LLMs): SEA-LION-v3-8B and Qwen2.5-7B. The test used a 3-shot prompting technique without any fine-tuning. The experiments were conducted on Google Colab using 4-bit quantization. The test dataset included 720 sentences, divided into two domains: General and Travel-Railway, with sample sizes of 10, 20, 50, and 100 sentences. The translation quality was measured using the BLEU score.The results show that no single model performs best in every scenario. In the General domain for Lao-to-Chinese (Lao→Zh) translation, Qwen outperformed SEA-LION across all data sizes, reaching a top score of 21.62. On the other hand, in the Travel-Railway domain (Lao→Zh), SEA-LION did significantly better than Qwen, with a maximum score of 10.99. This outcome aligns with the Domain Adaptation theory. However, both models faced an imbalance in translation directions. The Chinese-to-Lao (Zh→Lao) translation scores were much lower, mainly due to tokenizer limitations and the lack of a Lao corpus. In conclusion, this research proves that using 7-8B parameter LLMs with 4-bit quantization is highly possible in resource-limited environments. It also provides a useful guideline for developing better tokenizers or fine-tuning models with specific domain data in the future.