Natural Language Query to SQL query generation using LSTMs, Transformers, LLMs and Gen AI
DOI:
https://doi.org/10.58425/ajt.v4i1.352Keywords:
Natural Language Processing (NLP), SQL Query generation, Long Short-Term Memory (LSTM), Transformers, Large Language Models (LLM), generative AIAbstract
Aim: This study examined and assessed how these more sophisticated models, Long Short-Term Memory (LSTM) networks, Transformer architectures, and Large Language Models, can automate SQL query generation from natural language input.
Methods: Benchmark datasets from finance and e-commerce domains were drawn upon and compared to evaluate the models’ ability to respond to different query complexities.
Results: Transformer-based models with self-attention mechanisms outperformed LSTMs in handling long-range dependencies and complicated query structures by degrees, outperforming LSTMs by 87 % compared to 72 % of LSTMs. SQL generation performance was further enhanced by LLMs like GPT-3, which allowed them to produce more humanlike, context-aware query formulation even where inputs were ambiguous or incomplete. They demonstrated a reduction in the need for skilled professionals for database and increased operational efficiency in data-intensive industries.
Conclusion: The study shows the potential of generative AI for database querying systems and the importance of being responsible with its release.
Recommendations: Ensuring explainability standards, secure data handling protocols, and regular bias audits must be proactively enforced to address ethical and legal challenges, especially on data privacy, model transparency, and algorithmic bias. These results support the growing trend of applying AI to query systems that are more adaptive, scalable, and usable for nontechnical users in enterprise environments.
References
Ahuja, A., Rao, N., Katariya, S., Subbian, K., & Reddy, C. K. (2020, January). Language-agnostic representation learning for product search on e-commerce platforms. In Proceedings of the 13th International Conference on Web Search and Data Mining (pp. 7-15).
Alharbi, N. M., Alghamdi, N. S., Alkhammash, E. H., & Al Amri, J. F. (2021). Evaluation of sentiment analysis via word embedding and RNN variants for Amazon online reviews. Mathematical Problems in Engineering, 2021(1), 5536560.
Ashraf, M., & Haile, A. (2023, October). Data Protection and AI: Navigating Regulatory Compliance in AI-Driven Systems.
Balasubramaniam, N., Kauppinen, M., Rannisto, A., Hiekkanen, K., & Kujala, S. (2023). Transparency and explainability of AI systems: From ethical guidelines to requirements. Information and Software Technology, 159, 107197.
Bandi, A., Adapa, P. V. S. R., & Kuchi, Y. E. V. P. K. (2023). The power of generative AI: A review of requirements, models, input–output formats, evaluation metrics, and challenges. Future Internet, 15(8), 260.
Bertero, C., Roy, M., Sauvanaud, C., & Trédan, G. (2017, October). Experience report: Log mining using natural language processing and application to anomaly detection. In 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE) (pp. 351-360). IEEE.
Chavan, A. (2022). Importance of identifying and establishing context boundaries while migrating from a monolith to microservices. Helina. http://doi.org/10.47363/JEAST/2022(4)E168
Chavan, A. (2023). Managing scalability and cost in microservices architecture: Balancing infinite scalability with financial constraints. Journal of Artificial Intelligence & Cloud Computing, 2, E264. http://doi.org/10.47363/JAICC/2023(2)E264
Conrad, O. T. (2019). Database Solutions to Sports Applications (Master's thesis, NTNU).
Daouk, O. (2022). Thesis: Impact of AI on Sales and Marketing Activities in E-commerce. Science, 66(6), 2495-2522.
Dobriţa, G. (2023). Adaptive Microservices for Dynamic E-commerce: Enabling Personalized Experiences through Machine Learning and Real-time Adaptation. Economic Insights-Trends and Challenges, (1), 95-103.
Erazo-Bone, R., Vera¹, R. G., Chuchuca-Aguilar, F., Yagual¹, J. P. R., Lazo¹, C. A. P., & Escobar-Segovia, K. (2020, March). in Sucker Rod Pumping Systems to Improve Oil. In Applied Technologies: First International Conference, ICAT 2019, Quito, Ecuador, December 3–5, 2019, Proceedings, Part I (Vol. 1193, p. 110). Springer Nature.
Fischer, M., Balunovic, M., Drachsler-Cohen, D., Gehr, T., Zhang, C., & Vechev, M. (2019, May). DL2: training and querying neural networks with logic. In International Conference on Machine Learning (pp. 1931-1941). PMLR.
Fota, A. (2022). Online shopping intentions: antecedents and moderators of shopping intention formation in new fields of E-commerce. Springer Nature.
Gadde, H. (2022). Integrating AI into SQL Query Processing: Challenges and Opportunities. International Journal of Advanced Engineering Technologies and Innovations, 1(3), 194-219.
Gade, K. R. (2021). Data-driven decision making in a complex world. Journal of Computational Innovation, 1(1).
Gil, Y., Garijo, D., Khider, D., Knoblock, C. A., Ratnakar, V., Osorio, M., ... & Shu, L. (2021). Artificial intelligence for modeling complex systems: taming the complexity of expert models to improve decision making. ACM Transactions on Interactive Intelligent Systems, 11(2), 1-49.
Guagliardo, P., & Libkin, L. (2016, June). Making SQL queries correct on incomplete databases: A feasibility study. In Proceedings of the 35th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (pp. 211-223).
Gupta, M., & Gupta, R. (2020). Use of deep learning models for natural language processing tasks: A survey. Computational Intelligence and Neuroscience, 2020, 1-15.
Hartmann, B., Tamla, P., Freund, F., & Hemmje, M. (2023, December). Fine-Tune it Like I'm Five: Supporting Medical Domain Experts in Training NER Models Using Cloud, LLM, and Auto Fine-Tuning. In 2023, the 31st Irish Conference on Artificial Intelligence and Cognitive Science (AICS) (pp. 1-8). IEEE.
Hu, M., Zhang, Z., Zhao, S., Huang, M., & Wu, B. (2023). Uncertainty in natural language processing: Sources, quantification, and applications. arXiv preprint arXiv:2306.04459.
Kabacoff, R. (2022). R in action: data analysis and graphics with R and Tidyverse. Simon and Schuster.
Kabir, M. H., Hasan, K. F., Hasan, M. K., & Ansari, K. (2022). Explainable artificial intelligence for smart city application: a secure and trusted platform. In Explainable Artificial Intelligence for Cyber Security: Next Generation Artificial Intelligence (pp. 241-263). Cham: Springer International Publishing.
Karwa, K. (2023). AI-powered career coaching: Evaluating feedback tools for design students. Indian Journal of Economics & Business. https://www.ashwinanokha.com/ijeb-v22-4-2023.php
Kaswan, K. S., Gaur, L., Dhatterwal, J. S., & Kumar, R. (2021). AI-based natural language processing for the generation of meaningful information from electronic health record (EHR) data. In Advanced AI techniques and applications in bioinformatics (pp. 41-86). CRC Press.
Katsipoulakis, N. R., Tian, Y., Ozcan, F., Pirahesh, H., & Reinwald, B. (2015). A Generic Solution to Integrate SQL and Analytics for Big Data. In EDBT (pp. 671-676).
Kedwan, F. (2023). NLP Application: Natural Language Questions and SQL Using Computational Linguistics. CRC Press.
Kejriwal, M. (2023). Artificial intelligence for industries of the future. Springer International Publishing AG.
Keyvan, K., & Huang, J. X. (2022). How to approach ambiguous queries in conversational search: A survey of techniques, approaches, tools, and challenges. ACM Computing Surveys, 55(6), 1-40.
Khan, W., Daud, A., Khan, K., Muhammad, S., & Haq, R. (2023). Exploring the frontiers of deep learning and natural language processing: A comprehensive overview of key challenges and emerging trends. Natural Language Processing Journal, 4, 100026.
Khurana, R. (2021). Implementing Encryption and Cybersecurity Strategies across Client, Communication, Response Generation, and Database Modules in E-commerce Conversational AI Systems. Rahul Khurana 1 1Bothell, WA, USA.
Kim, H., So, B. H., Han, W. S., & Lee, H. (2020). Natural language to SQL: Where are we today? Proceedings of the VLDB Endowment, 13(10), 1737-1750.
Kumar, A. (2019). The convergence of predictive analytics in driving business intelligence and enhancing DevOps efficiency. International Journal of Computational Engineering and Management, 6(6), 118-142.
Kumar, A., Nagarkar, P., Nalhe, P., & Vijayakumar, S. (2022). Deep learning driven natural language.
text to SQL query conversion: a survey. arXiv preprint arXiv:2208.04415.
Li, F. (2017). Querying RDBMS using natural language (Doctoral dissertation).
Liapis, C. M., & Kotsiantis, S. (2023). Temporal convolutional networks and BERT-based multi-label emotion analysis for financial forecasting. Information, 14(11), 596.
Lin, F., Zhang, C., Liu, S., & Ma, H. (2020). A hierarchical, structured multi-head attention network for multi-turn response generation. IEEE Access, 8, 46802-46810.
Liu, P., Wei, F., & Liu, X. (2020). Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(3), 372-383.
Lynn, T., Fox, G., Gourinovitch, A., & Rosati, P. (2020). Understanding the determinants and future challenges of cloud computing adoption for high-performance computing. Future Internet, 12(8), 135.
Manto, V. (2022). Conversational access to structured knowledge exploiting large models.
Milo, T., & Somech, A. (2020, June). Automating exploratory data analysis via machine learning: An overview. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (pp. 2617-2622).
Nagabandi, A., Finn, C., & Levine, S. (2018). Deep online learning via meta-learning: Continual adaptation for model-based RL. arXiv preprint arXiv:1812.07671.
Nyati, S. (2018). Transforming telematics in fleet management: Innovations in asset tracking, efficiency, and communication. International Journal of Science and Research (IJSR), 7(10), 1804-1810.
Ordina, A. (2021). Classification Problem in Real Estate Corpora: Furniture Detection in Real Estate Listings (Master's thesis, Universidade NOVA de Lisboa (Portugal)).
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35, 27730-27744.
Pearson, D. L. (2021). Chaotic and unexplored: The complex relationship between security professionals, data breaches, and malicious actors. University of Fairfax.
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI Blog.
Raju, R. K. (2017). Dynamic memory inference network for natural language inference. International Journal of Science and Research (IJSR), 6(2). https://www.ijsr.net/archive/v6i2/SR24926091431.pdf
Ramzan, S., Bajwa, I. S., & Kazmi, R. (2018). An intelligent approach for handling complexity by migrating from conventional databases to big data. Symmetry, 10(12), 698.
Sarma, W., Nagavalli, S. P., & Sresth, V. (2020). Leveraging AI-Driven Algorithms to Address Real-World Challenges in E-commerce: Enhancing User Experience, Fraud Detection, and Operational Efficiency. International Journal Of Research And Analytical Reviews, 7, 2348-1269.
Singh, V. (2022). Visual question answering using transformer architectures: Applying transformer models to improve performance in VQA tasks. Journal of Artificial Intelligence and Cognitive Computing, 1(E228). https://doi.org/10.47363/JAICC/2022(1)E228
Singh, V. (2023). Enhancing object detection with self-supervised learning: Improving object detection algorithms using unlabeled data through self-supervised techniques. International Journal of Advanced Engineering and Technology. https://romanpub.com/resources/Vol%205%20%2C%20No%201%20-%2023.pdf
Sun, P. J. (2019). Privacy protection and data security in cloud computing: a survey, challenges, and solutions. IEEE Access, 7, 147420-147452.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence-to-sequence learning with neural networks. Advances in Neural Information Processing Systems, 27.
Tikkinen-Piri, C., Rohunen, A., & Markkula, J. (2018). EU General Data Protection Regulation: Changes and implications for personal data collecting companies. Computer Law & Security Review, 34(1), 134-153.
Tripathi, R., Sinha, P., Balabantaray, R., Pujari, S. P., Thind, P., & Kar, S. (2019, December). Generating Structured Database Queries Using Deeply Bidirectional Natural Language Encodings. International Conference on Information Technology (ICIT) (pp. 489-493). IEEE.
Trummer, I. (2022). CodexDB: Synthesizing code for query processing from natural language instructions using GPT-3 Codex. Proceedings of the VLDB Endowment, 15(11), 2921-2928.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. A., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Proceedings of NeurIPS 2017, 30, 5998-6008.
Wylot, M., Hauswirth, M., Cudré-Mauroux, P., & Sakr, S. (2018). RDF data storage and query processing schemes: A survey. ACM Computing Surveys (CSUR), 51(4), 1-36.
Yanisky-Ravid, S., & Hallisey, S. K. (2019). Equality and privacy by design: A new model of artificial intelligence data transparency via auditing, certification, and safe harbor regimes. Fordham Urb. LJ, 46, 428.
Yeung, J., Wong, S., & Wu, J. (2020). Graph databases for e-commerce platforms. In the International Conference on Internet Studies, Takamatsu, Japan
Zheng, Z., Lu, X. Z., Chen, K. Y., Zhou, Y. C., & Lin, J. R. (2022). Pretrained domain-specific language model for natural language processing tasks in the AEC domain. Computers in Industry, 142, 10373
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Rama Krishna Raju Samantapudi, Mukesh Reddy Dhanagari, Sai Tarun
This work is licensed under a Creative Commons Attribution 4.0 International License.
The authors retain the copyright and grant this journal right of first publication. This license allows other people to freely share and adapt the work but must give appropriate credit, provide a link to the license, and indicate if changes were made. They may do so in any reasonable manner, but not in any way that suggests the licensor endorses them or their use.
