Improving the Reliability of LLM-Generated Training Pipelines with Feedback-Driven Iterative Code Refinement
DOI:
https://doi.org/10.66238/pyk3aq74Keywords:
large language models, code generation, training pipeline automation, iterative refinement, feedback mechanisms, machine learning reliabilityAbstract
The emergence of large language models (LLMs) as automated code generation tools has introduced compelling opportunities for constructing machine learning (ML) training pipelines from natural language specifications. However, LLM-generated pipelines frequently suffer from syntactic errors, API incompatibilities, data flow mismatches, and numerical instability that undermine reliability in deployment. This paper proposes the Feedback-Driven Iterative Code Refinement (FDICR) framework, which integrates a containerized execution sandbox, a hierarchical error taxonomy, and a context-aware prompt reformulation module to enable structured multi-round refinement of LLM-generated training pipelines. The design of FDICR's iterative loop draws conceptual inspiration from controller-based architecture search, wherein a generator receives performance signals from an execution environment and updates its outputs accordingly. The multi-stage pipeline structure incorporates meta-level initialization, Bayesian optimization, and ensemble validation components that parallel established automated machine learning (AutoML) architectures. Experimental evaluation across three benchmark tasks demonstrates that FDICR reduces critical pipeline failures by 78.3% and improves first-pass execution success rates from 42.1% to 89.6% relative to single-pass baselines, yielding speedup magnitudes consistent with neural-augmented program synthesis systems. The framework supports PyTorch, TensorFlow, and scikit-learn backends and introduces a novel pipeline reliability scoring metric.
References
[1] Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., ... & Zaremba, W. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
[2] Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., ... & Sutton, C. (2021). Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
[3] Zhao, W., Chen, T., Yang, J. S., & Qiu, L. (2026). AutoML-Pipeline: A RAG-enhanced code generation framework with pre-validation for cloud-native machine learning workflows. IEEE Access.
[4] Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., ... & Xiong, C. (2022). Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474.
[5] Olausson, T. X., Inala, J. P., Wang, C., Gao, J., & Solar-Lezama, A. (2023). Is self-repair a silver bullet for code generation?. arXiv preprint arXiv:2306.09896.
[6] Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., ... & Vinyals, O. (2022). Competition-level code generation with alphacode. Science, 378(6624), 1092-1097.
[7] Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36, 8634-8652.
[8] Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590.
[9] Dakhel, A. M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., & Jiang, Z. M. J. (2023). Github copilot ai pair programmer: Asset or liability?. Journal of Systems and Software, 203, 111734.
[10] Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., & Yin, J. (2022, May). Unixcoder: Unified cross-modal pre-training for code representation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 7212-7225).
[11] Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., ... & Synnaeve, G. (2023). Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.
[12] Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., ... & Liang, W. (2024). DeepSeek-Coder: when the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196.
[13] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35, 24824-24837.
[14] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35, 27730-27744.
[15] Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., ... & Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
[16] Diao, S., Wang, P., Lin, Y., Pan, R., Liu, X., & Zhang, T. (2024, August). Active prompting with chain-of-thought for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1330-1350).
[17] Zhang, K., Li, Z., Li, J., Li, G., & Jin, Z. (2023, July). Self-edit: Fault-aware code editor for code generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 769-787).
[18] Jain, N., Vaidyanath, S., Iyer, A., Natarajan, N., Parthasarathy, S., Rajamani, S., & Sharma, R. (2022, May). Jigsaw: Large language models meet program synthesis. In Proceedings of the 44th International Conference on Software Engineering (pp. 1219-1231).
[19] Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., ... & Wen, J. (2024). A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6), 186345.
[20] Wang, Z., Yang, J. S., Shang, W., & Ding, J. (2026). FairPromote: Explainable and Fairness-Aware Talent Promotion Prediction via Adversarial Debiasing and SHAP-based Interpretation. IEEE Access.
[21] Ding, J., Shen, Z., & Liu, W. (2026). Game-Theoretic Cost-Sensitive Adversarial Training for Robust Cloud Intrusion Detection Against GAN-Based Evasion Attacks. Applied Sciences, 16(8), 3944.
[22] Ping, W., Jiao, Y., Fan, H., & Zhang, X. (2026). Multimodal Fraud Detection in Financial Statements: A Trimodal Attention Network with Contrastive Evidence Chain Construction. IEEE Access.
[23] Wang, B., Wang, Z., Zhao, W., Zhang, F., & Shang, W. (2026). DRL-Adapt: Deep Reinforcement Learning for Adaptive Routing Convergence Optimization in Large-Scale Networks. IEEE Open Journal of the Computer Society.
[24] Teng, D., Rhee, M., Qin, Y., Zi, B., & Liu, W. (2026). SW-SpeedDLM: Sliding-Window Speculative Decoding for Diffusion Language Models under Long-Context Constraints. Mathematics
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Haoyu Lin, Chenxi Guo (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
All articles published in the International Journal of Machine Learning and Engineering (IJMLE) are licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, reproduction, adaptation, and commercial use in any medium or format, provided the original author(s) and the source are properly credited.
Authors retain the copyright of their work while granting the journal the right of first publication.
To view a copy of this license, please visit:
https://creativecommons.org/licenses/by/4.0/
