Cognition-Aligned Sentence Embeddings for Cross-Domain Semantic Relation Classification
DOI:
https://doi.org/10.66238/pe60m322Keywords:
Sentence Embeddings, Cognitive Linguistics, Domain Adaptation, Semantic Relations, Natural Language ProcessingAbstract
Semantic relation classification is a foundational task in natural language processing, crucial for applications ranging from knowledge graph construction to complex question answering systems. Despite the remarkable success of heavily parameterized language models, these architectures frequently exhibit significant performance degradation when deployed in out-of-distribution target domains. This paper introduces a novel framework utilizing cognition-aligned sentence embeddings to address the pervasive challenge of cross-domain semantic relation classification. By integrating psycholinguistic signals, specifically eye-tracking fixation durations and neural activation patterns derived from reading comprehension tasks, our approach constrains the embedding space to reflect human-like semantic generalization. The proposed architecture employs a dual-stream processing mechanism where standard textual representations are systematically aligned with cognitive representations through a specialized contrastive learning objective. This ensures that the relational features learned in a source domain are deeply anchored in fundamental cognitive processing patterns, which are inherently more robust to domain-specific lexical variations. Extensive experimental evaluations across multiple distinct domains, including medical, legal, and general technical texts, demonstrate that the cognition-aligned model substantially outperforms traditional domain adaptation techniques. The findings suggest that incorporating human cognitive priors into dense sentence representations not only improves zero-shot and few-shot cross-domain transfer but also yields representations that are significantly more interpretable and stable under linguistic distributional shifts
References
1. Guo, H., Lu, C., Bao, F., Pang, T., Yan, S., Du, C., & Li, C. (2023). Gaussian mixture solvers for diffusion models. Advances in Neural Information Processing Systems, 36, 25598-25626.
2. Xu, X., Han, C., Zhang, Y., Tu, W., & Yang, Y. (2024, April). Curricular Contrastive Regularization for Speech Enhancement with Self-Supervised Representations. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 10486-10490). IEEE.
3. Dong, J., Koniusz, P., Chen, J., Xie, X., & Ong, Y. S. (2024). Adversarially robust few-shot learning via parameter co-distillation of similarity and class concept learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 28535-28544).
4. Ren, Y., Wu, Y., Fan, J. W., Khurana, A., Fu, S., Wu, D., ... & Huang, M. (2024). Automatic uncovering of patient primary concerns in portal messages using a fusion framework of pretrained language models. Journal of the American Medical Informatics Association, 31(8), 1714-1724.
5. Li, W. W., Ziser, Y., Xie, Y., Cohen, S. B., & Ma, T. (2025, July). Tsprank: Bridging pairwise and listwise methods with a bilinear travelling salesman model. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1 (pp. 707-718).
6. Tu, P., Huang, Y., Zheng, F., He, Z., Cao, L., & Shao, L. (2022, June). Guidedmix-net: Semi-supervised semantic segmentation by using labeled images as reference. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 2, pp. 2379-2387).
7. Xue, X., Yao, H. D., & Davidson, L. (2022). Synthetic turbulence generator for lattice Boltzmann method at the interface between RANS and LES. Physics of Fluids, 34(5).
8. Dong, J., Koniusz, P., Chen, J., Wang, Z. J., & Ong, Y. S. (2024). Robust distillation via untargeted and targeted intermediate adversarial samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 28432-28442).
9. Zhang, Jiaquan, et al. "Lightweight llm agent memory with small language models." arXiv preprint arXiv:2604.07798 (2026).
10. Sivarajkumar, S., Edupuganti, S., Lazris, D., Bhattacharya, M., Davis, M., Dressman, D., ... & Wang, Y. (2026). Extraction of Treatments and Responses From Non–Small Cell Lung Cancer Clinical Notes Using Natural Language Processing. JCO clinical cancer informatics, 10, e2500138.
11. Zhang, W., Zhang, C., Gu, C., Kou, J., Yuan, H., Fang, X., ... & Fang, Y. (2024, October). Hallucination in Large Language Models: From Mechanistic Understanding to Novel Control Frameworks. In 2024 7th International Conference on Universal Village (UV) (pp. 1-36). IEEE.
12. Wang J, Fan L, Li B, et al. A Dynamic Factor Gating Architecture with Market Regime Awareness for Stock Return Forecasting[J]. 2026.
13. Xu, X., Tu, W., Yang, Y., Li, J., Zhang, Y., & Chen, H. (2026). Contribution-aware Dynamic Multi-modal Balance for Audio-Visual Speech Separation. IEEE Transactions on Multimedia.
14. Wu, Yuxuan, et al. "Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraints." arXiv preprint arXiv:2407.03824 (2024).
15. Xu, Xinmeng, Weiping Tu, and Yuhong Yang. "Pcnn: A lightweight parallel conformer neural network for efficient monaural speech enhancement." arXiv preprint arXiv:2307.15251 (2023).
16. Zhu, Guoying, et al. "Enabling MoE on the Edge via Importance-Driven Expert Scheduling." arXiv preprint arXiv:2508.18983 (2025).
17. Chen, L. (2026). Beyond external constraints: The missing dimension of ai governance. Available at SSRN 6449738.
18. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., et al. (2023). LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
19. Zhang, Jiaquan, et al. "Learning global hypothesis space for enhancing synergistic reasoning chain." arXiv preprint arXiv:2602.09794 (2026).
20. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189–1232.
21. Xie, C., Zhu, D., Wang, Z., Zhang, H., & Wei, Z. (2026). Compliance-Aware Discharge Agent for Auditable ICU Discharge Planning: A Pilot Feasibility Study Using Structured eICU Records. Available at SSRN 6429758.
22. Dong, J., Moosavi-Dezfooli, S. M., Lai, J., & Xie, X. (2023). The enemy of my enemy is my friend: Exploring inverse adversaries for improving adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 24678-24687).
23. Huang, Z., Wang, J., Chen, L., Xiao, B., Cai, L., Zeng, Y., & Xu, J. (2025, October). MVISU-Bench: Benchmarking Mobile Agents for Real-World Tasks by Multi-App, Vague, Interactive, Single-App and Unethical Instructions. In Proceedings of the 33rd ACM International Conference on Multimedia (pp. 8797-8805).
24. Huang, T., Cui, Z., Du, C., & Chiang, C. E. (2025, June). CL-ISR: A Contrastive Learning and Implicit Stance Reasoning Framework for Misleading Text Detection on Social Media. In 2025 6th International Conference on Electronic Communication and Artificial Intelligence (ICECAI) (pp. 610-616). IEEE.
25. Wang, S., Yu, Y., Feldt, R., & Parthasarathy, D. (2025). Automating a complete software test process using llms: An automotive case study. arXiv preprint arXiv:2502.04008.
26. Yang, Huan, et al. "Kvshare: An llm service system with efficient and effective multi-tenant kv cache reuse." arXiv preprint arXiv:2503.16525 (2025).
27. Xue, Xiao, et al. "Uni-Flow: a unified autoregressive-diffusion model for complex multiscale flows." arXiv preprint arXiv:2602.15592 (2026).
28. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of EMNLP-IJCNLP.
29. Tang, Y., Zhang, G., Liu, J. K., & Qin, R. (2025). Weakly supervised land-cover classification of high-resolution images with low-resolution labels through optimized label refinement. International Journal of Remote Sensing, 46(5), 1913-1937.
30. Zhang, Y., Valentino, M., Carvalho, D., Pratt-Hartmann, I., & Freitas, A. (2024, June). Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders. In Findings of the Association for Computational Linguistics: NAACL 2024 (pp. 474-489).
31. Zhang, Luyan. "MCP: A Control-Theoretic Orchestration Framework for Synergistic Efficiency and Interpretability in Multimodal Large Language Models." arXiv preprint arXiv:2509.16597 (2025).
32. Yang, Tianyue, and Xiao Xue. "Meno: Meanflow-enhanced neural operators for dynamical systems." arXiv preprint arXiv:2604.06881 (2026).
33. Guo, J., Shuang, K., Zhang, K., Liu, Y., Li, J., & Wang, Z. (2023, June). Learning to imagine: distillation-based interactive context exploitation for dialogue state tracking. In Proceedings of the AAAI conference on artificial intelligence (Vol. 37, No. 11, pp. 12845-12853).
34. Xue, X., Wang, S., Yao, H. D., Davidson, L., & Coveney, P. V. (2024). Physics informed data-driven near-wall modelling for lattice Boltzmann simulation of high Reynolds number turbulent flows. Communications Physics, 7(1), 338.
35. Wang, Juyuan, et al. "HeadRank: Decoding-Free Passage Reranking via Preference-Aligned Attention Heads." arXiv preprint arXiv:2604.17237 (2026).
36. Lin, Liang, et al. "OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment." arXiv preprint arXiv:2509.24610 (2025).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Alberto Cathy, Donghyun Ryu, Albert Cathryn, Jihwan Moon (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the International Journal of Machine Learning and Engineering (IJMLE) are licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, reproduction, adaptation, and commercial use in any medium or format, provided the original author(s) and the source are properly credited.
Authors retain the copyright of their work while granting the journal the right of first publication.
To view a copy of this license, please visit:
https://creativecommons.org/licenses/by/4.0/
