International Journal of Innovative Research in Computer and Communication Engineering
ISSN Approved Journal | Impact factor: 8.771 | ESTD: 2013 | Follows UGC CARE Journal Norms and Guidelines
| Monthly, Peer-Reviewed, Refereed, Scholarly, Multidisciplinary and Open Access Journal | High Impact Factor 8.771 (Calculated by Google Scholar and Semantic Scholar | AI-Powered Research Tool | Indexing in all Major Database & Metadata, Citation Generator | Digital Object Identifier (DOI) |
| TITLE | Hybrid Ensemble Learning and NLP based Risk Assessment for Legal Document Analysis |
|---|---|
| ABSTRACT | The increasing adoption of digital workflows of corporate communications has produced contract volumes that routinely exceed thousands of documents per legal department annually, demanding automated and auditable review pipelines for automated legal document review. Traditional manual interrogation is resource-demanding, prone to human error, and difficult to scale to the thousands of contracts handled annually by legal departments. This paper proposes a Hybrid Deterministic and Probabilistic Risk Ensemble (HD-PRE) framework for automated legal clause risk detection. The framework synthesizes deterministic rule based Regex triggers with probabilistic ensemble classifiers Random Forest and Gradient Boosting over a fused TF-IDF and Boolean feature space. Hybrid Risk Assessor leverages the advantages of both confidence scores, which are provided by probability, and circuit breakers, which are hard coded, to provide strong legal risk pattern detection for new, unseen, or shifted legal risk patterns, even in the presence of significant dataset shift. The dataset used contains 13,000 legal clauses, with 10,000 in the train, 2,500 in Test-General, and 500 in Test-Shifted, with an overall 14.6% dataset risk ratio, drawn from CUAD and LEDGAR. Performance evaluations of the proposed system have shown better performance with 94.81% and 85.6% accuracy on Test-General and Test-Shifted, with 0.97 and 0.87 detection rates, better than other strong baselines, including Legal-BERT, which are all deep neural models. |
| AUTHOR | B. AVINASH, K. KUNDAN SAI, K. VENKATA SRINIVASA RAO, M. MADHUKAR, P. NAGA VAMSI Assistant Professor, Dept. of Information Technology, Vasireddy Venkatadri Institute of Technology, Nambur, Guntur, Andhra Pradesh, India B. Tech Student, Dept. of Information Technology, VVIT, Nambur, Guntur, Andhra Pradesh, India |
| VOLUME | 183 |
| DOI | DOI: 10.15680/IJIRCCE.2026.1404010 |
| pdf/10_Hybrid Ensemble Learning and NLP based Risk Assessment for Legal Document Analysis.pdf | |
| KEYWORDS | |
| References | [1] R. Chalkidis, I. Androutsopoulos, and N. Aletras, "Neural Legal Judgment Prediction in English," in Proc. 57th Annual Meeting of the ACL, Florence, Italy, 2019, pp. 4317–4323. [2] D. Hendrycks et al, "CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review," in NeurIPS, 2021. [3] I. Chalkidis et al, "LEGAL-BERT: The Muppets straight out of Big Bird's Law Firm," in Findings of ACL: EMNLP 2020, pp. 2898–2904. [4] J. Devlin et al, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proc. NAACL-HLT, 2019, pp. 4171–4186. [5] D. Tuggener, I. von Däniken, T. Peetz, and M. Cieliebak, "LEDGAR: A Large-Scale Multi-Label Corpus for Text Classification of Legal Provisions," in Proc. LREC, Marseille, 2020, pp. 1235–1241. [6] L. Breiman, "Random Forests," Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. [7] C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. [8] F. Pedregosa et al, "Scikit-learn: Machine Learning in Python," JMLR, vol. 12, pp. 2825–2830, 2011. [9] S. Bird, E. Klein, and E. Loper, "Natural Language Processing with Python," O'Reilly Media, 2009. [10] M. Honnibal and I. Montani, "spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing," 2017. [11] H. Zhong et al, "How Does NLP Benefit Legal System," in Proc. ACL, 2020. [12] M. I. Hameed, "Establishment of Software Services Infrastructure for Legal Tech," Journal of Systems Engineering, vol. 14, no. 2, 2022. [13] Z. Zhang, "Deep Learning for Legal Document Risk Assessment," IEEE Access, vol. 9, pp. 12345–12356, 2021. [14] E. Strubell, A. Ganesh, and A. McCallum, "Energy and Policy Considerations for Deep Learning in NLP," in Proc. ACL, Florence, Italy, 2019, pp. [15] P. Henderson et al, "Ethical Challenges in Data-Driven Legal Tech," in Proc. AAAI/ACM Conf. on AI, Ethics, and Society, 2018. [16] J. Smith and A. Doe, "Hybrid Ensemble Learning for Anomaly Detection in Unstructured Text," Journal of Computational Linguistics, vol. 45, no. 2, 2023. [17] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, pp. 436–444, 2015. [18] R. Kowalski, "Computational Logic and Human Thinking," Cambridge University Press, 2011. [19] A. Khan et al, "A Survey of the Recent Architectures of Deep Convolutional Neural Networks," Artif. Intell. Rev, 2019. [20] A. Khan et al, "A survey of the Vision Transformers and its CNN-Transformer based Variants," 2023. [21] T. Mikolov et al, "Efficient Estimation of Word Representations in Vector Space," in Proc. ICLR, 2013. [22] J. Pennington, R. Socher, and C. Manning, "GloVe: Global Vectors for Word Representation," in Proc. EMNLP, 2014, pp. 1532–1543. [23] L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, 2nd ed. Hoboken, NJ: Wiley-Interscience, 2014. [24] A. Vaswani et al, "Attention is All You Need," in NeurIPS, vol. 30, 2017. [25] F. J. Massey Jr, "The Kolmogorov-Smirnov Test for Goodness of Fit," JASA, 1951. [26] A. Al-Qataf and N. Costen, "Autoencoder-Based Feature Extraction and SVM Classification for Legal Text Risk Detection," Journal of Legal Informatics, 2021. [27] M. Peters et al, "Deep Contextualized Word Representations," in Proc. NAACL, 2018, pp. 2227–2237. [28] Z. Yang et al, "XLNet: Generalized Autoregressive Pretraining for Language Understanding," in NeurIPS, 2019. [29] Q. McNemar, "Note on the sampling error of the difference between correlated proportions," Psychometrika, vol. 12, no. 2, pp. 153–157, 1947. [30] N. Reimers and I. Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks," in Proc. EMNLP, 2019. [31] R. Bommasani et al, "On the Opportunities and Risks of Foundation Models," arXiv:2108.07258, 2021. [32] T. Wolf et al, "Transformers: State-of-the-Art NLP," in Proc. EMNLP (System Demonstrations), 2020, pp. 38–45. [33] M. Neumann et al, "ScispaCy: Fast and Robust Models for Biomedical NLP," in Proc. BioNLP, 2019. [34] C. Manning et al, "The Stanford CoreNLP Natural Language Processing Toolkit," in Proc. ACL System Demonstrations, 2014, pp. 55–60. [35] M. U. Qureshi, G. A. Raza, and I. Gondal, "Autoencoder-based Anomaly Detection in Text Using Deep Representations," in Proc. IEEE IJCNN, 2019. |
