Risk Graph-NLP: Graph-Assisted Language Modeling for Discovering Hidden Compliance Relationships in Enterprise Documents
DOI:
https://doi.org/10.71238/snnst.v1i01.181Abstract
Natural language processing is becoming a key tool for enterprise companies to deal with compliance obligations spread throughout contracts, policies and regulatory filings. Plain token-based pipelines do not easily bring to the surface relationships that cross clauses, sections, and/or files, resulting in a lack of compliance risk visibility and auditability. In this paper, we present a synthesis of a framework, RiskGraph-NLP, that proposes to combine contextual language models with graph neural networks to represent enterprise documents as heterogeneous compliance graphs, encompassing clauses, named entities, obligations, and regulations. The framework is grounded in three key research trends: transformer-based language representation, document-level relation extraction, and knowledge graph embedding, as well as graph-based fraud detection, with the goals of overcoming the limitations of sequential language models. The framework is inspired by three research directions, namely, transformer-based representation of language, document-level relation extraction, and knowledge graph embedding, along with graph-based fraud detection, with the aims of addressing the weaknesses of sequential language models. The synthesis shows that graph-assisted encoders are structurally more appropriate than sentence-bound models to show indirect or latent compliance dependencies, such as an obligation that is not explicitly referenced in the regulated text but is only implicitly referenced by an intermediate cross-reference. The topic is also explored from the architectural perspective, with a focus on the pros and cons of convolutional and attention-based graph encoders, as well as the ethical and regulatory implications of automated compliance inference in legal and financial applications. The paper concludes that hybrid language-graph architectures appear to be an exciting, but still developing, path for enterprise compliance analytics and provides methodological priorities for future empirical validation.
Downloads
References
F. Christopoulou, M. Miwa, and S. Ananiadou, “Connecting the Dots: Document-Level Neural Relation Extraction with Edge-Oriented Graphs,” in Proceedings of EMNLP-IJCNLP 2019, 2019, pp. 4925–4936. doi: 10.18653/v1/D19-1498.
S. K. Sahu, F. Christopoulou, M. Miwa, and S. Ananiadou, “Inter-Sentence Relation Extraction with Document-Level Graph Convolutional Neural Network,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4309–4316. doi: 10.18653/v1/P19-1423.
A. Vaswani et al., “Attention Is All You Need,” in Advances in Neural Information Processing Systems 30, 2017, pp. 5998–6008. doi: 10.48550/arXiv.1706.03762.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.
I. Chalkidis, M. Fergadiotis, P. Malakasiotis, N. Aletras, and I. Androutsopoulos, “LEGAL-BERT: The Muppets Straight Out of Law School,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 2898–2904. doi: 10.18653/v1/2020.findings-emnlp.261.
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A Comprehensive Survey on Graph Neural Networks,” IEEE Trans. Neural Networks Learn. Syst., pp. 4–24, 2021, doi: 10.1109/TNNLS.2020.2978386.
Y. Yang, M. C. S. Uy, and A. Huang, “FinBERT: A Pretrained Language Model for Financial Communications,” 2020. doi: 10.48550/arXiv.2006.08097.
Y. Xu, M. Li, L. Cui, S. Huang, F. Wei, and M. Zhou, “LayoutLM: Pre-training of Text and Layout for Document Image Understanding,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery, 2020, pp. 1192–1200. doi: 10.1145/3394486.3403172.
J. Li, A. Sun, J. Han, and C. Li, “A Survey on Deep Learning for Named Entity Recognition,” IEEE Trans. Knowl. Data Eng., pp. 50–70, 2022, doi: 10.1109/TKDE.2020.2981314.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph Attention Networks,” in International Conference on Learning Representations (ICLR), 2018. doi: 10.48550/arXiv.1710.10903.
Z. Zhang, P. Cui, and W. Zhu, “Deep Learning on Graphs: A Survey,” IEEE Trans. Knowl. Data Eng., pp. 249–270, 2022, doi: 10.1109/TKDE.2020.2981333.
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling, “Modeling Relational Data with Graph Convolutional Networks,” in The Semantic Web: ESWC 2018, 2018, pp. 593–607. doi: 10.1007/978-3-319-93417-4_38.
X. Wang et al., “Heterogeneous Graph Attention Network,” in The World Wide Web Conference, 2019, pp. 2022–2032. doi: 10.1145/3308558.3313562.
Q. Wang, Z. Mao, B. Wang, and L. Guo, “Knowledge Graph Embedding: A Survey of Approaches and Applications,” IEEE Trans. Knowl. Data Eng., pp. 2724–2743, 2017, doi: 10.1109/TKDE.2017.2754499.
S. Ji, S. Pan, E. Cambria, P. Marttinen, and P. S. Yu, “A Survey on Knowledge Graphs: Representation, Acquisition, and Applications,” IEEE Trans. Neural Networks Learn. Syst., pp. 494–514, 2022, doi: 10.1109/TNNLS.2021.3070843.
L. Yao, C. Mao, and Y. Luo, “Graph Convolutional Networks for Text Classification,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 7370–7377. doi: 10.1609/aaai.v33i01.33017370.
M. Weber et al., “Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics,” 2019. doi: 10.48550/arXiv.1908.02591.
O. Amaral Cejas, M. I. Azeem, S. Abualhaija, and L. C. Briand, “NLP-Based Automated Compliance Checking of Data Processing Agreements Against GDPR,” IEEE Trans. Softw. Eng., pp. 4282–4303, 2023, doi: 10.1109/TSE.2023.3288901.
T. N. Kipf and M. Welling, “Semi-Supervised Classification with Graph Convolutional Networks,” in International Conference on Learning Representations (ICLR), 2017. doi: 10.48550/arXiv.1609.02907.
R. Van Belle, C. Van Damme, H. Tytgat, and J. De Weerdt, “Inductive Graph Representation Learning for Fraud Detection,” Expert Syst. Appl., 2022, doi: 10.1016/j.eswa.2021.116463.
W. L. Hamilton, Z. Ying, and J. Leskovec, “Inductive Representation Learning on Large Graphs,” in Advances in Neural Information Processing Systems 30, 2017, pp. 1024–1034. doi: 10.48550/arXiv.1706.02216.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Varsha Shah (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Facebook 
Instagram 
Twitter 
Youtube 
+62 822 2101 3235
northpressmail@gmail.com