Preamble

A Survey of Unsupervised Graph Neural Networks: Interactive Clustering and Embedding

XIA LIU
School of Computer and Information Engineering, Henan Normal University
lx15518903750@163.com

WEI LI
Institute of Computing Technology, Chinese Academy of Sciences
liwei@ict.ac.cn

Abstract: The rise and application of neural networks has significantly advanced research in pattern recognition and data mining. In recent years, graph neural networks have attracted increasing attention, with applications spanning text classification, sequence annotation, neural machine translation, relation extraction, image classification, and other domains. This survey integrates existing research on semi-supervised and unsupervised graph neural networks, organizing contributions along three dimensions: research problems, methodological approaches, and evaluation measures. The primary research challenges addressed are the low-dimensional representation of graph nodes and the over-smoothing problem that occurs during message propagation. Methodologically, the literature focuses on graph embedding algorithms, including both probability graph-based approaches and deep learning-based techniques. Evaluation efforts primarily concentrate on the accuracy and efficiency of these algorithms and models. Finally, this paper identifies promising future research directions to guide readers in this rapidly evolving field.

Keywords: Graph neural network, unsupervised learning, network embedding, node clustering

1 Introduction

Graphs, also known as networks, represent a ubiquitous data structure that permeates daily life, manifesting in social networks, the World Wide Web, and numerous other domains. The advent and widespread adoption of neural networks have revolutionized pattern recognition and data mining, transforming traditionally feature engineering-intensive tasks such as object detection, machine translation, and speech recognition through end-to-end deep learning paradigms including Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and autoencoders. Graph Neural Networks (GNNs) constitute a class of connectionist models that capture dependencies within graphs through information propagation between nodes. While most GNNs are specifically designed for semi-supervised learning tasks (both transductive and inductive), where labeled nodes provide crucial supervisory signals, this reliance on supervision presents fundamental limitations. First, for unsupervised tasks such as node clustering and link prediction, supervised information is unavailable, and most existing unsupervised GNNs either reconstruct original information (adjacency and attribute matrices) or maximize mutual information to preserve as much data as possible. Consequently, the over-smoothing problem tends to be more severe in unsupervised GNNs compared to their semi-supervised counterparts. Second, although learning message aggregation from labeled nodes can mitigate smoothing issues, it often leads to serious over-fitting that severely impacts performance.

Through comprehensive literature review and analysis, we first examine the research objectives across various studies, finding that some focus on the low-dimensional representation of nodes in complex graphs while others analyze the over-smoothing problem during information propagation. We then investigate methodological approaches, revealing that most literature centers on graph embedding algorithms—some employing probability graph-based methods and others leveraging deep learning techniques. Additionally, we find that while many studies address semi-supervised and unsupervised graph neural networks, fully supervised approaches remain less common. Finally, we analyze evaluation methodologies, identifying works that measure classification accuracy and others that assess computational efficiency. By synthesizing these methodological and evaluative perspectives, we identify the distinctive characteristics and innovations of each contribution, and based on our analysis, propose future research directions and optimization opportunities for graph neural networks.

The remainder of this paper is organized as follows. Section 2 presents the classification of research objects in unsupervised graph neural networks. Section 3 discusses the classification of research methods. Section 4 reviews experimental analyses and comparisons in the literature. Section 5 explores future research opportunities, and Section 6 concludes the paper.

2 Classification of Research Objects

In recent years, graph neural networks have attracted considerable scholarly attention, with research spanning numerous domains. To clarify this landscape, we classify graph neural network studies using two distinct criteria: (1) network architecture type and (2) research topic focus.

Network Architecture Types: We identify three primary categories: graph convolutional networks, graph neural networks, and generative adversarial networks. Most existing graph neural networks are designed for semi-supervised tasks because supervised information can alleviate the over-smoothing problem during message propagation. Both graph convolutional networks and graph neural networks can learn low-dimensional node representations through graph embedding.

Research Topics: We distinguish two fundamental problems: low-dimensional node representation and over-smoothing during information transfer. Graph embedding aims to represent each node as a low-dimensional vector for downstream tasks. In semi-supervised learning, supervised information helps mitigate over-smoothing in message delivery. Unfortunately, due to the absence of such information, the over-smoothing problem becomes more acute in unsupervised tasks. Consequently, over-smoothing affects graph convolutional networks and generative adversarial networks, while low-dimensional representation challenges are present in both graph convolutional networks and general graph neural networks.

Based on these classification standards, we present the taxonomy in Table 1, with each category defined as follows:

Type I: Studies the low-dimensional representation of nodes in graph convolutional neural networks through graph embedding algorithms that learn representations in complex networks.

Type II: Investigates low-dimensional node representation in graph neural networks.

Type III: Addresses low-dimensional node representation in generative adversarial networks.

Type IV: Examines over-smoothing problems in information transfer within graph convolutional networks.

Type V: Studies over-smoothing issues in information transfer within graph neural networks.

Type VI: Focuses on over-smoothing problems in information transfer within graph generative networks.

Explanation of Different Types: References [9][15][19] belong to Type I. Reference [9] proposes methods to effectively capture and measure affinity relationships between nodes while learning low-dimensional representations of attributes and nodes in a unified semantic space. To obtain high-quality embeddings, it introduces a variational autoencoder that embeds Gaussian distributions characterized by mean and variance for each node and attribute. Reference [15] presents Deep Graph Infomax (DGI), a general approach for unsupervised learning of node representations in graph-structured data. DGI maximizes mutual information between patch representations and corresponding high-level graph summaries, both derived from established graph convolutional network architectures. Reference [19] explores Jumping Knowledge (JK) networks, which flexibly leverage different neighborhood ranges for each node to achieve improved structure-aware representations.

References [1][3] belong to Type II. Reference [1] introduces GraRep, a model for learning weighted graph vertex representations through low-dimensional vectors that integrate global structural information—addressing limitations of existing methods that cannot capture diverse connection patterns. Reference [3] proposes Node2vec, an algorithmic framework for learning continuous feature representations of nodes that maximizes preservation of network neighborhoods through a biased random walk process.

References [8][13][7] belong to Type IV. Reference [8] demonstrates that graph convolution in GCN models is a special form of Laplacian smoothing, which enables GCN performance but also introduces over-smoothing problems in deep architectures. To overcome shallow architecture limitations, it proposes co-training and self-training methods for GCNs. Reference [13] introduces DropEdge, a novel technique that alleviates over-smoothing and over-fitting by randomly removing edges at each training stage, functioning as both data augmentation and message deceleration. The paper theoretically proves that DropEdge can slow over-smoothing convergence and mitigate information loss, serving as a general enhancement for various backbone models. Reference [7] derives an improved propagation scheme based on personalized PageRank, establishing a relationship between GCN and PageRank. This scheme constructs the Personalized Propagation of Neural Predictions (PPNP) model and its fast approximation, APPNP.

Reference [16] belongs to Type VI. It proposes a framework addressing attentional limitations in graph attention networks and over-smoothing on decision boundaries. The work first theoretically proves GAT's over-smoothing behavior, then develops constrained attention weights based on class boundaries and feature aggregation patterns, with additional graph structure constraints to alleviate over-fitting.

3 Classification of Research Methods

Most existing graph neural networks are designed for semi-supervised learning tasks using deep learning, where supervised information mitigates over-smoothing. We classify research methods using two independent criteria: (1) machine learning paradigm and (2) graph embedding algorithm type.

Machine Learning Paradigms: We consider three categories: semi-supervised learning, unsupervised learning, and supervised learning. While many AI methods exist, graph neural network research predominantly employs machine learning, with semi-supervised approaches being most common because supervised information alleviates over-smoothing. This paper examines all three paradigms.

Graph Embedding Algorithms: We identify two approaches: probability model-based and deep learning-based methods. The literature employs these distinct graph embedding algorithms for graph clustering, providing another important distinguishing criterion.

Based on these classification standards, we present the taxonomy in Table 2, with each category defined as follows:

Type I: Addresses over-smoothing in semi-supervised graph neural networks using probability graph model-based embedding methods.

Type II: Tackles information transfer flatness in unsupervised graph neural networks using probability graph model-based embedding methods.

Type III: Overcomes information transfer challenges in fully supervised graph neural networks using probability graph model-based embedding methods.

Type IV: Mitigates over-smoothing in semi-supervised graph neural networks using deep learning model-based embedding methods.

Type V: Addresses over-smoothing in unsupervised graph neural networks using deep learning model-based embedding methods.

Type VI: Overcomes over-smoothing in fully supervised graph neural networks using probability graph model-based embedding methods.

Explanation of Different Types: References [15][10][11] belong to Type I. Reference [15] proposes Deep Graph Infomax (DGI), an unsupervised method for learning node representations in graph-structured data by maximizing mutual information between patch representations and high-level graph summaries derived from graph convolutional architectures. The learned patch representations summarize node-centered subgraphs for downstream tasks. Reference [10] introduces DeepWalk, which translates advances in language modeling and unsupervised feature learning to graphs by treating truncated random walks as sentences to learn latent representations from local information. Reference [11] employs Graphical Mutual Information (GMI) to measure correlations between input graphs and high-level hidden representations, extending traditional mutual information calculation from vector spaces to graph domains. GMI measures mutual information through node characteristics and topological structure while maintaining invariance to isomorphic transformations—an essential constraint in graph representation learning.

References [3][20] belong to Type II. Reference [3] presents Node2vec, which learns mappings from nodes to low-dimensional feature spaces that maximize preservation of network neighborhoods through a flexible biased random walk process. Reference [20] proposes a semi-supervised learning framework based on graph embedding with transductive and inductive variants that jointly predict class labels and neighborhood context. The transductive variant determines labels from learned embeddings and input features, while the inductive variant defines embeddings as parametric functions of feature vectors, enabling prediction for unseen instances.

Reference [1] belongs to Type III, proposing GraRep for learning weighted graph vertex representations through low-dimensional vectors that incorporate global structural information.

Reference [12] belongs to Type IV, introducing an adversarial graph data embedding framework that encodes topological structure and node content into compact representations. It trains decoders to reconstruct structural compositions and matches latent representations to prior distributions through adversarial training, developing Adversarially Regularized Graph Autoencoder (ARGA) and Adversarial Variational Graph Autoencoder (ARVGA).

References [9][4][7] belong to Type V. Reference [9] proposes a Collaborative Attributed Network (CAN) model that co-embeds attributes and nodes in the same semantic space to capture their similarity, using a variational autoencoder with Gaussian distribution embedding. Reference [4] addresses community structure detection, where network nodes cluster tightly with loose inter-group connections, proposing a method using centrality indices to identify community boundaries. Reference [7] derives PPNP and APPNP from the relationship between GCN and personalized PageRank, as previously described.

4 Review of Experimental Analysis

This section classifies evaluation metrics and system parameters, as shown in Table 3, which organizes experimental analyses according to these dimensions. Most references compare algorithmic superiority and effectiveness.

4.1 Metric of Evaluation

Accuracy measures the proportion of correctly classified nodes relative to actual classifications in social networks, calculated as:

Accuracy = Number of correctly categorized nodes / Number of actually categorized nodes

Efficiency refers to the time complexity of an algorithm or model being lower than existing alternatives, expressed as:

T(n) = O(f(n))

Other metrics include Macro-F1 Score, Micro-F1 Score, and Normalized Mutual Information (NMI).

4.2 System Parameters

Algorithm refers to the method used for node classification in social networks. Algorithms with lower time complexity enable more accurate node division than existing methods, though semi-supervised graph neural networks can mitigate certain limitations through supervised information.

Model denotes the architecture employed for node classification, where different models achieve varying effectiveness across problems. Additional parameters include experimental configurations for graph neural networks and comparative studies with existing models.

4.3 Experimental Comparison

Reference [15] evaluates its method on synthetic networks, reporting average classification accuracy (with standard deviation) at test nodes after 50 training sessions followed by logistic regression, demonstrating DGI's high accuracy. Reference [8] conducts extensive experiments on real benchmarks to validate its theory and methods, including joint training, self-training combinations, and cross-experiments, showing high accuracy in GCN node classification. Reference [6] evaluates BTLSC community detection performance across nine real-world networks with ground-truth communities, demonstrating high node classification accuracy. Reference [13] tests its model with 2, 8, and 32 layers on different datasets, showing DropEdge improves test accuracy. Reference [11] compares its method with other unsupervised approaches, with GMI-Mean and GMI-Adaptive achieving best classification accuracy across three datasets—benefiting from encoded representations that maximally preserve node features and topology.

Reference [1] experiments on clustering, classification, and visualization tasks across three graph types, showing GraRep effectively integrates different k-step local relation information into global graph representations for diverse applications. Reference [14] compares its LINE model with several scalable graph embedding methods on large networks, demonstrating LINE (second-order) outperforms DeepWalk's combination of first and second-order proximity on original networks in most cases, capturing most information as a highly effective and efficient method for both dense and sparse networks. Reference [7] experiments on GCN, N-GCN, GAT, BootStrapped Feature Propagation, and JK models, showing its PPNP model achieves high node classification accuracy. Reference [20] demonstrates its semi-supervised learning framework based on graph embedding achieves high accuracy in text classification compared to existing methods. Reference [17] evaluates its SGC model on Cora, Citeseer, and Pubmed datasets, showing better performance on Citeseer due to fewer parameters and reduced over-fitting. Reference [2] tests dual-Sinkhorn on the MNIST dataset, demonstrating high efficiency and low time complexity.

5 Discussion and Suggestion

Our analysis reveals that most graph neural network research targets semi-supervised tasks because supervised information can mitigate message smoothing problems. However, the over-smoothing problem becomes more severe in unsupervised settings where no supervision is available. Therefore, unsupervised graph neural network research represents an important direction. Deep learning has undoubtedly become the preferred methodology, yet most graph neural network research employs supervised and unsupervised graph embedding algorithms, with few studies investigating fully supervised deep learning-based methods. We propose the following future research directions:

1) Investigate low-dimensional node representation in graph attention networks using deep learning-based graph embedding algorithms, evaluated through node classification accuracy.

2) Conduct in-depth comparative studies on the accuracy and efficiency of various practical methods to identify optimal approaches for different problem domains.

Conclusions

Our analysis indicates that most graph neural network research focuses on semi-supervised learning tasks, while the over-smoothing problem in message delivery becomes more pronounced in unsupervised settings. Future work should further investigate graph neural networks in unsupervised modes, particularly deep learning-based graph embedding algorithms for learning low-dimensional representations in graph attention networks—an area currently underexplored. Such research methodologies warrant deeper investigation.

References

[1] S. Cao, W. Lu, and Q. Xu, "Grarep: Learning graph representations with global structural information," in CIKM, 2015, pp. 891–900.
[2] M. Cuturi, "Sinkhorn distances: Lightspeed computation of optimal transport," in NIPS, 2013, pp. 2292–2300.
[3] A. Grover and J. Leskovec, "node2vec: Scalable feature learning for networks," in SIGKDD, 2016, pp. 855–864.
[4] M. Girvan and M. E. J. Newman, "Community structure in social and biological networks," PNAS, vol. 99, no. 12, pp. 7821–7826, 2002.
[5] D. Jin, K. Wang, G. Zhang, P. Jiao, D. He, F. Fogelman-Soulie, and X. Huang, "Detecting communities with multiplex semantics by distinguishing background, general and specialized topics," 2019, pp. 1–1.
[6] T. N. Kipf and M. Welling, "Semi-supervised classification with graph convolutional networks," in ICLR, 2017.
[7] J. Klicpera, A. Bojchevski, and S. Gunnemann, "Predict then propagate: Graph neural networks meet personalized pagerank," in ICLR, 2019.
[8] Q. Li, Z. Han, and X. Wu, "Deeper insights into graph convolutional networks for semi-supervised learning," in AAAI, 2018, pp. 3538–3545.
[9] Z. Meng, S. Liang, H. Bao, and X. Zhang, "Co-embedding attributed networks," in WSDM, 2019, pp. 393–401.
[10] B. Perozzi, R. Al-Rfou, and S. Skiena, "Deepwalk: online learning of social representations," in SIGKDD, 2014, pp. 701–710.
[11] Z. Peng, W. Huang, M. Luo, Q. Zheng, Y. Rong, T. Xu, and J. Huang, "Graph representation learning via graphical mutual information maximization," in WWW, 2020, pp. 259–270.
[12] S. Pan, R. Hu, G. Long, J. Jiang, L. Yao, and C. Zhang, "Adversarially regularized graph autoencoder for graph embedding," in IJCAI, 2018, pp. 2609–
[13] Y. Rong, W. Huang, T. Xu, and J. Huang, "Dropedge: Towards deep graph convolutional networks on node classification," in ICLR, 2020.
[14] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, "LINE: large-scale information network embedding," in WWW, 2015, pp. 1067–1077.
[15] P. Velickovic, W. Fedus, W. L. Hamilton, P. Lio, Y. Bengio, and R. D. Hjelm, "Deep graph infomax," in ICLR, 2019.
[16] G. Wang, R. Ying, J. Huang, and J. Leskovec, "Improving graph attention networks with large margin-based constraints," NeurIPS Workshop, vol. abs/1910.11945, 2019.
[17] F. Wu, A. H. S. Jr., T. Zhang, C. Fifty, T. Yu, and K. Q. Weinberger, "Simplifying graph convolutional networks," in ICML, 2019, pp. 6861–6871.
[18] K. Xu, W. Hu, J. Leskovec, and S. Jegelka, "How powerful are graph neural networks?" in ICLR, 2019.
[19] K. Xu, C. Li, Y. Tian, T. Sonobe, K. Kawarabayashi, and S. Jegelka, "Representation learning on graphs with jumping knowledge networks," in ICML, 2018, pp. 5449–5458.
[20] Z. Yang, W. W. Cohen, and R. Salakhutdinov, "Revisiting semi-supervised learning with graph embeddings," in ICML, 2016, pp. 40–48.