How Attentive are Graph Attention Networks?
Explore this paper's citation graph
Summary
It is shown that GAT computes a very limited kind of attention: the ranking of the attention scores is unconditioned on the query node, and a simple fix is introduced by modifying the order of operations and proposed GATv2: a dynamic graph attention variant that is strictly more expressive than GAT.
- Type
- preprint
- Published
- 2021-05-30
- Cited by
- 2,025
- References
- 72
- Access
- Open access
- OpenAlex
- https://openalex.org/W3171970564
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:235254358
Keywords
Computer science, Graph, Theoretical computer science, Node (physics), Representation (politics)
References
- A new model for learning in graph domains
- Effective Approaches to Attention-based Neural Machine Translation
- On the approximate realization of continuous mappings by neural networks
- Approximation capabilities of multilayer feedforward networks
- Multilayer feedforward networks with a nonpolynomial activation function can approximate any function
- Quantum chemistry structures and properties of 134 kilo molecules
- Approximation by superpositions of a sigmoidal function
- The Graph Neural Network Model
- Multilayer feedforward networks are universal approximators
- Collective Classification in Network Data
- Approximation theory of the MLP model in neural networks
- Deep Residual Learning for Image Recognition
- Collective Classi!cation in Network Data
- Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs
- Geometric Deep Learning: Going beyond Euclidean data
- Neural Message Passing for Quantum Chemistry
- A simple neural network module for relational reasoning
- Graph Attention Networks
- Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
- Attention-based Graph Neural Network for Semi-supervised Learning
Cited by
- Attention-based fusion of multiple graphheat networks for structural to functional brain mapping
- Enel: Context-Aware Dynamic Scaling of Distributed Dataflow Jobs using Graph Propagation
- Graph Attention Network-Based Single-Pixel Compressive Direction of Arrival Estimation
- The CLRS Algorithmic Reasoning Benchmark
- JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints
- Effective use of sequence information to predict CRISPR-Cas9 off-target
- Duplicate Detection in a Knowledge Base with PIKA
- Passive Bimanual Skills Learning From Demonstration With Motion Graph Attention Networks
- Who's Next: Rising Star Prediction via Diffusion of User Interest in Social Networks
- Better Modeling the Programming World with Code Concept Graphs-augmented Multi-modal Learning
- Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions
- GraphWorld: Fake Graphs Bring Real Insights for GNNs
- Spatial Commonsense Graph for Object Localisation in Partial Scenes
- Robust RGB-T Tracking via Graph Attention-Based Bilinear Pooling
- Divide-and-Conquer: Post-User Interaction Network for Fake News Detection on Social Media
- Exploring Edge Disentanglement for Node Classification
- Graph Ordering Attention Networks
- Personalized Pagerank Graph Attention Networks
- Heterogeneous Information Network based Default Analysis on Banking Micro and Small Enterprise Users
- A Graph Attention Learning Approach to Antenna Tilt Optimization
Related papers
- LEARNING ON GRAPHS: ALGORITHMS FOR CLASSIFICATION AND SEQUENTIAL DECISIONS
- A Labeled Graph Approach to Support Analysis of Organizational Performance
- The Power of Local Information in Social Networks
- Live Exploration of Dynamic Rings
- Distributed exploration of dynamic rings
- Routing and search on large scale networks
- A Computational Approach to Organizational Structure