Anvesh Rao Vijjini

Hi! I am a third year PhD student advised by Prof. Snigdha Chaturvedi at UNC-Chapel Hill. Previously I was working as an NLP Researcher, helping make Bixby, Samsung‘s AI assistant, smarter.

And before that, I was a Computational Linguistics Master's student under the guidance of Prof. Radhika Mamidi with my thesis on Curriculum Learning in Sentiment Analysis at Language Technologies Research Center, IIIT-H.

I like to sketch, bike around, explore origami, and learn new languages in my free time.

CV     |   |   |   | 

• [Oct 2023] Paper accepted at EMNLP 2023 (Findings).
• [Apr 2023] I am interning at Bloomberg this summer.
• [Jan 2023] Paper accepted at EACL 2023 (Findings).
• [Oct 2022] Paper accepted at EMNLP 2022.
• [May 2022] I am joining Adobe as a Research Scientist intern for Summer 2022.
• [Apr 2021] I will be joining UNC-Chapel Hill as a PhD student in Fall 2021.
• [Dec 2020] Paper accepted at EACL 2021.
• [Aug 2019] Paper accepted at EMNLP-IJCNLP 2019.
• [Jul 2018] Paper accepted at COLING 2018.

I am interested in Creative NLP and Language Generation. Language Model's ability to generate narratives with controllable parameters reflects their generation capability. I have also researched building NLP models efficient in parameters and robust to variations in data. I consider interpreting models to be a crucial step in achieving this. Only when we find out what ML models learn; can we guide them to learn better. I also contribute to improving resource-scarce languages' representation by curating resources in manners that require minimal human intervention.

I have applied my research interests across NLP applications such as Text Generation, Sentiment Analysis, Reading Comprehension, Question Generation, and WER Estimation. Representative papers are highlighted.

Narrative Language Understanding and Generation

Story Generation has applications in education and interactive learning. I proposed ReLiST [1] for story generation with control over relationships. ReLiST addresses the most important challenge of this task - writing a story coherently while reflecting the given relationships.

Relevant Publications:

[1] PARROT, a zero-shot approach for enhancing narrative reading comprehension via parallel reading
Chao Zhao, Anvesh Rao Vijjini, Snigdha Chaturvedi
EMNLP, 2023 (Findings) [link]

[2] Towards Inter-character Relationship-driven Story Generation
Anvesh Rao Vijjini, Faeze Brahman, Snigdha Chaturvedi
EMNLP, 2022 [arXiv] [anthology] [ppt] [poster]

Model Interpretability

Interpreting Machine Learning models is the key to identifying potential challenges and improvements while also moving towards Responsible AI.

I have researched and identified the behavior of CNNs, especially what convolutional layers are good at learning and what they are not. To address this, I propose SCARN - a model effectively utilizing recurrent and convolutional structures for text classification [1]. I have also researched and identified when Curriculum Learning (CL) works, especially in sentiment analysis. Using BERT’s attention visualizations, we give qualitative explanations of how CL works; by breaking down a more challenging problem into multiple easier subproblems [2].

Relevant Publications:

[1] Sequential Learning of Convolutional Features for Effective Text Classification
Avinash Madasu, Anvesh Rao Vijjini
EMNLP-IJCNLP, 2019 [arXiv] [anthology] [ppt]

[2] Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axes
Anvesh Rao Vijjini*, Kaveri Anuranjana*, Radhika Mamidi
WASSA at EACL, 2021 [arXiv] [anthology] [ppt]

Lately, performance improvements in NLP tasks are being made based on increasing parameters rather than efficiently training a single model. I am interested in achieving efficiency in NLP models where performance improvements can be made while maintaining model sizes hence moving towards Green NLP.

Efficiency can be achieved in multiple ways; my work explores some of these methods. Curriculum Learning introduces an ordering to the training while using the same model [2]. We have introduced self-normalizing layers within CNN for text classification to achieve better generalization while reducing parameters [4]. I have proposed parameter less decay based weighting layers to weight words closer to aspect terms more than farther ones for Aspect-Based Sentiment Analysis [3]. More recently, I proposed WER-BERT for Automatic WER Estimation, which uses a custom loss function for exploiting the ordinal nature of the WER classification task [1]. I am also interested in distilling models and semi-supervised learning.

Relevant Publications:

[1] Curricular Next Conversation Prediction Pretraining for Transcript Segmentation
Anvesh Rao Vijjini, Hanieh Deilamsalehy, Franck Dernoncourt, Snigdha Chaturvedi
EACL, 2023 (Findings) [anthology]

[2] WER-BERT: Automatic WER Estimation with BERT in a Balanced Ordinal Classification Paradigm
Anvesh Rao Vijjini*, Akshay Krishna Sheshadri*, Sukhdeep Kharbanda
EACL, 2021 [arXiv] [anthology] [ppt] [poster]

[3] A SentiWordNet Strategy for Curriculum Learning in Sentiment Analysis
Anvesh Rao Vijjini*, Kaveri Anuranjana*, Radhika Mamidi
NLDB, 2020 [arXiv] [springer] [ppt] [code]

[4] A Position Aware Decay Weighted Network For Aspect Based Sentiment Analysis
Avinash Madasu, Anvesh Rao Vijjini
NLDB, 2020 [arXiv] [springer] [ppt]

[5] Effectiveness of Self Normalizing Neural Networks for Text Classification
Avinash Madasu, Anvesh Rao Vijjini
CICLing, 2019 [arXiv] [ppt] [poster]

Robust Natural Language Processing

My research at Samsung focused on Domain Adaptation, which helps build models that scale across domains. I have proposed gated convolutional architectures for this problem. The gated mechanism filters out domain-specific information [2]. I further refined this idea to present a Sequential Domain Adaptation framework [1]. I am also interested in exploring Robust NLP for building models that are impervious to noise and are unbiased.

Relevant Publications:

[1] Sequential Domain Adaptation through Elastic Weight Consolidation for Sentiment Analysis
Avinash Madasu, Anvesh Rao Vijjini
ICPR, 2020 [arXiv] [ppt] [poster]

[2] Gated Convolutional Neural Networks for Domain Adaptation
Avinash Madasu, Anvesh Rao Vijjini
NLDB, 2019 [arXiv] [springer]

Modern machine learning algorithms rely heavily on data, and this reliance has adverse effects in Resource-Scarce Settings. I improved two pre-existing lexicon-based resources in Telugu [3,4,6]. We also created Hindi resources for Question Answering. To aid Educational Applications, we divided the resource into academic grades based on difficulty [5].

While resource creation for scarce languages is essential, hand-curating datasets can be time and labor-intensive. Moving towards Automated Resource Creation, we developed a solution to exploit the massive emoji-rich Social Media presence of Indian languages to automate Multilingual Sentiment Analysis datasets [1]. We also made a linguistically informed rule-based system that relies on semantic roles identified via dependency parsing to convert Hindi sentences into questions to facilitate HindiQA [2].

Relevant Publications:

[1] Twitter corpus of Resource-Scarce Languages for Sentiment Analysis and Multilingual Emoji Prediction
Nurendra Choudary, Rajat Singh, Anvesh Rao Vijjini, Manish Shrivastava
COLING, 2018 [anthology] [data]

[2] Hindi Question Generation Using Dependency Structures
Anvesh Rao Vijjini*, Kaveri Anuranjana*, Radhika Mamidi
Humanizing AI (HAI) at IJCAI, 2019 [arXiv] [poster]

[3] Towards Automation of Sense-type Identification of Verbs in OntoSenseNet
Sreekavitha Parupalli, Anvesh Rao Vijjini, Radhika Mamidi
SocialNLP at ACL, 2018 [arXiv] [anthology]

[4] Towards Enhancing Lexical Resource and Using Sense-annotations of OntoSenseNet for Sentiment Analysis
Sreekavitha Parupalli, Anvesh Rao Vijjini, Radhika Mamidi
SemDeep-3 at COLING, 2018 [arXiv] [anthology] [code]

[5] HindiRC: A Dataset for Reading Comprehension in Hindi
Anvesh Rao Vijjini*, Kaveri Anuranjana*, Radhika Mamidi
CICLing, 2019 [researchgate] [poster] [data]

[6] BCSAT: A Benchmark Corpus for Sentiment Analysis in Telugu UsingWord-level Annotations
Sreekavitha Parupalli, Anvesh Rao Vijjini, Radhika Mamidi
Student Research Workshop (SRW) at ACL, 2018 [arXiv] [anthology]

Website design inspired from Jon Barron's website