Sarah Finch

Logo

★ ★ ★ ★ ★
Ph.D., Computer Science (2024)
Emory University

AI/LLM engineer and researcher with 5+ years of experience in machine learning model development and evaluation for human-computer interaction, text processing, and data generation.

Currently working as a Post-Doctoral Researcher for developing LLM systems for the classification and understanding of nursing assessment videos.

View My GitHub Profile

Portfolio

1. – Emora Chatbot: Winner of Amazon Alexa Prize 3 (2020)

Emora is a chatbot developed during the Amazon Alexa Prize Socialbot Grand Challenge which won the competition in 2020, advancing through two elimination rounds based on user ratings and then receiving the highest overall rating from the panel of final invited judges.

An example conversation Emora can hold:

The system architecture of the Emora Chatbot:

The user ratings Emora received during the quarter and semifinal rounds of the competition:

More Information:

2. – Article QABot using Generative AI

Article QABot is an article-grounded conversational question-answering dialogue system that ingests online FAQ documents in order to offer customer support. It is a modular system that incorporates numerous dialogue-relevant tasks, including information-retrieval, hallucination-detection, and response generation, and leverages prompt-based large language model approaches.

An example prompt for response generation:

Overall Article QABot architecture:

Measured response correctness for Article QABot:

3. – ConvoSenseGenerator

ConvoSenseGenerator is a fine-tuned T5 model that generates commonsense inferences for a provided dialogue context. It is fine-tuned on a new dialogue commonsense dataset, ConvoSense, collected using GPT that boasts greater contextual novelty, a higher volume of inferences per example, and substantially enriched detail compared to previous datasets.

Example commonsense inference outputs of the ConvoSenseGenerator:

Illustration of the ConvoSense ChatGPT framework including an example of the prompt:

Empirical results from human evaluation demonstrating the superiority of the ConvoSenseGenerator:

More Information:

4. – Explicit Reasoning over Commonsense for Dialogue Response Generation

ConvoSense-E (CS-E) is a commonsense-grounded dialogue system that leverages explicit reasoning similar to chain-of-thought prompting to integrate ConvoSenseGenerator outputs into dialogue response generation using GPT. It is highly preferred by human judges compared to alternative dialogue systems.

Example dialogue showing motivation of explicit reasoning over commonsense inferences to guide follow-up response generation:

Illustrative responses generated from the proposed approach (CS-E) and alternatives:

Human evaluation results showing superiority of proposed approach (ConvoSense-E) against alternatives:

5. – Annotation of Behaviors in Chat Evaluation (ABC-Eval): Dialogue System Evaluation Framework

ABC-Eval is a novel evaluation framework for chat-oriented dialogue systems that measures the rate of 16 different dialogue behaviors that can be expressed by chatbots. It is a web-based annotation platform that was built on top of the ParlAI Javascript framework with major modifications to support the annotation requirements of the 16 ABC-Eval tasks.

The online interface for annotating the usage of Correct Facts and Incorrect Facts:

The online interface for annotating consistency mistakes:

More Information:

Page template forked from evanca