Sijie's Homepage
Sijie's Homepage
Home
News
Experience
Publications
Projects
Awards
Light
Dark
Automatic
1
OpenChat: Advancing Open-source Language Models with Mixed-Quality Data
OpenChat is an innovative library of open-source language models, fine-tuned with C-RLFT - a strategy inspired by offline reinforcement learning. Our models learn from mixed-quality data without preference labels, delivering exceptional performance on par with ChatGPT, even with a 7B model which can be run on a consumer GPU (e.g. RTX 3090). Despite our simple approach, we are committed to developing a high-performance, commercially viable, open-source large language model, and we continue to make significant strides toward this vision.
Guan Wang
,
Sijie Cheng
,
Xianyuan Zhan
,
Xiangang Li
,
Sen Song
,
Yang Liu
PDF
Cite
Code
Project
Modeling Adversarial Attack on Pre-trained Language Models as Sequential Decision Making
Pre-trained language models (PLMs) have been widely used to underpin various downstream tasks. However, the adversarial attack task has …
Xuanjie Fang
,
Sijie Cheng
,
Yang Liu
,
Wei Wang
PDF
Cite
Prompt-Guided Retrieval Augmentation for Non-Knowledge-Intensive Tasks
Retrieval-augmented methods have received increasing attention to support downstream tasks by leveraging useful information from …
Zhicheng Guo
,
Sijie Cheng
,
Yile Wang
,
Peng Li
,
Yang Liu
PDF
Cite
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge
Large language models (LLMs) have been widely studied for their ability to store and utilize positive knowledge. However, negative …
Jiangjie Chen
,
Wei Shi
,
Ziquan Fu
,
Sijie Cheng
,
Lei Li
,
Yanghua Xiao
PDF
Cite
Code
Unsupervised Explanation Generation via Correct Instantiations
We introduce Neon, an unsupervised framework for generating explanations of incorrect statements by leveraging large pre-trained language models. Neon outperforms baselines on standard explanation benchmarks and demonstrates generalizability across different scenarios.
Sijie Cheng
,
Zhiyong Wu
,
Jiangjie Chen
,
Zhixing Li
,
Yang Liu
,
Lingpeng Kong
PDF
Cite
Code
Can Pre-trained Language Models Interpret Similes as Smart as Human?
Simile interpretation is a crucial task in natural language processing. Nowadays, pre-trained language models (PLMs) have achieved …
Qianyu He
,
Sijie Cheng
,
Zhixu Li
,
Rui Xie
,
Yanghua Xiao
PDF
Cite
Code
Learning What You Need from What You Did: Product Taxonomy Expansion with User Behaviors Supervision
We present a self-supervised and user behavior-oriented framework for automatically expanding product taxonomies by appending new concepts. Experimental results on a real-world e-commerce platform demonstrate the superiority of our framework, achieving significant taxonomy expansion with high precision.
Sijie Cheng
,
Zhouhong Gu
,
Bang Liu
,
Rui Xie
,
Wei Wu
,
Yanghua Xiao
PDF
Cite
Unsupervised Editing for Counterfactual Stories
Creating what-if stories requires reasoning about prior statements and possible outcomes of the changed conditions. One can easily …
Jiangjie Chen
,
Chun Gan
,
Sijie Cheng
,
Hao Zhao
,
Yanghua Xiao
,
Lei Li
PDF
Cite
Code
On commonsense cues in bert for solving commonsense tasks
This study examines how BERT utilizes commonsense cues when solving commonsense tasks. We find that BERT incorporates structural commonsense knowledge, positively impacting its accuracy in these tasks.
Leyang Cui
,
Sijie Cheng
,
Yu Wu
,
Yue Zhang
PDF
Cite
NEARM: Natural Language Enhanced Association Rules Mining
Knowledge bases (KBs), which are typical heterogeneous graphs that contain numerous triple facts of various types and relations, have …
Shiya Ren
,
Zhixing Li
,
Huaming Wang
,
Yuan Li
,
Ke Shen
,
Sijie Cheng
PDF
Cite
Cite
×