Jiachen Lian

Jiachen Lian (连甲琛)

I am PhD Candidate at Berkeley EECS and I am affiliated at Berkeley Artificial Intelligence Research (BAIR) where I am advised by Prof. Gopala Krishna Anumanchipalli. I also collaborate closely with Prof. Maria Luisa Gorno Tempini to revolutionize language screening for children and deliver speech AI solutions to every individual with dyslexia and aphasia worldwide for both clinical and education efforts. I am also a Visiting Researhcer at Meta AI Seamless Team.

Research

Generally, my interest is the things that are still interesting 5 years from now.

Specifically, I am currently working on:

Boosting Conversational/Healthy/Clinical/Educational system with rich spoken language understanding .

Essay

Why do so few scientists make significant contributions and so many are forgotten in the long run [1]?
[1] https://www.cs.virginia.edu/~robins/YouAndYourResearch.pdf

News

• Starting from 2025, as approved by the California State Government, public schools will adopt our language screener, where I developed the first and state-of-the-art speech dysfluency transcriber [UDM][SSDM], serving 1 million kids! See reports.

Industrial

Meta AI, CA, USA
Visiting Researcher • Sep 2024 to Now
With: Seamless Team

Meta AI, CA, USA
Research Intern • July 2024 to Sep 2024
With: Vimal Manohar

Meta AI, CA, USA
FAIR-BAIR Student Researcher • Oct 2023 to May 2024
With: Wei-Ning Hsu

Meta AI, CA, USA
Research Intern • May 2022 to Dec. 2022
With: Alexei Baevski, Wei-Ning Hsu, Michael Auli

Speech and NLP Group, Tencent AI Lab(American) , WA, USA
Research Intern • Dec 2021 to Feb. 2022
With: Chunlei Zhang, Dong Yu

Speech and NLP Group, Tencent AI Lab(American) , WA, USA
Research Intern • April 2021 to Aug. 2021
With: Chunlei Zhang, Dong Yu

Teaching

Introduction to Robotics, Berkeley EECS , CA, USA Reader • Fall 2024 Lab GSI. Instructor: Prof. Roberto Horowitz
Audio Signal Processing in Humans and Machines, Berkeley EECS , CA, USA Reader • Fall 2022 Designed ASR Lab. Instructor: Prof. Gopala Krishna Anumanchipalli
Introduction to Deep Learning, CMU LTI , PA, USA Teaching Assistant • Fall 2020 Independently designed face recognition assignment from scratch (one of the heaviest assignments). [Write-up] [Kaggle] Instructor: Prof. Bhiksha Raj

Selected Publications

	Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities Guan-Ting Lin, Jiachen Lian, Tingle Li, Qirui Wang, Gopala Krishna Anumanchipalli, Alexander H. Liu, Hung-yi Lee, Tech Report. The first benchmark for end-to-end full-duplex spoken dialogue system. [Project Page] [Code]
	SSDM 2.0: Time-Accurate Speech Rich Transcription with Non-Fluenciesg Jiachen Lian, Xuanru Zhou, Zoe Ezzes, Jet Vonk, Brittany Morin, David Baquirin, Zachary Miller, Maria Luisa Gorno Tempini and Gopala Krishna Anumanchipalli, Tech Report. An efficient AI Agent for Language Screening and Spoken Language Learning. [Project Page]
	SSDM: Scalable Speech Dysfluency Modeling Jiachen Lian, Xuanru Zhou, Zoe Ezzes, Jet Vonk, Brittany Morin, David Baquirin, Zachary Miller, Maria Luisa Gorno Tempini and Gopala Krishna Anumanchipalli, 2024 NeurIPS. An AI Agent for Speech Therapy and Spoken Language Learning. A foundation model for scientific research, engineering deployment and business development . [Project Page] ( NeurIPs Scholar Award )
	Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection Xuanru Zhou, Jiachen Lian, Cheol Jun Cho, Zoe Ezzes, Jet M.J. Vonk, Brittany T. Morin, David Paul Galang Baquirin, Zachary A. Miller, Maria Luisa Gorno-Tempini, Gopala Anumanchipalli, Tech Report. Open Source Benchmarking Dysfluency Modeling** [Project Page]
	Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection Xuanru Zhou, Cheol Jun Cho, Ayati Sharma, Brittany Morin, David Baquirin, Jet Vonk, Zoe Ezzes, Zachary Miller, Maria Luisa Gorno Tempini, Jiachen Lian, and Gopala Krishna Anumanchipalli, 2024 SLT . Multi-lingual Co-Dysfluency Detector with Articulatory Simulation [Code] ( Student Grant Award )
	YOLO-Stutter: End-to-End Region-Wise Speech Dysfluency Detection Xuanru Zhou, Anshul Kashyap, Steve Li, Ayati Sharma, Brittany Morin, David Baquirin, Jet Vonk, Zoe Ezzes, Zachary Miller, Maria Luisa Gorno Tempini, Jiachen Lian, and Gopala Krishna Anumanchipalli, 2024 Interspeech . Dysfluency Modeling as Object Detection . [Code] ( ICSA Student Grant Award ).
	Towards Hierarchical Spoken Language Dysfluency Modeling Jiachen Lian, and Gopala Krishna Anumanchipalli, 2024 EACL (Oral Presentation). Hierarchical extension of UDM with monotonicity injection. .
	Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection Jiachen Lian, Carly Feng, Naasir Farooqi, Steve Li, Anshul Kashyap, Cheol Jun Cho, Peter Wu, Robbie Netzorg, Tingle Li, and Gopala Krishna Anumanchipalli, 2023 ASRU (Best Paper Nomination ). First work to detect both type and time of dys(dis)fluencies. 2024 Sevin Rosen Funds Award
	Deep Speech Synthesis from MRI-Based Articulatory Representations Peter Wu, Tingle Li, Yijing Lu, Yubin Zhang, Jiachen Lian, Alan Black, Louis Goldstein, Shinji Watanabe, and Gopala Krishna Anumanchipalli, 2023 Interspeech.
	AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations Jiachen Lian, Alexei Baevski, Wei-Ning Hsu, and Michael Auli, 2023 ASRU.
	Articulatory Representation Learning Via Joint Factor Analysis and Neural Matrix Factorization Jiachen Lian, Alan W Black, Yijing Lu, Louis Goldstein, Shinji Watanabe, and Gopala Krishna Anumanchipalli, 2023 ICASSP.
	UTTS: Unsupervised TTS with Conditional Disentangled Sequential Variational Auto-encoder Jiachen Lian, Chunlei Zhang, Gopala Krishna Anumanchipalli, and Dong Yu. IEEE/ACM Transactions on Audio, Speech, and Language Processing [Project Page]
	Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition Jiachen Lian, Alan W Black, Louis Goldstein, and Gopala Krishna Anumanchipalli. 2022 Interspeech [code]
	Towards Improved Voice Conversion with Conditional DSVAE Jiachen Lian, Chunlei Zhang, Gopala Krishna Anumanchipalli, and Dong Yu. 2022 Interspeech [Project Page]
	Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion Jiachen Lian, Chunlei Zhang, and Dong Yu. 2022 ICASSP [Demo]
	Masked Proxy Loss For Text-Independent Speaker Verification Jiachen Lian, Aiswarya Vinod Kumar, Hira Dhamyal, Bhiksha Raj, and Rita Singh. 2021 Interspeech (ISCA Student Grant Award) [Code]
	Detection and Evaluation of Human and Machine Generated Speech in Spoofing Attacks on Automatic Speaker Verification Systems Yang Gao, Jiachen Lian, Bhiksha Raj, and Rita Singh. 2021 SLT.