Postdoc in Large Language Models

Author:

The University of California, San Francisco, is hiring for a postdoctoral fellow.

The unprecedented ability of large language models (LLMs) to interpret text data with human-like reasoning is poised to transform many fields. Nevertheless, for LLMs to be safe and effective for use in high-risk domains like healthcare, it is crucial to understand biases embedded in this technology, as it has been shown to vary in performance across subgroups and even discriminate against minorities. This project aims to study and develop red-teaming solutions to audit LLMs and understand the limits of current approaches. Methodologies developed in this project will be tested on real-world clinical data, including unstructured notes. This project supplements our PCORI, “Diagnostic Tools for Quality Improvement of Machine Learning-Based Clinical Decision Support Systems.”

RESPONSIBILITIES

We are seeking a highly motivated postdoc researcher to join our team. The primary responsibilities are:

  • Rigorously analyze and evaluate existing red-teaming algorithms for LLMs
  • Develop new statistical methods/frameworks for comprehensive red-teaming of LLMs
  • Develop an explanation framework and statistical inference procedures to understand the systematic limitations of LLMs
  • Write, edit, and publish research manuscripts in collaboration with the team

Our predictive analytics team is highly collaborative and includes team members with wide-ranging expertise, including machine learning, biostatistics, computer science, healthcare, and regulation.

QUALIFICATIONS

The position requires a Ph.D. or equivalent in data science, (bio)statistics, computer science, or another relevant field. We are looking for someone who:

  • has experience in training and testing ML algorithms for large datasets
  • has experience in natural language processing and working with LLMs
  • has experience in methodological development and can perform independent research with a strong and relevant publication record
  • has a strong software engineering background (e.g. python, torch, hugging face, git-based workflows, high-performance computing, SQL, spark)
  • can work collaboratively with a team

The position has two years of guaranteed funding. Screening of applicants will begin immediately and continue as needed throughout the recruitment period.

If you are interested, please submit the following materials to jean.feng@ucsf.edu

  1. A cover letter
  2. A CV summarizing your education and work experience so far
  3. The names and email addresses of three references
  4. A code sample on GitHub
  5. One representative publication

Best wishes

Jean Feng

Assistant Professor
Department of Epidemiology and Biostatistics
University of California, San Francisco
https://www.jeanfeng.com/

 

Categories: POSTDOC