PhD Internship - Enhancing Small Language Models for Accurate and Reliable Information Extraction

About AstraZeneca

AstraZeneca is a global, science-led biopharmaceutical business and its innovative medicines are used by millions of patients worldwide. AstraZeneca has long been an advocate of student work placement training! These placements immerse students in the pharmaceutical industry, allowing the opportunity to contribute to our diverse pipeline of medicines whether in the lab or outside of it. You will feel trusted and empowered to take on new challenges, but with all the help and guidance you need to succeed. At AstraZeneca, you will engage in meaningful work within a pioneering research and development organization. This placement will help you develop essential skills, expand your knowledge, and build a network that will set you up for future success. You will be surrounded by curious, passionate, and open-minded professionals eager to learn and follow the science, fostering your growth in a truly collaborative and global team.

Introduction

Recent developments in language modeling have produced powerful Large Language Models that demonstrate impressive performance across various application domains. However, their dependency on large-scale computational resources makes them impractical for many real-world scenarios.

Small Language Models (SLMs) offer a promising and efficient alternative with their ability to provide faster inference with comparatively lower computational costs[1,2]. Despite their potential, SLMs struggle with tasks that require fine-grained information extraction and advanced reasoning. With the recent research trend towards developing more powerful open-source SLMs [2,3,4], we propose to take one step further in this direction by exploring the potential of these models to enhance the efficiency and interpretability of the model predictions. By focusing on test-time compute scaling [5], we aim to develop resource-efficient SLMs with improved reasoning and accuracy, particularly for applications in the healthcare domain.

The key objectives are,

• Evaluate and optimize SLMs for information extraction on semi/un-structured datasets

• Explore techniques to enable advanced reasoning capabilities in SLMs

• Design and implement mechanisms to improve the test-time compute scaling for SLMs

• Conduct extensive experiments and analyze the performance on both internal and open-source datasets and publish the research findings.

Applications will be open until 28/02/2025. Start date 02/06/2025 - 29/08/2025 and you can expect to hear from us by the end of March 2025.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment.

AstraZeneca embraces diversity and equality of opportunity. We are committed to building an inclusive and diverse team representing all backgrounds, with as wide a range of perspectives as possible, and harnessing industry-leading skills. We believe that the more inclusive we are, the better our work will be. We welcome and consider applications to join our team from all qualified candidates, regardless of their characteristics. We comply with all applicable laws and regulations on non-discrimination in employment (and recruitment), as well as work authorization and employment eligibility verification requirements.