Qiyu Han

Ph.D Candidate in Quantitative Method

Purdue University

About Me

I am currently in my final year as a Ph.D. candidate in Quantitative Methods at the Daniel Business School, Purdue University. Under the guidance of Professor Will Wei Sun , I have focused my studies on the theoretical and practical aspects of machine learning algorithms. My primary interests include statistical inference in Reinforcement Learning and Bandit settings, as well as exploring the intersections of Causal Inference and Machine Learning.

Interests

Statistical Reinforcement Learning
Tensor Learning
Online Learning
Causal Inference
Risk Quantification in Reinforcement Learning

Education

Ph.D in Quantitative Method (Statistics track), 2018 - 2024
Purdue University
Master of Science in Statistics and Computer Science, 2018 - 2024
Purdue University
Master of Science in Mathematics in Finance, 2018
New York University
Bachelor of Science in Mathematics, 2016
New York University

Teaching

Currently, I am involved in both instructing and assisting in various method-focused courses. My teaching experience includes:

As an instructor for MGMT 571 - Data Mining

I design engaging assignments and craft stimulating Kaggle competitions for Master of Science in Business Analytics and Information Management students.

In my role as a teaching assistant for several core graduate courses

In MGMT 571 - Data Mining, I collaborated with the instructor on the design of homework problems and solutions, led and designed a challenging Kaggle competition;
In MGMT 670 - Business Analytics, I taught student applying analytical methods in the business world, evaluated and graded homeworks and exams;
In MGMT 305 - Introduction to Business Statistics, I introduced students to statistics in the context of business, I lead weekly recitation sessions.

Publication Preprint

Qiyu Han, Will Wei Sun, Yichen Zhang

December, 2022 In AOS

Online Statistical Inference for Matrix Contextual Bandit.

Collaboration:
- This is a collaborative work with Will Wei Sun and Yichen Zhang. Under first round revision for Annals of Statistics (AOS).
Key Contributions:
- Introduced a novel methodology enabling online estimation and debiasing for low-rank estimators under a contextual bandit framework with a wide range of decision-making policies.
- Developed new error bound on sequential low-rank SGD estimation with adaptively collected data.
- Constructed valid confidence interval for both low-rank parameters and optimal value with data collected via a wide range of bandit policies.

Pratik Ramprasad, Qiyu Han, Zhengling Qi, Will Wei Sun

Off-Policy Evaluation For Low-Rank Tensor Markov Decision Processes.

Collaboration:
- This is a collaborative work with Pratik Ramprasad, Zhengling Qi, and Will Wei Sun. To be submitted to JASA (Journal of the American Statistical Association), manuscript available upon request.
Key Contributions:
- Developed a batch off-policy evaluation method to capture the low-rank structure in the tensor param- eter for the state-action value Q function.
- Developed novel concentration inequalities for the suprema of certain matrix-valued empirical processes under Markovian noise.

Ongoing Research Projects

Active Learning in RL with Human Feedback

Develop frameworks to incorporate human feedback into reinforcement learning to refine reward struc- tures, ensuring agent behaviors align with domain-specific quality and safety standards.
Advance research on the critical role of human feedback in establishing action constraints for safety and fairness, aiming to improve RL algorithm adaptability in real-world applications.

Statistical Inference for Contextual Bandit with Delayed Action Effects

Investigate the delayed effects of actions in critical settings, such as healthcare, where the impact of medications emerges over time and is linked to past decisions.
Conduct statistical inference on delayed feedback in contextual bandit frameworks, aiming to rigorously evaluate drug effects and optimize decision-making in healthcare applications.