Research

Testing software with models and language models

My research sits in software engineering, mostly in automated testing: how to get trustworthy test oracles, how to derive tests from models of safety-critical systems, and how to explore large configuration spaces. Most of it is collaborative work with colleagues in FBK's Software Engineering unit.

01

LLM-generated test oracles

Automatically generated tests often encode what the code does rather than what it should do. This work asks large language models to infer intended behaviour from documentation and turn it into assertions, and measures when that actually helps.

Projects: AugmenTest, EvoOracle

Journal articleSCP · 2026

AugmenTest: A tool for improving test quality through automatic assertion generation

Shaker Mahmud Khandaker, Fitsum Kifetew, Davide Prandi and Angelo Susi

Science of Computer Programming, vol. 254, article 103531. Elsevier.

Cite (BibTeX)
@article{khandaker2026augmentest,
  title = {AugmenTest: A tool for improving test quality through automatic assertion generation},
  author = {Shaker Mahmud Khandaker and Fitsum Kifetew and Davide Prandi and Angelo Susi},
  journal = {Science of Computer Programming},
  year = {2026},
  volume = {254},
  pages = {103531},
  publisher = {Elsevier},
  doi = {10.1016/j.scico.2026.103531}
}
Conference paperICST 2025 · 2025

AugmenTest: Enhancing Tests with LLM-Driven Oracles

Shaker Mahmud Khandaker, Fitsum Kifetew, Davide Prandi and Angelo Susi

2025 IEEE Conference on Software Testing, Verification and Validation (ICST), Naples, Italy, pp. 279–289. IEEE.

Cite (BibTeX)
@inproceedings{khandaker2025augmentest,
  title = {AugmenTest: Enhancing Tests with LLM-Driven Oracles},
  author = {Shaker Mahmud Khandaker and Fitsum Kifetew and Davide Prandi and Angelo Susi},
  booktitle = {2025 IEEE Conference on Software Testing, Verification and Validation (ICST), Naples, Italy},
  year = {2025},
  pages = {279--289},
  publisher = {IEEE},
  doi = {10.1109/ICST62969.2025.10988926}
}
ThesisM.Sc. thesis · 2023

Exploring the Feasibility and Effectiveness of Large Language Models for Automated Test Oracle Generation in Software Testing

Shaker Mahmud Khandaker

Master's thesis, M.Sc. in Data Science, University of Trento. Supervisors: Fitsum Meshesha Kifetew, Paolo Rota.Academic year 2022/2023

Cite (BibTeX)
@mastersthesis{khandaker2023exploring,
  title = {Exploring the Feasibility and Effectiveness of Large Language Models for Automated Test Oracle Generation in Software Testing},
  author = {Shaker Mahmud Khandaker},
  howpublished = {Master's thesis, M.Sc. in Data Science, University of Trento. Supervisors: Fitsum Meshesha Kifetew, Paolo Rota},
  year = {2023}
}

02

Model-based testing of safety-critical systems

Deriving tests from models of system behaviour, applied to railway interlocking systems.

Conference paperISoLA 2024 · 2024

Model-Based Testing of Railway Interlocking Systems

Alessandro Cimatti, Shaker Khandaker, Fitsum Kifetew, Lorenzo Leone, Davide Prandi, Giuseppe Scaglione, Angelo Susi and Orazio Turboli

Leveraging Applications of Formal Methods, Verification and Validation. Application Areas (ISoLA 2024), Lecture Notes in Computer Science, pp. 112–126. Springer.

Cite (BibTeX)
@inproceedings{cimatti2024model,
  title = {Model-Based Testing of Railway Interlocking Systems},
  author = {Alessandro Cimatti and Shaker Khandaker and Fitsum Kifetew and Lorenzo Leone and Davide Prandi and Giuseppe Scaglione and Angelo Susi and Orazio Turboli},
  booktitle = {Leveraging Applications of Formal Methods, Verification and Validation. Application Areas (ISoLA 2024), Lecture Notes in Computer Science},
  year = {2024},
  pages = {112--126},
  publisher = {Springer},
  doi = {10.1007/978-3-031-75390-9_8}
}

03

Configuration optimisation with feature models

Using feature models to explore and interactively optimise the configuration space of complex systems.

Conference paperIn conference programmeQUATIC 2026 · 2026

Interactive Optimization of System Configurations Using Feature Models

Shaker Mahmud Khandaker, Fitsum Meshesha Kifetew, Davide Prandi and Angelo Susi

International Conference on the Quality of Information and Communications Technology (QUATIC 2026), Requirements Engineering track.Full paper in the QUATIC 2026 scientific programme (9 September 2026). Proceedings details to follow.

Cite (BibTeX)
@inproceedings{khandaker2026interactive,
  title = {Interactive Optimization of System Configurations Using Feature Models},
  author = {Shaker Mahmud Khandaker and Fitsum Meshesha Kifetew and Davide Prandi and Angelo Susi},
  booktitle = {International Conference on the Quality of Information and Communications Technology (QUATIC 2026), Requirements Engineering track},
  year = {2026}
}

04

Assistive technology

Earlier work on low-cost hardware: a refreshable braille display built from micro servos.

Conference paperICIEV 2018 · 2018

A Portable Braille Refreshable Display Using Micro Servos

A M Muntasir Rahman, Shaker Mahmud Khandaker, Nasif Noor Saleheen, Tasnia Nobi Afee, Naba Afrin and Md. Ashraful Alam

2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 212–217. IEEE.

Cite (BibTeX)
@inproceedings{rahman2018portable,
  title = {A Portable Braille Refreshable Display Using Micro Servos},
  author = {A M Muntasir Rahman and Shaker Mahmud Khandaker and Nasif Noor Saleheen and Tasnia Nobi Afee and Naba Afrin and Md. Ashraful Alam},
  booktitle = {2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR)},
  year = {2018},
  pages = {212--217},
  publisher = {IEEE},
  doi = {10.1109/ICIEV.2018.8641028}
}