The 10th International Workshop on Empirical Software Engineering in Practice

December 13-14, 2019, Tokyo, Japan

Image copyrights yu.kleides

Welcome to IWESEP 2019

IWESEP 2019 is over. Thank you for your contribution and see you soon in other conferences or workshops!

Best Research Paper Award 🎉

Categorizing and visualizing issue tickets to better understand the features implemented in existing software systems
Ryo Ishizuka, Hironori Washizaki, Yoshiaki Fukazawa, Shinobu Saito and Saori Ouji

Best Poster Award 🎉

Identifying publication citations in source code comments using BERT
Supatsara Wattanakriengkrai, Hideaki Hata and Kenichi Matsumoto

About IWESEP 2019

The 10th International Workshop on Empirical Software Engineering in Practice (IWESEP 2019) will be held in Tokyo, Japan, on December 13-14, 2019.

IWESEP 2019 is a research forum to discuss research methods and tools on software engineering as well as findings from applying these technologies in academic or industrial settings. It encourages researchers, engineers, and practitioners to exchange their research ideas and expectations on empirical software engineering. We believe that this workshop is a good opportunity for academic researchers and industrial engineers/practitioners to develop and deepen collaborative networks among them.

IWESEP 2019 will be held in conjunction with IPSJ SIGSE technical meeting. IPSJ SIGSE will host an invited talk session on Dec. 13 as a part of the technical meeting program, and the attendees of IWESEP 2019 can join it thanks to the kindness of IPSJ SIGSE. Don't miss the event!

We have a special section on Empirical Software Engineering of IEICE Transactions on Information and System that will be published in January 2021.

It is our great honor to announce that this workshop is supported by Global Software Engineering Laboratory, Waseda University and Nara Institute of Science and Technology. We are deeply grateful for their kind support.

Important Dates

Abstract submission

September 27th, 2019 (AOE) (extended)

Research paper submission

October 4th, 2019 (AOE) (extended)


November 1st, 2019 (AOE)

Camera Ready

November 8th, 2019 (AOE)

Abstract submission (Poster)

November 5th, 2019 (AOE)

Notifications (Poster)

November 10th, 2019 (AOE)


December 4th, 2019 (AOE)

Call for Papers

The 10th International Workshop on Empirical Software Engineering in Practice (IWESEP) aims to foster the development of the field by providing a forum where researchers and practitioners reports on and discuss research results and applications in the area of empirical software engineering. The workshop encourages the exchange of ideas within the international community to better understand, from an empirical viewpoint, the strengths and weaknesses of novel and current technology, with the expectation of furthering the field of software engineering.

The workshop welcomes both original and replicated studies, varying from controlled experiments to field studies, from quantitative to qualitative. The workshop also welcomes reports on the application of existing software engineering technologies in industrial settings.

Submission Categories

The workshop provides two submission categories: Full Research Papers and Posters.

All the submissions must be in English, and must be submitted electronically before the due date via EasyChair.
Note that both of the categories share the same submission page, and the categories are differentiated by the Topics attribute.
Please make sure that you have selected the correct category before submission, otherwise your submission might not be reviewed.

Full Research Papers

Papers in this category must not exceed 6 pages. At least two program committee members will evaluate each submission. All the accepted papers will be published in the workshop proceedings with an IEEE catalog number and ISBN. The proceedings will be submitted to IEEE Xplore for publication. For each accepted paper, at least one of the authors must register for and attend the workshop to make an oral presentation of the paper.

All submissions in this category must come in PDF format and conform, at time of submission, to the IEEE Conference Proceedings Formatting Guidelines, title in 24pt font and full text in 10pt font, LaTeX users must use \documentclass[10pt,conference]{IEEEtran} without including the compsoc or compsocconf option. Also, papers must comply with the IEEE Policy on Authorship.


Authors in this category should submit an abstract (max 700 words) on the content to be presented in the poster, describing the motivation, methodology, results, and contribution of the study. The submitted abstracts will be quickly reviewed by the program committee chairs to check whether they meet the scope of the workshop. The authors of the accepted submissions will be requested to prepare the posters and bring them to the workshop venue. Note that the submissions in this category will not appear in the workshop proceedings.

Authors need to fill in the abstracts of the submissions on EasyChair submission page, in the Title and Abstract field. Poster submissions do NOT require any files to describe the abstracts.

Topics of Interest

Topics of interest include, but are not limited to:

  • Empirical studies of software process and product
  • Comparison of cost estimation techniques
  • Analysis of the effects and characteristics of design methods
  • Evaluation of the readability of coding styles
  • Development, derivation, or comparison of organizational models of software development
  • Evaluation of testing methodologies
  • Evaluation of automated patch generation techniques
  • Reports on the benefits derived from using software development environments
  • Development or comparison of predictive models for software quality (defects, vulnerabilities, and reliability) from real data
  • Infrastructure issues, such as measurement theory, experimental design, qualitative modeling, and analysis approaches
  • Experiences with research methods such as grounded theory, protocol studies, or families of experiments
  • Industrial experience in process improvement
  • Quality measurement
  • Process modeling and applications in industry
  • Experience management


All participants are kindly requested to fill out the registration form and submit it by December 4, 2019 (AOE). ​ The registration has been closed. Thank you for your registration.

The registration fee will be collected on-site. Only cash payments (no credit cards) in Japanese yen (JPY) can be acceptable. ​

  • SIGSE Member: JPY 0
  • SIGSE is a Japanese community in the research field of software engineering.
  • Non-Member: JPY 1,000

This conference does not include lunch, but we plan to hold dinner in December 13th. Please check off the box in the form if you want to attend it.

If you need visa for Japan and its application documents for your local Japanese embassy or consulate, please check off the box in the form. We will contact you to send the documents. Visa Information

For questions about the registration, please contact hatano.tomomi[at]fujitsu.com .


The 10th International Workshop on Empirical Software Engineering in Practice (IWESEP 2019) will be held at Nishiwaseda Campus of Waseda University. The workshop room will be located on the ground floor of Building No. 55, N Tower.

Nishiwaseda campus is located at a very convenient place in Tokyo. There are three train stations near that campus.

  • [Tokyo Metro] Fukutoshin line, Nishi-Waseda (西早稲田 / F11) Station (3 min on foot)
  • [JR] Yamanote line, Takadanobaba (高田馬場 / JY15) Station (13 min on foot)
  • [JR] Yamanote line, Shin-Okubo (新大久保 / JY16) Station (15 min on foot)

Using Nishi-Waseda station is the most convenient option to get to the workshop venue. If you are getting off at Nishi-Waseda station, please leave from Exit 3 (not Exit 1 or 2). This exit will take you directly to the campus. Building No. 55, where the workshop will be held, stands very closely to that exit. Be aware that it will take a few minutes walk if you use other exits.


Talk 20 mins, QA 5 mins.

Time Session
December 13th
10:00 - 11:30 Special Talk Hosted by IPSJ SIGSE
11:30 - 13:00 Lunch
13:00 - 13:20 Opening
13:20 - 14:10 Session 1 (2 talks)

Session chair: Kiyoshi Honda

An empirical study of source code detection using image classification

Juntong Hong, Osamu Mizuno and Masanari Kondo

An Exploratory Study on how Contributors Impact Code Naturalness: The Case of 50 Python Projects

Thanadon Bunkerd, Thanwadee Sunetnanta, Dong Wang, Raula Gaikovina, Takashi Ishio, Chaiyong Ragkhitwetsagul, Morakot Choetkiertikul and Kenichi Matsumoto

14:10 - 14:30 Break
14:30 - 15:20 Session 2 (2 talks)

Session chair: Masanari Kondo

Industrial Case Study on Time Series Analysis of Metrics Changes Based on GQM Models

Kiyoshi Honda, Hironori Washizaki, Yoshiaki Fukazawa, Masahiro Taga and Akira Matsuzaki

Software Team Member Configurations: A Study of Team Effectiveness in Moodle

Noppadol Assavakamhaenghan, Morakot Choetkiertikul, Suppawong Tuarob, Raula Gaikovina Kula, Hideaki Hata, Chaiyong Ragkhitwetsagul, Thanwadee Sunetnanta and Kenichi Matsumoto

15:20 - 15:40 Break
15:40 - 16:40 Poster Session

1. Visualizing Phase Transition for Real-Time 3D Profiler

Jonathan Komala, Tetsuya Kanda, Tsuyoshi Mizouchi and Katsuro Inoue

2. Identifying publication citations in source code comments using BERT

Supatsara Wattanakriengkrai, Hideaki Hata and Kenichi Matsumoto

3. Source Code Neutralization for Mining Software Repositories

Nozomi Nakajima, Shinsuke Matsumoto and Shinji Kusumoto

4. Identification of Future Active Contributors based on Activities in Outside OSS Community

Tomoki Koguchi and Akinori Ihara

5. Investigating mislabeling in making a dataset for automated NFR classification

Yuta Kimura and Masao Ohira

6. An Analysis of Computational Thinking to Implement Scratch Programs

Ryota Ando and Akinori Ihara

7. An Analysis of Patch Acceptance Using Test Case Generation Tool

Haruki Fukumoto, Akinori Ihara, Takashi Ishio, and Yuki Ueda

8. An Analysis of the Different Github Projects that Attract Newbies

Ifraz Rehman, Raula Gaikovina Kula and Kenichi Matsumoto

9. An Interactive Environment for Tailoring Automatically Untangled Changes

Satoshi Yamashita, Shinpei Hayashi and Motoshi Saeki

10. Investigating Effective Usages of Code Smell Information for Bug Localization

Aoi Takahashi, Natthawute Sae-Lim, Shinpei Hayashi and Motoshi Saeki

11. Understanding Usages, Updates, and Risks of Static Analysis Tool in Open Source Software Projects

Yuta Minami, Akinori Ihara and Haruki Fukumoto

12. The Impact of CI Bots Installation on Code Reviews in OSS Development

Tsubasa Sunada, Yuki Ueda, Takashi Ishio, Toshiki Hirao and Kenichi Matsumoto

13. Toward Automated Refactoring of Clone Groups

Yutaro Otani, Motoshi Saeki and Shinpei Hayashi

December 14th
9:30 - 10:20 Session 3 (2 talks)

Session chair: Haruki Yokoyama

Categorizing and visualizing issue tickets to better understand the features implemented in existing software systems

Ryo Ishizuka, Hironori Washizaki, Yoshiaki Fukazawa, Shinobu Saito and Saori Ouji

Improving Clone Detection Precision using Machine Learning Techniques

Vara Arammongkolvichai, Rainer Koschke, Chaiyong Ragkhitwetsagul, Morakot Choetkiertikul and Thanwadee Sunetnanta

10:20 - 10:40 Break
10:40 - 12:10 MSR Asia Summit Session

Session chair: Akinori Ihara

12:10 - 13:30 Lunch
13:30 - 14:45 Session 4 (3 talks)

Session chair: Yutaro Kashiwa

GenProg meets Cluster Computing

Junnosuke Matsumoto, Yoshiki Higo, Hiroyuki Matsuo, Ryo Arima, Shinsuke Matsumoto and Shinji Kusumoto

Visualizing the Usage of Pythonic Idioms Over Time: A Case Study of the with open Idiom

Tattiya Sakulniwat, Raula Gaikovina Kula, Takashi Ishio, Chaiyong Ragkhitwetsagul, Morakot Choetkiertikul, Thanwadee Sunetnanta, Dong Wang and Kenichi Matsumoto

Studying Software Engineering Patterns for Designing Machine Learning Systems

Hironori Washizaki, Hiromu Uchida, Foutse Khomh and Yann-Gaël Guéhéneuc

14:45 - 15:05 Break
15:05 - 15:15 Spcial Talk from Prof. Kenichi Matsumoto
15:15 - 15:35 Award Ceremony & Closing

Special Talk Hosted by IPSJ SIGSE

This year, Special Interest Group on Software Engineering (SIGSE) of Information Processing Society of Japan (IPSJ) will host an invited talk by Dr. Mukul Prasad from Fujitsu Laboratories of America, as a part of the program of the 203rd SIGSE technical meeting. ​

Thanks to the kindness of IPSJ SIGSE, all the attendees of IWESEP 2019 can join this special talk session with no additional fee. ​ This special talk will be held on the morning of Dec. 13 at the same location of IWESEP 2019, and will be in English.

About the Talk

  • Dr. Mukul Prasad

    Intelligence-Driven Software Engineering

    Dr. Mukul Prasad

    Director in the Advanced Software & Algorithms Laboratory at Fujitsu Laboratories of America | website

    Talk Abstract

    The present-day system of developing and distributing software relies on a sophisticated ecosystem of repositories that catalog virtually every aspect and every kind of artifact of a software's lifecycle. They include version control systems for source code, to bug tracking systems, to online developer and user discussion forums. These repositories, commonly referred to as Big Code, encapsulate vast amounts of human software development expertise, experience, and opinions. This has motivated a burgeoning field of research within software engineering that uses Artificial Intelligence (AI) technologies, such as machine learning, to mine Big Code for relevant insights, i.e., "intelligence", and use it to power next-generation software engineering solutions. In this talk I will provide an overview of research conducted at Fujitsu Laboratories of America, in the area of intelligence-driven software engineering, in the context of the larger body of work in this upcoming area. In particular, I will discuss our research in the areas of automated testing, debugging and patching, as well as our efforts to translate this research into practical solutions to help real users to be more productive and to develop more correct and reliable software.

    Speaker Biography

    Dr. Mukul Prasad is a Director in the Advanced Software & Algorithms Laboratory at Fujitsu Laboratories of America. His team's research aims to develop automated tools and methodologies to empower software development teams to be more productive and to develop more correct and reliable software. He has thrice been a co-recipient of the Fujitsu Laboratories President Award for innovation. ​ He has authored over 50 conference and journal papers, including 1 best paper award and 2 best paper nominations, 1 book, and holds over 50 patents. He has served on more than 20 technical program committees of international conferences and workshops. ​ He has a Ph.D. from the University of California at Berkeley and a B.Tech from the Indian Institute of Technology at Delhi. He is a Senior Member of ACM and a Senior Member of IEEE.


Please note that IWESEP attendees are allowed to join only the special talk session. If you would like to attend the other sessions than the special talk session of the SIGSE technical meeting, you need to register for and pay for the SIGSE technical meeting.
The special talk will be in English. However, make sure that the other sessions of the SIGSE technical meeting will be mainly in Japanese since the technical meeting itself is a domestic research forum.

MSR Asia Summit Session

IWESEP 2019 will have MSR Asia Summit session, in which we will invite leading researchers in the field of MSR (Mining Software Repositories). It provides participants with opportunities to learn the cutting edge of the research related to MSR.
This year, we have invited two speakers, Dr. Fabio Palomba and Dr. Maurício Aniche. You can find the talk abstracts and the speaker biographies below.

  • Dr. Fabio Palomba

    Test code quality: A New Measure of Test Code Effectiveness

    Dr. Fabio Palomba

    Senior Research Associate at the Zurich Empirical Software Engineering Team (ZEST) of the University of Zurich | website

    Talk Abstract

    Test cases form the first line of defense against the introduction of software faults. In the past, researchers have been studying methods and tools to support developers during testing activities, mainly focusing on how to (1) measure the capabilities of test cases in finding real defects in production code and (2) provide them with automatically written tests. Despite this notable effort, the impact of test code quality has been mostly ignored. In this talk, I will show that this aspect has a critical impact on the effectiveness of tests and, particularly, on the presence of flaky tests as well as on the code coverage achievable by test cases automatically generated. Furthermore, I will discuss how test code quality can be employed for predictive analytics and assist developers when assessing test code effectiveness.

    Speaker Biography

    Dr. Fabio Palomba is an Assistant Professor of Software Engineering at the University of Salerno, Italy. He received the European PhD degree in Management & Information Technology in 2017. His PhD Thesis was the recipient of the 2017 IEEE Computer Society Best PhD Thesis Award (Italy section). His research interests include software maintenance and evolution, software testing, empirical software engineering, source code quality, and mining software repositories. He was the recipient of two ACM/SIGSOFT and one IEEE/TCSE Distinguished Paper Awards at ASE'13, ICSE'15, and ICSME'17, respectively, and Best Paper Awards at CSCW'18 and SANER'18. In 2019 he was the recipient of an SNSF Ambizione grant, one of the most prestigious individual research grants in Europe. He serves and has served as a program committee member of various international conferences (e.g., MSR, ICPC, ICSME), and as referee for various international journals (e.g., TSE, EMSE, JSS) in the field of software engineering. Fabio will be the program co-chair of ICPC 2021, which will be held in Madrid, Spain. Since 2016 he is Review Board Member of EMSE and, since 2019, Editorial Board Member of TOSEM, JSS, and SCICO. He was the recipient of five Distinguished/Outstanding Reviewer Awards for his reviewing activities conducted for EMSE, IST, and JSS between 2015 and 2019.

  • Dr. Maurício Aniche

    "Testing in production": A Gentle Walk on Monitoring Research

    Dr. Maurício Aniche

    Assistant Professor at TU Delft | website

    Talk Abstract

    "Testing in production" used to be a joke among developers. However, given the complexity of the large and distributed systems that take care of important parts of our lives, "testing in development", or, in other words, prevention, might not be enough anymore. In this talk, I'll discuss the importance of systems monitoring, logging, and log analysis to modern software systems. I'll reflect on the current state-of-the-art in industry and research fields, as well as the current open challenges. A great part of this talk is based on the research we conducted at Adyen, a large-scale payment company, that serves companies such as Facebook, Uber, and Spotify.

    Speaker Biography

    Dr. Maurício Aniche is an Assistant Professor in Software Engineering at Delft University of Technology, the Netherlands. His line of work focuses on how to make developers more productive during maintenance and testing. His research has been published in top-tier conferences (ICSE, FSE, ASE) and journals (TSE, EMSE). He always had a foot in industry. During his MSc, he co-founded Alura, the biggest e-learning platform for software engineers in Brazil. Because of Alura, he has given training and consultancy on software development and testing to 27 different companies, from 2010 to 2015. Moreover, he published three books focused on practitioners ("OOP and SOLID for ninjas", "Test-Driven Development in the real world", and "A Practical Guide on Software Testing"), which, altogether, have sold 10k copies. All these activities have given him a very particular vision on software engineering and testing should be done in practice. Now, fully dedicated to academia, he still (desires and) partners up with companies. In the last two years, he has been working closely with Adyen, a Dutch payment unicorn. His work with the company has been published in prestigious venues such as FSE, as well as ICSE's and ICSME's industry tracks.


General Co-Chairs

  • Kazuhiro Yamashita (FUJITSU LABORATORIES LTD.)

PC Co-Chairs

Publication Co-Chairs

  • Masanari Kondo (Kyoto Institute of Technology)
  • Dong Wang (Nara Institute of Science and Technology)

Registration Co-Chairs

  • Yuki Ueda (Nara Institute of Science and Technology)

Finance Chair

Publicity Co-Chairs

Web Chair


Local Arrangement Co-Chairs

  • Kazuya Aizawa (Waseda University)

Steering Committee Members

Advisory Committee Members

Program Committee Members