Introduction
This article is based on a dialogue between Professor Tsuchiya (left photo) of Osaka University, who developed the test case generation engine CIT-BACH for pairwise testing, and Senior Researcher Ishihara (right photo), who is now in charge of joint research at Baltes. is.
About “Qumias”
A high-speed combination test case generation tool that uses the combination test case generation engine CIT-BACH (sit back) developed by Professor Tsuchiya of Osaka University.
The conventional combination test case generation tool has problems such as extremely slow processing and hang-up when the prohibition becomes complicated and the number of factors/levels increases, so be careful when using it in actual business. I needed it.
In this joint research, Professor Tsuchiya of Osaka University has developed a combination test case generation engine CIT-BACH (sit back) with a powerful calculation algorithm, which has greatly improved the conventional problems. Baltes tested CIT-BACH using the factor levels and prohibitions used in the actual software development site and confirmed that the engine was practical.
Baltes also designed an easy-to-use GUI, which was made possible by an add-in that runs on Excel, which many IT engineers are accustomed to. This makes it a tool that can fully utilize the performance of the simple GUI-based CIT-BACH.
Improving the problems of conventional tools, making them more practical tools
Ishihara: We are collaborating with Baltes on combination testing, but what other research is Professor Tsuchiya doing?
Tsuchiya: The name of the laboratory is “Dependability Engineering Course”. We are researching highly reliable technologies centered on software systems.
- Originally I was doing theoretical research on distributed systems and fault tolerance, but starting from there, software testing theory and model checking are called, but it seems to be a kind of so-called formal verification, formal method. I am studying things.
Ishihara: What is your most enthusiastic research?
Tsuchiya: That would be CIT-BACH.
Ishihara: Thank you.
- Then, how is CIT-BACH positioned in “Dependability” (high reliability), which is also the name of Professor Tsuchiya’s laboratory?
Tsuchiya: There are two main ways to improve the reliability of a system: failure resistance, which makes it safe even if a failure occurs, and failure-free, in the first place. CIT-BACH will be useful research for the latter.
Ishihara: I see, my specialized software testing tends to be biased toward preventing failures, but if you don’t work on testing with an awareness of dependability, you won’t be able to achieve a truly high-quality system. hey.
Tsuchiya: Well, dependability is a big concept that includes all the attributes of trust, so it’s very important.
The advantages of CIT-BACH
Ishihara: You already knew that PICT was already available as a combination test case generation tool, but why did you decide to develop a new CIT-BACH?
Tsuchiya: I think that kinsoku short is important for the generation of combinations. Kinsoku Shori can be done with PICT, but there were some problems. But even if you try to improve it, how it is realized is not disclosed. Recently, other researchers announced a combination method that clarified the method, which is different from PICT. After seeing that method, I was wondering if there was a better way than that, and I started development to make it happen.
Ishihara: That’s amazing. If the research results are announced, it will be a big topic.
Tsuchiya: No, how about it? I hope that happens.
Ishihara: What are the advantages of CIT-BACH?
Tsuchiya: With the conventional method, we first created a combination test case without considering the prohibition, then checked whether the prohibition was satisfied one by one, and if it was satisfied, we eliminated it. CIT-BACH has the prohibition information in advance so that you can see whether the test case meets the prohibition or not in almost an instant when you refer to it without checking it one by one.
Ishihara: I knew from experience that kinky short would be extremely slow when the level of PICT increased, but that was the case. But then CIT-BACH would take a long time to create the first ban information, wouldn’t it?
Tsuchiya: I’m in the process of verifying what’s going on there. However, there are already known examples of PICT becoming extremely slow, but it’s fast.
Ishihara: Even in the test for CIT-BACH that we conducted, I had the impression that it can be used with confidence even if we include factor levels and prohibited data that we were worried about so far. Was the development done alone?
Tsuchiya: Yes, I did all the tool development myself. I asked the students to check if it would be faster.
This joint research is the first time that we have created a tool together.
Ishihara: What was the ingenuity or the part that you struggled with?
Tsuchiya: The engine part was completed unexpectedly quickly. Where I handle prohibited information, I used a Java library that can handle BDD with a graph structure. The reason why I had a hard time, or that it took a long time, is that it has nothing to do with the output file or such an engine, considering that it is called from the GUI.
Ishihara: You also responded to the request I made.
Tsuchiya: Yes, we did as much as we could. Has Mr. Ishihara ever done such joint research with a university?
Ishihara: Up until now, I have been involved with university teachers by inviting teachers to give lectures and teach them various theories. Professor Tsuchiya studied a lot this time as well, but this is the first time that we have been able to provide data and create tools together. ..
Tsuchiya: The GUI of Qumias is very nice, isn’t it? It’s easy to use if you have a GUI. I think everyone wants something easy to use in actual work, but GUI development is a difficult part of university research. It would be helpful if the company could take on such a part. What did you struggle with in developing Qumias?
Ishihara: It’s a balance between high functionality and convenience. Since we have been using combination testing as such, we want to be able to do various things, but doing so increases the number of setting screens and impairs convenience. I just want to create a combination table that covers two factors from the factor level table, but I don’t want to use it when I see various setting dialogs. Therefore, there are some functions that CIT-BACH has, such as the prototype sheet usage function, that cannot be used from Qumias.
Tsuchiya: I see. However, the prototype sheet may not be usable now, but I feel that it can be made relatively easily by expanding the function of the model file output by Qumias.
Ishihara: Yes, I made it with the idea that it is easy to use, but if there is a big voice that I want to use the method of prototype sheet or decision table on Qumias, I can put it in the next version. I’m thinking about it.
Tsuchiya: That’s wonderful. What should I do? I don’t want to have a bug in CIT-BACH.
Ishihara: We also tested CIT-BACH, so it’s okay. By the way, please tell us about the name CIT-BACH.
Tsuchiya: CIT is a commonly used abbreviation for Combinatorial Interaction Testing. Pairwise, Three-way, etc. are collectively called CIT. So, as the name, BACH seems to have forcibly stuck together in an attempt to resemble the sound of sitting back, which means “relax”. For the time being, BDD Assisted Constraint Handler means a constraint processing device that uses BDD.
Future outlook
Ishihara: Do you have any plans for future improvements to CIT-BACH?
Tsuchiya: I devised the prohibition and constraint processing, but I made the test case generation algorithm with a well-known type. I want to refine it so that faster and smaller test sets come out. I have prepared some algorithms, and after that, I am thinking of expanding it by looking at the given factor level and forbidden pattern, estimating the optimum algorithm, and switching automatically.
Ishihara: We will be conducting joint research for the next two years, and we would like to give feedback to Professor Tsuchiya about the results of using it in the actual field.
Tsuchiya: There is no “actual problem” that is convenient because there is CIT-BACH in the laboratory, and it is a problem if it is not. Therefore, you can’t study unless you formalize the problem that came up from the actual business like Mr. Baltes and thinks about the solution to it.
Ishihara: At the same time as having a Baltes test engineer use it, we will make it possible to download Qumias on a software-quality portal site called Qbook. I wish I could improve both Qumias and CIT-BACH while collecting voices and data from the field.
Tsuchiya: I would be grateful if you could do so. I think it would be nice if you could feel free to tell us that it was late under these conditions, various requests, for example, to make it faster.
Ishihara: It would be great if we could use the improved tools and research results, not just tools, to make actual work tests more accurate and more efficient. It’s a big advantage for us too. That is also the goal of joint research, so we look forward to your continued support.
Tsuchiya: Thank you for your cooperation.
Ishihara: Thank you for your time today.
If you ever want to know about similar things, check out the Facebook page Maga Techs.
